We Keep Coding

What is the correct way to write a class that pulls in sub-classes (methods) from several files

I have written a class as a central data management engine for a register map. I have written sub-classes (well more like extensions) which provide various input parsers and output formatters. However they rely on being able to access the internal variables of the main class in order to operate. I would like to keep the extensions separate from the main class but be able to advertise all the methods to the instantiator of the main class.
It all works fine actually except that I get many errors from Pylint of the form:
E:275, 8: Instance of 'RegMapXLS' has no 'log' member (no-member)
where log is a shared function in the main class.
# typical "extension" class
class RegMapXLS():
''' The part of RegisterMap class which contains the xlsx source file
parsing functions
'''
def parse_config(self, configsheet):
# main class including all "extension class inherritance"
class RegisterMap(RegMapXLS, RegMapPy, RegMapDocx, RegMapVhdl):
'''Register map bundling settings as instances of the class Setting.
'''
# only this class has an __init__ call to instantiate it and its code
def __init__(self, name, output_path, report=None):
I could of course write the whole thing as one big class but for maintainability reasons I have split them up into several files. The other way would be to write a "base class" which all the "sister"-classes inherit and create an empty class which simply inherits all of the parts (though its not clear then how I call the init method 2 levels down). My question is what is the best/Pythonic way and for maintenance and readability.
What I want in the end is a simple UI where I can write:
with RegisterMap(udi, output_path=output_path) as regmap:
regmap.parse_source(regmap_source)
#these methods are pulled in from other source files/authors
regmap.export_HDF5(output_path)
regmap.to_vhdl(regmap_reg_pkg_template)
regmap.to_pin()
regmap.to_docx(doc_template=doc_template)

Can I use a python metaclass to keep track of subclasses in separate files?

I am new to the community. In my Django project, I have a module called "social" in which I have several files like facebook.py, and twitter.py, each with classes that are subclasses of "SocialProvider"
Example
class Facebook(SocialProvider):
Also in my social module is a file called "helper.py" which contains a SocialProviderHelper, which I would like to keep track of all SocialProvider subclasses, and generate different lists, and identifiers for them. For example.. the SocialProviderHelper could "find" Facebook and Twitter and add them to various lists like cool_providers = [] popular_providers = [] old_providers, etc, etc. The goal is to make a universal SocialProviderHelper that other areas of code can import and use... for example.. in a view I may want to get all the "popular_providers" names.. so I would import SocialProviderHelper.. and do something like
helper = SocialProviderHelper()
helper.getOldProviders()
In my SocialProviderHelper, I have:
class SocialProviderHelper(object):
_providers = SocialProvider.__subclasses__()
but
subclasses()
returns an empty list seemingly because the "providers" are defined in other files that haven't been imported, and subclasses() only maintains weak references to the class if they are "alive".
Is there any way around this ?
**The end goal is to be able to whip up another subclass of SocialProvider anywhere simply by doing..
import SocialProvider
class NewSocialNetwork(SocialProvider)
...and have the SocialProviderHelper know of its existence.**
I have tried to implement "subclasses() with a metaclass that keeps a registry of subclasses, but a similar problem persists.

This could be done with a decorator:
from SocialProvider import SocialProvider
#SocialProvider.Register
class NewSocialProvider( SocialProvider ):
pass
I've assumed SocialProvider is a superclass written by you. If so, in your superclass file, the Register helper could be defined as a class method:
class SocialProvider:
subclasses = []
#classmethod
def Register( cls, subcl ):
cls.subclasses.append( subcl )
return subcl
If not, then you could still define a function somewhere, to be used as a decorator, but it and the list it appends to would have to be defined somewhere else, separate from the SocialProvider class. You mention the problem of not-yet-imported files. There's nothing you can do to detect a class that hasn't been defined yet, so you'd have to make sure that the files with the subclass definitions are imported (e.g. via statements like import facebook etc in the ___init__.py file of your module).

Call a method in an instance that a calling class implements - but is otherwise unrelated

Say I have this file structure:
project
gui
gui.py
widgets
base_widg.py
midi
main.py
main.py contains a class called Engine, and a method inside that, called midi_out
base_widg.py contains a class called BaseWidg
gui.py is the file that I run when I want to test the entire program, and implements both an instance of the Engine class in main.py and an instance of the BaseWidg class in base_widg.py
There is a method inside BaseWidg that needs to call the midi_out function in the implemented instance of Engine
EDIT
in the init function of Engine:
self.midiout = rtmidi.MidiOut()
This is important because many of the other methods need this declaration, including midi_out, because:
def midi_out(self, note, vel, state):
# figure out what channel is (irrelevant)
self.midiout.send_message([channel, note, vel)
Unfortunately this tends to suggest to me I cannot use #staticmethod...
What is the most pythonic and efficient way to do this? Do I need to change the structure of my program?
Potentially irrelevant: I am using python 2.7, pygtk 2 and python_rtmidi.

One option is to have the constructor of BaseWidg take an instance of Engine and store it as an instance variable. Then you can just call the midi_out() method of that instance of Engine from within BaseWidg.

GAE, Python: How to define db object classes in only one place?

I have a main.py which contains class definitions for objects that are fetched from db and displayed.
I also have a scrape.py that fetches these same sorts of objects from the web, and stores them to the db.
How do I avoid having to have class definitions for these objects in both main.py and scrape.py?

Put the classes in a separate module (file) and import them in both of the other files.
models.py (new file)
class MyModel(object):
pass # Implementation here
scrape.py or main.py
from models import MyModel, SomeOtherModel
m = MyModel
m.put()

Make a seperate .py file called something like classes and variables, place all classes you use in your code in that file and call it upon startup for both files.

Laying out MVC classes in Python

I'm working on a project in Python, and I'm trying to follow a somewhat-strict MVC pattern for practice. My idea was to separate things out into packages named model, view and controller, plus have a Model, View and Controller class for each. Each package would have the supporting files for each piece.
My problem is that I want to be able to use them like:
from controller import Controller
And then in the file with the Controller class I can:
from controller.someclass import SomeClass
But if I put them in packages with the same name, I get problems. I read up about how modules work, and realized I needed to name them controller/__init__.py, model/__init__.py and view/__init__.py, but it seems weird to put them in that file, and it's kind of annoying that all of them show up gedit as __init__.py
Is there any better way of doing this? Am I going about this the right way?

I've seen some black magic in the django source that pulls classes from a base.py file into the __init__.py namespace. However I'm not sure how that's done. ( See comments for how to do this. )
From what I do know, you can do one of two things.
A -
inside bar/controller/__init__.py
import os,sys
# Make sure the interpreter knows were your files are.
sys.path.append(os.path.join(os.path.dirname(__file__),'../')
from bar.controller import Controller
from bar.model import Model
from bar.view import View
class Controller(object):
model = Model()
view = View()
And now you make bar/model/__init__.py and bar/view/__init__.py
B -
inside bar/controller/__init__.py
class Model(object):
pass
class View(object):
pass
class Controller(object):
model = Model()
view = View()
Edit:...
After reading your comment, a third option comes to mind. A package doesn't litertly translate into a module in python. I think your desired result is to create a directory structure like this:
bar/
__init__.py
controller.py
model.py
view.py
Then inside controller.py
import os,sys
from bar.controller import Controller
from bar.model import Model
from bar.view import View
class Controller(object):
model = Model()
view = View()
This was a huge hurdle for me to get coming from java. Your class file names do not have to match the class name. Think of them as a step, you step into the folder(module) and then into the file(.py) and then you import your class.(Model(object))

If I understand correctly, all you're interested in doing here is having this happen:
from controller import Controller
without having the Controller class defined in controller/__init__.py is that right?
If so, then just do this:
In controller/base.py: (notice there is a file called base.py or something else)
class Controller(BaseClass):
# define Controller here
In controller/__init__.py:
from base import Controller
Now you can have the exact syntax you are looking for.