The primary problem I'm trying to solve is how to detect all the subclasses of a particular class. The reason I'm unable to use __subclasses__ is that the child classes aren't yet accessible in the context from which I'm attempting to access them.
The folder structure I'm working with looks like this:
main.py
projects/
__init__.py
project.py
some_project_child.py
What I'd like to do is get a list of all subclasses of Project (defined in project.py) from main.py.
I'm able to do this by doing:
from projects.project import Project
from projects.some_project_child import SomeProjectChild
Project.__subclasses__
The aspect of this approach I'd like to avoid is having to add an import line every time I add a new project file/class. I realize I can do this by iterating over the files in the directory and import each one's contents but is there a cleaner more pythonic manner of handling this?
It is the nature of a non-compiled language like Python that it is impossible to do anything like this without importing. There is simply no way for Python to know what subclasses any class has without executing the files they are defined in.
Related
This is probably a vestige from my days as a c# programmer, and I probably just need to learn the "pythonic" way of doing things. I'm relatively new to python, but have been coding in c# for years.
I have a python module within my project named 'applib' where I store a number of classes. Each class is contained within its own file of the same name (e.g. Foo.py has a class named Foo, Bar.py has a class named Bar, etc).
When I want to use a class, I have to say
import applib.Foo
followed by something like
my_foo=applib.Foo.Foo()
This results in having to specify "Foo" twice in my declaration (once for the module name, and then again for the class). In c# I could control this with a namespace directive at the top of the class file, but in python I don't know of a way to control this except for putting all my classes in a single file. I know I can also import the class by using
from applib.Foo import Foo
but that's still a duplication of namespace elements, which I want to avoid completely.
Can anyone provide me with some recommendations on a more pythonic methodology for namespace structure within an app?
Obviously, registering classes in Python is a major use-case for metaclasses. In this case, I've got a serialization module that currently uses dynamic imports to create classes and I'd prefer to replace that with a factory pattern.
So basically, it does this:
data = #(Generic class based on serial data)
moduleName = data.getModule()
className = data.getClass()
aModule = __import__(moduleName)
aClass = getattr(aModule, className)
But I want it to do this:
data = #(Generic class based on serial data)
classKey = data.getFactoryKey()
aClass = factory.getClass(classKey)
However, there's a hitch: If I make the factory rely on metaclasses, the Factory only learns about the existence of classes after their modules are imported (e.g., they're registered at module import time). So to populate the factory, I'd have to either:
manually import all related modules (which would really defeat the purpose of having metaclasses automatically register things...) or
automatically import everything in the whole project (which strikes me as incredibly clunky and ham-fisted).
Out of these options, just registering the classes directly into a factory seems like the best option. Has anyone found a better solution that I'm just not seeing? One option might be to automatically generate the imports required in the factory module by traversing the project files, but unless you do that with a commit-hook, you run the risk of your factory getting out of date.
Update:
I have posted a self-answer, to close this off. If anyone knows a good way to traverse all Python modules across nested subpackages in a way that will never hit a cycle, I will gladly accept that answer rather than this one. The main problem I see happening is:
\A.py (import Sub.S2)
\Sub\S1.py (import A)
\Sub\S2.py
\Sub\S3.py (import Sub.S2)
When you try to import S3, it first needs to import Main (otherwise it won't know what a Sub is). At that point, it tries to import A. While there, the __init__.py is called, and tries to register A. At this point, A tries to import S1. Since the __init__.py in Sub is hit, it tries to import S1, S2, and S3. However, S1 wants to import A (which does not yet exist, as it is in the process of being imported)! So that import fails. You can switch how the traversal occurs (i.e., depth first rather than breadth first), but you hit the same issues. Any insight on a good traversal approach for this would be very helpful. A two-stage approach can probably solve it (i.e., traverse to get all module references, then import as a flat batch). However, I am not quite sure of the best way to handle the final stage (i.e., to know when you are done traversing and then import everything). My big restriction is that I do not want to have a super-package to deal with (i.e., an extra directory under Sub and A). If I had that, it could kick off traversal, but everything would need to import relative to that for no good reason (i.e., all imports longer by an extra directory). Thusfar, adding a special function call to sitecustomize.py seems like my only option (I set the root directory for the package development in that file anyway).
The solution I found to this was to do all imports on the package based off of a particular base directory and have special __init__.py functions for all of the ones that might have modules with classes that I'd want to have registered. So basically, if you import any module, it first has to import the base directory and proceeds to walk every package (i.e., folder) with a similar __init__.py file.
The downside of this approach is that the same modules are sometimes imported multiple times, which is annoying if anyone leaves code with side effects in a module import. However, that's bad either way. Unfortunately, some major packages (cough, cough: Flask) have serious complaints with IDLE if you do this (IDLE just restarts, rather than doing anything). The other downside is that because modules import each other, sometimes it attempts to import a module that it is already in the process of importing (an easily caught error, but one I'm still trying to stamp out). It's not ideal, but it does get the job done. Additional details on the more specific issue are attached, and if anyone can offer a better answer, I will gladly accept it.
I an writing an app to record data given by the user, calculate and print out bills. I have written a script usage.py in MainFolder and want to pull its methods or functions in MainFolder/file/MainFile.py . I have already used the methods or functions in usage.py in MainWin.py
The scripts are in the 3 py extensions are pasted here
How can I use them in methods in MainFolder/usage.py in MainFolder/file/MainFile.py ?
If you make the folder a package that will allow you to use the methods in either location. But, to give further advice it would make more sense to move the methods that are in usage.py into the MainFile.py from a design standpoint. It doesn't make a lot of sense to use functions in a top level file in a nested file.
If you make it a module you can use:
from some.package.usage import policySize
Module documentation:
http://docs.python.org/tutorial/modules.html
I'm developing a Python application for the GAE.
The application consists of a bunch of classes and functions which are at the moment all in the same file main.py.
The application is running without problems.
Now, I want to refactor the application and outsource all the classes. Every class should be in her own file. The files shall be arranged in directories like this:
main.py
/directory1/class1.py
/directory1/class2.py
/directory2/class1.py
My problem is that inside these outsourced classes, I cannot use the functions of main.py.
I tried this inside the class-files.
from main import name_of_function
But the compiler says
from main import name_of_function
ImportError: cannot import name name_of_function
What did I wrong?
The name of the funktion is login. Maybe this causes the problem?
Try moving the extra functions from main.py into a separate file.
main.py
library.py # contains login() and other functions from main
/directory1/class1.py
/directory1/class2.py
/directory2/class1.py
Sometimes it is good to leave classes in same module not separate without purpose if they belong together.
The problem of using function from main is sign that you should refactor one module say common_utils.py out of those functions and separate it from main. You can import that to your modules, which use those. Do not think classes only think whole use case.
If you could give pseudo code of your program's logic, we could check the refactoring better together.
I'm learning Python and I have been playing around with packages. I wanted to know the best way to define classes in packages. It seems that the only way to define classes in a package is to define them in the __init__.py of that package. Coming from Java, I'd kind of like to define individual files for my classes. Is this a recommended practice?
I'd like to have my directory look somewhat like this:
recursor/
__init__.py
RecursionException.py
RecursionResult.py
Recursor.py
So I could refer to my classes as recursor.Recursor, recursor.RecursionException, and recursor.RecursionResult. Is this doable or recommended in Python?
Go ahead and define your classes in separate modules. Then make __init__.py do something like this:
from RecursionException import RecursionException
from RecursionResult import RecursionResult
from Recursor import Recursor
That will import each class into the package's root namespace, so calling code can refer to recursor.Recursor instead of recursor.Recursor.Recursor.
I feel the need to echo some of the other comments here, though: Python is not Java. Rather than creating a new module for every class under the sun, I suggest grouping closely related classes into a single module. It's easier to understand your code that way, and calling code won't need a bazillion imports.
This is perfectly doable. Just create a new class module for each of those classes, and create exactly the structure you posted.
You can also make a Recursion.py module or something similar, and include all 3 classes in that file.
(I'm also new to Python from Java, and I haven't yet put anything in my __init__.py files...)
In Python you're not restricted to defining 1 class per file and few do that. You can if you want to though - it's totally up to you. A Package in Python is just a directory with an
__init__.py
file. You don't have to put anything in that file you can to control what gets imported etc.