I am new to python and found that I can import a module without importing any of the classes inside it. I have the following structure --
myLib/
__init__.py
A.py
B.py
driver.py
Inside driver.py I do the following --
import myLib
tmp = myLib.A()
I get the following error trying to run it.
AttributeError: 'module' object has no attribute A
Eclipse does not complain when I do this, in fact the autocomplete shows A when I type myLib.A.
What does not it mean when I import a module and not any of the classes inside it?
Thanks
P
Python is not Java. A and B are not classes. They are modules. You need to import them separately. (And myLib is not a module but a package.)
The modules A and B might themselves contain classes, which might or might not be called A and B. You can have as many classes in a module as you like - or even none at all, as it is quite possible to write a large Python program with no classes.
To answer your question though, importing myLib simply places the name myLib inside your current namespace. Anything in __init__.py will be executed: if that file itself defines or imports any names, they will be available as attributes of myLib.
If you do from myLib import A, you have now imported the module A into the current namespace. But again, any of its classes still have to be referenced via the A name: so if you do have a class A there, you would instantiate it via A.A().
A third option is to do from myLib.A import A, which does import the class A into your current namespace. In this case, you can just call A() to instantiate the class.
You need to do
from mylib import A
Because A is not an attribute of __init__.py inside mylib
When you do import mylib it imports __init__.py
See my answer.
About packages
Related
I am trying to simplify my package by making importing straightforward (like requests module does).
For that, I thought using __init__.py would be the best choice. Since, when user imports my package, __init__.py is called. Then i add little code inside, which imports object from specific module.
Please imagine, That my package and class object both have same names:
# __init__.py
from index import packagename # This is class object
myclass = myclass # This is just example to be substituted, I know it has no effect
print(myclass)
Whenever package is imported, Python logs the type of myclass:
<class 'packagename.index.packagename'> # "packagename" in the beginning is my package, "packagename" in the end is class object
However, New instance to the class is not set:
<module 'packagename' from 'packagename/__init__.pyc'>
From my research i couldn't find any significantly reliable answer for my question yet (apologies if i missed something), However, in short, How could i define variable from __init__.py so it can be used by user?
So whenever user imports packagename, variable packagename will be instance of class object and not a module.
Thanks!
Found it out.
requests package uses the same technique as i did, Their functions are imported from __init__.py right away, single example:
from .api import request, get, head, post, patch, put, delete, options
However, This functions are imported only inside from package, So they still should be called from package:
requests.post
So in my case, Package and Class Object have the same names, But unfortunately, i don't think they can replace each other in a neat way.
If my package was called Apple and Class Object inside index.py module was called Apple too.
If i have imported Apple Class Object from __init__.py, I would call it like this:
Apple.Apple (where Apple[0] is the name of package and Apple[1] is the name of class object)
I have a base class, and several sub classes that inherit from it. I am trying to detect dynamically which sub classes inherit from the base class dynamically. I am currently doing it by dynamically importing all the sub classes in the base class __init__(), and then using the __subclasses__() method.
I have the following file structure:
proj/
|-- __init__.py
|-- base.py
`-- sub
|-- __init__.py
|-- sub1.py
|-- sub2.py
`-- sub3.py
base.py:
import importlib
class Base(object):
def __init__(self):
importlib.import_module('sub.sub1')
importlib.import_module('sub.sub2')
importlib.import_module('sub.sub3')
#classmethod
def inheritors(cls):
print(cls.__subclasses__())
b = Base()
b.inheritors()
sub1.py:
import sys
import os
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from base import Base
class Sub1(Base):
pass
sub2.py:
import sys
import os
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from base import Base
class Sub2(Base):
pass
and finally sub3.py:
import sys
import os
class Sub3(object):
pass
You will notice that sub.sub1.Sub1 and sub.sub2.Sub2 both inherit from base.Base while sub.sub3.Sub3 does not.
When I open IPython3, and run import base I get the following output:
In [1]: import base
[<class 'sub.sub1.Sub1'>, <class 'sub.sub2.Sub2'>]
The output above is exactly as I would expect it to be. It gets weird when I run base.py using Python command line:
python3 base.py
[<class 'sub.sub2.Sub2'>]
[]
Now I think that I understand that there are two prints in the second case because the Python importer initially does not see base.py in the sys.modules global variable, so when a subclass is imported it will import base.py again and the code will be executed a second time. This explanation does not explain why the first time it prints [<class 'sub.sub2.Sub2'>] and not [<class 'sub.sub1.Sub1'>] as sub.sub1.Sub1 is imported first, and it does not explain why only sub.sub2.Sub2 appears in the __subclasses__() while sub.sub1.Sub1 does not.
Any explanation that would help me understand how Python works in this regard will be greatly appreciated!
EDIT: I would like to run the module using python base.py, so maybe I can be pointed in the correct direction for that?
You made a knot.
A complicated, uneeded knot. I could figure it out - but I don't know if I can keep it in mind to explain what is going on in a clear way :-)
But one thing first: this has less to do with "inheritance detection", andvall to do with the import system - which you tied in a complicated knot.
So, you get the unexpected result because when you do python base.py, the contents of base are recorded as the module named __main__ in sys.modules.
Ordinarily, Python will never import the module and run the same code again: upon fiding an import statement that tries to import an existing module, it just creates a new variable poiting to the existing module. If that module did not finish the execution of its body yet, not all classes or variables will be seem on the place where there is the second import statement. Calls to importlib do no better - they just don t automate the variable biding part. When you do circular imports, change the import path, and import a module named base from another file, Python does not know this is the same base that is __main__. So, the new one gets a new fresh import, and a second entry in sys.modules,as base.
If you just print the __class__ in your inheritors method, it will be clear:
#classmethod
def inheritors(cls):
print("At class {}. Subclasses: {}".format(__class__, cls.__subclasses__()))
Then you will see that "base.Base" has the "sub2" subclass and __main__.Base has no subclasses.
Now, let me try to put the timeline for it:
base.py is imported as __main__ and runs up to the line b =
Base(). At this point the __init__ method of Base will import the
submodules
submodule sub1 is run, changes the sys.path, and
re-imports base.py as the base module.
The contents of the
base module are run until the __init__ method in base.Base is met;
therein, it imports sub.sub1,and Python finds out this module has
already been imported and is in sys.modules. Its code has not been
completed, and the Sub1 base is not yet defined, though.
Inside the sub1 import of base, __init__ tries to import sub.sub2. That
is a new module to Python, so it is imported
On the import of
sub2, when import base is met, Python recognizes the module as
imported already (although, again, not all the initialization code
is complete)- it just brings the name alias to sub2 globals, and
keeps on
Sub2 is defined as subclass of base.Base
sub.sub2 import finishes, and Python resumes to the __init__ method on step (4); Python imports sub.sub3 and resumes to the b.inheritors() call
(from base, not from main). At this point the only subclass of
base.Base is sub2 - that is printed
The importing of
base.py as base finishes, and Python resumes executing the bodu
of sub.sub1- class Sub1 is defined as a subclass of base.Base
Python resumes the __main__.base.__init__ execution, imports
sub.sub2 - but it is already run, the same for sub.sub3
__main__.Base.inheritors is called in __main__, and prints no
sub-classes.
And that is the end of a complicated history.
What you should be doing
first: if you need to do the sys.path.append trickery, there is something wrong with your package. Let your package be proj, and point proj.__init__ to import base if you want that to be run (and dynamically import the other modules) - but stop fidling with sys.path to find things in your own package.
second:
the cls.__subclasses__ call is of little use, as it will only tell you about the imediate subclasses of cls - if there is a grand-chid subclass it will go unoticed,
The most usual pattern is to have a register of subclasses of your Base - an as they are created, just add the new classes to this record. This can be done with a metaclass, in Python < 3.6, or with the __init_subclass__ method on Python 3.6 and on.
Let's have this __init__.py in a Python3 package:
from .mod1 import *
from .mod2 import *
from .mod3 import *
__all__ = mod1.__all__ + mod2.__all__ + mod3.__all__
The code looks quite simple and does what is expected: it imports from modules mod1, mod2 and mod3 all symbols that these modules have put into their __all__ list and then a summary of all three __all__ lists is created.
I tried to run the very same code in a module, i.e. not in the __init__.py. It imported the three modules, but mod1, mod2 and mod3 were undefined variables.
(BTW, if you run pylint on the original __init__.py, you will get this error too.)
The same statement from .mod1 import * creates a mod1 object when executed in the __init__.py, but does not create it elsewhere. Why?
__init__.py is a special file, but till now, I thought only its name was special.
According to the documentation, this is expected behaviour:
When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module’s namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.
In other words, when you do a from .whatever import something within a module, you will magically get a whatever attribute bound to the module. Naturally, you can access module's own attributes within __init__.py as if they were defined as variables there. When you are in another module you cannot do it. In this sense __init__.py is special indeed.
I have a python source file with a class defined in it, and a class from another module imported into it. Essentially, this structure:
from parent import SuperClass
from other import ClassA
class ClassB(SuperClass):
def __init__(self): pass
What I want to do is look in this module for all the classes defined in there, and only to find ClassB (and to overlook ClassA). Both ClassA and ClassB extend SuperClass.
The reason for this is that I have a directory of plugins which are loaded at runtime, and I get a full list of the plugin classes by introspecting on each .py file and loading the classes which extend SuperClass. In this particular case, ClassB uses the plugin ClassA to do some work for it, so is dependent upon it (ClassA, meanwhile, is not dependent on ClassB). The problem is that when I load the plugins from the directory, I get 2 instances of ClassA, as it gets one from ClassA's file, and one from ClassB's file.
For packages there is the approach:
__all__ = ['module_a', 'module-b']
to explicitly list the modules that you can import, but this lives in the __init__.py file, and each of the plugins is a .py file not a directory in its own right.
The question, then, is: can I limit access to the classes in a .py file, or do I have to make each one of them a directory with its own init file? Or, is there some other clever way that I could distinguish between these two classes?
You meant "for packages there is the approach...". Actually, that works for every module (__init__.py is a module, just with special semantics). Use __all__ inside the plugin modules and that's it.
But remember: __all__ only limits what you import using from xxxx import *; you can still access the rest of the module, and there's no way to avoid that using the standard Python import mechanism.
If you're using some kind of active introspection technique (eg. exploring the namespace in the module and then importing classes from it), you could check if the class comes from the same file as the module itself.
You could also implement your own import mechanism (using importlib, for example), but that may be overkill...
Edit: for the "check if the class come from the same module":
Say that I have two modules, mod1.py:
class A(object):
pass
and mod2.py:
from mod1 import A
class B(object):
pass
Now, if I do:
from mod2 import *
I've imported both A and B. But...
>>> A
<class 'mod1.A'>
>>> B
<class 'mod2.B'>
as you see, the classes carry information about where did they originate. And actually you can check it right away:
>>> A.__module__
'mod1'
>>> B.__module__
'mod2'
Using that information you can discriminate them easily.
Here's a python module. foo is in sys.path.
foo\
__init__.py
bar\
__init__.py
base.py
class Base(object)
derived.py
import foo.bar.base as base
class Derived(base.Base)
I've got nothing fancy going on yet. If I want to instantiate the Derived class from the derived module, I can do that easily enough:
import foo.bar.derived as derived
print(derived.Derived())
However, I'd like to just import the bar module and call bar.Derived(), because I plan to have lots of classes within lots of different modules, and I don't want to deal with all these tentacular import paths. My understanding is that I can simply import Derived into the namespace of the bar module, by modifying my project like so:
foo\
__init__.py
bar\
__init__.py
from foo.bar.derived import Derived
base.py
class Base(object)
derived.py
import foo.bar.base as base
class Derived(base.Base)
Now I should be able to do the following:
import foo.bar as bar
print(bar.Derived())
But I get an AttributeError complaining that the foo module has no submodule called bar:
test.py (1): import foo.bar
foo\bar\__init__.py (1): from foo.bar.derived import Derived
foo\bar\derived.py (1): import foo.bar.base as base
AttributeError: 'module' object has no attribute 'bar'
In fact, my original test code (at top) doesn't work either! As soon as I try to import foo.bar, I get errors.
What I can gleam from this error is that the import statement in __init__.py causes derived.py to be executed before bar is fully loaded, and therefore it can't import the module (also from bar) which contains its own base class. I'm coming from the C++ world, where ultra-nested namespaces aren't as integral and a simple forward declaration would negate this problem, but I've been led to believe that what I'm looking for is possible and at least a somewhat acceptably Pythonic solution. What am I doing wrong? What's the correct way to make classes from a submodule available in the parent module's namespace?
If you're working with Python 2.5 or later, try using explicit relative imports (http://www.python.org/dev/peps/pep-0328/#guido-s-decision):
test.py (1): import foo.bar
foo\bar\__init__.py (1): from .derived import Derived
foo\bar\derived.py (1): from . import base
(Note that if you are indeed working with Python 2.5 or 2.6, you'll need to include from __future__ import absolute_import in your modules.)
in derived.py, use this:
EDIT: as JAB pointed out, implicit relative imports are deprecated, to the following isn't recommended (although it does work still in Python 2.7 - with no deprecation errors!).
import base # this is all you need - it's in the current directory
Instead, use:
from . import base #
(or)
from foo.bar import base
instead of:
import foo.bar.base as base
This will solve both your errors (since they're from the same issue). Your import doesn't work since there is no base function or class inside the foo.bar.base module.