How can one get the fully qualified name of a module in Python, at the module level?
For example, let's say this is the file structure:
foo
├── setup.py
└── bar
└── baz
└── spam.py
From within the spam.py file at the module level, how can I get the fully qualified module name "bar.baz.spam"?
There is __file__, which one can pair with inspect.getmodulename(__file__) to get the module name "spam". However, how can I turn this into "bar.baz.spam"?
Try something like this:
from bar.baz import spam
print(spam.__name__)
If the name being imported is not a module, you can get the module name and object name like this:
from bar.baz.spam import MyClass
print(MyClass.__module__, MyClass.__name__)
Related
Say I have ClassA, ClassB and ClassC each within their own .py files called ClassA.py, ClassB.py and ClassC.py respectively.
# Inside ClassA.py
class ClassA:
pass
In my __init__.py I end up with
from .ClassA import ClassA
from .ClassB import ClassB
from .ClassC import ClassC
Every time I add a new class (using the same structure), I have to change the __init__.py. Is there a recommended way to automate this, given that all files in this package follow the same (not-quite-pythonic-but-oh-well) structure?
It is possible, but not recommended.
Consider the following directory structure:
├── test_module
│ ├── ClassA.py
│ ├── ClassB.py
│ └── __init__.py
We want to import ClassA.ClassA and ClassB.ClassB into __init__.py programmatically.
Assume these are the contents of ClassA.py:
class ClassA:
pass
ClassB.py is identical, save for the name of the class.
Now, say that in __init__.py we want to traverse the root directory non-recursively (not that it matters, since we have no directories) and import all classes from all modules within.
__init__.py:
import os
from importlib import import_module
my_location = os.path.dirname(__file__)
module_list = [file
for file in os.listdir(my_location)
if os.path.splitext(file)[1] == '.py'
and file != '__init__.py']
modules = [import_module(f'.{os.path.splitext(module)[0]}', __name__)
for module in module_list]
After running import test_module, the classes from ClassA.py and ClassB.py will be imported into the working namespace as test_module.ClassA.ClassA and test_module.ClassB.ClassB respectively.
To illustrate:
>>> import test_module
>>> test_module.ClassA.ClassA()
<test_module.ClassA.ClassA object at 0x7f1e66181fd0>
For completeness, if you want this script to mimic the behaviour of from X import Y:
globals().update({name: getattr(module, name)
for module in modules
for name in module.__dict__
if not name.startswith('_')})
Importing:
>>> import test_module
>>> test_module.ClassA()
<test_module.ClassA.ClassA object at 0x7fb8edb9dfd0>
These will make those names accessible as test_module.ClassA etc. (because you are importing from test_module, we add an additional layer of indirection. In test_module's scope, they are accessible directly as unqualified name.
There are additional bells and whistles we could add, such as checking each module's __all__ attribute and performing recursive traversal of sub-directories, but that is rather out of scope of this question, and I must emphasise that to my mind, it would be better to refactor your code such that this is not needed in the first place, rather than tack on functionality that plays with the internals of Python where it does not seem to be essential.
Yes, you can. This is what the importlib package is for. It contains the lower level calls that actually do the importing. It lets you do imports by calling the methods it provides.
So you could use Python os.path.listdir or whatever to learn what your subpackages are, and then make calls to importlib to import each of them.
I haven't done much of this myself. Just enough to know this exists. Maybe someone else can give you more details. Also, this link might be helpful to you:
https://dev.to/0xcrypto/dynamic-importing-stuff-in-python--1805
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.
I have a python package main and other_python_files which are like:
main/
__init__.py
lib.py
other_python_files/
__init__.py
test.py
Let lib.py contain a class called MyClass. When I do from main import lib.py and use MyClass inside test.py I get the error that MyClass is not defined.
I tried doing from main import MyClass inside the init file in the main directory but I still get the same error. What should I do to achieve importing a specific class from the lib.py file ?
You either have to import that class out of lib:
from main.lib import MyClass
Or use lib.MyClass in place of MyClass.
You can also import MyClass inside of the __init__.py file that's in main, which lets you import it the way you originally tried:
__all__ = ['MyClass']
from lib import MyClass
You can read about __all__ here: Can someone explain __all__ in Python?