I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.
Related
I have a class file fil1.py and in this file, there is only one class called f. I want to build this class file into a module for which I've tried organizing my files like this:
foo/
foo/
__init__.py
file1.py # where class resides
setup.py
README.md
LICENSE
In the __init__.py, I have one sentence:
from file1 import f
Then, when I publish on pypi and then install back to my local computer. When I try:
from foo import f
I got an error message:
ImportError: cannot import name 'f' from 'foo'
I also tried
import foo
foo.f
The error message was:
foo has no attribute f
I am not sure how to achieve what I want.
There is no such thing as a "class file" in Python. A file is a module, and modules may contain zero or more classes, functions, or valid python statements not enclosed in a class or function (which run just one time when the module is imported). That said, I think you want:
foo/file1.py:
class f:
[...]
foo/__init__.py:
from foo.file1 import f
then all of the following are valid (assuming foo is available on sys.path, which includes the current working directory automatically as the first entry):
import foo
import foo.file1
from foo import f
from foo.file1 import f
foo is a package (because it contains __init__.py) with one module called file1. Alternatively, you could use a relative import inside __init__.py:
from .file1 import f
but that can get messy and is usually not appreciated in larger projects.
To achieve the absolute import from the package foo, I did is move my class definition into py.py. See Using pip to install single class package without module.pyfile import.
root/
foo/
__init__.py # where class resides
setup.py
README.md
LICENSE
Then I can do
from foo import f #f is the class previously in the file1.py
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.
Imports in an __init__.py seem to behave differently when the file is run, to when it is imported.
If we have the following files:
run.py:
import test
test/b.py:
class B(object):
pass
test/__init__.py:
from b import B
print B
print b
If we run __init__.py we get an error as I expect:
% python test/__init__.py
<class 'b.B'>
Traceback (most recent call last):
File "test/__init__.py", line 6, in <module>
print b
NameError: name 'b' is not defined
But if we run.py then we don't:
% python run.py
<class 'test.b.B'>
<module 'test.b' from '~/temp/test/b.py'>
I would expect the behaviour to be the same. Why does this work?
This only works if we do it in an __init__.py. If we:
mv __init__.py a.py
touch __init__.py
and make run.py:
import test.a
Then we do get the error.
The situation is the following: you have a script (run.py), a package test and its submodule test.b.
Whenever you import a submodule in Python, the name of that submodule is automatically stored into the parent package. So that when you do import collections.abc (or from collections.abc import Iterable, or similar), the package collections automatically gets the attribute abc.
This is what's happening here. When you do:
from b import B
the name b is automatically loaded into test, because b is a submodule of the test package.
Even if you don't do import b explicitly, whenever you import that module, the name is placed into test. Because b is a submodule of test, and it belongs to test.
Side node: your code won't work with Python 3, because relative imports have been removed. To make your code work with Python 3, you would have to write:
from test.b import B
This syntax is perfectly identical to from b import B, but is much more explicit, and should help you understand what's going on.
Reference: the Python reference documentation also explains this behavior, and includes a helpful example, very similar to this situation (the difference is just that an absolute import is used, instead of a relative import).
When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module's namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.
Let's say you have the following directory structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python's familiar name binding rules this might seem surprising, but it's actually a fundamental feature of the import system. The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former.
Let's have this __init__.py in a Python3 package:
from .mod1 import *
from .mod2 import *
from .mod3 import *
__all__ = mod1.__all__ + mod2.__all__ + mod3.__all__
The code looks quite simple and does what is expected: it imports from modules mod1, mod2 and mod3 all symbols that these modules have put into their __all__ list and then a summary of all three __all__ lists is created.
I tried to run the very same code in a module, i.e. not in the __init__.py. It imported the three modules, but mod1, mod2 and mod3 were undefined variables.
(BTW, if you run pylint on the original __init__.py, you will get this error too.)
The same statement from .mod1 import * creates a mod1 object when executed in the __init__.py, but does not create it elsewhere. Why?
__init__.py is a special file, but till now, I thought only its name was special.
According to the documentation, this is expected behaviour:
When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module’s namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.
In other words, when you do a from .whatever import something within a module, you will magically get a whatever attribute bound to the module. Naturally, you can access module's own attributes within __init__.py as if they were defined as variables there. When you are in another module you cannot do it. In this sense __init__.py is special indeed.
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.