Imports in an __init__.py seem to behave differently when the file is run, to when it is imported.
If we have the following files:
run.py:
import test
test/b.py:
class B(object):
pass
test/__init__.py:
from b import B
print B
print b
If we run __init__.py we get an error as I expect:
% python test/__init__.py
<class 'b.B'>
Traceback (most recent call last):
File "test/__init__.py", line 6, in <module>
print b
NameError: name 'b' is not defined
But if we run.py then we don't:
% python run.py
<class 'test.b.B'>
<module 'test.b' from '~/temp/test/b.py'>
I would expect the behaviour to be the same. Why does this work?
This only works if we do it in an __init__.py. If we:
mv __init__.py a.py
touch __init__.py
and make run.py:
import test.a
Then we do get the error.
The situation is the following: you have a script (run.py), a package test and its submodule test.b.
Whenever you import a submodule in Python, the name of that submodule is automatically stored into the parent package. So that when you do import collections.abc (or from collections.abc import Iterable, or similar), the package collections automatically gets the attribute abc.
This is what's happening here. When you do:
from b import B
the name b is automatically loaded into test, because b is a submodule of the test package.
Even if you don't do import b explicitly, whenever you import that module, the name is placed into test. Because b is a submodule of test, and it belongs to test.
Side node: your code won't work with Python 3, because relative imports have been removed. To make your code work with Python 3, you would have to write:
from test.b import B
This syntax is perfectly identical to from b import B, but is much more explicit, and should help you understand what's going on.
Reference: the Python reference documentation also explains this behavior, and includes a helpful example, very similar to this situation (the difference is just that an absolute import is used, instead of a relative import).
When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module's namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.
Let's say you have the following directory structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python's familiar name binding rules this might seem surprising, but it's actually a fundamental feature of the import system. The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former.
Related
Should I use
from foo import bar
OR
import foo.bar as bar
when importing a module and and there is no need/wish for changing the name (bar)?
Are there any differences? Does it matter?
Assuming that bar is a module or package in foo, there is no difference*, it doesn't matter. The two statements have exactly the same result:
>>> import os.path as path
>>> path
<module 'posixpath' from '/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc'>
>>> from os import path
>>> path
<module 'posixpath' from '/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc'>
If bar is not a module or package, the second form will not work; a traceback is thrown instead:
>>> import os.walk as walk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named walk
* In Python 3.6 and before, there was a bug with the initialization ordering of packages containing other modules, where in the loading stage of the package using import contained.module.something as alias in a submodule would fail where from contained.module import something as alias would not. See Imports in __init__.py and `import as` statement for a very illustrative example of that problem, as well as Python issues #23203 and #30024.
This is a late answer, arising from what is the difference between 'import a.b as b' and 'from a import b' in python
This question has been flagged as a duplicate, but there is an important difference between the two mechanisms that has not been addressed by others.
from foo import bar imports any object called bar from namespace foo into the current namespace.
import foo.bar as bar imports an importable object (package/module/namespace) called foo.bar and gives it the alias bar.
What's the difference?
Take a directory (package) called foo which has an __init__.py containing:
# foo.__init__.py
class myclass:
def __init__(self, var):
self.__var = var
def __str__(self):
return str(self.__var)
bar = myclass(42)
Meanwhile, there is also a module in foo called bar.py.
from foo import bar
print(bar)
Gives:
42
Whereas:
import foo.bar as bar
print(bar)
Gives:
<module 'foo.bar' from '/Users//..../foo/bar.py'>
So it can be seen that import foo.bar as bar is safer.
Both are technically different:
import torch.nn as nn will only import a module/package torch.nn, wheres
from torch import nn can and will prefer to import an attribute .nn from the torch module/package. Importing a module/package torch.nn is a fall.back.
In practice, it is bad style to have the same fully qualified name refer to two separate things. As such, torch.nn should only refer to a module/package. In this common case, both import statements are functionally equivalent: The same object is imported and bound to the same name.
Which one to choose comes down to preference if the target always is a module. There are practical differences when refactoring:
import torch.nn as nn guarantees .nn is a module/package. It protects against accidentally shadowing with an attribute.
from torch import nn does not care what .nn is. It allows to transparently change the implementation.
7.11. The import statement
The basic import statement (no from clause) is executed in two steps:
find a module, loading and initializing it if necessary
define a name or names in the local namespace for the
scope where the import statement occurs.
[...]
The from form uses a slightly more complex process:
find the module specified in the from clause, loading and initializing it if necessary;
for each of the identifiers specified in the import clauses:
check if the imported module has an attribute by that name
if not, attempt to import a submodule with that name and then check the imported module again for that attribute
You can use as to rename modules suppose you have two apps that have views and you want to import them
from app1 import views as views1
from app2 import views as views2
if you want multiple import use comma separation
>>> from datetime import date as d, time as t
>>> d
<type 'datetime.date'>
>>> t
<type 'datetime.time'>
The only thing I can see for the second option is that you will need as many lines as things you want to import. For example :
import foo.bar as bar
import foo.tar as tar
import foo.zar as zar
Instead of simply doing :
from foo import bar, tar, zar
The difference between import module and from module import is mainly subjective. Pick the one you like best and be consistent in your use of it. Here are some advantage of both to help you decide.
advantage of import
Less maintenance of your import statements.
Don't need to add any additional imports to start using another item from the module
advantage of from torch import nn
Less typing to use the nn
More control over which items of a module can be accessed
I am creating a custom Python module that I wanted to work similarly to something like numpy where most of the functionality can be accessed by calling np.function() or what have you. However, in creating my submodules and __init__.py file, I think that I've run into a circular import reference. Here's what I'm trying to do and the error I'm seeing, any tips to help me get this working the way I envisioned are greatly appreciated:
Let's call the module "foobar" and say I have a folder named "foobar" on the path and in this folder are three py files: "__init__.py", "foo.py", and "bar.py".
An example of submodule 1, "foo.py":
import bar
class Foo():
def __init__(self):
pass
def runfunc(self):
bar.func()
An example of submodule 2, "bar.py":
import foo
def func():
f=foo.F()
An example of __init__.py:
import numpy
from foo import *
from bar import *
__all__ = ['foo', 'bar']
So what I perceive to be my issue is that when foo is imported, it imports bar, which imports foo, which creates a loop (in my mind). I am also confused as to the use of __init__.py if, when it is called and imports other modules they cannot see each other when executing (i.e. I need to import bar into foo even though __init__.py did this).
When I run something from a script trying to import the entire module, I get the error in bar.py "ImportError: cannot import name foo". I am actually also having this issue elsewhere in the same module where I have a file with the base class "Base" and the extended class "Extended" where base.py needs to import extended.py so it can spawn instances of the expanded class and extended.py needs to import base.py so it can inherit the class.
Thanks for any help in advance!
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.
I have a Python project with the following structure:
testapp/
├── __init__.py
├── api
│ ├── __init__.py
│ └── utils.py
└── utils.py
All of the modules are empty except testapp/api/__init__.py which has the following code:
from testapp import utils
print "a", utils
from testapp.api.utils import x
print "b", utils
and testapp/api/utils.py which defines x:
x = 1
Now from the root I import testapp.api:
$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>
The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:
The from form does not bind the module name: it goes through the list
of identifiers, looks each one of them up in the module found in step
(1), and binds the name in the local namespace to the object thus
found.
And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:
>>> from os.path import abspath
>>> path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined
I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.
What is happening here?
From the import system documentation:
When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:
spam/
__init__.py
foo.py
bar.py
and spam/__init__.py has the following lines in it:
from .foo import Foo
from .bar import Bar
then executing the following puts a name binding to foo and bar in
the spam module:
>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>
Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.
If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.