Let's say I have 3 files:
a.py
from d import d
class a:
def type(self):
return "a"
def test(self):
try:
x = b()
except:
print "EXCEPT IN A"
from b import b
x = b()
return x.type()
b.py
import sys
class b:
def __init__(self):
if "a" not in sys.modules:
print "Importing a!"
from a import a
pass
def type(self):
return "b"
def test(self):
for modules in sys.modules:
print modules
x = a()
return x.type()
c.py
from b import b
import sys
x = b()
print x.test()
and run python c.py
Python comes back complaining:
NameError: global name 'a' is not
defined
But, a IS in sys.modules:
copy_reg
sre_compile
locale
_sre
functools
encodings
site
__builtin__
operator
__main__
types
encodings.encodings
abc
errno
encodings.codecs
sre_constants
re
_abcoll
ntpath
_codecs
nt
_warnings
genericpath
stat
zipimport
encodings.__builtin__
warnings
UserDict
encodings.cp1252
sys
a
codecs
os.path
_functools
_locale
b
d
signal
linecache
encodings.aliases
exceptions
sre_parse
os
And I can alter b.py such that:
x = a()
changes to
x = sys.modules["a"].a()
And python will happily run that.
A couple questions arise from this:
Why does python say it doesn't know what a is, when it is in sys.modules?
Is using sys.modules a "proper" way to access class and function definitions?
What is the "right" way to import modules?
ie
from module import x
or
import module
I guess it's a problem of scoping, if you import a module in your constructor you can only use it in your constructor, after the import statement.
According to the Python documentation,
Import statements are executed in two steps: (1) find a module, and initialize it if necessary; (2) define a name or names in the local namespace (of the scope where the import statement occurs).
So the problem is that even though module a has been imported, the name a has only been bound in the scope of the b.__init__ method, not the entire scope of b.py. So in the b.test method, there is no such name a, and so you get a NameError.
You might want to read this article on importing Python modules, as it helps to explain best practices for working with imports.
In your case, a is in sys.modules.. but not everything in sys.modules is in b's scope. If you want to use re, you'd have to import that as well.
Conditional importing is occasionally acceptable, but this isn't one of those occasions. For one thing, the circular dependency between a and b in this case is unfortunate, and should be avoided (lots of patterns for doing so in Fowler's Refactoring).. That said, there's no need to conditionally import here.
b ought to simply import a. What were you trying to avoid by not importing it directly at the top of the file?
It is bad style to conditionally import code modules based on program logic. A name should always mean the same thing everywhere in your code. Think about how confusing this would be to debug:
if (something)
from office import desk
else
from home import desk
... somewhere later in the code...
desk()
Even if you don't have scoping issues (which you most likely will have), it's still confusing.
Put all your import statements at the top of your file. That's where other coders will look for them.
As far as whether to use "from foo import bar" verses just "import foo", the tradeoff is more typing (having to type "foo.bar()" or just type "bar()") verses clearness and specificity. If you want your code to be really readable and unambiguous, just say "import foo" and fully specify the call everywhere. Remember, it's much harder to read code than it is to write it.
Related
Should I use
from foo import bar
OR
import foo.bar as bar
when importing a module and and there is no need/wish for changing the name (bar)?
Are there any differences? Does it matter?
Assuming that bar is a module or package in foo, there is no difference*, it doesn't matter. The two statements have exactly the same result:
>>> import os.path as path
>>> path
<module 'posixpath' from '/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc'>
>>> from os import path
>>> path
<module 'posixpath' from '/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc'>
If bar is not a module or package, the second form will not work; a traceback is thrown instead:
>>> import os.walk as walk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named walk
* In Python 3.6 and before, there was a bug with the initialization ordering of packages containing other modules, where in the loading stage of the package using import contained.module.something as alias in a submodule would fail where from contained.module import something as alias would not. See Imports in __init__.py and `import as` statement for a very illustrative example of that problem, as well as Python issues #23203 and #30024.
This is a late answer, arising from what is the difference between 'import a.b as b' and 'from a import b' in python
This question has been flagged as a duplicate, but there is an important difference between the two mechanisms that has not been addressed by others.
from foo import bar imports any object called bar from namespace foo into the current namespace.
import foo.bar as bar imports an importable object (package/module/namespace) called foo.bar and gives it the alias bar.
What's the difference?
Take a directory (package) called foo which has an __init__.py containing:
# foo.__init__.py
class myclass:
def __init__(self, var):
self.__var = var
def __str__(self):
return str(self.__var)
bar = myclass(42)
Meanwhile, there is also a module in foo called bar.py.
from foo import bar
print(bar)
Gives:
42
Whereas:
import foo.bar as bar
print(bar)
Gives:
<module 'foo.bar' from '/Users//..../foo/bar.py'>
So it can be seen that import foo.bar as bar is safer.
Both are technically different:
import torch.nn as nn will only import a module/package torch.nn, wheres
from torch import nn can and will prefer to import an attribute .nn from the torch module/package. Importing a module/package torch.nn is a fall.back.
In practice, it is bad style to have the same fully qualified name refer to two separate things. As such, torch.nn should only refer to a module/package. In this common case, both import statements are functionally equivalent: The same object is imported and bound to the same name.
Which one to choose comes down to preference if the target always is a module. There are practical differences when refactoring:
import torch.nn as nn guarantees .nn is a module/package. It protects against accidentally shadowing with an attribute.
from torch import nn does not care what .nn is. It allows to transparently change the implementation.
7.11. The import statement
The basic import statement (no from clause) is executed in two steps:
find a module, loading and initializing it if necessary
define a name or names in the local namespace for the
scope where the import statement occurs.
[...]
The from form uses a slightly more complex process:
find the module specified in the from clause, loading and initializing it if necessary;
for each of the identifiers specified in the import clauses:
check if the imported module has an attribute by that name
if not, attempt to import a submodule with that name and then check the imported module again for that attribute
You can use as to rename modules suppose you have two apps that have views and you want to import them
from app1 import views as views1
from app2 import views as views2
if you want multiple import use comma separation
>>> from datetime import date as d, time as t
>>> d
<type 'datetime.date'>
>>> t
<type 'datetime.time'>
The only thing I can see for the second option is that you will need as many lines as things you want to import. For example :
import foo.bar as bar
import foo.tar as tar
import foo.zar as zar
Instead of simply doing :
from foo import bar, tar, zar
The difference between import module and from module import is mainly subjective. Pick the one you like best and be consistent in your use of it. Here are some advantage of both to help you decide.
advantage of import
Less maintenance of your import statements.
Don't need to add any additional imports to start using another item from the module
advantage of from torch import nn
Less typing to use the nn
More control over which items of a module can be accessed
This question already has answers here:
What can I do about "ImportError: Cannot import name X" or "AttributeError: ... (most likely due to a circular import)"?
(17 answers)
Closed 6 months ago.
I know the issue of circular imports in python has come up many times before and I have read these discussions. The comment that is made repeatedly in these discussions is that a circular import is a sign of a bad design and the code should be reorganised to avoid the circular import.
Could someone tell me how to avoid a circular import in this situation?: I have two classes and I want each class to have a constructor (method) which takes an instance of the other class and returns an instance of the class.
More specifically, one class is mutable and one is immutable. The immutable class is needed
for hashing, comparing and so on. The mutable class is needed to do things too. This is similar to sets and frozensets or to lists and tuples.
I could put both class definitions in the same module. Are there any other suggestions?
A toy example would be class A which has an attribute which is a list and class B which has an attribute which is a tuple. Then class A has a method which takes an instance of class B and returns an instance of class A (by converting the tuple to a list) and similarly class B has a method which takes an instance of class A and returns an instance of class B (by converting the list to a tuple).
Consider the following example python package where a.py and b.py depend on each other:
/package
__init__.py
a.py
b.py
Types of circular import problems
Circular import dependencies typically fall into two categories depending
on what you're trying to import and where you're using it inside each
module. (And whether you're using python 2 or 3).
1. Errors importing modules with circular imports
In some cases, just importing a module with a circular import dependency
can result in errors even if you're not referencing anything from the
imported module.
There are several standard ways to import a module in python
import package.a # (1) Absolute import
import package.a as a_mod # (2) Absolute import bound to different name
from package import a # (3) Alternate absolute import
import a # (4) Implicit relative import (deprecated, python 2 only)
from . import a # (5) Explicit relative import
Unfortunately, only the 1st and 4th options actually work when you
have circular dependencies (the rest all raise ImportError
or AttributeError). In general, you shouldn't be using the
4th syntax, since it only works in python2 and runs the risk of
clashing with other 3rd party modules. So really, only the first
syntax is guaranteed to work.
EDIT: The ImportError and AttributeError issues only occur in
python 2. In python 3 the import machinery has been rewritten and all
of these import statements (with the exception of 4) will work, even with
circular dependencies. While the solutions in this section may help refactoring python 3 code, they are mainly intended
for people using python 2.
Absolute Import
Just use the first import syntax above. The downside to this method is
that the import names can get super long for large packages.
In a.py
import package.b
In b.py
import package.a
Defer import until later
I've seen this method used in lots of packages, but it still feels
hacky to me, and I dislike that I can't look at the top of a module
and see all its dependencies, I have to go searching through all the
functions as well.
In a.py
def func():
from package import b
In b.py
def func():
from package import a
Put all imports in a central module
This also works, but has the same problem as the first method, where
all the package and submodule calls get super long. It also has two
major flaws -- it forces all the submodules to be imported, even if
you're only using one or two, and you still can't look at any of the
submodules and quickly see their dependencies at the top, you have to
go sifting through functions.
In __init__.py
from . import a
from . import b
In a.py
import package
def func():
package.b.some_object()
In b.py
import package
def func():
package.a.some_object()
2. Errors using imported objects with circular dependencies
Now, while you may be able to import a module with a circular import
dependency, you won't be able to import any objects defined in the module
or actually be able to reference that imported module anywhere
in the top level of the module where you're importing it. You can,
however, use the imported module inside functions and code blocks that don't
get run on import.
For example, this will work:
package/a.py
import package.b
def func_a():
return "a"
package/b.py
import package.a
def func_b():
# Notice how package.a is only referenced *inside* a function
# and not the top level of the module.
return package.a.func_a() + "b"
But this won't work
package/a.py
import package.b
class A(object):
pass
package/b.py
import package.a
# package.a is referenced at the top level of the module
class B(package.a.A):
pass
You'll get an exception
AttributeError: module 'package' has no attribute 'a'
Generally, in most valid cases of circular dependencies, it's possible
to refactor or reorganize the code to prevent these errors and move
module references inside a code block.
Only import the module, don't import from the module:
Consider a.py:
import b
class A:
def bar(self):
return b.B()
and b.py:
import a
class B:
def bar(self):
return a.A()
This works perfectly fine.
We do a combination of absolute imports and functions for better reading and shorter access strings.
Advantage: Shorter access strings compared to pure absolute imports
Disadvantage: a bit more overhead due to extra function call
main/sub/a.py
import main.sub.b
b_mod = lambda: main.sub.b
class A():
def __init__(self):
print('in class "A":', b_mod().B.__name__)
main/sub/b.py
import main.sub.a
a_mod = lambda: main.sub.a
class B():
def __init__(self):
print('in class "B":', a_mod().A.__name__)
Python has complicated namespaces and modules notion, so i unsure about this. Normally python module and something that is imported from it has different names or only module is imported and it's content used by fully qualified name:
import copy # will use copy.copy
from time import localtime # "localtime" has different name from "time".
But what if module has same name as something that i'm importing from it? For example:
from copy import copy
copy( "something" )
Is it safe? Maybe it's some complicated consequences that i can't see?
From PEP8 ( http://www.python.org/dev/peps/pep-0008/#imports):
When importing a class from a class-containing module, it's usually okay to spell this:
from myclass import MyClass
from foo.bar.yourclass import YourClass
If this spelling causes local name clashes, then spell them
import myclass
import foo.bar.yourclass
and use "myclass.MyClass" and "foo.bar.yourclass.YourClass".
Does it matter where modules are loaded in a code?
Or should they all be declared at the top, since during load time the external modules will have to be loaded regardless of where they are declared in the code...?
Example:
from os import popen
try:
popen('echo hi')
doSomethingIllegal;
except:
import logging #Module called only when needed?
logging.exception("Record to logger)
or is this optimized by the compiler the same way as:
from os import popen
import logging #Module will always loaded regardless
try:
popen('echo hi')
doSomethingIllegal;
except:
logging.exception("Record to logger)
This indicates it may make a difference:
"import statements can be executed just about anywhere. It's often useful to place them inside functions to restrict their visibility and/or reduce initial startup time. Although Python's interpreter is optimized to not import the same module multiple times, repeatedly executing an import statement can seriously affect performance in some circumstances."
These two OS questions, local import statements? and import always at top of module? discuss this at length.
Finally, if you are curious about your specific case you could profile/benchmark your two alternatives in your environment.
I prefer to put all of my import statements at the top of the source file, following stylistic conventions and for consistency (also it would make changes easier later w/o having to hunt through the source file looking for import statements scattered throughout)
The general rule of thumb is that imports should be at the top of the file, as that makes code easier to follow, and that makes it easier to figure out what a module will need without having to go through all the code.
The Python style guide covers some basic guidelines for how imports should look: http://www.python.org/dev/peps/pep-0008/#imports
In practice, though, there are times when it makes sense to import from within a particular function. This comes up with imports that would be circular:
# Module 1
from module2 import B
class A(object):
def do_something(self):
my_b = B()
...
# Module 2
from module1 import A
class B(object):
def do_something(self):
my_a = A()
...
That won't work as is, but you could get around the circularity by moving the import:
# Module 1
from module2 import B
class A(object):
def do_something(self):
my_b = B()
...
# Module 2
class B(object):
def do_something(self):
from module1 import A
my_a = A()
...
Ideally, you would design the classes such that this would never come up, and maybe even include them in the same module. In that toy example, having each import the other really doesn't make sense. However, in practice, there are some cases where it makes more sense to include an import for one method within the method itself, rather than throwing everything into the same module, or extracting the method in question out to some other object.
But, unless you have good reason to deviate, I say go with the top-of-the-module convention.
I have some problem, that look like some spaces in my understanding how python does import modules. For example i have module which called somemodule with two submodules a.py and b.py.
content of a.py:
from somemodule.b import b
def a():
b()
print "I'am A"
content b.py
from somemodule.a import a
def b():
a()
print "I'am B"
Now if i would like to invoke any module i get ImportError:
ImportError: cannot import name b
Whats wrong with me?
You've got a circular reference. You import module a which then imports module b. But module b imports function a from module a. But at the time it tries to do so, a has not been defined. Remember that Python import effectively executes the module.
The solution would appear to be to move the function definitions so that they appear before the imports.
Or, as #lazyr suggests, move the import statements to be inside the functions so that the import happens when the function is called, not at module import time.
You have recursive importing here:
a imports b which imports a which imports b which .....
In addition, please make sure you have an __init__.py file in the folder somemodule