I have three files in total in python 2.7:
A module file in some directory (e.g. module1.py)
A different module in the same directory, which imports this module (e.g. worker.py)
A main script in the top-level directory importing worker.py
When the file worker.py looks as the following
import module1 as mod
everything works as expected (i.e. I can use worker.mod.XXX in my main code). But when I replace the content of worker.py as follows (which I expected to do the same):
mod = __import__('module1')
I get an error: ImportError: No module named module1. I need the latter way to handle things to automate importing modules from a list.
What am I missing here?
To be more precise: I am just looking for a way to replace the statement
import module1 as mod
by an expression in which module1 is a string. I have e.g. modulname='module1', and want to import the module with the name of the module given in the string modulname. How to do that?
__import__(name, globals={}, locals={}, fromlist=[], level=-1) -> module
Import a module. Because this function is meant for use by the Python
interpreter and not for general use it is better to use
importlib.import_module() to programmatically import a module.
The globals argument is only used to determine the context;
they are not modified. The locals argument is unused. The fromlist
should be a list of names to emulate from name import ..., or an
empty list to emulate import name.
When importing a module from a package, note that __import__('A.B', ...)
returns package A when fromlist is empty, but its submodule B when
fromlist is not empty. Level is used to determine whether to perform
absolute or relative imports. -1 is the original strategy of attempting
both absolute and relative imports, 0 is absolute, a positive number
is the number of parent directories to search relative to the current module.
try this:
mod = __import__('module1', globals=globals())
Related
I have a module some_module.py which contains the following code:
def testf():
print(os.listdir())
Now, in a file named test.py, I have this code:
import os
from some_module import testf
testf()
But executing test.py gives me NameError: name 'os' is not defined. I've already imported os in test.py, and testf is in the namespace of test.py. So why does this error occur?
import is not the same as including the content of the file as if you had typed it directly in place of the import statement. You might think it works this way if you're coming from a C background, where the #include preprocessor directive does this, but Python is different.
The import statement in Python reads the content of the file being imported and evaluates it in its own separate context - so, in your example, the code in some_module.py has no access to or knowledge of anything that exists in test.py or any other file. It starts with a "blank slate", so to speak. If some_module.py's code wants to access the os module, you have to import it at the top of some_module.py.
When a module is imported in Python, it becomes an object. That is, when you write
import some_module
one of the first things Python does is to create a new object of type module to represent the module being imported. As the interpreter goes through the code in some_module.py, it assigns any variables, functions, classes, etc. that are defined in that file to be attributes of this new module object. So in your example, the module object will have one attribute, testf. When the code in the function testf wants to access the variable os, it looks in the function itself (local scope) and sees that os is not defined there, so it then looks at the attributes of the module object which testf belongs to (this is the "global" scope, although it's not truly global). In your example, it will not see os there, so you get an error. If you add
import os
to some_module.py, then that will create an attribute of the module under the name os, and your code will find what it needs to.
You may also be interested in some other answers I've written that may help you understand Python's import statement:
Why import when you need to use the full name?
Does Python import statement also import dependencies automatically?
The name testf is in the namespace of test. The contents of the testf function are still in some_module, and don't have access to anything in test.
If you have code that needs a module, you need to import that module in the same file where that code is. Importing a module only imports it into the one file where you import it. (Multiple imports of the same module, in different files, won't incur a meaningful performance penalty; the actual loading of the module only happens once, and later imports of the same module just get a reference to the already-imported module.)
Importing a module adds its name as an attribute of the current scope. Since different modules have independent scopes, any code in some_module cannot use names in __main__ (the executed script) without having imported it first.
I am having an issue with the __import__ method. It seems to only import the base directory of the module, but not the file.
For instance I have:
test_suite/assert_array_length.py
when I pass this into __import__:
moduleLocation = "test_suite.assert_array_length"
module = __import__(moduleLocation)
print module
I am getting:
[sub_directories]/test_suite/__init__.pyc
The call sequence is going from run_tests.py to test_runner.py. test_runner.py then imports assert_array_length.py. They are laid out like this:
run_tests.py
|-----------test_runner.py
|-----------assert_array_length.py
because it's importing the __init__.py, I can't get what I need from the assert_array_length.py file.
__import__ imports the module you asked for. However, if you checked the documentation, you would find the following:
When the name variable is of the form package.module, normally, the top-level package (the name up till the first dot) is returned, not the module named by name.
You may prefer importlib.import_module, which will return package.module instead of package if you tell it to import package.module.
I need a function that returns the name of the package of the module from which the function was called. Getting the module's name is easy:
import inspect
module_name = inspect.currentframe().f_back.f_globals['__name__']
And stripping the last part to get the module's package is also easy:
package_name = '.'.join(module_name.split('.')[:-1])
But if the function is called from a package's __init__.py, the last part of the name should not be stripped. E.g. if called from foo/bar/__init__.py, module_name in the above example will be set to 'foo.bar', which is already the name of the package.
How can I check, from the module name or the module object, whether it refers to a package or a module?
The best way I found is getting the module object's __file__ attribute, if it exists, and check whether it points to a file whose name is __init__ plus extension. But this seems very brittle to me.
from the module object:
module.__package__
is available for a couple that I looked at, and appears to be correct. it may not give you exactly what you want though:
import os.path
import httplib2
import xml.etree.ElementTree as etree
os.path.__package__
<blank... this is a builtin>
httplib2.__package__
'httplib2'
etree.__package__
'xml.etree'
you can use
function.__module__
also, but that gives you the module name, not the actual module - not sure if there is an easier way to get the module object itself than importing again to a local variable.
etree.parse.__module__
'xml.etree.ElementTree'
os.path.split.__module__
'ntpath'
The good thing is this appears to be more correct, following the actual location of things, e.g.:
httplib2.Response.__module__
'httplib2'
httplib2.copy.copy.__module__
'copy'
httplib2.urlparse.ParseResult.__module__
'urlparse'
etc.
This is what importlib.__import__() does, which needs to re-implement most of the Python's built-in import logic and needs to find a module's package to support relative imports:
# __package__ is not guaranteed to be defined or could be set to None
# to represent that it's proper value is unknown
package = globals.get('__package__')
if package is None:
package = globals['__name__']
if '__path__' not in globals:
package = package.rpartition('.')[0]
module = _gcd_import(name, package, level)
So it seems that there is no reliable "direct" way to get a module's package.
consider this:
/
test.py
lib/
L __init__.py
+ x/
L __init__.py
L p.py
with p.py:
class P():
pass
p1 = P()
With test.py:
import sys
import os
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), "lib"))
import lib.x.p
import x.p
print(id(lib.x.p.p1))
print(id(x.p.p1))
Here I get different object IDs though I am importing the same object from the same package/module Can someone please explain this behaviour, as it is very confusing, and I did not find any documentation about it.
Thanks!
Modules are cached in the dicitonary sys.modules using their dotted names as keys. Since you are importing the same module by two different dotted names, you end up with two copies of this module, and also with two copies of everything inside them.
The solution is easy: Don't do this, and try to avoid messing around with sys.path.
x.p and lib.x.p aren't the same module. They come from the same file, but Python doesn't determine a module's identity by its file; a module's identity is based on its package-qualified name. The module search logic may have found the same file for both modules, but they're still loaded and executed separately, and objects created in one module are distinct from objects created in another.
I've loaded one of my modules <module my_modules.mymodule from .../my_modules/mymodule.pyc> with __import__.
Now I'm having the module saved in a variable, but I'd like to create an instance of the class mymodule. Thing is - I've gotten the module name passed as string into my script.
So I've got a variable containing the module, and I've got the module name but as string.
variable_containing_module.string_variable_with_correct_classname() doesn't work. Because it says there is no such thing in the module as "string_variable_with_correct_classname" - because it doesn't evaluate the string name. How can I do so?
Your problem is that __import__ is designed for use by the import internals, not really for direct usage. One symptom of this is that when importing a module from inside a package, the top-level package is returned rather than the module itself.
There are two ways to deal with this. The preferred way is to use importlib.import_module() instead of using __import__ directly.
If you're on an older version of Python that doesn't provide importlib, then you can create your own helper function:
import sys
def import_module(module_name):
__import__(module_name)
return sys.modules[module_name]
Also see http://bugs.python.org/issue9254