I know I can use importlib to import modules via a string. How can I recreate the import * functionality using this library? Basically, I want something like this:
importlib.import_module('path.to.module', '*')
My reasons for not name-spacing the imported attributes are deliberate.
Here is a solution: import the module, then one by one make alias in the current namespace:
import importlib
# Import the module
mod = importlib.import_module('collections')
# Determine a list of names to copy to the current name space
names = getattr(mod, '__all__', [n for n in dir(mod) if not n.startswith('_')])
# Copy those names into the current name space
g = globals()
for name in names:
g[name] = getattr(mod, name)
Here is shorten version for #HaiVu answer which refers to this solution from #Bakuriu
import importlib
# import the module
mod = importlib.import_module('collections')
# make the variable global
globals().update(mod.__dict__)
Note:
This will import lots of things beside the user-defined variables
#HaiVu solution did the best of it ie. only import user-defined variables
Related
I'd like to load a module dynamically, given its string name (from an environment variable). I'm using Python 2.7. I know I can do something like:
import os, importlib
my_module = importlib.import_module(os.environ.get('SETTINGS_MODULE'))
This is roughly equivalent to
import my_settings
(where SETTINGS_MODULE = 'my_settings'). The problem is, I need something equivalent to
from my_settings import *
since I'd like to be able to access all methods and variables in the module. I've tried
import os, importlib
my_module = importlib.import_module(os.environ.get('SETTINGS_MODULE'))
from my_module import *
but I get a bunch of errors doing that. Is there a way to import all methods and attributes of a module dynamically in Python 2.7?
If you have your module object, you can mimic the logic import * uses as follows:
module_dict = my_module.__dict__
try:
to_import = my_module.__all__
except AttributeError:
to_import = [name for name in module_dict if not name.startswith('_')]
globals().update({name: module_dict[name] for name in to_import})
However, this is almost certainly a really bad idea. You will unceremoniously stomp on any existing variables with the same names. This is bad enough when you do from blah import * normally, but when you do it dynamically there is even more uncertainty about what names might collide. You are better off just importing my_module and then accessing what you need from it using regular attribute access (e.g., my_module.someAttr), or getattr if you need to access its attributes dynamically.
Not answering precisely the question as worded, but if you wish to have a file as proxy to a dynamic module, you can use the ability to define __getattr__ on the module level.
import importlib
import os
module_name = os.environ.get('CONFIG_MODULE', 'configs.config_local')
mod = importlib.import_module(module_name)
def __getattr__(name):
return getattr(mod, name)
My case was a bit different - wanted to dynamically import the constants.py names in each gameX.__init__.py module (see below), cause statically importing those would leave them in sys.modules forever (see: this excerpt from Beazley I picked from this related question).
Here is my folder structure:
game/
__init__.py
game1/
__init__.py
constants.py
...
game2/
__init__.py
constants.py
...
Each gameX.__init__.py exports an init() method - so I had initially a from .constants import * in all those gameX.__init__.py which I tried to move inside the init() method.
My first attempt in the lines of:
## -275,2 +274,6 ## def init():
# called instead of 'reload'
+ yak = {}
+ yak.update(locals())
+ from .constants import * # fails here
+ yak = {x: y for x,y in locals() if x not in yak}
+ globals().update(yak)
brec.ModReader.recHeader = RecordHeader
Failed with the rather cryptic:
SyntaxError: import * is not allowed in function 'init' because it contains a nested function with free variables
I can assure you there are no nested functions in there. Anyway I hacked and slashed and ended up with:
def init():
# ...
from .. import dynamic_import_hack
dynamic_import_hack(__name__)
Where in game.__init__.py:
def dynamic_import_hack(package_name):
print __name__ # game.init
print package_name # game.gameX.init
import importlib
constants = importlib.import_module('.constants', package=package_name)
import sys
for k in dir(constants):
if k.startswith('_'): continue
setattr(sys.modules[package_name], k, getattr(constants, k))
(for setattr see How can I add attributes to a module at run time? while for getattr How can I import a python module function dynamically? - I prefer to use those than directly access the __dict__)
This works and it's more general than the approach in the accepted answer cause it allows you to have the hack in one place and use it from whatever module. However I am not really sure it's the best way to implement it - was going to ask a question but as it would be a duplicate of this one I am posting it as an answer and hope to get some feedback. My questions would be:
why this "SyntaxError: import * is not allowed in function 'init'" while there are no nested functions ?
dir has a lot of warnings in its doc - in particular it attempts to produce the most relevant, rather than complete, information - this complete worries me a bit
is there no builtin way to do an import * ? even in python 3 ?
My problem is quite easy but I can not find a good answer because the search engines are ambiguous on the term "module".
What I want do to is roughly this :
Module : a.py
x = 2
Module : b.py
import a
Now, I want to be able to access x from b without using qualified name (i.e. without typing a.x, just with x). In my situation I cannot use :
from a import x
because I don't know which elements a will contains. I can not use
from a import *
neither. Is there any simple way to merge or join the modules (I mean the Object Modules) ?
This is not a good idea, but you can use:
globals().update(vars(a))
to add all names defined in the a module to your local namespace. This is almost the same as from a import *. To emulate from a import * exactly, without using from a import * itself, you'd have to use:
globals().update(p for p in vars(a).items() if p[0] in getattr(a, '__all__', dir(a)))
You normally just would use x = a.x or from a import x.
If you are using zipimport you don't have to do any of this. Just add the path to the archive to your sys.path:
import sys
sys.path.insert(0, '/path/to/archive.zip')
from test import x
I have a long module name and I want to avoid having to type it all over many times in my document. I can simply do import long_ass_module_name as lamn and call it that way. However, this module has many submodules that I wish to import and use as well.
In this case I won't be able to write import lamn.sub_module_1 because python import does not recognize this alias I made for my long_ass_module_name. How can I achieve this?
Should I simply automatically import all submodules in my main module's __init__.py?
An aliased object still changes when you import submodules,
import my_long_module_name as mlmn
import my_long_module_name.submodule
mlmn.submodule.function()
The import statement always takes the full name of the module. The module is just an object, and importing a submodule will add an attribute to that object.
This (highly unrecommendable) way of importing all the members of an object to the current namespace works by looking up the vars() dictionary:
import my_bad_ass_long_module.bafd as b
# get __dict__ of current namespace
myn = vars()
for k,v in vars(b).items():
# populate this namespace with the all the members of the b namespace (overwriting!)
myn[k] = v
How can I make this import:
from module import *
with imp module?
Disclaimer:
I need use imp module because I need make several statements and I want to do it dynamicly
I need use * because I need that in the file I made the import, the variables and methods defined in module be available directly, i mean without module.method or module.variable. And I want import all variables and methods in the module because I don't know what methods or variables can be in the module in the future
Here!
def load_everything_from(module_names):
g = globals()
for module_name in module_names:
m = __import__(module_name)
names = getattr(m, '__all__', None)
if names is None:
names = [name for name in dir(m) if not name.startswith('_')]
for name in names:
g[name] = getattr(m, name)
I am kind of making things up there a little bit with trying to find an __all__ symbol first and then, if that files, doing a dir() and grabbing symbols that look non-private — you would have to look at the implementation of import * to know if that resembles Python's actual logic closely enough for your purposes.
If you are using django (as mentionned in comments), something like this should work
from django.utils import importlib
my_module = importlib.import_module('app.my_module')
from my_module import *
But I agree that it may be dangerous
You can do it by:
from imp import *
but remember:
Explicit is better than implicit.
(from The Zen of Python - read it by using the following command: import this)
I have a Python module that I want to dynamically import given only a string of the module name. Normally I use importlib or __import__ and this works quite well given that I know which objects I want to import from the module, but is there a way to do the equivalent of import * dynamically. Or is there a better approach?
I know in general its bad practice to use import * but the modules I'm trying to import are automatically generated on the fly and I have no way of knowing the exact module which contains the class I'm addressing.
Thanks.
Use update for dicts:
globals().update(importlib.import_module('some.package').__dict__)
Note, that using a_module.__dict__ is not the same as from a_module import *, because all names are "imported", not only those from __all__ or not starting with _.
I came up with some ugly hacky code, it works in python 2.6. I'm not sure if this is the smartest thing to do though, perhaps some other people here have some insight:
test = __import__('os',globals(),locals())
for k in dir(test):
globals()[k] = test.__dict__[k]
You probably want to put a check here to make sure you aren't overwriting anything in the global namespace. You could probably avoid the globals part and just look through each dynamically imported module for your class of interest. This would probably be much better than polluting the global namespace with everything you are importing.
For example, say your class is named Request from urllib2
test = __import__('urllib2',globals(),locals())
cls = None
if 'Request' in dir(test):
cls = test.__dict__['Request']
# you found the class now you can use it!
cls('http://test.com')
The following is highly sinful and will condemn you to purgatory or worse
# module_a.py
myvar = "hello"
# module_b.py
import inspect
def dyn_import_all(modpath):
"""Incredibly hackish way to load into caller's global namespace"""
exec('from ' + modpath + ' import *', inspect.stack()[1][0].f_globals)
# module_c.py
from module_b import dyn_import_all
def print_from(modpath):
dyn_import_all(modpath)
print(myvar)
Demo:
>>> import module_c
>>> module_c.print_from("module_a")
hello