How can I make this import:
from module import *
with imp module?
Disclaimer:
I need use imp module because I need make several statements and I want to do it dynamicly
I need use * because I need that in the file I made the import, the variables and methods defined in module be available directly, i mean without module.method or module.variable. And I want import all variables and methods in the module because I don't know what methods or variables can be in the module in the future
Here!
def load_everything_from(module_names):
g = globals()
for module_name in module_names:
m = __import__(module_name)
names = getattr(m, '__all__', None)
if names is None:
names = [name for name in dir(m) if not name.startswith('_')]
for name in names:
g[name] = getattr(m, name)
I am kind of making things up there a little bit with trying to find an __all__ symbol first and then, if that files, doing a dir() and grabbing symbols that look non-private — you would have to look at the implementation of import * to know if that resembles Python's actual logic closely enough for your purposes.
If you are using django (as mentionned in comments), something like this should work
from django.utils import importlib
my_module = importlib.import_module('app.my_module')
from my_module import *
But I agree that it may be dangerous
You can do it by:
from imp import *
but remember:
Explicit is better than implicit.
(from The Zen of Python - read it by using the following command: import this)
Related
I know I can use importlib to import modules via a string. How can I recreate the import * functionality using this library? Basically, I want something like this:
importlib.import_module('path.to.module', '*')
My reasons for not name-spacing the imported attributes are deliberate.
Here is a solution: import the module, then one by one make alias in the current namespace:
import importlib
# Import the module
mod = importlib.import_module('collections')
# Determine a list of names to copy to the current name space
names = getattr(mod, '__all__', [n for n in dir(mod) if not n.startswith('_')])
# Copy those names into the current name space
g = globals()
for name in names:
g[name] = getattr(mod, name)
Here is shorten version for #HaiVu answer which refers to this solution from #Bakuriu
import importlib
# import the module
mod = importlib.import_module('collections')
# make the variable global
globals().update(mod.__dict__)
Note:
This will import lots of things beside the user-defined variables
#HaiVu solution did the best of it ie. only import user-defined variables
I am having a code like that:
>> from xmodule import defines
>> print defines.VARIABLE_ONE
>>> 123
What I'd like to achieve is making the VARIABLE_ONE defined in the defines module like it would be defined in my main file so I could simply use:
>> from xmodule import defines
>> print VARIABLE_ONE
>>> 123
Is this possible, or I'd rather have to always point to the defines module?
You want this:
from xmodule.defines import VARIABLE_ONE
Assuming xmodule is package and defines is a module in that package.
See the Packages section of the documentation for more details on importing packages and submodules of packages.
Note that all the normal rules for importing names from modules apply here; you can import all the names from defines using *:
from xmodule.defines import *
Note that in general it's not a good idea to use the from <module> import * pattern. Though it seems that the defines module is just defining a bunch of constants, which is one case where it's usually ok to use that pattern.
I'd like to load a module dynamically, given its string name (from an environment variable). I'm using Python 2.7. I know I can do something like:
import os, importlib
my_module = importlib.import_module(os.environ.get('SETTINGS_MODULE'))
This is roughly equivalent to
import my_settings
(where SETTINGS_MODULE = 'my_settings'). The problem is, I need something equivalent to
from my_settings import *
since I'd like to be able to access all methods and variables in the module. I've tried
import os, importlib
my_module = importlib.import_module(os.environ.get('SETTINGS_MODULE'))
from my_module import *
but I get a bunch of errors doing that. Is there a way to import all methods and attributes of a module dynamically in Python 2.7?
If you have your module object, you can mimic the logic import * uses as follows:
module_dict = my_module.__dict__
try:
to_import = my_module.__all__
except AttributeError:
to_import = [name for name in module_dict if not name.startswith('_')]
globals().update({name: module_dict[name] for name in to_import})
However, this is almost certainly a really bad idea. You will unceremoniously stomp on any existing variables with the same names. This is bad enough when you do from blah import * normally, but when you do it dynamically there is even more uncertainty about what names might collide. You are better off just importing my_module and then accessing what you need from it using regular attribute access (e.g., my_module.someAttr), or getattr if you need to access its attributes dynamically.
Not answering precisely the question as worded, but if you wish to have a file as proxy to a dynamic module, you can use the ability to define __getattr__ on the module level.
import importlib
import os
module_name = os.environ.get('CONFIG_MODULE', 'configs.config_local')
mod = importlib.import_module(module_name)
def __getattr__(name):
return getattr(mod, name)
My case was a bit different - wanted to dynamically import the constants.py names in each gameX.__init__.py module (see below), cause statically importing those would leave them in sys.modules forever (see: this excerpt from Beazley I picked from this related question).
Here is my folder structure:
game/
__init__.py
game1/
__init__.py
constants.py
...
game2/
__init__.py
constants.py
...
Each gameX.__init__.py exports an init() method - so I had initially a from .constants import * in all those gameX.__init__.py which I tried to move inside the init() method.
My first attempt in the lines of:
## -275,2 +274,6 ## def init():
# called instead of 'reload'
+ yak = {}
+ yak.update(locals())
+ from .constants import * # fails here
+ yak = {x: y for x,y in locals() if x not in yak}
+ globals().update(yak)
brec.ModReader.recHeader = RecordHeader
Failed with the rather cryptic:
SyntaxError: import * is not allowed in function 'init' because it contains a nested function with free variables
I can assure you there are no nested functions in there. Anyway I hacked and slashed and ended up with:
def init():
# ...
from .. import dynamic_import_hack
dynamic_import_hack(__name__)
Where in game.__init__.py:
def dynamic_import_hack(package_name):
print __name__ # game.init
print package_name # game.gameX.init
import importlib
constants = importlib.import_module('.constants', package=package_name)
import sys
for k in dir(constants):
if k.startswith('_'): continue
setattr(sys.modules[package_name], k, getattr(constants, k))
(for setattr see How can I add attributes to a module at run time? while for getattr How can I import a python module function dynamically? - I prefer to use those than directly access the __dict__)
This works and it's more general than the approach in the accepted answer cause it allows you to have the hack in one place and use it from whatever module. However I am not really sure it's the best way to implement it - was going to ask a question but as it would be a duplicate of this one I am posting it as an answer and hope to get some feedback. My questions would be:
why this "SyntaxError: import * is not allowed in function 'init'" while there are no nested functions ?
dir has a lot of warnings in its doc - in particular it attempts to produce the most relevant, rather than complete, information - this complete worries me a bit
is there no builtin way to do an import * ? even in python 3 ?
I have a Python module that I want to dynamically import given only a string of the module name. Normally I use importlib or __import__ and this works quite well given that I know which objects I want to import from the module, but is there a way to do the equivalent of import * dynamically. Or is there a better approach?
I know in general its bad practice to use import * but the modules I'm trying to import are automatically generated on the fly and I have no way of knowing the exact module which contains the class I'm addressing.
Thanks.
Use update for dicts:
globals().update(importlib.import_module('some.package').__dict__)
Note, that using a_module.__dict__ is not the same as from a_module import *, because all names are "imported", not only those from __all__ or not starting with _.
I came up with some ugly hacky code, it works in python 2.6. I'm not sure if this is the smartest thing to do though, perhaps some other people here have some insight:
test = __import__('os',globals(),locals())
for k in dir(test):
globals()[k] = test.__dict__[k]
You probably want to put a check here to make sure you aren't overwriting anything in the global namespace. You could probably avoid the globals part and just look through each dynamically imported module for your class of interest. This would probably be much better than polluting the global namespace with everything you are importing.
For example, say your class is named Request from urllib2
test = __import__('urllib2',globals(),locals())
cls = None
if 'Request' in dir(test):
cls = test.__dict__['Request']
# you found the class now you can use it!
cls('http://test.com')
The following is highly sinful and will condemn you to purgatory or worse
# module_a.py
myvar = "hello"
# module_b.py
import inspect
def dyn_import_all(modpath):
"""Incredibly hackish way to load into caller's global namespace"""
exec('from ' + modpath + ' import *', inspect.stack()[1][0].f_globals)
# module_c.py
from module_b import dyn_import_all
def print_from(modpath):
dyn_import_all(modpath)
print(myvar)
Demo:
>>> import module_c
>>> module_c.print_from("module_a")
hello
This question already has answers here:
How can I import a module dynamically given its name as string?
(10 answers)
Closed 3 months ago.
I'm trying to dynamically load modules I've created.
Right now this works properly:
import structures.index
But if I try the same thing by importing it dynamically, it fails.
struct = __import__("structures.index")
Error supplied is:
Error ('No module named structures.index',)
Any ideas why?
Edit: When using full scope (it sort of works?):
struct = __import__("neoform.structures.index")
This doesn't throw any errors, however, it isn't loading the index module, it's loading the "neoform" module instead.
The result of "struct" is:
<module 'neoform' from '/neoform/__init__.py'>
Also, as a side question, how can I then instantiate a class within a dynamically loaded module? (assuming all the modules contain a common class name).
Edit: Solution: (thanks coonj & Rick) This ended up being what worked. Not sure why (yet), but the fromlist had to be something "anything apparently, since it worked when I put the letter "a" as a value (strange, given that the file only had 1 class in it).
def get_struct_module(self, name):
try:
return = __import__("neoform.structures." + name, fromlist='*')
except ImportError, e:
self.out.add("Could not load struct: neoform.structure." + name + "\n\n" + "Error " + str(e.args))
I'm not sure what "it fails" means, so I'll just mention that __import__('structures.index') should, in fact, work, but it doesn't assign the module name in the current scope. To do that (and then use a class in the dynamically imported module), you'll have to use:
structures = __import__('structures.index')
structures.index.SomeClass(...)
The complete details on __import__ are available here.
Edit: (based on question edit)
To import neoform.structures.index, and return the index module, you would do the following:
structures = __import__('neoform.structures.index',
fromlist=['does not in fact matter what goes here!'])
So if you have a list of package names packages, you can import their index modules and instantiate some MyClass class for each using the following code:
modules = [ __import__('neoform.%s.index' % pkg, fromlist=['a'])
for pkg in packages ]
objects = [ m.MyClass() for m in modules ]
To import sub-modules, you need to specify them in the fromlist arg of __import__() Fo example, the equivalent of:
import structures.index
is:
structures = __import__('structures', fromlist=['index'])
To do this in a map is a little more tricky...
import mod1.index
import mod2.index
import mod3.index
For those imports, you would want to define a new function to get the index sub-module from each module:
def getIndexMods(mod_names):
mod_list = map(lambda x: __import__(x, fromlist='index'))
index_mods = [mod.index for mod in mod_list]
return index_mods
Now, you can do this to get references to all index modules:
index_mods = getIndexMods(['mod1', 'mod2', 'mod3'])
Also, if you want to grab sub-modules that are not named 'index' then you could do this:
mod1, mod2, mod3 = map(lambda x,y: __import__(x, fromlist=y),
['mod1', 'mod2', 'mod3'], ['index1', 'index2', 'index3'])
Use full scope ("neoform.structures.index") with this helper method.
def import_module(name):
mod = __import__(name)
components = name.split('.')
for comp in components[1:]:
mod = getattr(mod, comp)
return mod
module = import_module("neoform.structures.index")
# do stuff with module
>>> import imp
>>> fm = imp.find_module('index', ['./structures']) # for submodule
>>> mymod = imp.load_module('structures.index', *fm)
>>> mymod
<module 'structures.index' from './structures/index.pyc'>
>>> x = mymod.insideIndex()
Initialising index class...
Voila!
Java programmer here, but I think you need the imp module
Why on earth would you replace
import structures.index
with
map(__import__, ["structures.index"])
The first one (a) works, (b) is dynamic and (c) is directly supported. What possible use case is there for replacing easy-to-change, plain-text source with something more complex?
In short: don't do this. It doesn't have any value.
Edit
The "I'm getting the import from a database" is a noble effort, but still not sensible. What code block depends on those imports? That whole code block -- imports and all -- is what you want to execute. That whole code block -- imports, statements and everything -- should be a plain old python module file.
Import that block of code from the file system. Use the database to identify which file, the author of the file -- anything you want to use the database for. But simply import and execute the module the simplest possible way.
Really late post here. But I was searching for this question on google. I did some trial and error. Not sure if this snippet will help but here it is. Using it for Flask site.
modules = ['frontend', 'admin']
for module in modules:
mod = __init__('controllers.%s' % module, fromlist=[module])
app.register_blueprint(mod.blueprint_mod)
# or
from importlib import import_module
modules = ['frontend', 'admin']
for module in modules:
mod = import_module('controllers.%s' % module)
app.regitster_blueprint(mod.blueprint_mod)