In Python, if you want to dynamically import a module (such as from a string name) you can use the module importlib and the function importlib.import_module("foo"), which essentially gives the same result as import foo (but it's dynamic).
Anyway, in my program, I'm using a function to import a module from a list, so it looks something like this:
# Note: this code does not produce the desired result.
# Please see the snippet below, for the working version
module_list = ["os"]
def import_module(name):
exec("global {}".format(name))
exec("import {}".format(name))
for item in module_list:
import_module(item)
I haven't seen this type of solution anywhere else on the web. What I'm asking, is why? Is it bad practice because I'm using the exec() function (as I've read not to do countless times) or is it because It's simply more confusing
Edit: I feel like it's relevant to note that it's not my exact code above, but it's the part that's actually relevant to this question, instead of confusing people
Edit (2): thanks to user Aran-Fey for discovering that my code doesn't work. I hadn't properly tested this specific snippet. Here's a version that works in python 3.6:
module_list = ["os"]
def import_module(name):
exec("global {}".format(name), globals())
exec("import {}".format(name), globals())
for item in module_list:
import_module(item)
Related
As i was going through python basics and introduction one thing that really confuses me is namespaces. I cant seem to understand how they work.
For instance in Cs50 course they brush over the concept but i wanted to get clearer understanding of it if possible because it seems very important to grasp. For example this code:
import cs50
x = get_int("x: ")
y = get_int("y: ")
print(x + y)
Causes this error:
python calculator.py
Traceback (most recent call last):
File "/workspaces/20377622/calculator.py", line 3, in
x = get_int("x: ")
NameError: name 'get_int' is not defined
What is wonder is why when cs50.get_int() is written instead interpreter doesn't throw an error? Is it because cs50 library is seen as its own namespace structure and . goes into that structure to get_int location? What does . operator do exactly here in terms of namespaces ?
You import cs50, so you have a name "cs50", you can use
cs50.get_int()
but namespaces has no name get_int.
You can use from cs50 import get_int to import name "get_int"
To answer this question, let's talk about modules.
In Python, "module" is used to refer to two things. First, a piece of code, usually a .py file. Second, the object that is created for the namespace of that code when it is run.
When you do import foo, a couple of things happen.
Python checks if foo has already been imported. If so, it skips to the last step.
Python looks up where it can find foo, for example if there is a foo.py in the right place.
Python creates a fresh namespace object and stores a reference to it in sys.modules['foo']
Python runs the code that it found, using that namespace object as its "global" namespace.
Python goes back to the importing module, and adds a global variable foo in the namespace of importing module that points to sys.modules['foo'].
You can then access any global variable bar that was created in the foo.py module by using foo.bar.
You could also use from cs50 import get_int which works like this:
import cs50
get_int = cs50.get_int
... except that the name cs50 is not assigned to.
If you're asking why it works that way: this way different modules can define the same name for classes, functions or constants, and they won't conflict with each other.
Now, if you know you're only using get_int from cs50, and you don't have any other get_int either in your main module or that you imported, the from ... import syntax is very useful to not have to write cs50.get_int every time you want to use it.
Inside a function, I have to import a variable (dict) from a module dynamically:
exec("from ctrl_%s import default_settings" % get_version_id(iid))
which doesnt work. When referencing this variable later, it says: UnboundLocalError: local variable 'default_settings' referenced before assignment
The variable is in the global scope of the module to import.
But:
This all works, if I hardcode this statement without exec(). The string is correctly formed, I can print it out.
Someone knows what to do?
I highly would discourage to use exec in the first place, it often does not do what you want especially if some special syntax is involved like here.
But fortunately there are some tricks:
e.g. you can import the module and use the dict or getattr:
import math
getattr(math,"sin")
math.__dict__['sin']
Edit just checked my answer and I saw you wanted to import a module ...
But there is also a trick for this:
https://docs.python.org/3/library/functions.html#__import__
Look also at this question for some examples:
How to import a module given its name as string?
I found the following code snippet that I can't seem to make work for my scenario (or any scenario at all):
def load(code):
# Delete all local variables
globals()['code'] = code
del locals()['code']
# Run the code
exec(globals()['code'])
# Delete any global variables we've added
del globals()['load']
del globals()['code']
# Copy k so we can use it
if 'k' in locals():
globals()['k'] = locals()['k']
del locals()['k']
# Copy the rest of the variables
for k in locals().keys():
globals()[k] = locals()[k]
I created a file called "dynamic_module" and put this code in it, which I then used to try to execute the following code which is a placeholder for some dynamically created string I would like to execute.
import random
import datetime
class MyClass(object):
def main(self, a, b):
r = random.Random(datetime.datetime.now().microsecond)
a = r.randint(a, b)
return a
Then I tried executing the following:
import dynamic_module
dynamic_module.load(code_string)
return_value = dynamic_module.MyClass().main(1,100)
When this runs it should return a random number between 1 and 100. However, I can't seem to get the initial snippet I found to work for even the simplest of code strings. I think part of my confusion in doing this is that I may misunderstand how globals and locals work and therefore how to properly fix the problems I'm encountering. I need the code string to use its own imports and variables and not have access to the ones where it is being run from, which is the reason I am going through this somewhat over-complicated method.
You should not be using the code you found. It is has several big problems, not least that most of it doesn't actually do anything (locals() is a proxy, deleting from it has no effect on the actual locals, it puts any code you execute in the same shared globals, etc.)
Use the accepted answer in that post instead; recast as a function that becomes:
import sys, imp
def load_module_from_string(code, name='dynamic_module')
module = imp.new_module(name)
exec(code, mymodule.__dict__)
return module
then just use that:
dynamic_module = load_module_from_string(code_string)
return_value = dynamic_module.MyClass().main(1, 100)
The function produces a new, clean module object.
In general, this is not how you should dynamically import and use external modules. You should be using __import__ within your function to do this. Here's a simple example that worked for me:
plt = __import__('matplotlib.pyplot', fromlist = ['plt'])
plt.plot(np.arange(5), np.arange(5))
plt.show()
I imagine that for your specific application (loading from code string) it would be much easier to save the dynamically generated code string to a file (in a folder containing an __init__.py file) and then to call it using __import__. Then you could access all variables and functions of the code as parts of the imported module.
Unless I'm missing something?
Introduction
Pydev is a great eclipse plugin that let us write python code easily.
It can even give autocompletion suggestion when I do this:
from package.module import Random_Class
x = Random_Class()
x. # the autocompletion will be popped up,
# showing every method & attribute inside Random_Class
That is great !!!
The Problem (And My Question)
However, when I don't use explicit import, and use __import__ for example, I can't have the same autocompletion effect.
import_location = ".".join(('package', 'module'))
__import__(import_location, globals(), locals(), ['*'])
My_Class = getattr(sys.modules[import_location], 'Random_Class')
x = My_Class()
x. # I expect autocompletion, but nothing happened
Question: is there any way (in pydev or any IDE) to make the second one also
show autocompletion?
Why do I need to do this?
Well, I make a simple MVC framework, and I want to provide something like load_model, load_controller, and load_view which is still work with autocompletion (or at least possible to work)
So, instead of leave users do this (although I don't forbid them to do so):
from applications.the_application.models.the_model import The_Model
x = The_Model()
x. # autocompletion works
I want to let users do this:
x = load_model('the_application', 'the_model')()
x. # autocompletion still works
The "applications" part is actually configurable by another script, and I don't want users to change all of their importing model/controller part everytime they change the configuration. Plus, I think load_model, load_controller, and load_view make MVC pattern shown more obvious.
Unexpected Answer
I know some tricks such as doing this (as what people do with
web2py):
import_location = ".".join(('package', 'module'))
__import__(import_location, globals(), locals(), ['*'])
My_Class = getattr(sys.modules[import_location], 'Random_Class')
x = My_Class()
if 0:
from package.module import Random_Class
x = Random_Class()
x. # Now autocompletion is going to work
and I don't expect to do this, since it will only add unnecessary
extra work.
I don't expect any don't try to be clever comments. I have enough of them
I don't expect dynamic import is evil comments. I'm not a purist.
I don't expect any just use django, or pylons, or whatever comments. Such as comments even unrelated to my question.
I have done this before. This may be slightly different from your intended method, so let me know if it doesn't apply.
I dynamically import different modules that all subclass a master class, using similar code to your example. Because the subclassing module already imports the master, I don't need to import it in the main module.
To get highlighting, the solution was to import the master class into the main module first, even though it wasn't used directly. In my case it was a good fallback if the particular subclass didn't exist, but that's an implementation detail.
This only works if your classes all inherit from one parent.
Not really an answer to my own question. However, I can change the approach. So, instead of provide "load_model()", I can use relative import. Something like this:
from ..models.my_model import Model_Class as Great_Model
m = Great_Model()
Ive searched the web and this site and cant find an answer to this problem. Im sure its right in front of me somewhere but cant find it.
I need to be able to import a module based on a string. Then execute a function within that module while passing arguments.
I can import based on the string and then execute using eval() but I know this is not the best way to handle this. I also cant seem to pass arguments that way.
My current module that would be set based on a string is named TestAction.py and lives in a folder called Tasks.
This is the content of TestAction.py:
def doSomething(var):
print var
This is the code I am executing to import TestAction and execute.
module = "Tasks.TestAction"
import Tasks
mymod = __import__(module)
eval(module + ".doSomething()")
How can I make this code #1 not use eval() and #2 pass the var argument to doSomething()?
Thanks in advance!
Thanks everyone for the help. it looks like importlib combined with getattr was what I needed. For future reference here is the exact code that is working for me.
module = "FarmTasks.TestAction"
mymod = importlib.import_module(module)
ds = getattr(mymod, "doSomething")
ds("stuff")
Is the function name also variable? If not, just use your imported module:
mymod.doSomething('the var argument')
if it is, use getattr:
fun = 'doSomething'
getattr(mymod, fun)('the var argument')
According to the documentation
it is better to use importlib.import_module() to programmatically import a module.
Using this you can retrieve your module like this:
import importlib
TestAction = importlib.import_module("TestAction", package="Tasks")
After that you can simply call functions normally or by name:
TestAction.some_known_function(arg1, arg2)
getattr(TestAction, "some_other_function_name")(arg1, arg2)
I hope this answered your question, feel free to elaborate if you are still missing something.
If you use Python 2.7 or 3.1+ the easiest way to go is to use the importlib module in combination with getattr. Your code would look like that then:
import importlib
module = "Tasks.TestAction"
mymod = importlib.import_module(module)
myfunc = getattr(mymod, "doSomething")
myfunc()
I recently wrote a simple function that imports a given function, it seems to work for me:
def import_function(name):
"""Import a function by name: module.function or
module.submodule.function, etc. Return the function object."""
mod, f = name.rsplit('.', 1)
return getattr(__import__(mod, fromlist=[f]), f)
You can use it as:
f = import_function('Tasks.TestAction.doSometing')
f()
or just
import_function('Tasks.TestAction.doSometing')()