How do I execute a string containing Python code in Python?
Do not ever use eval (or exec) on data that could possibly come from outside the program in any form. It is a critical security risk. You allow the author of the data to run arbitrary code on your computer. If you are here because you want to create multiple variables in your Python program following a pattern, you almost certainly have an XY problem. Do not create those variables at all - instead, use a list or dict appropriately.
For statements, use exec(string) (Python 2/3) or exec string (Python 2):
>>> my_code = 'print("hello world")'
>>> exec(my_code)
Hello world
When you need the value of an expression, use eval(string):
>>> x = eval("2+2")
>>> x
4
However, the first step should be to ask yourself if you really need to. Executing code should generally be the position of last resort: It's slow, ugly and dangerous if it can contain user-entered code. You should always look at alternatives first, such as higher order functions, to see if these can better meet your needs.
In the example a string is executed as code using the exec function.
import sys
import StringIO
# create file-like string to capture output
codeOut = StringIO.StringIO()
codeErr = StringIO.StringIO()
code = """
def f(x):
x = x + 1
return x
print 'This is my output.'
"""
# capture output and errors
sys.stdout = codeOut
sys.stderr = codeErr
exec code
# restore stdout and stderr
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
print f(4)
s = codeErr.getvalue()
print "error:\n%s\n" % s
s = codeOut.getvalue()
print "output:\n%s" % s
codeOut.close()
codeErr.close()
eval and exec are the correct solution, and they can be used in a safer manner.
As discussed in Python's reference manual and clearly explained in this tutorial, the eval and exec functions take two extra parameters that allow a user to specify what global and local functions and variables are available.
For example:
public_variable = 10
private_variable = 2
def public_function():
return "public information"
def private_function():
return "super sensitive information"
# make a list of safe functions
safe_list = ['public_variable', 'public_function']
safe_dict = dict([ (k, locals().get(k, None)) for k in safe_list ])
# add any needed builtins back in
safe_dict['len'] = len
>>> eval("public_variable+2", {"__builtins__" : None }, safe_dict)
12
>>> eval("private_variable+2", {"__builtins__" : None }, safe_dict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'private_variable' is not defined
>>> exec("print \"'%s' has %i characters\" % (public_function(), len(public_function()))", {"__builtins__" : None}, safe_dict)
'public information' has 18 characters
>>> exec("print \"'%s' has %i characters\" % (private_function(), len(private_function()))", {"__builtins__" : None}, safe_dict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'private_function' is not defined
In essence you are defining the namespace in which the code will be executed.
Remember that from version 3 exec is a function!
so always use exec(mystring) instead of exec mystring.
Avoid exec and eval
Using exec and eval in Python is highly frowned upon.
There are better alternatives
From the top answer (emphasis mine):
For statements, use exec.
When you need the value of an expression, use eval.
However, the first step should be to ask yourself if you really need to. Executing code should generally be the position of last resort: It's slow, ugly and dangerous if it can contain user-entered code. You should always look at alternatives first, such as higher order functions, to see if these can better meet your needs.
From Alternatives to exec/eval?
set and get values of variables with the names in strings
[while eval] would work, it is generally not advised to use variable names bearing a meaning to the program itself.
Instead, better use a dict.
It is not idiomatic
From http://lucumr.pocoo.org/2011/2/1/exec-in-python/ (emphasis mine)
Python is not PHP
Don't try to circumvent Python idioms because some other language does it differently. Namespaces are in Python for a reason and just because it gives you the tool exec it does not mean you should use that tool.
It is dangerous
From http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html (emphasis mine)
So eval is not safe, even if you remove all the globals and the builtins!
The problem with all of these attempts to protect eval() is that they are blacklists. They explicitly remove things that could be dangerous. That is a losing battle because if there's just one item left off the list, you can attack the system.
So, can eval be made safe? Hard to say. At this point, my best guess is that you can't do any harm if you can't use any double underscores, so maybe if you exclude any string with double underscores you are safe. Maybe...
It is hard to read and understand
From http://stupidpythonideas.blogspot.it/2013/05/why-evalexec-is-bad.html (emphasis mine):
First, exec makes it harder to human beings to read your code. In order to figure out what's happening, I don't just have to read your code, I have to read your code, figure out what string it's going to generate, then read that virtual code. So, if you're working on a team, or publishing open source software, or asking for help somewhere like StackOverflow, you're making it harder for other people to help you. And if there's any chance that you're going to be debugging or expanding on this code 6 months from now, you're making it harder for yourself directly.
eval() is just for expressions, while eval('x+1') works, eval('x=1') won't work for example. In that case, it's better to use exec, or even better: try to find a better solution :)
It's worth mentioning that exec's brother exists as well, called execfile, if you want to call a Python file. That is sometimes good if you are working in a third party package which have terrible IDE's included and you want to code outside of their package.
Example:
execfile('/path/to/source.py')
or:
exec(open("/path/to/source.py").read())
You accomplish executing code using exec, as with the following IDLE session:
>>> kw = {}
>>> exec( "ret = 4" ) in kw
>>> kw['ret']
4
As the others mentioned, it's "exec" ..
but, in case your code contains variables, you can use "global" to access it, also to prevent the compiler to raise the following error:
NameError: name 'p_variable' is not defined
exec('p_variable = [1,2,3,4]')
global p_variable
print(p_variable)
I tried quite a few things, but the only thing that worked was the following:
temp_dict = {}
exec("temp_dict['val'] = 10")
print(temp_dict['val'])
output:
10
Use eval.
Check out eval:
x = 1
print eval('x+1')
->2
The most logical solution would be to use the built-in eval() function .Another solution is to write that string to a temporary python file and execute it.
Ok .. I know this isn't exactly an answer, but possibly a note for people looking at this as I was. I wanted to execute specific code for different users/customers but also wanted to avoid the exec/eval. I initially looked to storing the code in a database for each user and doing the above.
I ended up creating the files on the file system within a 'customer_filters' folder and using the 'imp' module, if no filter applied for that customer, it just carried on
import imp
def get_customer_module(customerName='default', name='filter'):
lm = None
try:
module_name = customerName+"_"+name;
m = imp.find_module(module_name, ['customer_filters'])
lm = imp.load_module(module_name, m[0], m[1], m[2])
except:
''
#ignore, if no module is found,
return lm
m = get_customer_module(customerName, "filter")
if m is not None:
m.apply_address_filter(myobj)
so customerName = "jj"
would execute apply_address_filter from the customer_filters\jj_filter.py file
I found the following code snippet that I can't seem to make work for my scenario (or any scenario at all):
def load(code):
# Delete all local variables
globals()['code'] = code
del locals()['code']
# Run the code
exec(globals()['code'])
# Delete any global variables we've added
del globals()['load']
del globals()['code']
# Copy k so we can use it
if 'k' in locals():
globals()['k'] = locals()['k']
del locals()['k']
# Copy the rest of the variables
for k in locals().keys():
globals()[k] = locals()[k]
I created a file called "dynamic_module" and put this code in it, which I then used to try to execute the following code which is a placeholder for some dynamically created string I would like to execute.
import random
import datetime
class MyClass(object):
def main(self, a, b):
r = random.Random(datetime.datetime.now().microsecond)
a = r.randint(a, b)
return a
Then I tried executing the following:
import dynamic_module
dynamic_module.load(code_string)
return_value = dynamic_module.MyClass().main(1,100)
When this runs it should return a random number between 1 and 100. However, I can't seem to get the initial snippet I found to work for even the simplest of code strings. I think part of my confusion in doing this is that I may misunderstand how globals and locals work and therefore how to properly fix the problems I'm encountering. I need the code string to use its own imports and variables and not have access to the ones where it is being run from, which is the reason I am going through this somewhat over-complicated method.
You should not be using the code you found. It is has several big problems, not least that most of it doesn't actually do anything (locals() is a proxy, deleting from it has no effect on the actual locals, it puts any code you execute in the same shared globals, etc.)
Use the accepted answer in that post instead; recast as a function that becomes:
import sys, imp
def load_module_from_string(code, name='dynamic_module')
module = imp.new_module(name)
exec(code, mymodule.__dict__)
return module
then just use that:
dynamic_module = load_module_from_string(code_string)
return_value = dynamic_module.MyClass().main(1, 100)
The function produces a new, clean module object.
In general, this is not how you should dynamically import and use external modules. You should be using __import__ within your function to do this. Here's a simple example that worked for me:
plt = __import__('matplotlib.pyplot', fromlist = ['plt'])
plt.plot(np.arange(5), np.arange(5))
plt.show()
I imagine that for your specific application (loading from code string) it would be much easier to save the dynamically generated code string to a file (in a folder containing an __init__.py file) and then to call it using __import__. Then you could access all variables and functions of the code as parts of the imported module.
Unless I'm missing something?
I've noticed some functions are called using the var.func() format as in var1.split() while other functions are called using the func(var) format as in sorted(list1).
Here's a very simple program to illustrate the question. I've also noticed the same behavior with open and read functions.
str1 = "This is a string"
list1 = str1.split()
print str1.split(' ')
print sorted(list1)
I'm very new to programming so any help would be greatly appreciated!
Everything in python is an object. Thus when doing something like this:
s = "some string"
s is an str object and you can call all the str methods on it. You can also do things like this:
"some string".split()
and it will give you a list of splitted strings.
This difference has to do with issues of scope. Functions which can be called directly, such as sorted(list1) in your example above, are either builtin functions, or else defined at the top level of one of your imported libraries (for example when using from simpy import *, you can call test() directly to run the built in test suite for the simpy library). Functions which are accessed through the dot operator are functions which are defined for the particular data type that you are applying them to. Remember that each data type in python is an object, and therefore an instance of a class. Those functions, such as split() are defined in that data type's class definition. Additionally, to use the example of test() from the simpy library again, if you were to import a library with only import simpy, you would have to use simpy.test() to call that method.
from simpy import *
test()
vs
import simpy
simpy.test()
The first works because you've imported all methods and classes from the top level of the simpy library, whereas the second works because you've explicitly dived into the scope of the simpy library.
var.func() just means that the function belongs to the object.
For instance, x.sort(). lists (like x) have a function sort.
When you call func(var), func is not a function of lists.
For instance, sorted(x).
This isn't Python specific. You will see the same idea in other languages (e.g. Java).
In var.func() the func() is meant to be used with the var object.
e.g. split() on a string object but cannot use on a list
But func(var) is not confined to a single var object type. you can use it with any appropriate var object.
e.g. sorted() can be used with any iterable like lists, tuples, dicts...
Following TraxusIV's line of thought, I tried the following and it worked!
from string import split
str1 = "This is a string"
list1 = str1.split()
print split(str1)
print sorted(list1)
I would like to invoke the following python script in C :
#!/usr/bin/python2.7
import parser
def evaluate(text):
code = parser.expr(text).compile()
return eval(code)
as explained in the following page https://docs.python.org/2/extending/embedding.html, i can call this script from C using the pwd(path) of the file.
However, i would like to know if it's possible to not load the script by calling python on a C string directly, defining the script.
For example, i would like to let's say i put :
#define PYTHON_SCRIPT ((char*)(\
import parser\
\
def evaluate(text):\
code = parser.expr(text).compile()\
return eval(code)\
))
is it possible to call the python interpreter directly on that string?
Indeed, knowing that i need to pass text as a variable, i can't use this Pyrun_SimpleString function, and i was not able to find something to answer this question.
As mentioned in the comment there is no Pyrun_SimpleString. How to execute Python functions from C is covered here. One way to do it:
Compile your script using Py_CompileString
Create a dictionary for globals/locals.
Extract the function from your globals dict by using PyDict_GetItemString(name)
Build your arguments tuple with PyArg_ParseTuple
Execute your function object by using PyObject_CallFunction.
Take a look at Weave, it allows you to include C code directly in Python code.
I can source a Tcl script, and run a proc from said script like so:
import Tkinter
>>> tclsh = Tkinter.Tcl()
>>> tclsh.eval('source {myscript.tcl}')
>>> tclsh.eval('myproc')
...[output of proc]
>>>
However, should I want to pass variables to this proc, I have to do it like so (assuming the proc takes a dict as a parameter:
>>> tclsh.eval('dict set spec num 10000')
>>> tclsh.eval('dict set spec time 10000')
>>> tclsh.eval('dict set spec rate 10')
Is there an easier, more-Pythonic way to do this from the context of Tkinter? I've seen the variable classes, but they don't seem to have a dict-style variable, or even tie into the Tcl interpreter part of the code at all.
The variable classes are a good idea, but there is no specific dict version of those available, which is a bit ugly, but you can simply use a string version (and take a performance hit due to it, but thats fixable).
So the easy way first. A Tcl dict has a string representation and is convertible from and to its string rep automatically, so if you have a proc that needs a dict, you can simply pass in the string rep for the dict and it just works.
interp = tkinter.Tcl()
myvar = tkinter.StringVar()
def pydict2tcldict(d):
return tkinter._stringify(list(d.items()))
d = {'num': 10000, 'time': 10000, 'rate': 10}
myvar.set(pydict2tcldict(d))
interp.eval("""source {myscript.tcl}
myproc $%s""" % myvar._name)
You can of course make things a bit nicer and faster by providing a special dict variable wrapper instead of the slow round trip through the string rep, see the implementation of the variable classes.
But fundamentally tkinter is just missing a few conversion functions in the _tkinter.c module (see AsObj/FromObj/CallArgs) if one added the appropriate code for mappings (trivial), you could simply do this and be done (and it would be reasonably fast):
interp.call("myproc", d)
The patch to modules/_tkinter.c should be pretty trivial, after reading the Tcl dict C API manpage and the Python mapping C-API (https://www.tcl.tk/man/tcl8.6/TclLib/DictObj.htm and https://docs.python.org/2/c-api/mapping.html ).