How do I execute a string containing Python code in Python?
Do not ever use eval (or exec) on data that could possibly come from outside the program in any form. It is a critical security risk. You allow the author of the data to run arbitrary code on your computer. If you are here because you want to create multiple variables in your Python program following a pattern, you almost certainly have an XY problem. Do not create those variables at all - instead, use a list or dict appropriately.
For statements, use exec(string) (Python 2/3) or exec string (Python 2):
>>> my_code = 'print("hello world")'
>>> exec(my_code)
Hello world
When you need the value of an expression, use eval(string):
>>> x = eval("2+2")
>>> x
4
However, the first step should be to ask yourself if you really need to. Executing code should generally be the position of last resort: It's slow, ugly and dangerous if it can contain user-entered code. You should always look at alternatives first, such as higher order functions, to see if these can better meet your needs.
In the example a string is executed as code using the exec function.
import sys
import StringIO
# create file-like string to capture output
codeOut = StringIO.StringIO()
codeErr = StringIO.StringIO()
code = """
def f(x):
x = x + 1
return x
print 'This is my output.'
"""
# capture output and errors
sys.stdout = codeOut
sys.stderr = codeErr
exec code
# restore stdout and stderr
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
print f(4)
s = codeErr.getvalue()
print "error:\n%s\n" % s
s = codeOut.getvalue()
print "output:\n%s" % s
codeOut.close()
codeErr.close()
eval and exec are the correct solution, and they can be used in a safer manner.
As discussed in Python's reference manual and clearly explained in this tutorial, the eval and exec functions take two extra parameters that allow a user to specify what global and local functions and variables are available.
For example:
public_variable = 10
private_variable = 2
def public_function():
return "public information"
def private_function():
return "super sensitive information"
# make a list of safe functions
safe_list = ['public_variable', 'public_function']
safe_dict = dict([ (k, locals().get(k, None)) for k in safe_list ])
# add any needed builtins back in
safe_dict['len'] = len
>>> eval("public_variable+2", {"__builtins__" : None }, safe_dict)
12
>>> eval("private_variable+2", {"__builtins__" : None }, safe_dict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'private_variable' is not defined
>>> exec("print \"'%s' has %i characters\" % (public_function(), len(public_function()))", {"__builtins__" : None}, safe_dict)
'public information' has 18 characters
>>> exec("print \"'%s' has %i characters\" % (private_function(), len(private_function()))", {"__builtins__" : None}, safe_dict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'private_function' is not defined
In essence you are defining the namespace in which the code will be executed.
Remember that from version 3 exec is a function!
so always use exec(mystring) instead of exec mystring.
Avoid exec and eval
Using exec and eval in Python is highly frowned upon.
There are better alternatives
From the top answer (emphasis mine):
For statements, use exec.
When you need the value of an expression, use eval.
However, the first step should be to ask yourself if you really need to. Executing code should generally be the position of last resort: It's slow, ugly and dangerous if it can contain user-entered code. You should always look at alternatives first, such as higher order functions, to see if these can better meet your needs.
From Alternatives to exec/eval?
set and get values of variables with the names in strings
[while eval] would work, it is generally not advised to use variable names bearing a meaning to the program itself.
Instead, better use a dict.
It is not idiomatic
From http://lucumr.pocoo.org/2011/2/1/exec-in-python/ (emphasis mine)
Python is not PHP
Don't try to circumvent Python idioms because some other language does it differently. Namespaces are in Python for a reason and just because it gives you the tool exec it does not mean you should use that tool.
It is dangerous
From http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html (emphasis mine)
So eval is not safe, even if you remove all the globals and the builtins!
The problem with all of these attempts to protect eval() is that they are blacklists. They explicitly remove things that could be dangerous. That is a losing battle because if there's just one item left off the list, you can attack the system.
So, can eval be made safe? Hard to say. At this point, my best guess is that you can't do any harm if you can't use any double underscores, so maybe if you exclude any string with double underscores you are safe. Maybe...
It is hard to read and understand
From http://stupidpythonideas.blogspot.it/2013/05/why-evalexec-is-bad.html (emphasis mine):
First, exec makes it harder to human beings to read your code. In order to figure out what's happening, I don't just have to read your code, I have to read your code, figure out what string it's going to generate, then read that virtual code. So, if you're working on a team, or publishing open source software, or asking for help somewhere like StackOverflow, you're making it harder for other people to help you. And if there's any chance that you're going to be debugging or expanding on this code 6 months from now, you're making it harder for yourself directly.
eval() is just for expressions, while eval('x+1') works, eval('x=1') won't work for example. In that case, it's better to use exec, or even better: try to find a better solution :)
It's worth mentioning that exec's brother exists as well, called execfile, if you want to call a Python file. That is sometimes good if you are working in a third party package which have terrible IDE's included and you want to code outside of their package.
Example:
execfile('/path/to/source.py')
or:
exec(open("/path/to/source.py").read())
You accomplish executing code using exec, as with the following IDLE session:
>>> kw = {}
>>> exec( "ret = 4" ) in kw
>>> kw['ret']
4
As the others mentioned, it's "exec" ..
but, in case your code contains variables, you can use "global" to access it, also to prevent the compiler to raise the following error:
NameError: name 'p_variable' is not defined
exec('p_variable = [1,2,3,4]')
global p_variable
print(p_variable)
I tried quite a few things, but the only thing that worked was the following:
temp_dict = {}
exec("temp_dict['val'] = 10")
print(temp_dict['val'])
output:
10
Use eval.
Check out eval:
x = 1
print eval('x+1')
->2
The most logical solution would be to use the built-in eval() function .Another solution is to write that string to a temporary python file and execute it.
Ok .. I know this isn't exactly an answer, but possibly a note for people looking at this as I was. I wanted to execute specific code for different users/customers but also wanted to avoid the exec/eval. I initially looked to storing the code in a database for each user and doing the above.
I ended up creating the files on the file system within a 'customer_filters' folder and using the 'imp' module, if no filter applied for that customer, it just carried on
import imp
def get_customer_module(customerName='default', name='filter'):
lm = None
try:
module_name = customerName+"_"+name;
m = imp.find_module(module_name, ['customer_filters'])
lm = imp.load_module(module_name, m[0], m[1], m[2])
except:
''
#ignore, if no module is found,
return lm
m = get_customer_module(customerName, "filter")
if m is not None:
m.apply_address_filter(myobj)
so customerName = "jj"
would execute apply_address_filter from the customer_filters\jj_filter.py file
I'm attempting to instantiate an object from a string. Specifically, I'm trying to change this:
from node.mapper import Mapper
mapper = Mapper(file)
mapper.map(src, dst)
into something like this:
with open('C:.../node/mapper.py', 'r') as f:
mapping_script = f.read()
eval(mapping_script)
mapper = Mapper(file)
mapper.map(src, dst)
The motivation for this seemingly bizarre task is to be able to store different versions of mapping scripts in a database and then retrieve/use them as needed (with emphasis on the polymorphism of the map() method).
The above does not work. For some reason, eval() throws SyntaxError: invalid syntax. I don't understand this since it's the same file that's being imported in the first case. Is there some reason why eval() cannot be used to define classes?
I should note that I am aware of the security concerns around eval(). I would love to hear of alternative approaches if there are any. The only other thing I can think of is to fetch the script, physically save it into the node package directory, and then import it, but that seems even crazier.
You need to use exec:
exec(mapping_script)
eval() works only for expressions. exec() works for statements. A typical Python script contains statements.
For example:
code = """class Mapper: pass"""
exec(code)
mapper = Mapper()
print(mapper)
Output:
<__main__.Mapper object at 0x10ae326a0>
Make sure you either call exec() (Python 3, in Python 2 it is a statement) at the module level. When you call it in a function, you need to add globals(), for example exec(code, globals()), to make the objects available in the global scope and to the rest of the function as discussed here.
I realize that this may be a fragile approach, but I'm looking for a way to intercept global name lookups (and also to provide a value/binding when the lookup fails) under 'exec'.
Use case: I want to provide a restricted execution environment for some external scripts written by users. I am trying to tailor the script conventions and namespace construction to very unsophisticated users, so I'd like them to be able to call a bunch of functions as if they were "global" without having to construct the entire dictionary by hand ahead of time.
Ergo, I'd like to intercept the global/module namespace lookup of SomeIdentifierNameTheyMayUse, and to dynamically bind that name to something computed rather than something already bound in the namespace.
Is something like this possible in general?
I managed to get something sort-of working, but it has problems, as you can see below:
class mydict( dict ):
def __missing__( self, key ):
print "__missing__:", key
return 99
d = mydict()
d[ '__builtins__' ] = {}
code = """
# triggers __missing__ call as desired, prints 99
print this_bad_sym_is_ok
def action1():
print 'action1!'
# does not trigger __missing__. Why? And how can I fix it?
print this_bad_sym_is_not
"""
exec code in d
print "d=", d
exec 'action1()' in d
which currently produces:
__missing__: this_bad_sym_is_ok
99
d= {'__builtins__': {}, 'action1': <function action1 at 0x107d6b2a8>}
action1!
Traceback (most recent call last):
File "t.py", line 25, in <module>
exec 'action1()' in d
File "<string>", line 1, in <module>
File "<string>", line 10, in action1
NameError: global name 'this_bad_sym_is_not' is not defined
Even if it's not possible to do something similar to this, I'd still like to understand why it's not working.
Thanks!
Maybe this helps: https://wiki.python.org/moin/SandboxedPython
It explains the restricted execution environment.
This is an implementation: https://pypi.python.org/pypi/pysandbox/
I have a problem similar to the first problem in this question, which as far as I can see went unanswered.
I have a file "config.py" which contains a lot of parameters to be used by a class (this config.py file will change), however I can't get these to propagate into the class via execfile.
In an example piece of code:
class Class():
def __init__(self):
execfile("config.py")
print x
# config.py
x = "foo"
>>> t = Class()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in __init__
NameError: global name 'x' is not defined
Any help welcome, or any better methods of retrieving parameters from a file to be used in a class.
Many Thanks.
I don't get what you're trying to do (but i don't like it, and this is just me) but to fix your problem do (test in python2.6):
class Class():
def __init__(self):
execfile('config.py', locals()) # Not recommanded, maybe you want globals().
print x
But from the doc:
Note
The default locals act as described
for function locals() below:
modifications to the default locals
dictionary should not be attempted.
Pass an explicit locals dictionary if
you need to see effects of the code on
locals after function execfile()
returns. execfile() cannot be used
reliably to modify a function’s
locals.
and about :
Any help welcome, or any better
methods of retrieving parameters from
a file to be used in a class.
You can use import.
Even though it might be convenient to keep configuration settings in a Python file I would recommend against it. I think it opens up a whole set of problems that you don't really want to have do deal with. Anything could be placed in your configuration file, including malicious code.
I would use either the json module or the ConfigParser module to hold my configuration.
If you have trouble choosing between those two I would recommend the json module. Json is a simple yet flexible format for structured data.
What's the correct way to prevent invoking (creating an instance of) a C type from Python?
I've considered providing a tp_init that raises an exception, but as I understand it that would still allow __new__ to be called directly on the type.
A C function returns instances of this type -- that's the only way instances of this type are intended to be created.
Edit: My intention is that users of my type will get an exception if they accidentally use it wrongly. The C code is such that calling a function on an object incorrectly created from Python would crash. I realise this is unusual: all of my C extension types so far have worked nicely when instantiated from Python. My question is whether there is a usual way to provide this restriction.
Simple: leave the tp_new slot of the type empty.
>>> Foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: cannot create 'foo.Foo' instances
>>> Foo.__new__(Foo)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object.__new__(foo.Foo) is not safe, use foo.Foo.__new__()
If you inherit from a type other than the base object type, you will have to set tp_new to NULL after calling PyType_Ready().
Don't prevent them from doing it. "We're all consenting adults here."
Nobody is going to do it unless they have a reason, and if they have such a reason then you shouldn't stop them just because you didn't anticipate every possible use of your type.
There is a fantastically bulletproof way. Let people create the object, and have Python crash. That should stop them doing it pretty efficiently. ;)
Also you can underscore the class name, to indicate that it should be internal. (At least, I assume you can create underscored classnames from C too, I haven't actually ever done it.)
"The type is a return type of another C function - that's the only way instances of this type are intended to be created" -- that's rather confusing. I think you mean "A C function returns instances of this type -- that's the only way etc etc".
In your documentation, warn the caller clearly against invoking the type. Don't export the type from your C extension. You can't do much about somebody who introspects the returned instances but so what? It's their data/machine/job at risk, not yours.
[Update (I hate the UI for comments!)]
James: "type ...just only created from C": again you are confusing the type and its instances. The type is created statically in C. Your C code contains the type and also a factory function that users are intended to call to obtain instances of the type. For some reason that you don't explain, if users obtain an instance by calling the type directly, subsequent instance.method() calls will crash (I presume that's what you mean by "calling functions on the object". Call me crazy, but isn't that a bug that you should fix?
Re "don't export": try "don't expose".
In your C code, you will have something like this where you list out all the APIs that your module is providing, both types and functions:
static struct PyMethodDef public_functions[] = {
{"EvilType", (PyCFunction) py_EvilType, ......},
/* omit above line and punters can't call it directly from Python */
{"make_evil", (PyCFunction) py_make_evil, ......},
......,
};
module = Py_InitModule4("mymodule", public_functions, module_doc, ...