I have code like this:
globals_defined = {'add': my_add_fn, 'divide': my_divide_fn}
eval_result = eval(<some code>, {data: {'name_1': 'NAME1', 'name_2': 'NAME2'}, globals_defined)
I would like set a global variable inside the eval and then be able to access it afterwards. So like:
globals_defined = {'add': my_add_fn, 'divide': my_divide_fn, count_iterations: 0}
eval_result = eval(<some code>, {data: {'name_1': 'NAME1', 'name_2': 'NAME2'}, globals_defined)
print 'iterations: ' + str(globals_defined['count_iterations'])
And ideally that would print a modified value of count_iterations. Inside the eval, the my_add_fn would do something like the below to increment it:
def my_add_fn():
global count_iterations
count_terations += 1
return 'blah!'
Edit: I should have added this at first. Yes, I need to use eval. This eval is from user input originally but has been parsed into an Abstract Syntax Tree that rejects all but a few mathematical operations. Then, that AST is what is being eval'd with some custom function definitions defined.
Sounds like I can't do it this way though.
You should not be using eval to start with.
But since you don't show the code you are running in there, there is no way to suggest alternatives.
The fact is Python's eval only executes expressions - and assignments in Python are statements - you can't, ordinarily, have an assignment inside an eval.
The way to fix that is just to use exec instead of eval. By default, exec will use the global variables of the module where it is run, so, any assignment in the executed code will affect the global namespace.
If you pass a dictionary as a second argument to exec it will use that dictionary as its globals dictionary - and then you can have it modify the values inside that dict with assignments.
That are a general statement about eval and exec - your specific problem is that you are (likely) trying to call a function inside eval - add() - which is exposed to it in the globals_defined dictionary itself. The problem is your add function is defined in your module - outside the eval (or exec) scope. The Global variables for the my_add_fn function are not the globals passed to eval, rather the module globals themselves. So the variable count_iterations it accesses is on the module scope, not inside the globals_defined dictionary.
Of course there are ways to fix that. The best is: not to use eval (or exec).
Your print idiom on the last line suggests you are new to Python - otherwise you would know about the powerfull ways of string formatting in existence.
That said: you can call functions stored in a dictionary just as you can access any element on a dictionary - that is:
globals_defined["add"] () will all your my_add_fn function - this is a hint of how you can get rid of the need to use "eval".
And finally, back to your problem as is - one way to solve it, is to have your function not rely on its global namespace, and rather receive and return parameters - and you then make assignments of the values returned by the function outside it - on your exec string scope - thus:
def my_add_fn(count):
return 'blah!', count+1
global_dict = {"add": my_add_fn, "count_iterations": 0}
exec("result, count_iterations = add()", global_dict)
print "Count = {}".format(global_dict["count_iterations"])
Why try to use a global namespace? Why not construct a specific local namespace to pass the functions to the eval? The problem with passing globals() as the global namespace to eval or exec is that you are giving away control of your namespace to the code you exit, so (e.g.) if it binds to a name you are already using your value will be overwritten.
Related
When using inner function, it reads variable defined in outer function. But somehow it fails when using eval(). It seems to be related to how locals() works... but I'm not sure how and why...
def main():
aaa = 'print this'
def somethingelse():
print(locals())
#print(aaa)
print(eval('aaa'))
print(locals())
somethingelse()
main()
The above codes wouldn't work, giving error message:
File "", line 1, in
NameError: name 'aaa' is not defined
But if unmark the print(aaa) so both print lines exists, then both of them will work.
I tried to print locals() before and after this print(aaa) command, it turns out that if the print(aaa) line is marked, both locals() would be empty {}. But if unmarked, then both locals() would be {aaa: 'print this'}
This is puzzling to me...
When your Python code is compiled, the compiler has to do special things to allow a local variable to be accessible from inside a nested function. This makes all access to the variable slower so it only does it for variables that it knows are used in the inner function. Other local variables from the outer function just don't exist in the inner function's namespace.
It can't analyse inside the string you use for eval so it doesn't know that code is attempting to access a variable that otherwise wouldn't exist in the inner function. You need to access the variable directly from inside the inner function for the compiler to add it to the local variables for that function.
You probably don't want to be using eval anyway, there are extremely few cases where it is a good idea to use it.
Is there a way to limit function so that it would only have access to local variable and passed arguments?
For example, consider this code
a = 1
def my_fun(x):
print(x)
print(a)
my_fun(2)
Normally the output will be
2
1
However, I want to limit my_fun to local scope so that print(x) would work but throw an error on print(a). Is that possible?
I feel like I should preface this with: Do not actually do this.
You (sort of) can with functions, but you will also disable calls to all other global methods and variables, which I do not imagine you would like to do.
You can use the following decorator to have the function act like there are no variables in the global namespace:
import types
noglobal = lambda f: types.FunctionType(f.__code__, {})
And then call your function:
a = 1
#noglobal
def my_fun(x):
print(x)
print(a)
my_fun(2)
However this actually results in a different error than you want, it results in:
NameError: name 'print' is not defined
By not allowing globals to be used, you cannot use print() either.
Now, you could pass in the functions that you want to use as parameters, which would allow you to use them inside the function, but this is not a good approach and it is much better to just keep your globals clean.
a = 1
#noglobal
def my_fun(x, p):
p(x)
p(a)
my_fun(2, print)
Output:
2
NameError: name 'a' is not defined
Nope. The scoping rules are part of a language's basic definition. To change this, you'd have to alter the compiler to exclude items higher on the context stack, but still within the user space. You obviously don't want to limit all symbols outside the function's context, as you've used one in your example: the external function print. :-)
exec() from inside a function gives a different output, even I pass all the parameter needed to that function. Consider this code:
def list_filter(expr,datalist,includelist) :
exec(expr)
print outlist
datalist=['try to kill me','black image']
includelist=['me']
expr="outlist = [i for i in datalist if all(j in i for j in includelist) ]"
exec(expr)
print outlist
list_filter(expr,datalist,includelist)
I have checked the similar case here : How does exec work with locals?
But this is a different error, where I'm using version 2.7.13 and I check under general condition, the exec() normally has no error at all. I found this problem shows up when there's a 'nested loop' inside list comprehension statement, such as using all() or any(). As in this case , if I remove the if condition from the list comprehension (make it to be expr = "outlist = [i for i in datalist ]") then I will get the correct output as usual.
Any idea why?
Almost always it's a bad idea to use exec in this case you probably shouldn't use it at all.
But since you asked: It works correctly if you pass in the variables from the local scope:
def list_filter(expr, datalist, includelist):
exec(expr, locals())
print outlist
I'm not very familiar with the scope rules for exec but I often found that you need to pass in the variables explicitly (especially if the exec isn't in the global scope).
In your case you could even pass them in explicitly:
def list_filter(expr, datalist, includelist):
exec(expr, {'datalist': datalist, 'includelist': includelist})
print outlist
It's even stated in the documentation that you may need to pass the variables from the scope to exec:
The built-in functions globals() and locals() return the current global and local dictionary, respectively, which may be useful to pass around for use by exec.
I would like to write a function which receives a local namespace dictionary and update it. Something like this:
def UpdateLocals(local_dict):
d = {'a':10, 'b':20, 'c':30}
local_dict.update(d)
When I call this function from the interactive python shell it works all right, like this:
a = 1
UpdateLocals(locals())
# prints 20
print a
However, when I call UpdateLocals from inside a function, it doesn't do what I expect:
def TestUpdateLocals():
a = 1
UpdateLocals(locals())
print a
# prints 1
TestUpdateLocals()
How can I make the second case work like the first?
UPDATE:
Aswin's explanation makes sense and is very helpful to me. However I still want a mechanism to update the local variables. Before I figure out a less ugly approach, I'm going to do the following:
def LoadDictionary():
return {'a': 10, 'b': 20, 'c': 30}
def TestUpdateLocals():
a = 1
for name, value in LoadDictionary().iteritems():
exec('%s = value' % name)
Of course the construction of the string statements can be automated, and the details can be hidden from the user.
You have asked a very good question. In fact, the ability to update local variables is very important and crucial in saving and loading datasets for machine learning or in games. However, most developers of Python language have not come to a realization of its importance. They focus too much on conformity and optimization which is nevertheless important too.
Imagine you are developing a game or running a deep neural network (DNN), if all local variables are serializable, saving the entire game or DNN can be simply put into one line as print(locals()), and loading the entire game or DNN can be simply put into one line as locals().update(eval(sys.stdin.read())).
Currently, globals().update(...) takes immediate effect but locals().update(...) does not work because Python documentation says:
The default locals act as described for function locals() below:
modifications to the default locals dictionary should not be
attempted. Pass an explicit locals dictionary if you need to see
effects of the code on locals after function exec() returns.
Why they design Python in such way is because of optimization and conforming the exec statement into a function:
To modify the locals of a function on the fly is not possible without
several consequences: normally, function locals are not stored in a
dictionary, but an array, whose indices are determined at compile time
from the known locales. This collides at least with new locals added
by exec. The old exec statement circumvented this, because the
compiler knew that if an exec without globals/locals args occurred in
a function, that namespace would be "unoptimized", i.e. not using the
locals array. Since exec() is now a normal function, the compiler does
not know what "exec" may be bound to, and therefore can not treat is
specially.
Since global().update(...) works, the following piece of code will work in root namespace (i.e., outside any function) because locals() is the same as globals() in root namespace:
locals().update({'a':3, 'b':4})
print(a, b)
But this will not work inside a function.
However, as hacker-level Python programmers, we can use sys._getframe(1).f_locals instead of locals(). From what I have tested so far, on Python 3, the following piece of code always works:
def f1():
sys._getframe(1).f_locals.update({'a':3, 'b':4})
print(a, b)
f1()
However, sys._getframe(1).f_locals does not work in root namespace.
The locals are not updated here because, in the first case, the variable declared has a global scope. But when declared inside a function, the variable loses scope outside it.
Thus, the original value of the locals() is not changed in the UpdateLocals function.
PS: This might not be related to your question, but using camel case is not a good practice in Python. Try using the other method.
update_locals() instead of UpdateLocals()
Edit To answer the question in your comment:
There is something called a System Stack. The main job of this system stack during the execution of a code is to manage local variables, make sure the control returns to the correct statement after the completion of execution of the called function etc.,
So, everytime a function call is made, a new entry is created in that stack,
which contains the line number (or instruction number) to which the control has to return after the return statement, and a set of fresh local variables.
The local variables when the control is inside the function, will be taken from the stack entry. Thus, the set of locals in both the functions are not the same. The entry in the stack is popped when the control exits from the function. Thus, the changes you made inside the function are erased, unless and until those variables have a global scope.
is it possible to add a local varible to a function, just before calling it ? if yes how ?
EDIT:REASON
i noticed that all my views in django are using
render_to_response(template_name,locals())
now i created a middleware and i wanted to add one more local variable using the
def process_view():
method of it .so that i don't have to modify the views .
The local scope for a function does not exist until the function is called, so it's not possible to do this. You could do this for a closure, but the next person to have to maintain the code would hunt you down and kill you.
Although I also think it is pretty useless, I thought that you may enclose the function in either a 'with' statement or another function, like the code below. Of course, this approach can be accomplished directly within the function of interest. In fact, you are adding the local variable 'during' the function declaration. See if this fits your needs!
#!/usr/bin/python
def my_funct(_local):
"""My function of interest
"""
print "Local argument was %s" % str(_local)
return "Finished"
def localize(fct, local_var):
"""
"""
return fct(_local = local_var)
## Use function to 'localize' variable
localize(my_funct, local_var="LOCAL_VARIABLE")
## Same effect without supplementary function :
my_funct(_local="LOCAL_VARIABLE")
try:
print local_var
except:
print "No such global variable"
Just some thoughts :)
Cheers
So if you’re one of those lazy
programmers and you like keeping code
particularly concise, you can take
advantage of a built-in Python
function called locals(). It returns a
dictionary mapping all local variable
names to their values, where “local”
means all variables that have been
defined within the current scope.
source
It is a trick in order to not have to explicitly list all of the variables you need to pass in to the function. In this case, you need to explicitly state a variable to pass in. Therefore, you should not be using locals() in the calls you are making in your middle-ware, as the trick was not designed to be used like that.
i mangaged to do that using decorators.