Say I have the following dict:
In [6]: scope
Out[6]: {'bar': <function bar>, 'foo': <function foo>}
And foo and bar are:
def foo():
return 5
def bar(x):
return foo() + x
I want to run bar(1), but it will need to find foo(). Is there any way to run bar() in the scope namespace so it finds foo()?
I don't know precisely which function in scope bar will need, so I need a general method of running bar in the scope namespace. I don't have the source code for and cannot modify either function to accept a dict.
It seems like functions have a __closure__ attribute, but it's immutable. There is also a __globals__ attribute, but that just points to globals(). I've seen some answers on SO that update locals() but I'd like to leave locals() untouched.
I tried an eval in scope, but get a NameError for foo:
In [12]: eval(scope['bar'](1), scope)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-12-f6305634f1da> in <module>()
----> 1 eval(scope['bar'](1), scope)
<string> in bar(x)
NameError: global name 'foo' is not defined
Is there some simple way I'm missing?
One approach is to create new functions with a shared globals dictionary:
from types import FunctionType
def scopify(**kwargs):
scoped = dict()
for name, value in kwargs.items():
if isinstance(value, FunctionType):
scoped[name] = FunctionType(value.__code__, scoped, name)
else:
scoped[name] = value
return scoped
scope = scopify(
foo = lambda: baz,
bar = lambda x: foo() + x,
baz = 5,
)
>>> scope['foo']
5
>>> scope['bar'](10)
15
>>> scope['baz'] = 100
>>> scope['bar'](10)
110
Using eval() will work, however there are two problems with the way you tried to use it. For starters, its first argument, expression, should be a string. Secondly, since you're referencing scope in this expression and passing it to eval() as the globals namespace argument, you'll need to add it to itself.
This illustrates what I mean:
def foo():
return 5
def bar(x):
return foo() + x
scope = {'scope': {'bar': bar, 'foo':foo}} # define self-referential scope
expression = "scope['bar'](42)" # explicit reference to `scope`
print('eval({!r}, scope) returns {}'.format(expression, eval(expression, scope)))
However, in this specific case, it would be simpler to just let eval() directly look-up the value of bar in the scope dictionary namespace itself (instead of via scope['scope']['bar']):
scope = {'bar': bar, 'foo':foo}
expression = 'bar(42)'
print('eval({!r}, scope) returns {}'.format(expression, eval(expression, scope)))
Related
I have a function that I define inside another function.
def foo():
x = something
def bar(list=[]):
list.append(x)
return x
return bar
I have two questions:
As I return bar from foo, how does bar keep access to x? At this point, x is only known inside of foo, but when we exit foo, how will bar know what x is?
bar has a mutable default argument. When the returned function bar goes out of scope, is this argument, list, deleted from memory? It will grow and grow every time I call bar, and I want to make sure it is deleted when no longer used.
bar keeps access to x by creating a closure:
>>> def foo():
... x = 42
... def bar(list=[]):
... list.append(x)
... return x
... return bar
...
>>> func = foo()
>>> func
<function foo.<locals>.bar at 0x7fcae19ac050>
>>> func.__closure__
(<cell at 0x7fcae1b19910: int object at 0x1080d99a0>,)
As for the default argument, you ask "When the returned function bar goes out of scope, is this argument, list, deleted from memory?".
Functions are objects. Objects don't go out of scope, scope is a property of variables. If your function object is no longer referenced, then like any other object, it is available for garbage collection. In CPython, this is immediately after a reference count reaches zero. The default values of function arguments are just stored as an attribute, like in any other object:
>>> func.__defaults__
([],)
>>>
So yes, all of this will be cleaned up like normal. If the function object is the only object that references it, then when the function object ceases to exist, the list's reference count reaches zero, and then it becomes available for garbage collection.
You can show this to yourself by defining a verbose finalizer and using that as a default value:
>>> class Bar:
... def __del__(self):
... print('goodbye from ', self)
...
>>> def foo():
... def bar(x=Bar()):
... x.baz = 42
... return bar
...
>>> func = foo()
>>> func = foo()
goodbye from <__main__.Bar object at 0x7fcae1b19950>
>>> func = foo()
goodbye from <__main__.Bar object at 0x7fcae1b19910>
What is the difference between globals(), locals(), and vars()? What do they return? Are updates to the results useful?
Each of these return a dictionary:
globals() always returns the dictionary of the module namespace
locals() always returns a dictionary of the current namespace
vars() returns either a dictionary of the current namespace (if called with no argument) or the dictionary of the argument.
locals and vars could use some more explanation. If locals() is called inside a function, it updates a dict with the values of the current local variable namespace (plus any closure variables) as of that moment and returns it. Multiple calls to locals() in the same stack frame return the same dict each time - it's attached to the stack frame object as its f_locals attribute. The dict's contents are updated on each locals() call and each f_locals attribute access, but only on such calls or attribute accesses. It does not automatically update when variables are assigned, and assigning entries in the dict will not assign the corresponding local variables:
import inspect
def f():
x = 1
l = locals()
print(l)
locals()
print(l)
x = 2
print(x, l['x'])
l['x'] = 3
print(x, l['x'])
inspect.currentframe().f_locals
print(x, l['x'])
f()
gives us:
{'x': 1}
{'x': 1, 'l': {...}}
2 1
2 3
2 2
The first print(l) only shows an 'x' entry, because the assignment to l happens after the locals() call. The second print(l), after calling locals() again, shows an l entry, even though we didn't save the return value. The third and fourth prints show that assigning variables doesn't update l and vice versa, but after we access f_locals, local variables are copied into locals() again.
Two notes:
This behavior is CPython specific -- other Pythons may allow the updates to make it back to the local namespace automatically.
In CPython 2.x it is possible to make this work by putting an exec "pass" line in the function. This switches the function to an older, slower execution mode that uses the locals() dict as the canonical representation of local variables.
If locals() is called outside a function it returns the actual dictionary that is the current namespace. Further changes to the namespace are reflected in the dictionary, and changes to the dictionary are reflected in the namespace:
class Test(object):
a = 'one'
b = 'two'
huh = locals()
c = 'three'
huh['d'] = 'four'
print huh
gives us:
{
'a': 'one',
'b': 'two',
'c': 'three',
'd': 'four',
'huh': {...},
'__module__': '__main__',
}
So far, everything I've said about locals() is also true for vars()... here's the difference: vars() accepts a single object as its argument, and if you give it an object it returns the __dict__ of that object. For a typical object, its __dict__ is where most of its attribute data is stored. This includes class variables and module globals:
class Test(object):
a = 'one'
b = 'two'
def frobber(self):
print self.c
t = Test()
huh = vars(t)
huh['c'] = 'three'
t.frobber()
which gives us:
three
Note that a function's __dict__ is its attribute namespace, not local variables. It wouldn't make sense for a function's __dict__ to store local variables, since recursion and multithreading mean there can be multiple calls to a function at the same time, each with their own locals:
def f(outer):
if outer:
f(False)
print('Outer call locals:', locals())
print('f.__dict__:', f.__dict__)
else:
print('Inner call locals:', locals())
print('f.__dict__:', f.__dict__)
f.x = 3
f(True)
which gives us:
Inner call locals: {'outer': False}
f.__dict__: {'x': 3}
Outer call locals: {'outer': True}
f.__dict__: {'x': 3}
Here, f calls itself recursively, so the inner and outer calls overlap. Each one sees its own local variables when it calls locals(), but both calls see the same f.__dict__, and f.__dict__ doesn't have any local variables in it.
My understanding of mutability and immutability in Python is, say we have a variable foo, if there exists a way to change how foo looks like (by using print) without changing its id, then foo is mutable. Otherwise, it's immutable.
For example, you can do this for a list,
foo = [1, 2, 3]
print(foo, id(foo))
foo[0] = 100
print(foo, id(foo))
but no way for int.
But what about function? First of all, is my definitions of mutability and immutability given above correct? If yes, can you find a way to mutate function without changing its id in order to prove it's mutable?
You can explicitly change the code of a function without affecting its id (here is code using python 2.7):
>>> def f():
... print "f"
...
>>> def g():
... print "g"
...
>>> id(f)
140305904690672
>>> f()
f
>>> f.func_code = g.func_code
>>> id(f)
140305904690672
>>> f()
g
Is it possible to modify values for a dictionary inside a function without passing the dictionary as a parameter.
I would like not to return a dictionary, but only to modify its values.
That's possible, but not necessarily advisable, i can't imagine why you wouldn't like to pass or return the dictionary, if you merely don't want to return the dictionary, but could pass it, you can modify it to reflect in the original dictionary without having to return it, for example:
dict = {'1':'one','2':'two'}
def foo(d):
d['1'] = 'ONE'
print dict['1'] # prints 'one' original value
foo(dict)
print dict['1'] # prints 'ONE' ie, modification reflects in original value
# so no need to return it
However, if you absolutely cannot pass it for whatever reason, you can use a global dictionary as follows:
global dict # declare dictionary as global
dict = {'1':'one','2':'two'} # give initial value to dict
def foo():
global dict # bind dict to the one in global scope
dict['1'] = 'ONE'
print dict['1'] # prints 'one'
foo(dict)
print dict['1'] # prints 'ONE'
I'd recommend the first method demonstrated in the first code block, but feel free to use the second if absolutely necessary.
Enjoy :)
Yes you can, dictionary is an mutable object so they can be modified within functions, but it must be defined before you actually call the function.
To change the value of global variable pointing to an immutable object you must use the global statement.
>>> def func():
... dic['a']+=1
...
>>> dic = {'a':1} #dict defined before function call
>>> func()
>>> dic
{'a': 2}
For immutable objects:
>>> foo = 1
>>> def func():
... global foo
... foo += 3 #now the global variable foo actually points to a new value 4
...
>>> func()
>>> foo
4
What is the difference between globals(), locals(), and vars()? What do they return? Are updates to the results useful?
Each of these return a dictionary:
globals() always returns the dictionary of the module namespace
locals() always returns a dictionary of the current namespace
vars() returns either a dictionary of the current namespace (if called with no argument) or the dictionary of the argument.
locals and vars could use some more explanation. If locals() is called inside a function, it updates a dict with the values of the current local variable namespace (plus any closure variables) as of that moment and returns it. Multiple calls to locals() in the same stack frame return the same dict each time - it's attached to the stack frame object as its f_locals attribute. The dict's contents are updated on each locals() call and each f_locals attribute access, but only on such calls or attribute accesses. It does not automatically update when variables are assigned, and assigning entries in the dict will not assign the corresponding local variables:
import inspect
def f():
x = 1
l = locals()
print(l)
locals()
print(l)
x = 2
print(x, l['x'])
l['x'] = 3
print(x, l['x'])
inspect.currentframe().f_locals
print(x, l['x'])
f()
gives us:
{'x': 1}
{'x': 1, 'l': {...}}
2 1
2 3
2 2
The first print(l) only shows an 'x' entry, because the assignment to l happens after the locals() call. The second print(l), after calling locals() again, shows an l entry, even though we didn't save the return value. The third and fourth prints show that assigning variables doesn't update l and vice versa, but after we access f_locals, local variables are copied into locals() again.
Two notes:
This behavior is CPython specific -- other Pythons may allow the updates to make it back to the local namespace automatically.
In CPython 2.x it is possible to make this work by putting an exec "pass" line in the function. This switches the function to an older, slower execution mode that uses the locals() dict as the canonical representation of local variables.
If locals() is called outside a function it returns the actual dictionary that is the current namespace. Further changes to the namespace are reflected in the dictionary, and changes to the dictionary are reflected in the namespace:
class Test(object):
a = 'one'
b = 'two'
huh = locals()
c = 'three'
huh['d'] = 'four'
print huh
gives us:
{
'a': 'one',
'b': 'two',
'c': 'three',
'd': 'four',
'huh': {...},
'__module__': '__main__',
}
So far, everything I've said about locals() is also true for vars()... here's the difference: vars() accepts a single object as its argument, and if you give it an object it returns the __dict__ of that object. For a typical object, its __dict__ is where most of its attribute data is stored. This includes class variables and module globals:
class Test(object):
a = 'one'
b = 'two'
def frobber(self):
print self.c
t = Test()
huh = vars(t)
huh['c'] = 'three'
t.frobber()
which gives us:
three
Note that a function's __dict__ is its attribute namespace, not local variables. It wouldn't make sense for a function's __dict__ to store local variables, since recursion and multithreading mean there can be multiple calls to a function at the same time, each with their own locals:
def f(outer):
if outer:
f(False)
print('Outer call locals:', locals())
print('f.__dict__:', f.__dict__)
else:
print('Inner call locals:', locals())
print('f.__dict__:', f.__dict__)
f.x = 3
f(True)
which gives us:
Inner call locals: {'outer': False}
f.__dict__: {'x': 3}
Outer call locals: {'outer': True}
f.__dict__: {'x': 3}
Here, f calls itself recursively, so the inner and outer calls overlap. Each one sees its own local variables when it calls locals(), but both calls see the same f.__dict__, and f.__dict__ doesn't have any local variables in it.