In this code snippet I can print the value of counter from inside the bar function
def foo():
counter = 1
def bar():
print("bar", counter)
return bar
bar = foo()
bar()
But when I try to increment counter from inside the bar function I get an UnboundLocalError error.
UnboundLocalError: local variable 'counter' referenced before assignment
Code snippet with increment statement in.
def foo():
counter = 1
def bar():
counter += 1
print("bar", counter)
return bar
bar = foo()
bar()
Do you only have read access to variables in the outer function in a Python closure?
You can't re-bind closure variables in Python 2. In Python 3, which you appear to be using due to your print(), you can declare them nonlocal:
def foo():
counter = 1
def bar():
nonlocal counter
counter += 1
print("bar", counter)
return bar
bar = foo()
bar()
Otherwise, the assignment within bar() makes the variable local, and since you haven't assigned a value to the variable in the local scope, trying to access it is an error.
In Python 2, my favorite workaround looks like this:
def foo():
class nonlocal:
counter = 1
def bar():
nonlocal.counter += 1
print("bar", nonlocal.counter)
return bar
bar = foo()
bar()
This works because mutating a mutable object doesn't require changing what the variable name points to. In this case, nonlocal is the closure variable and it is never reassigned; only its contents are changed. Other workarounds use lists or dictionaries.
Or you could use a class for the whole thing, as #naomik suggests in a comment. Define __call__() to make the instance callable.
class Foo(object):
def __init__(self, counter=1):
self.counter = counter
def __call__(self):
self.counter += 1
print("bar", self.counter)
bar = Foo()
bar()
Why can't Python increment variable in closure?
I offer a couple of solutions here.
Using a function attribute (uncommon, but works pretty well)
Using a closure with nonlocal (ideal, but Python 3 only)
Using a closure over a mutable object (idiomatic of Python 2)
Using a method on a custom object
Directly calling instance of the object by implementing __call__
Use an attribute on the function.
Set a counter attribute on your function manually after creating it:
def foo():
foo.counter += 1
return foo.counter
foo.counter = 0
And now:
>>> foo()
1
>>> foo()
2
>>> foo()
3
Or you can auto-set the function:
def foo():
if not hasattr(foo, 'counter'):
foo.counter = 0
foo.counter += 1
return foo.counter
Similarly:
>>> foo()
1
>>> foo()
2
>>> foo()
3
These approaches are simple, but uncommon, and unlikely to be quickly grokked by someone viewing your code without you present.
More common ways what you wish to accomplish is done varies depending on your version of Python.
Python 3, using a closure with nonlocal
In Python 3, you can declare nonlocal:
def foo():
counter = 0
def bar():
nonlocal counter
counter += 1
print("bar", counter)
return bar
bar = foo()
And it would increment
>>> bar()
bar 1
>>> bar()
bar 2
>>> bar()
bar 3
This is probably the most idiomatic solution for this problem. Too bad it's restricted to Python 3.
Python 2 workaround for nonlocal:
You could declare a global variable, and then increment on it, but that clutters the module namespace. So the idiomatic workaround to avoid declaring a global variable is to point to a mutable object that contains the integer on which you wish to increment, so that you're not attempting to reassign the variable name:
def foo():
counter = [0]
def bar():
counter[0] += 1
print("bar", counter)
return bar
bar = foo()
and now:
>>> bar()
('bar', [1])
>>> bar()
('bar', [2])
>>> bar()
('bar', [3])
I do think that is superior to the suggestions that involve creating classes just to hold your incrementing variable. But to be complete, let's see that.
Using a custom object
class Foo(object):
def __init__(self):
self._foo_call_count = 0
def foo(self):
self._foo_call_count += 1
print('Foo.foo', self._foo_call_count)
foo = Foo()
and now:
>>> foo.foo()
Foo.foo 1
>>> foo.foo()
Foo.foo 2
>>> foo.foo()
Foo.foo 3
or even implement __call__:
class Foo2(object):
def __init__(self):
self._foo_call_count = 0
def __call__(self):
self._foo_call_count += 1
print('Foo', self._foo_call_count)
foo = Foo2()
and now:
>>> foo()
Foo 1
>>> foo()
Foo 2
>>> foo()
Foo 3
As you can't modify immutable objects in Python, there is a workaround in Python 2.X:
Use List
def counter(start):
count = [start]
def incr():
count[0] += 1
return count[0]
return incr
a = counter(100)
print a()
b = counter(200)
print b()
Related
Example of what I'm asking about:
def foo(bar):
"""Do a print function bar times"""
count = 0
while count < bar:
def baz():
return "This is baz"
print(baz())
count += 1
Does the function declaration in the middle of the while loop slow down the execution time of foo?
To expand on one of the comments, you are adding additional work to your loop. Every time you declare baz() the compile is doing work and allocating memory. Is there any particular reason you wanted to do it this way?
More efficient code:
def foo(bar):
"""Do a print function bar times"""
count = 0
def baz():
return "This is baz"
while count < bar:
print(baz())
count += 1
Most efficient code:
def baz():
return "This is baz"
def foo(bar):
"""Do a print function bar times"""
count = 0
while count < bar:
print(baz())
count += 1
I can create a function of the following format.
def bar():
if not hasattr(bar, 'c'):
bar.c = 0
bar.c += 1
return bar.c
When run it produces the following output, as intended:
>>> bar()
1
>>> bar()
2
>>> bar()
3
But if I suddenly move this function to a class, Python gives me an error.
class Foo(object):
def bar(self):
if not hasattr(self.bar, 'c'):
self.bar.c = 0
self.bar.c += 1
return self.bar.c
Now I get
>>> foo = Foo()
>>> foo.bar()
...
AttributeError: 'method' object has no attribute 'c'
It's telling me it has no attribute, but I'm trying to create the attribute. What's going on here?
Taken from the python documentation (https://docs.python.org/2/library/stdtypes.html?highlight=instancemethod) "However, since method attributes are actually stored on the underlying function object (meth.im_func), setting method attributes on either bound or unbound methods is disallowed."
In other words, we could have tried Foo.bar.im_func.c = 0, but unfortunately, it is read-only, so it doesn't work.
Therefore, for what you try to accomplish, I would suggest
class Foo(object):
c = 0
def bar(self):
Foo.c += 1
return Foo.c
f = Foo()
print f.bar()
print f.bar()
print f.bar()
Works on python2 and python3.
The only "drawback" is that c would be available to any other method of class Foo.
In python a function is a first class object. A class can be called. So you can replace a function with a class. But can you make a function behave like a class? Can you add and remove attributes or call inner functions( then called methods) in a function?
I found a way to do this via code inspection.
import inspect
class AddOne(object):
"""class definition"""
def __init__(self, num):
self.num = num
def getResult(self):
"""
class method
"""
def addOneFunc(num):
"inner function"
return num + 1
return addOneFunc(self.num);
if __name__ == '__main__':
two = AddOne(1);
two_src = '\n'.join([line[4:] for line in inspect.getsource(AddOne.getResult).split('\n')])
one_src = '\n'.join([line[4:] for line in two_src.split('\n')
if line[:4] == ' ' and line[4:8] == ' ' or line[4:8] == 'def '])
one_co = compile(one_src, '<string>', 'exec')
exec one_co
print addOneFunc(5)
print addOneFunc.__doc__
But is there a way to access the local variables and functions defined in a class in a more direct way?
EDIT
The question is about how to access the inner structure of python to get a better understanding. Of course I wouldn't do this in normal programming. The question arose when we had a discussion about private variables in python. My opinion was this to be against the philosophy of the language. So someone came up with the example above. At the moment it seems he is right. You cannot access the function inside a function without the inspect module, rendering this function private. With co_varnames we are awfully close because we already have the name of the function. But where is the namespace dictionary to hold the name. If you try to use
getResult.__dict__
it is empty. What I like to have is an answer from python like
function addOneFunc at <0xXXXXXXXXX>
You can consider a function to be an instance of a class that only implements __call__, i.e.
def foo(bar):
return bar
is roughly equivalent to
class Foo(object):
def __call__(self, bar):
return bar
foo = Foo()
Function instances have a __dict__ attribute, so you can freely add new attributes to them.
Adding an attribute to a function can be used, for example, to implement a memoization decorator, which caches previous calls to a function:
def memo(f):
#functools.wraps(f)
def func(*args):
if args not in func.cache: # access attribute
func.cache[args] = f(*args)
return func.cache[args]
func.cache = {} # add attribute
return func
Note that this attribute can also be accessed inside the function, although it can't be defined until after the function.
You could therefore do something like:
>>> def foo(baz):
def multiply(x, n):
return x * n
return multiply(foo.bar(baz), foo.n)
>>> def bar(baz):
return baz
>>> foo.bar = bar
>>> foo.n = 2
>>> foo('baz')
'bazbaz'
>>> foo.bar = len
>>> foo('baz')
6
(although it's possible that nobody would thank you for it!)
Note, however, that multiply, which was not made an attribute of foo, is not accessible from outside the function:
>>> foo.multiply(1, 2)
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
foo.multiply(1, 2)
AttributeError: 'function' object has no attribute 'multiply'
The other question addresses exactly what you're trying to do:
>>> import inspect
>>> import new
>>> class AddOne(object):
"""Class definition."""
def __init__(self, num):
self.num = num
def getResult(self):
"""Class method."""
def addOneFunc(num):
"inner function"
return num + 1
return addOneFunc(self.num)
>>> two = AddOne(1)
>>> for c in two.getResult.func_code.co_consts:
if inspect.iscode(c):
print new.function(c, globals())
<function addOneFunc at 0x0321E930>
Not sure if the following is what you're thinking about, but you can do this:
>>> def f(x):
... print(x)
...
>>> f.a = 1
>>> f.a
1
>>> f(54)
54
>>> f.a = f
>>> f
<function f at 0x7fb03579b320>
>>> f.a
<function f at 0x7fb03579b320>
>>> f.a(2)
2
So you can assign attributes to a function, and those attributes can be variables or functions (note that f.a = f was chosen for simplicity; you can assign f.a to any function of course).
If you want to access the local variables inside the function, I think then it's more difficult, and you may indeed need to revert to introspection. The example below uses the func_code attribute:
>>> def f(x):
... a = 1
... return x * a
...
>>> f.func_code.co_nlocals
2
>>> f.func_code.co_varnames
('x', 'a')
>>> f.func_code.co_consts
(None, 1)
I want to use a variable that will be independent of any objects the class may have :
class A:
var = 5
ob1 = A()
ob2 = A()
ob1.var += 2
print ob2.var
I want that once ob1 modifies the value of var from 5 to 7, ob2.var also shows up 7 instead of 5. How do I do this?
When you do ob1.var += 2 you are not modifying the class attribute, you are creating a new instance variable with the same name as the class attribute. For example:
>>> ob1 = A()
>>> ob1.var += 2
>>> ob1.__dict__
{'var': 7}
>>> ob1.__class__.__dict__
{'var': 5, '__module__': '__main__', '__doc__': None}
If you want to modify the class attribute, just use A.var += 2. If you need to do this from an instance you can use ob1.__class__.var += 2.
Here is how you could implement the behavior you want using properties on a new-style class:
class A(object):
_var = 5
#property
def var(self):
return self.__class__._var
#var.setter
def var(self, value):
self.__class__._var = value
Now with this class when you get or set the var variable on an instance of A, it will actually be getting or setting a class attribute. For example:
>>> ob1 = A()
>>> ob2 = A()
>>> ob1.var += 2
>>> ob2.var
7
you are already close
class MyData:
my_data_var = 4
class A:
def fun1():
MyData.my_data_var += 2
def fun2():
return MyData.my_data_var
def main():
print MyData.my_data_var
a = A()
a.fun1()
print a.fun2()
b = A()
b.fun1()
print b.fun2()
print a.fun1()
main()
these are static class variables ... there are other ways to simillarly utilize them ... but this was a simple quick example
I suppose you could use a property to do something like this:
class Foo(object):
_var = 5
#property
def var(self):
return self._var
#var.setter
def var(self,value):
self.__class__._var = value
a = Foo()
b = Foo()
a.var += 3
assert a.var == 8
assert b.var == 8
c = Foo()
assert c.var == 8 #(un-instantiated objects pick up the change as well)
class Bar(Foo):
pass
bar = Bar()
assert bar.var == 8
a.var += 1
#changes to Foo.var affect Bar until a bar object makes it's own changes
assert bar.var == 9
# changes to a subclass's property don't have any effect on the parent class
bar.var += 2
assert bar.var == 11
assert a.var == 9
As noted in the comments, this could get a little tedious if you have a lot of properties which behave this way. A slightly more general solution would be to do something like:
class ClassAttribute(object):
def __init__(self,vname,default=None):
self.vname = '_'+str(vname)
self.default = default
def __get__(self,instance,owner):
return getattr(owner,self.vname,self.default)
def __set__(self,instance,value):
setattr(instance.__class__,self.vname,value)
class Foo(object):
var = ClassAttribute("var",5)
a = Foo()
b = Foo()
a.var += 2
print a.var
print b.var
here's a sample code:
def foo():
def bar():
foobar = 'foobaz'
foobar = 'foobar'
print foobar
bar()
print foobar
foo()
I want to change variable foobar inside foo by function bar. The code above will not work, since foobar inside bar is in separate namespace with foobar in foo. A simple workaround would be making a global foobar and have both foo and bar can access it, but I hope there would be simpler workarounds.
On python 3.x you can use nonlocal and for python 2.x try using function attributes:
def foo():
def bar():
foo.foobar = 'foobaz' #change the function attribute
foo.foobar = 'foobar' #declare as function attribute
print foo.foobar
bar()
print foo.foobar
foo()
output:
foobar
foobaz
You are looking for the nonlocal keyword, which exists in 3.x.
def f():
x = None
def g():
nonlocal x
x = 1
If you are stuck in 2.x, you can do it by having a list or similar mutable data container and accessing that as a work around.
def f():
x = [None]
def g():
x[0] = 1
This works as variables do fall into scope, but won't leak out of scope. With mutable objects, we can change them inside the scope, and those changes propagate out.
Not possible in python 2.7. In python 3:
def foo():
def bar():
nonlocal foobar
foobar = 'foobaz'
foobar = 'foobar'
print foobar
bar()
print foobar
foo()
In 2.x, you can do:
def foo():
foobar = []
def bar():
foobar[0] = 'foobaz'
foobar[0] = 'foobar'
print foobar[0]
bar()
print foobar[0]
foo()
def foo():
def bar():
foobar = 'foobaz'
return foobar
foobar = 'foobar'
print foobar
foobar = bar()
print foobar
foo()
Even though functions are already first class objects in Python, you can create your own "functor" or function object something like this:
class Foo(object):
def bar(self):
self.foobar = 'foobaz'
def __call__(self):
self.foobar = 'foobar'
print self.foobar
self.bar()
print self.foobar
foo = Foo()
foo()