I'm implementing a python decorator with an internal memory (represented by counter below).
It seams the decorator variables are shared across instances of a dataclass while being different for instances of a common class.
Why is so? And is there a cleaner/easier solution other than checking if f belongs to a class or not and if so, if the class is a dataclass?
import dataclasses
def decorator(f):
counter = {}
def wrapper(*args, **kwargs):
key = repr([f.__name__, args, kwargs])
counter[key] = counter.setdefault(key, 0)+1
result = f(*args, **kwargs)
print(f"{counter[key]}", end=" ")
return result
return wrapper
#dataclasses.dataclass
class D:
#decorator
def foo(self):
pass
class C:
#decorator
def foo(self):
pass
Despide C and D being very similar, the code below shows that the instances of normal object have different counter each:
>>> for i in range(5):
... c = C()
... c.foo()
1 1 1 1 1
While when using the dataclass instead, the counter is shared:
>>> for i in range(5):
... c = D()
... c.foo()
1 2 3 4 5
Decorator syntax is a shortcut for function application, so each use of #decorator is a separate call to decorator, each of which creates a new dict associated with the decorated function.
So it's one counter per decorated function, and there is one decorated function per class in your example.
But then there's another problem.
Your key depends on each class's __repr__ function, as *args includes the object itself.
For C, __repr__ is not defined, so object.__repr__ is used, producing a unique key for each instance.
For D, D.__repr__ returns a generic string 'D()' for every instance, so you aren't getting unique keys for instances of D.
The solution is to be more explicit in constructing the key. Perhaps something like
from collections import Counter
def decorator(f):
counter = Counter()
def wrapper(*args, **kwargs):
key = repr([id(f.__name__), [id(x) for x in args], [id(x) for x in kwargs.items()]])
counter[key] += 1
result = f(*args, **kwargs)
# print(f"{counter[key]}", end=" ")
return result
return wrapper
As you're decorating methods rather than functions, the value of *args in wrapper(*args, **kwargs) will be a one element tuple containing the implicit self.
Your key value would then look like this ['foo', (<__main__.C object at 0x7fe8945403c8>,), {}].
As those instances of C and D get garbage collected, sometimes Python will reuse the same memory address and sometimes not, leading to different keys.
I'm not sure why Python would reuse dataclass addresses more than for regular classes.
If you change wrapper to expect self, you should get consistent results.
def decorator(f):
counter = {}
def wrapper(self, *args, **kwargs):
key = repr([f.__name__, args, kwargs])
counter[key] = counter.setdefault(key, 0)+1
result = f(self, *args, **kwargs)
print(f"{counter[key]}", end=" ")
return result
return wrapper
how to implement a function that will be invoked in the following way sum_numbers(2)(3)(4)......(n) in python?
the result should be 2+3+4+.....+n
The hint that I have is since functions are object in pythons there is way to do those using a nested function but I am not sure.
def sum_number(x):
def sum_number_2(y):
def sum_number_3(z):
....................
def sum_number_n(n)
return n
return sum_number_n
return sum_number_3
return sum_number_2
return sum_number
But instead of writing so many nested functions we should be able to do it in couple nested functions to compute sum of n values when invoked in the following way sum_numbers(2)(3)(4)......(n)
Use Python's data model features to convert the result into the desired type.
class sum_number(object):
def __init__(self, val):
self.val = val
def __call__(self, val):
self.val += val
return self
def __float__(self):
return float(self.val)
def __int__(self):
return int(self.val)
print '{}'.format(int(sum_number(2)(3)(8)))
print '{}'.format(float(sum_number(2)(3)(8)))
You could create a subclass of int that is callable:
class sum_numbers (int):
def __new__ (cls, *args, **kwargs):
return super().__new__(cls, *args, **kwargs)
def __call__ (self, val):
return sum_numbers(self + val)
That way, you have full compatibility with a normal integer (since objects of that type are normal integers), so the following examples work:
>>> sum_numbers(2)(3)(4)(5)
14
>>> isinstance(sum_numbers(2)(3), int)
True
>>> sum_numbers(2)(3) + 4
9
Of course, you may want to override additional methods, e.g. __add__ so that adding a normal integer will still return an object of your type. Otherwise, you would have to call the type with the result, e.g.:
>>> sum_numbers(sum_numbers(2)(3) + 5)(6)
16
If your function is returning another function, you can't just chain calls together and expect a human readable result. If you want a function that does what you want without the final result, this works:
def sum(x):
def f(y):
return sum(x+y)
return f
If you're fine with printing out the operations you can try this:
def sum(x):
print(x)
def f(y):
return sum(x+y)
return f
If you absolutely, absolutely need a return value then this is a dirty, horrible hack you could try:
def sum(x, v):
v[0] = x
def f(y, v):
return sum(x+y, v)
return f
v = [0]
sum(1,v)(2,v)(3,v)
print(v[0]) # Should return 6
Here's another solution that uses classes:
class sum(object):
def __init__(self, x=0):
self.x=x
def __call__(self, *y):
if len(y) > 0:
self.x += y[0]
return self
return self.x
print(sum(1)(2)(3)()) # Prints 6
What you're asking for is not possible in Python since you aren't providing a way to determine the end of the call chain, as cricket_007 mentions in the comments. However, if you do provide a way to indicate that there are no more calls then the function is easy to code. One way to indicate the end of the chain is to make the last call with no arguments.
I'm using rsum (recursive sum) as the name of the function in my code because sum is a built-in function and unnecessarily shadowing the Python built-ins is not a good coding practice: it makes the code potentially confusing, or at least harder to read because you have to keep remembering that the name isn't referring to what you normally expect it to refer to, and can lead to subtle bugs.
def rsum(val=None, tot=0):
if val is None:
return tot
tot += val
return lambda val=None, tot=tot: rsum(val, tot)
print rsum(42)()
print rsum(1)(2)()
print rsum(4)(3)(2)(1)()
print rsum(4100)(310000)(9)(50)()
output
42
3
10
314159
class MetaSum(type):
def __repr__(cls):
sum_str = str(cls.sum)
cls.sum = 0
return sum_str
def __call__(cls, *args):
for arg in args:
cls.sum += arg
return cls
class sum_numbers(object, metaclass = MetaSum):
sum = 0
print (sum_numbers(2)(3)(4)) # this only works in python 3
For instanse, I have the following code snippet:
def func1(num):
print(num)
def func2(num):
func1(num)
def func3(num):
func2(num)
func1(num)
def begin():
pass
def print_graph():
pass
def main():
begin()
func3(3)
print_graph()
Is there any simple way to print something like that:
func3(1)
func2(1)
func1(1)
func1(1)
I believe, that I have to use globals(), but I don't know, what I do next. It is some sort of study task, therefore I cant use any libraries.
I can go one better than #jme. Here's a version of his decorator that indents and dedents according to your location in the call stack:
import functools
# a factory for decorators
def create_tracer(tab_width):
indentation_level = 0
def decorator(f): # a decorator is a function which takes a function and returns a function
#functools.wraps(f)
def wrapper(*args): # we wish to extend the function that was passed to the decorator, so we define a wrapper function to return
nonlocal indentation_level # python 3 only, sorry
msg = " " * indentation_level + "{}({})".format(f.__name__, ", ".join([str(a) for a in args]))
print(msg)
indentation_level += tab_width # mutate the closure so the next function that is called gets a deeper indentation level
result = f(*args)
indentation_level -= tab_width
return result
return wrapper
return decorator
tracer = create_tracer(4) # create the decorator itself
#tracer
def f1():
x = f2(5)
return f3(x)
#tracer
def f2(x):
return f3(2)*x
#tracer
def f3(x):
return 4*x
f1()
Output:
f1()
f2(5)
f3(2)
f3(40)
The nonlocal statement allows us to mutate the indentation_level in the outer scope. Upon entering a function, we increase the indentation level so that the next print gets indented further. Then upon exiting we decrease it again.
This is called decorator syntax. It's purely 'syntactic sugar'; the transformation into equivalent code without # is very simple.
#d
def f():
pass
is just the same as:
def f():
pass
f = d(f)
As you can see, # simply uses the decorator to process the decorated function in some way, and replaces the original function with the result, just like in #jme's answer. It's like Invasion of the Body Snatchers; we are replacing f with something that looks similar to f but behaves differently.
If you're stuck on Python 2, you can simulate the nonlocal statement by using a class with an instance variable. This might make a bit more sense to you, if you've never used decorators before.
# a class which acts like a decorator
class Tracer(object):
def __init__(self, tab_width):
self.tab_width = tab_width
self.indentation_level = 0
# make the class act like a function (which takes a function and returns a function)
def __call__(self, f):
#functools.wraps(f)
def wrapper(*args):
msg = " " * self.indentation_level + "{}({})".format(f.__name__, ", ".join([str(a) for a in args]))
print msg
self.indentation_level += self.tab_width
result = f(*args)
self.indentation_level -= self.tab_width
return result
return wrapper
tracer = Tracer(4)
#tracer
def f1():
# etc, as above
You mentioned that you're not allowed to change the existing functions. You can retro-fit the decorator by messing around with globals() (though this generally isn't a good idea unless you really need to do it):
for name, val in globals().items(): # use iteritems() in Python 2
if name.contains('f'): # look for the functions we wish to trace
wrapped_func = tracer(val)
globals()[name] = wrapped_func # overwrite the function with our wrapped version
If you don't have access to the source of the module in question, you can achieve something very similar by inspecting the imported module and mutating the items it exports.
The sky's the limit with this approach. You could build this into an industrial-strength code analysis tool by storing the calls in some sort of graph data structure, instead of simply indenting and printing. You could then query your data to answer questions like "which functions in this module are called the most?" or "which functions are the slowest?". In fact, that's a great idea for a library...
If you don't want to use modify code, you can always use sys.settrace. Here is a simple sample:
import sys
import inspect
class Tracer(object):
def __init__(self):
self._indentation_level = 0
#property
def indentation_level(self):
return self._indentation_level
#indentation_level.setter
def indentation_level(self, value):
self._indentation_level = max(0, value)
def __enter__(self):
sys.settrace(self)
def __exit__(self, exc_type, exc_value, traceback):
sys.settrace(None)
def __call__(self, frame, event, args):
frameinfo = inspect.getframeinfo(frame)
filename = frameinfo.filename
# Use `in` instead of comparing because you need to cover for `.pyc` files as well.
if filename in __file__:
return None
if event == 'return':
self.indentation_level -= 1
elif event == 'call':
print "{}{}{}".format(" " * self.indentation_level,
frameinfo.function,
inspect.formatargvalues(*inspect.getargvalues(frame)))
self.indentation_level += 1
else:
return None
return self
Usage:
from tracer import Tracer
def func1(num):
pass
def func2(num):
func1(num)
def func3(num):
func2(num)
func1(num)
def main():
with Tracer():
func3(1)
And results:
func3(num=1)
func2(num=1)
func1(num=1)
func1(num=1)
How about using decorators to print a function's name when it is called? Something like this:
from functools import wraps
def print_on_entry(fn):
#wraps(fn)
def wrapper(*args):
print "{}({})".format(fn.func_name, ", ".join(str(a) for a in args))
fn(*args)
return wrapper
Then you can wrap each of your functions up:
func1 = print_on_entry(func1)
func2 = print_on_entry(func2)
func3 = print_on_entry(func3)
So that:
>>> func3(1)
func3(1)
func2(1)
func1(1)
1
func1(1)
1
Of course there are a lot of assumptions in the above code -- the arguments can be converted to strings, for example -- but you get the picture.
I have a set of arrays that are very large and expensive to compute, and not all will necessarily be needed by my code on any given run. I would like to make their declaration optional, but ideally without having to rewrite my whole code.
Example of how it is now:
x = function_that_generates_huge_array_slowly(0)
y = function_that_generates_huge_array_slowly(1)
Example of what I'd like to do:
x = lambda: function_that_generates_huge_array_slowly(0)
y = lambda: function_that_generates_huge_array_slowly(1)
z = x * 5 # this doesn't work because lambda is a function
# is there something that would make this line behave like
# z = x() * 5?
g = x * 6
While using lambda as above achieves one of the desired effects - computation of the array is delayed until it is needed - if you use the variable "x" more than once, it has to be computed each time. I'd like to compute it only once.
EDIT:
After some additional searching, it looks like it is possible to do what I want (approximately) with "lazy" attributes in a class (e.g. http://code.activestate.com/recipes/131495-lazy-attributes/). I don't suppose there's any way to do something similar without making a separate class?
EDIT2: I'm trying to implement some of the solutions, but I'm running in to an issue because I don't understand the difference between:
class sample(object):
def __init__(self):
class one(object):
def __get__(self, obj, type=None):
print "computing ..."
obj.one = 1
return 1
self.one = one()
and
class sample(object):
class one(object):
def __get__(self, obj, type=None):
print "computing ... "
obj.one = 1
return 1
one = one()
I think some variation on these is what I'm looking for, since the expensive variables are intended to be part of a class.
The first half of your problem (reusing the value) is easily solved:
class LazyWrapper(object):
def __init__(self, func):
self.func = func
self.value = None
def __call__(self):
if self.value is None:
self.value = self.func()
return self.value
lazy_wrapper = LazyWrapper(lambda: function_that_generates_huge_array_slowly(0))
But you still have to use it as lazy_wrapper() not lazy_wrapper.
If you're going to be accessing some of the variables many times, it may be faster to use:
class LazyWrapper(object):
def __init__(self, func):
self.func = func
def __call__(self):
try:
return self.value
except AttributeError:
self.value = self.func()
return self.value
Which will make the first call slower and subsequent uses faster.
Edit: I see you found a similar solution that requires you to use attributes on a class. Either way requires you rewrite every lazy variable access, so just pick whichever you like.
Edit 2: You can also do:
class YourClass(object)
def __init__(self, func):
self.func = func
#property
def x(self):
try:
return self.value
except AttributeError:
self.value = self.func()
return self.value
If you want to access x as an instance attribute. No additional class is needed. If you don't want to change the class signature (by making it require func), you can hard code the function call into the property.
Writing a class is more robust, but optimizing for simplicity (which I think you are asking for), I came up with the following solution:
cache = {}
def expensive_calc(factor):
print 'calculating...'
return [1, 2, 3] * factor
def lookup(name):
return ( cache[name] if name in cache
else cache.setdefault(name, expensive_calc(2)) )
print 'run one'
print lookup('x') * 2
print 'run two'
print lookup('x') * 2
Python 3.2 and greater implement an LRU algorithm in the functools module to handle simple cases of caching/memoization:
import functools
#functools.lru_cache(maxsize=128) #cache at most 128 items
def f(x):
print("I'm being called with %r" % x)
return x + 1
z = f(9) + f(9)**2
You can't make a simple name, like x, to really evaluate lazily. A name is just an entry in a hash table (e.g. in that which locals() or globals() return). Unless you patch access methods of these system tables, you cannot attach execution of your code to simple name resolution.
But you can wrap functions in caching wrappers in different ways.
This is an OO way:
class CachedSlowCalculation(object):
cache = {} # our results
def __init__(self, func):
self.func = func
def __call__(self, param):
already_known = self.cache.get(param, None)
if already_known:
return already_known
value = self.func(param)
self.cache[param] = value
return value
calc = CachedSlowCalculation(function_that_generates_huge_array_slowly)
z = calc(1) + calc(1)**2 # only calculates things once
This is a classless way:
def cached(func):
func.__cache = {} # we can attach attrs to objects, functions are objects
def wrapped(param):
cache = func.__cache
already_known = cache.get(param, None)
if already_known:
return already_known
value = func(param)
cache[param] = value
return value
return wrapped
#cached
def f(x):
print "I'm being called with %r" % x
return x + 1
z = f(9) + f(9)**2 # see f called only once
In real world you'll add some logic to keep the cache to a reasonable size, possibly using a LRU algorithm.
To me, it seems that the proper solution for your problem is subclassing a dict and using it.
class LazyDict(dict):
def __init__(self, lazy_variables):
self.lazy_vars = lazy_variables
def __getitem__(self, key):
if key not in self and key in self.lazy_vars:
self[key] = self.lazy_vars[key]()
return super().__getitem__(key)
def generate_a():
print("generate var a lazily..")
return "<a_large_array>"
# You can add as many variables as you want here
lazy_vars = {'a': generate_a}
lazy = LazyDict(lazy_vars)
# retrieve the variable you need from `lazy`
a = lazy['a']
print("Got a:", a)
And you can actually evaluate a variable lazily if you use exec to run your code. The solution is just using a custom globals.
your_code = "print('inside exec');print(a)"
exec(your_code, lazy)
If you did your_code = open(your_file).read(), you could actually run your code and achieve what you want. But I think the more practical approach would be the former one.
What is the idiomatic Python equivalent of this C/C++ code?
void foo()
{
static int counter = 0;
counter++;
printf("counter is %d\n", counter);
}
specifically, how does one implement the static member at the function level, as opposed to the class level? And does placing the function into a class change anything?
A bit reversed, but this should work:
def foo():
foo.counter += 1
print "Counter is %d" % foo.counter
foo.counter = 0
If you want the counter initialization code at the top instead of the bottom, you can create a decorator:
def static_vars(**kwargs):
def decorate(func):
for k in kwargs:
setattr(func, k, kwargs[k])
return func
return decorate
Then use the code like this:
#static_vars(counter=0)
def foo():
foo.counter += 1
print "Counter is %d" % foo.counter
It'll still require you to use the foo. prefix, unfortunately.
(Credit: #ony)
You can add attributes to a function, and use it as a static variable.
def myfunc():
myfunc.counter += 1
print myfunc.counter
# attribute must be initialized
myfunc.counter = 0
Alternatively, if you don't want to setup the variable outside the function, you can use hasattr() to avoid an AttributeError exception:
def myfunc():
if not hasattr(myfunc, "counter"):
myfunc.counter = 0 # it doesn't exist yet, so initialize it
myfunc.counter += 1
Anyway static variables are rather rare, and you should find a better place for this variable, most likely inside a class.
One could also consider:
def foo():
try:
foo.counter += 1
except AttributeError:
foo.counter = 1
Reasoning:
much pythonic ("ask for forgiveness not permission")
use exception (thrown only once) instead of if branch (think StopIteration exception)
Many people have already suggested testing 'hasattr', but there's a simpler answer:
def func():
func.counter = getattr(func, 'counter', 0) + 1
No try/except, no testing hasattr, just getattr with a default.
Other answers have demonstrated the way you should do this. Here's a way you shouldn't:
>>> def foo(counter=[0]):
... counter[0] += 1
... print("Counter is %i." % counter[0]);
...
>>> foo()
Counter is 1.
>>> foo()
Counter is 2.
>>>
Default values are initialized only when the function is first evaluated, not each time it is executed, so you can use a list or any other mutable object to store static values.
Python doesn't have static variables but you can fake it by defining a callable class object and then using it as a function. Also see this answer.
class Foo(object):
# Class variable, shared by all instances of this class
counter = 0
def __call__(self):
Foo.counter += 1
print Foo.counter
# Create an object instance of class "Foo," called "foo"
foo = Foo()
# Make calls to the "__call__" method, via the object's name itself
foo() #prints 1
foo() #prints 2
foo() #prints 3
Note that __call__ makes an instance of a class (object) callable by its own name. That's why calling foo() above calls the class' __call__ method. From the documentation:
Instances of arbitrary classes can be made callable by defining a __call__() method in their class.
Here is a fully encapsulated version that doesn't require an external initialization call:
def fn():
fn.counter=vars(fn).setdefault('counter',-1)
fn.counter+=1
print (fn.counter)
In Python, functions are objects and we can simply add, or monkey patch, member variables to them via the special attribute __dict__. The built-in vars() returns the special attribute __dict__.
EDIT: Note, unlike the alternative try:except AttributeError answer, with this approach the variable will always be ready for the code logic following initialization. I think the try:except AttributeError alternative to the following will be less DRY and/or have awkward flow:
def Fibonacci(n):
if n<2: return n
Fibonacci.memo=vars(Fibonacci).setdefault('memo',{}) # use static variable to hold a results cache
return Fibonacci.memo.setdefault(n,Fibonacci(n-1)+Fibonacci(n-2)) # lookup result in cache, if not available then calculate and store it
EDIT2: I only recommend the above approach when the function will be called from multiple locations. If instead the function is only called in one place, it's better to use nonlocal:
def TheOnlyPlaceStaticFunctionIsCalled():
memo={}
def Fibonacci(n):
nonlocal memo # required in Python3. Python2 can see memo
if n<2: return n
return memo.setdefault(n,Fibonacci(n-1)+Fibonacci(n-2))
...
print (Fibonacci(200))
...
Use a generator function to generate an iterator.
def foo_gen():
n = 0
while True:
n+=1
yield n
Then use it like
foo = foo_gen().next
for i in range(0,10):
print foo()
If you want an upper limit:
def foo_gen(limit=100000):
n = 0
while n < limit:
n+=1
yield n
If the iterator terminates (like the example above), you can also loop over it directly, like
for i in foo_gen(20):
print i
Of course, in these simple cases it's better to use xrange :)
Here is the documentation on the yield statement.
Other solutions attach a counter attribute to the function, usually with convoluted logic to handle the initialization. This is inappropriate for new code.
In Python 3, the right way is to use a nonlocal statement:
counter = 0
def foo():
nonlocal counter
counter += 1
print(f'counter is {counter}')
See PEP 3104 for the specification of the nonlocal statement.
If the counter is intended to be private to the module, it should be named _counter instead.
Using an attribute of a function as static variable has some potential drawbacks:
Every time you want to access the variable, you have to write out the full name of the function.
Outside code can access the variable easily and mess with the value.
Idiomatic python for the second issue would probably be naming the variable with a leading underscore to signal that it is not meant to be accessed, while keeping it accessible after the fact.
Using closures
An alternative would be a pattern using lexical closures, which are supported with the nonlocal keyword in python 3.
def make_counter():
i = 0
def counter():
nonlocal i
i = i + 1
return i
return counter
counter = make_counter()
Sadly I know no way to encapsulate this solution into a decorator.
Using an internal state parameter
Another option might be an undocumented parameter serving as a mutable value container.
def counter(*, _i=[0]):
_i[0] += 1
return _i[0]
This works, because default arguments are evaluated when the function is defined, not when it is called.
Cleaner might be to have a container type instead of the list, e.g.
def counter(*, _i = Mutable(0)):
_i.value += 1
return _i.value
but I am not aware of a builtin type, that clearly communicates the purpose.
A little bit more readable, but more verbose (Zen of Python: explicit is better than implicit):
>>> def func(_static={'counter': 0}):
... _static['counter'] += 1
... print _static['counter']
...
>>> func()
1
>>> func()
2
>>>
See here for an explanation of how this works.
_counter = 0
def foo():
global _counter
_counter += 1
print 'counter is', _counter
Python customarily uses underscores to indicate private variables. The only reason in C to declare the static variable inside the function is to hide it outside the function, which is not really idiomatic Python.
def staticvariables(**variables):
def decorate(function):
for variable in variables:
setattr(function, variable, variables[variable])
return function
return decorate
#staticvariables(counter=0, bar=1)
def foo():
print(foo.counter)
print(foo.bar)
Much like vincent's code above, this would be used as a function decorator and static variables must be accessed with the function name as a prefix. The advantage of this code (although admittedly anyone might be smart enough to figure it out) is that you can have multiple static variables and initialise them in a more conventional manner.
After trying several approaches I ended up using an improved version of #warvariuc's answer:
import types
def func(_static=types.SimpleNamespace(counter=0)):
_static.counter += 1
print(_static.counter)
The idiomatic way is to use a class, which can have attributes. If you need instances to not be separate, use a singleton.
There are a number of ways you could fake or munge "static" variables into Python (one not mentioned so far is to have a mutable default argument), but this is not the Pythonic, idiomatic way to do it. Just use a class.
Or possibly a generator, if your usage pattern fits.
A static variable inside a Python method
class Count:
def foo(self):
try:
self.foo.__func__.counter += 1
except AttributeError:
self.foo.__func__.counter = 1
print self.foo.__func__.counter
m = Count()
m.foo() # 1
m.foo() # 2
m.foo() # 3
Another (not recommended!) twist on the callable object like https://stackoverflow.com/a/279598/916373, if you don't mind using a funky call signature, would be to do
class foo(object):
counter = 0;
#staticmethod
def __call__():
foo.counter += 1
print "counter is %i" % foo.counter
>>> foo()()
counter is 1
>>> foo()()
counter is 2
Soulution n +=1
def foo():
foo.__dict__.setdefault('count', 0)
foo.count += 1
return foo.count
A global declaration provides this functionality. In the example below (python 3.5 or greater to use the "f"), the counter variable is defined outside of the function. Defining it as global in the function signifies that the "global" version outside of the function should be made available to the function. So each time the function runs, it modifies the value outside the function, preserving it beyond the function.
counter = 0
def foo():
global counter
counter += 1
print("counter is {}".format(counter))
foo() #output: "counter is 1"
foo() #output: "counter is 2"
foo() #output: "counter is 3"
Using a decorator and a closure
The following decorator can be used create static function variables. It replaces the declared function with the return from itself. This implies that the decorated function must return a function.
def static_inner_self(func):
return func()
Then use the decorator on a function that returns another function with a captured variable:
#static_inner_self
def foo():
counter = 0
def foo():
nonlocal counter
counter += 1
print(f"counter is {counter}")
return foo
nonlocal is required, otherwise Python thinks that the counter variable is a local variable instead of a captured variable. Python behaves like that because of the variable assignment counter += 1. Any assignment in a function makes Python think that the variable is local.
If you are not assigning to the variable in the inner function, then you can ignore the nonlocal statement, for example, in this function I use to indent lines of a string, in which Python can infer that the variable is nonlocal:
#static_inner_self
def indent_lines():
import re
re_start_line = re.compile(r'^', flags=re.MULTILINE)
def indent_lines(text, indent=2):
return re_start_line.sub(" "*indent, text)
return indent_lines
P.S. There is a deleted answer that proposed the same. I don't know why the author deleted it.
https://stackoverflow.com/a/23366737/195417
Prompted by this question, may I present another alternative which might be a bit nicer to use and will look the same for both methods and functions:
#static_var2('seed',0)
def funccounter(statics, add=1):
statics.seed += add
return statics.seed
print funccounter() #1
print funccounter(add=2) #3
print funccounter() #4
class ACircle(object):
#static_var2('seed',0)
def counter(statics, self, add=1):
statics.seed += add
return statics.seed
c = ACircle()
print c.counter() #1
print c.counter(add=2) #3
print c.counter() #4
d = ACircle()
print d.counter() #5
print d.counter(add=2) #7
print d.counter() #8
If you like the usage, here's the implementation:
class StaticMan(object):
def __init__(self):
self.__dict__['_d'] = {}
def __getattr__(self, name):
return self.__dict__['_d'][name]
def __getitem__(self, name):
return self.__dict__['_d'][name]
def __setattr__(self, name, val):
self.__dict__['_d'][name] = val
def __setitem__(self, name, val):
self.__dict__['_d'][name] = val
def static_var2(name, val):
def decorator(original):
if not hasattr(original, ':staticman'):
def wrapped(*args, **kwargs):
return original(getattr(wrapped, ':staticman'), *args, **kwargs)
setattr(wrapped, ':staticman', StaticMan())
f = wrapped
else:
f = original #already wrapped
getattr(f, ':staticman')[name] = val
return f
return decorator
Instead of creating a function having a static local variable, you can always create what is called a "function object" and give it a standard (non-static) member variable.
Since you gave an example written C++, I will first explain what a "function object" is in C++. A "function object" is simply any class with an overloaded operator(). Instances of the class will behave like functions. For example, you can write int x = square(5); even if square is an object (with overloaded operator()) and not technically not a "function." You can give a function-object any of the features that you could give a class object.
# C++ function object
class Foo_class {
private:
int counter;
public:
Foo_class() {
counter = 0;
}
void operator() () {
counter++;
printf("counter is %d\n", counter);
}
};
Foo_class foo;
In Python, we can also overload operator() except that the method is instead named __call__:
Here is a class definition:
class Foo_class:
def __init__(self): # __init__ is similair to a C++ class constructor
self.counter = 0
# self.counter is like a static member
# variable of a function named "foo"
def __call__(self): # overload operator()
self.counter += 1
print("counter is %d" % self.counter);
foo = Foo_class() # call the constructor
Here is an example of the class being used:
from foo import foo
for i in range(0, 5):
foo() # function call
The output printed to the console is:
counter is 1
counter is 2
counter is 3
counter is 4
counter is 5
If you want your function to take input arguments, you can add those to __call__ as well:
# FILE: foo.py - - - - - - - - - - - - - - - - - - - - - - - - -
class Foo_class:
def __init__(self):
self.counter = 0
def __call__(self, x, y, z): # overload operator()
self.counter += 1
print("counter is %d" % self.counter);
print("x, y, z, are %d, %d, %d" % (x, y, z));
foo = Foo_class() # call the constructor
# FILE: main.py - - - - - - - - - - - - - - - - - - - - - - - - - - - -
from foo import foo
for i in range(0, 5):
foo(7, 8, 9) # function call
# Console Output - - - - - - - - - - - - - - - - - - - - - - - - - -
counter is 1
x, y, z, are 7, 8, 9
counter is 2
x, y, z, are 7, 8, 9
counter is 3
x, y, z, are 7, 8, 9
counter is 4
x, y, z, are 7, 8, 9
counter is 5
x, y, z, are 7, 8, 9
This answer builds on #claudiu 's answer.
I found that my code was getting less clear when I always had
to prepend the function name, whenever I intend to access a static variable.
Namely, in my function code I would prefer to write:
print(statics.foo)
instead of
print(my_function_name.foo)
So, my solution is to :
add a statics attribute to the function
in the function scope, add a local variable statics as an alias to my_function.statics
from bunch import *
def static_vars(**kwargs):
def decorate(func):
statics = Bunch(**kwargs)
setattr(func, "statics", statics)
return func
return decorate
#static_vars(name = "Martin")
def my_function():
statics = my_function.statics
print("Hello, {0}".format(statics.name))
Remark
My method uses a class named Bunch, which is a dictionary that supports
attribute-style access, a la JavaScript (see the original article about it, around 2000)
It can be installed via pip install bunch
It can also be hand-written like so:
class Bunch(dict):
def __init__(self, **kw):
dict.__init__(self,kw)
self.__dict__ = self
I personally prefer the following to decorators. To each their own.
def staticize(name, factory):
"""Makes a pseudo-static variable in calling function.
If name `name` exists in calling function, return it.
Otherwise, saves return value of `factory()` in
name `name` of calling function and return it.
:param name: name to use to store static object
in calling function
:type name: String
:param factory: used to initialize name `name`
in calling function
:type factory: function
:rtype: `type(factory())`
>>> def steveholt(z):
... a = staticize('a', list)
... a.append(z)
>>> steveholt.a
Traceback (most recent call last):
...
AttributeError: 'function' object has no attribute 'a'
>>> steveholt(1)
>>> steveholt.a
[1]
>>> steveholt('a')
>>> steveholt.a
[1, 'a']
>>> steveholt.a = []
>>> steveholt.a
[]
>>> steveholt('zzz')
>>> steveholt.a
['zzz']
"""
from inspect import stack
# get scope enclosing calling function
calling_fn_scope = stack()[2][0]
# get calling function
calling_fn_name = stack()[1][3]
calling_fn = calling_fn_scope.f_locals[calling_fn_name]
if not hasattr(calling_fn, name):
setattr(calling_fn, name, factory())
return getattr(calling_fn, name)
Building on Daniel's answer (additions):
class Foo(object):
counter = 0
def __call__(self, inc_value=0):
Foo.counter += inc_value
return Foo.counter
foo = Foo()
def use_foo(x,y):
if(x==5):
foo(2)
elif(y==7):
foo(3)
if(foo() == 10):
print("yello")
use_foo(5,1)
use_foo(5,1)
use_foo(1,7)
use_foo(1,7)
use_foo(1,1)
The reason why I wanted to add this part is , static variables are used not only for incrementing by some value, but also check if the static var is equal to some value, as a real life example.
The static variable is still protected and used only within the scope of the function use_foo()
In this example, call to foo() functions exactly as(with respect to the corresponding c++ equivalent) :
stat_c +=9; // in c++
foo(9) #python equiv
if(stat_c==10){ //do something} // c++
if(foo() == 10): # python equiv
#add code here # python equiv
Output :
yello
yello
if class Foo is defined restrictively as a singleton class, that would be ideal. This would make it more pythonic.
I write a simple function to use static variables:
def Static():
### get the func object by which Static() is called.
from inspect import currentframe, getframeinfo
caller = currentframe().f_back
func_name = getframeinfo(caller)[2]
# print(func_name)
caller = caller.f_back
func = caller.f_locals.get(
func_name, caller.f_globals.get(
func_name
)
)
class StaticVars:
def has(self, varName):
return hasattr(self, varName)
def declare(self, varName, value):
if not self.has(varName):
setattr(self, varName, value)
if hasattr(func, "staticVars"):
return func.staticVars
else:
# add an attribute to func
func.staticVars = StaticVars()
return func.staticVars
How to use:
def myfunc(arg):
if Static().has('test1'):
Static().test += 1
else:
Static().test = 1
print(Static().test)
# declare() only takes effect in the first time for each static variable.
Static().declare('test2', 1)
print(Static().test2)
Static().test2 += 1
Miguel Angelo's self-redefinition solution is even possible without any decorator:
def fun(increment=1):
global fun
counter = 0
def fun(increment=1):
nonlocal counter
counter += increment
print(counter)
fun(increment)
fun() #=> 1
fun() #=> 2
fun(10) #=> 12
The second line has to be adapted to get a limited scope:
def outerfun():
def innerfun(increment=1):
nonlocal innerfun
counter = 0
def innerfun(increment=1):
nonlocal counter
counter += increment
print(counter)
innerfun(increment)
innerfun() #=> 1
innerfun() #=> 2
innerfun(10) #=> 12
outerfun()
The plus of the decorator is that you don't have to pay extra attention to the scope of your construction.
Sure this is an old question but I think I might provide some update.
It seems that the performance argument is obsolete.
The same test suite appears to give similar results for siInt_try and isInt_re2.
Of course results vary, but this is one session on my computer with python 3.4.4 on kernel 4.3.01 with Xeon W3550.
I have run it several times and the results seem to be similar.
I moved the global regex into function static, but the performance difference is negligible.
isInt_try: 0.3690
isInt_str: 0.3981
isInt_re: 0.5870
isInt_re2: 0.3632
With performance issue out of the way, it seems that try/catch would produce the most future- and cornercase- proof code so maybe just wrap it in function