I have some function which uses outside variables. A (substantially) simplified example:
a = 2
b = 3
def f(x):
return x * a + b
While I need a and b in f, I don't need them anywhere else. In particular, one can write a = 5, and that will change the behavior of f. How should I make a and b invisible to the outside?
Other languages allow me to write roughly the following code:
let f =
a = 2
b = 3
lambda x: x * a + b
What I want:
f must work as intended and have the same signature
a and b must be computed only once
a and b must not exist in the scope outside of f
Assignments a = ... and b = ... don't affect f
The cleanest way to do this. E.g. the following solution formally works, but it introduces g and then deletes it, which I don't like (e.g. there is a risk of overriding an existing g and I believe that it's simply ugly):
def g():
a = 2
b = 3
return lambda x: x * a + b
f = g()
del g
One method is to simply use a class. This allows you to place a and b in the scope of the class while f can still access them.
custom class
class F:
def __init__(self):
self.a = 2
self.b = 3
def __call__(self, x):
return x * self.a + self.b
f = F()
f(1)
# returns:
5
If you don't like having to call the class constructor, you can override __new__ to essentially create a callable with internal stored variables. This is an antipattern though and not very pythonic.
custom callable
class f:
a = 2
b = 3
def __new__(cls, x):
return x * cls.a + cls.b
f(1)
# returns:
5
This approach is based on the answers provided in this thread, though scoped to the specific problem above. You can use a decorator to update the global variables available to the function while also storin a and b within a closure.
decorator with closure
from functools import wraps
def dec_ab(fn):
a = 2
b = 3
#wraps(fn)
def wrapper(*args, **kwargs):
# get global scope
global_scope = f.__globals__
# copy current values of variables
var_list = ['a', 'b']
current_vars = {}
for var in var_list:
if var in global_scope:
current_vars[var] = global_scope.get(var)
# update global scope
global_scope.update({'a': a, 'b': b})
try:
out = fn(*args, **kwargs)
finally:
# undo the changes to the global scope
for var in var_list:
global_scope.pop(var)
global_scope.update(current_vars)
return out
return wrapper
#dec_ab
def f(x):
"""hello world"""
return x * a + b
This preserves the functions signature and keeps a and b from being altered
f(1)
# returns:
5
a
# raises:
NameError: name 'a' is not defined
You can use default arguments to accomplish this. Default arguments are only computed once, when the closure is created (that is why if you have mutable objects as default arguments, the state is retained between invocations).
def f(x, a=2, b=3):
return x * a + b
Related
Given:
A set of computations to calculate interdependent variables
A set of desired outputs chosen from the variables
I would like to:
Compute only the variables I need (lazy computation)
Compute each variable at most once (caching)
Get rid of the variables that are no longer needed in either the output or the computation of the remaining outputs (garbage collection)
BONUS: Order the computations so that the biggest variables to be removed can be removed first, in order to reduce memory usage to the max
For example:
a = 1
b = 2
c = b ** 10
d = a + b
In this example:
If a is the only required output, then b, c, and d never need to be computed
If c and d are required, then b should only be calculated once
If c and d are required, then a can be forgotten as soon as d has been computed
Since a can be eventually garbage collected, we try to arrange for that ASAP and therefore compute d first (or maybe we should start with c if the ** operation will temporarily take up more memory? Not completely sure about this...)
When writing the computations as above, as a plain sequence of statements, properties 1 and 4 are not observed.
Meanwhile, it is possible to obtain properties 1 and 3 (missing 2 and 4) using #property:
class DataGetter:
#property
def a(self): return 1
#property
def b(self): return 2
#property
def c(self): return self.b ** 10
#property
def d(self): return self.a + self.b
Likewise, it is possible to obtain properties 1 and 2 (missing 3 and 4) using #cached_property:
class DataGetter:
#cached_property
def a(self): return 1
#cached_property
def b(self): return 2
#cached_property
def c(self): return self.b ** 10
#cached_property
def d(self): return self.a + self.b
Is there a way to ensure all the first 3 properties are met? (And possibly also the 4th?)
If we wrap each variable in an instance, then laziness can be achieved by deferring the computation with lambda:, and caching can be achieved in the normal way. Since the lambdas are closures, each will hold "cells" which retain only the local variables from the outer function that the lambda actually uses (see this Q&A), allowing garbage collection to work as desired.
class LazyValue:
def __init__(self, f):
self._f = f
#cached_property
def value(self):
v = self._f()
self._f = None
return v
Setting self._f = None is required for garbage collection to work as desired; if a value has already been computed then we don't need or want to retain references to any other LazyValue instances which self._f closes over.
Usage:
def compute():
a = LazyValue(lambda: 1)
b = LazyValue(lambda: 2)
c = LazyValue(lambda: b.value ** 10)
d = LazyValue(lambda: a.value + b.value)
# return whichever results are required
return d
print(compute().value) # 3
I would like to define a Python function in which a parameter is external if it is declared when the function is called and it is internal if it is not mentioned into the function calling.
An easy example will clarify my issue:
def func(a, b):
c = 4
try:
b
except:
b= c
return a + b
Now I want something in which I can write
func(2, 1)
and provide me the result (2 + 1 = 3), and I can even write
func(2)
and the function will generate a solution that is c + a = 6.
How can I do that?
You are describing a default argument. Python does that for you (see the docs):
def func(a, b=4):
return a + b
Alternatively, if more logic is involved, you might want to consider:
def func(a, b=None):
if b is None:
try:
b = fancy_function()
except FancyException:
b = 4
return a + b
The same holds for mutable default values:
def func(a, b=None):
if b is None:
b = []
b.append(a)
return b
Simply write
def func(a, b=4):
return a + b
>>> func(2)
6
>>> func(2, 1)
3
If the default value of b is not known beforehand, you can do the following:
def func(a, b=None):
if b is None:
b = some_function_that_generates_b()
return a + b
Consider the following function, which we'd like to not be constant on integers a, but which always returns (1,2):
def foo(a):
b = 1
c = 2
def bar(b,c):
b = b + a
c = c + a
bar(b,c)
return (b,c)
If I understand correctly, the idiomatic way to implement bar(b,c) would be to give it no parameters at all and declare b and c nonlocal inside its definition. I'm curious, however: how do we make an inner function have nonlocal effects on its parameters?
As stated in this answer for Python 2.x:
Python doesn't allow you to reassign the value of a variable from an
outer scope in an inner scope (unless you're using the keyword
"global", which doesn't apply in this case).
This will return (2,3):
def foo(a):
b = 1
c = 2
def bar(b,c):
b = b + a
c = c + a
return b, c
b, c = bar(b,c)
return (b,c)
print(foo(1))
Make b and c function attributes.
def foo(a):
foo.b = 1
foo.c = 2
def bar():
foo.b = foo.b + a
foo.c = foo.c + a
bar()
return (foo.b,foo.c)
Notice you no longer pass b or c into the function bar.
Function parameters are always locals. You could pass in mutable objects however, and apply your operations to the indirectly referenced value:
def foo(a):
b = [1]
c = [2]
def bar(b, c):
b[0] += a
c[0] += a
bar(b, c)
return (b[0], c[0])
The changes you make to the mutable object are shared with any other references to those objects, including locals in foo().
However, if you want something to be a closure, just make it a closure. There are no use cases for this that cannot be handled by nonlocal values and return.
Lets say I have a function with a variable duration inside it.
Is there any way to set the Duration` value outside of the function in some other nonparent function without adjusting any parameter?
With Python 3.x you can declare it with the nonlocal keyword
def make_slow(action):
slowDuration = None
def slowAction(self, colony):
nonlocal slowDuration
slowDuration = 10 # It is changing the variable from the scope above
If you want to change a value from somewhere else and you mustn't return the value, try going global... Note this may pollute your current namespace.
For a more pythonic approach, you should use something like self.slowDuration. That's what objects are for.
slowDuration is a local variable of the slowAction function. The point of local variables is that they are only accessible inside the function.
You should change the slowAction function so it uses a slowDuration variable that is defined somewhere else, for example as a member variable of the class that make_slow apparently belongs to.
You can also make slowAction be an instance of a class that overrides the __call__ method.
>>> class Counter:
... def __init__(self):
... self.count = 0
... def __call__(self, delta):
... self.count += delta
... print(self.count)
... def set_count(self, c):
... self.count = c
...
>>> c = Counter()
>>> c(1)
1
>>> c(3)
4
>>> c(3)
7
>>> c(3)
10
>>> c.set_count(42)
>>> c(-2)
40
You could also use some trickery to make the shared variable available on the function object itself:
def makeCounter():
counter = None
def counter_func():
counter.count += 1
print(counter.count)
counter = counter_func
counter.count = 0
return counter
and use it like this:
>>> c = makeCounter()
>>> c()
1
>>> c()
2
>>> c()
3
>>> c()
4
>>> c.count = 42
>>> c()
43
>>> c()
44
>>> c()
45
But in general, "clever" code like that should be avoided unless you have a very good reason to use it, because it makes the code base harder to understand.
How can I make a Python function default value be evaluated each time the function is called?
Take this dummy code:
b=0
def a():
global b
return b
def c(d=a()):
return d
What I would expect as output:
>>> c()
0
>>> b=1
>>> a()
1
>>> c()
1
What I actually get:
>>> c()
0
>>> b=1
>>> a()
1
>>> c()
0
One more solution, in closer resemblance to your original answer.
b = 0
def a():
return b
def c(d=a): # When it's a parameter, the call will be evaluated and its return
# value will be used. Instead, just use the function name, because
return d() # within the scope of the function, the call will be evaluated every time.
When a function name is paired with the parentheses and its parameters, like f(x), it is assumed your intention is to call it at that time
d=a() is evaluated at start of program when function c is defined (ie a() gets called while it returns 0 ...)
def c(d=None):
if d == None: d=a()
return d
will cause it to be evaluated at the time you want
The problem here is, as you probably already know, that the d=a() (default argument assignement) is evaluated when the function is defined.
To change that, it is pretty common to use eg. None as default argument and evaluate it in the body of the function:
b=0
def a():
global b
return b
def c(d=None):
if d is None:
d = a()
return d
I'll give a slight variation on the above:
b = 0
def a():
# unless you are writing changes to b, you do not have to make it a global
return b
def c(d=None, get_d=a):
if d is None:
d = get_d()
return d