I am trying to write a recursive function that needs to store and modify an object (say a set) as it recurses. Should I use a global name inside the function? Another option is to modify or inherit the class of the parameter of the function so that it can keep this persistent object but I don't find it elegant. I could also use a stack if I would forgo the recursion altogether...
Is there a pythonic way of doing this? Could a generator do the trick?
Just pass through your persistent object through the recursive method.
def recursivemethod(obj_to_act_on, persistent_obj=None):
if persistent_obj == None:
persistent_obj = set()
# Act on your object
return recursivemethod(newobj, persistent_obj)
Objects are passed by reference. If you're only modifying an object, you can do that from within a recursive function and the change will be globally visible.
If you need to assign a variable inside a recursive function and see it after the function returns, then you can't just assign a local variable with =. What you can do is update a field of another object.
class Accumulator: pass
def foo():
# Create accumulator
acc = Accumulator()
acc.value = 0
# Define and call a recursive function that modifies accumulator
def bar(n):
if (n > 0): bar(n-1)
acc.value = acc.value + 1
bar(5)
# Get accumulator
return acc.value
Pass the set into the recursive method as an argument, then modify it there before passing it to the next step. Complex objects are passed by reference.
If it's a container (not an immutable data type), you can pass the object through:
import random
def foo(bar=None, i=10):
if bar is None:
bar = set()
if i == 0:
return bar
bar |= set(random.randint(1, 1000) for i in xrange(10))
return foo(bar, i - 1)
random_numbers_set = foo()
(Don't ask me what that's meant to do... I was just typing random things :P)
If the object you pass is mutable then changes to it in deeper recursions will be seen in earlier recursions.
Use a variable global to the function.
Pass the object around as an accumulator:
def recurse(foo, acc=None):
acc = {}
recurse(acc)
Related
I recently studied how decorators work in python, and found an example which integrates decorators with nested functions.
The code is here :
def integer_check(method):
def inner(ref):
if not isinstance(ref._val1, int) or not isinstance(ref._val2, int):
raise TypeError('val1 and val2 must be integers')
else:
return method(ref)
return inner
class NumericalOps(object):
def __init__(self, val1, val2):
self._val1 = val1
self._val2 = val2
#integer_check
def multiply_together(self):
return self._val1 * self._val2
def power(self, exponent):
return self.multiply_together() ** exponent
y = NumericalOps(1, 2)
print(y.multiply_together())
print(y.power(3))
My question is how the inner function argument("ref") accesses the instance attributes (ref._val1 and ref._val2)?
It seems like ref equals the instance but i have no idea how it happenes.
Let's first recall how a decorator works:
Decorating the method multiply_together with the decorator #integer_check is equivalent to adding the line: multiply_together = integer_check(multiply_together), and by the definition of multiply_together, this is equivalent to multiply_together = inner.
Now, when you call the method multiply_together, since this is an instance method, Python implicitly adds the class instance used to invoke the method as its first (an only, in this case) argument. But multiply_togethet is, actually,inner, so, in fact, inner is invoked with the class instance as an argument. This instance is mapped to the parameter ref, and through this parameter the function gets access to the required instance attributes.
well one explanation I found some time ago about the self argument was that this:
y.multiply_together()
is roughly the same as
NumericalOps.multiply_together(y)
So now that you use that decorator it returns the function inner which requires the ref argument so I see that roughly happen like this (on a lower level):
NumericalOps.inner(y)
Because inner "substitutes" multiply_together while also adding the extra functionality
inner replaces the original function as the value of the class attribute.
#integer_check
def multiply_together(self):
return self._val1 * self._val2
# def multiply_together(self):
# ...
#
# multiply_together = integer_check(multiply_together)
first defines a function and binds it to the name multiply_together. That function is then passed as the argument to integer_check, and then the return value of integer_check is bound to the name multiply_together. The original function is now only refernced by the name ref that is local to inner/multiply_together.
The definition of inner implies that integer_check can only be applied to functions whose first argument will have attributes named _val1 and _val2.
How can I pass an integer by reference in Python?
I want to modify the value of a variable that I am passing to the function. I have read that everything in Python is pass by value, but there has to be an easy trick. For example, in Java you could pass the reference types of Integer, Long, etc.
How can I pass an integer into a function by reference?
What are the best practices?
It doesn't quite work that way in Python. Python passes references to objects. Inside your function you have an object -- You're free to mutate that object (if possible). However, integers are immutable. One workaround is to pass the integer in a container which can be mutated:
def change(x):
x[0] = 3
x = [1]
change(x)
print x
This is ugly/clumsy at best, but you're not going to do any better in Python. The reason is because in Python, assignment (=) takes whatever object is the result of the right hand side and binds it to whatever is on the left hand side *(or passes it to the appropriate function).
Understanding this, we can see why there is no way to change the value of an immutable object inside a function -- you can't change any of its attributes because it's immutable, and you can't just assign the "variable" a new value because then you're actually creating a new object (which is distinct from the old one) and giving it the name that the old object had in the local namespace.
Usually the workaround is to simply return the object that you want:
def multiply_by_2(x):
return 2*x
x = 1
x = multiply_by_2(x)
*In the first example case above, 3 actually gets passed to x.__setitem__.
Most cases where you would need to pass by reference are where you need to return more than one value back to the caller. A "best practice" is to use multiple return values, which is much easier to do in Python than in languages like Java.
Here's a simple example:
def RectToPolar(x, y):
r = (x ** 2 + y ** 2) ** 0.5
theta = math.atan2(y, x)
return r, theta # return 2 things at once
r, theta = RectToPolar(3, 4) # assign 2 things at once
Not exactly passing a value directly, but using it as if it was passed.
x = 7
def my_method():
nonlocal x
x += 1
my_method()
print(x) # 8
Caveats:
nonlocal was introduced in python 3
If the enclosing scope is the global one, use global instead of nonlocal.
Maybe it's not pythonic way, but you can do this
import ctypes
def incr(a):
a += 1
x = ctypes.c_int(1) # create c-var
incr(ctypes.ctypes.byref(x)) # passing by ref
Really, the best practice is to step back and ask whether you really need to do this. Why do you want to modify the value of a variable that you're passing in to the function?
If you need to do it for a quick hack, the quickest way is to pass a list holding the integer, and stick a [0] around every use of it, as mgilson's answer demonstrates.
If you need to do it for something more significant, write a class that has an int as an attribute, so you can just set it. Of course this forces you to come up with a good name for the class, and for the attribute—if you can't think of anything, go back and read the sentence again a few times, and then use the list.
More generally, if you're trying to port some Java idiom directly to Python, you're doing it wrong. Even when there is something directly corresponding (as with static/#staticmethod), you still don't want to use it in most Python programs just because you'd use it in Java.
Maybe slightly more self-documenting than the list-of-length-1 trick is the old empty type trick:
def inc_i(v):
v.i += 1
x = type('', (), {})()
x.i = 7
inc_i(x)
print(x.i)
A numpy single-element array is mutable and yet for most purposes, it can be evaluated as if it was a numerical python variable. Therefore, it's a more convenient by-reference number container than a single-element list.
import numpy as np
def triple_var_by_ref(x):
x[0]=x[0]*3
a=np.array([2])
triple_var_by_ref(a)
print(a+1)
output:
7
The correct answer, is to use a class and put the value inside the class, this lets you pass by reference exactly as you desire.
class Thing:
def __init__(self,a):
self.a = a
def dosomething(ref)
ref.a += 1
t = Thing(3)
dosomething(t)
print("T is now",t.a)
In Python, every value is a reference (a pointer to an object), just like non-primitives in Java. Also, like Java, Python only has pass by value. So, semantically, they are pretty much the same.
Since you mention Java in your question, I would like to see how you achieve what you want in Java. If you can show it in Java, I can show you how to do it exactly equivalently in Python.
class PassByReference:
def Change(self, var):
self.a = var
print(self.a)
s=PassByReference()
s.Change(5)
class Obj:
def __init__(self,a):
self.value = a
def sum(self, a):
self.value += a
a = Obj(1)
b = a
a.sum(1)
print(a.value, b.value)// 2 2
In Python, everything is passed by value, but if you want to modify some state, you can change the value of an integer inside a list or object that's passed to a method.
integers are immutable in python and once they are created we cannot change their value by using assignment operator to a variable we are making it to point to some other address not the previous address.
In python a function can return multiple values we can make use of it:
def swap(a,b):
return b,a
a,b=22,55
a,b=swap(a,b)
print(a,b)
To change the reference a variable is pointing to we can wrap immutable data types(int, long, float, complex, str, bytes, truple, frozenset) inside of mutable data types (bytearray, list, set, dict).
#var is an instance of dictionary type
def change(var,key,new_value):
var[key]=new_value
var =dict()
var['a']=33
change(var,'a',2625)
print(var['a'])
I have a class. This class has a list of functions that are to be evaluated by a different program.
class SomeClass(object):
def __init__(self, context):
self.functions_to_evaluate = []
There is a function that adds functions to an instance of SomeClass, via something like:
new_function = check_number(5)
SomeClassInstance.functions_to_evaluate.append(new_function)
Where check_number is a function that will check if number is greater than 10, let's say.
If I take SomeClassInstance.functions_to_evaluate and print it, I get a bunch of python objects, like so:
<some_library.check_number object at 0x07B35B90>
I am wondering if it is possible for me to extract the input given to check_number, so something like:
SomeClassInstance.functions_to_evaluate[0].python_feature() that will return "5" or whatever the input to check_number was to me.
You can use the standard library functools.partial, which creates a new partially applied function *.
>>> from functools import partial
>>> def check_number(input):
... return input > 10
>>> fn = partial(check_number, 5)
>>> fn.args # this attribute gives you back the bound arguments, as a tuple.
(5,)
>>> fn() # calls the function with the bound arguments.
False
*: actually the partial object is not a function instance, but it is a callable, and from a duck-type perspective it's a function.
If new_function = check_number(5) is a closure, then you can extract this value using __closure__[0].cell_contents:
Example:
def foo(x):
def inn(y):
return x
return inn
s = foo(5)
print(s.__closure__[0].cell_contents)
Output:
5
I understand your confusion, but:
new_function = check_number(5)
Is calling the function, and the new_function variable gets assigned the return value of the function.
If you have this check_number function:
def check_number(input):
return input > 10
Then it will return False, and new_function will be False. Never <some_library.check_number object at 0x07B35B90>.
If you're getting <some_library.check_number object at 0x07B35B90> then your check_number() function is returning something else.
There are probably several ways to skin this cat. But I'd observe first and foremost that you're not adding python function objects to the functions_to_evaluate list, you're adding the evaluations of functions.
You could simply add a tuple of function, args to the list:
SomeClassInstace.functions_to_evaluate.append((check_number, 5))
And then you can:
for f, args in SomeClassInstance.functions_to_evaluate:
print(args)
I'm using a recursive function to sort a list in Python, and I want to keep track of the number of sorts/merges as the function continues. However, when I declare/initialize the variable inside the function, it becomes a local variable inside each successive call of the function. If I declare the variable outside the function, the function thinks it doesn't exist (i.e. has no access to it). How can I share this value across different calls of the function?
I tried to use the "global" variable tag inside and outside the function like this:
global invcount ## I tried here, with and without the global tag
def inv_sort (listIn):
global invcount ## and here, with and without the global tag
if (invcount == undefined): ## can't figure this part out
invcount = 0
#do stuff
But I cannot figure out how to check for the undefined status of the global variable and give it a value on the first recursion call (because on all successive recursions it should have a value and be defined).
My first thought was to return the variable out of each call of the function, but I can't figure out how to pass two objects out of the function, and I already have to pass the list out for the recursion sort to work. My second attempt to resolve this issue involved me adding the variable invcount to the list I'm passing as the last element with an identifier, like "i27". Then I could just check for the presence of the identifier (the letter i in this example) in the last element and if present pop() it off at the beginning of the function call and re-add it during the recursion. In practice this is becoming really convoluted and while it may work eventually, I'm wondering if there is a more practical or easier solution.
Is there a way to share a variable without directly passing/returning it?
There's couple of things you can do. Taking your example you should modify it like this:
invcount = 0
def inv_sort (listIn):
global invcount
invcount += 1
# do stuff
But this approach means that you should zero invcount before each call to inv_sort.
So actually its better to return invcount as a part of result. For example using tuples like this:
def inv_sort(listIn):
#somewhere in your code recursive call
recursive_result, recursive_invcount = inv_sort(argument)
# this_call_invcount includes recursive_invcount
return this_call_result, this_call_invcount
There's no such thing as an "undefined" variable in Python, and you don't need one.
Outside the function, set the variable to 0. Inside the loop, use the global keyword, then increment.
invcount = 0
def inv_sort (listIn):
global invcount
... do stuff ...
invcount += 1
An alternative might be using a default argument, e.g.:
def inv_sort(listIn, invcount=0):
...
invcount += 1
...
listIn, invcount = inv_sort(listIn, invcount)
...
return listIn, invcount
The downside of this is that your calls get slightly less neat:
l, _ = inv_sort(l) # i.e. ignore the second returned parameter
But this does mean that invcount automatically gets reset each time the function is called with a single argument (and also provides the opportunity to inject a value of invcount if necessary for testing: assert result, 6 == inv_sort(test, 5)).
Assuming that you don't need to know the count inside the function, I would approach this using a decorator function:
import functools
def count_calls(f):
#functools.wraps(f)
def func(*args):
func.count += 1
return f(*args)
func.count = 0
return func
You can now decorate your recursive function:
#count_calls
def inv_sort(...):
...
And check or reset the count before or after calling it:
inv_sort.count = 0
l = inv_sort(l)
print(inv_sort.count)
I have a class (list of dicts) and I want it to sort itself:
class Table(list):
…
def sort (self, in_col_name):
self = Table(sorted(self, key=lambda x: x[in_col_name]))
but it doesn't work at all. Why? How to avoid it? Except for sorting it externally, like:
new_table = Table(sorted(old_table, key=lambda x: x['col_name'])
Isn't it possible to manipulate the object itself? It's more meaningful to have:
class Table(list):
pass
than:
class Table(object):
l = []
…
def sort (self, in_col_name):
self.l = sorted(self.l, key=lambda x: x[in_col_name])
which, I think, works.
And in general, isn't there any way in Python which an object is able to change itself (not only an instance variable)?
You can't re-assign to self from within a method and expect it to change external references to the object.
self is just an argument that is passed to your function. It's a name that points to the instance the method was called on. "Assigning to self" is equivalent to:
def fn(a):
a = 2
a = 1
fn(a)
# a is still equal to 1
Assigning to self changes what the self name points to (from one Table instance to a new Table instance here). But that's it. It just changes the name (in the scope of your method), and does affect not the underlying object, nor other names (references) that point to it.
Just sort in place using list.sort:
def sort(self, in_col_name):
super(Table, self).sort(key=lambda x: x[in_col_name])
Python is pass by value, always. This means that assigning to a parameter will never have an effect on the outside of the function. self is just the name you chose for one of the parameters.
I was intrigued by this question because I had never thought about this. I looked for the list.sort code, to see how it's done there, but apparently it's in C. I think I see where you're getting at; what if there is no super method to invoke? Then you can do something like this:
class Table(list):
def pop_n(self, n):
for _ in range(n):
self.pop()
>>> a = Table(range(10))
>>> a.pop_n(3)
>>> print a
[0, 1, 2, 3, 4, 5, 6]
You can call self's methods, do index assignments to self and whatever else is implemented in its class (or that you implement yourself).