This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 2 years ago.
As far as I know, python passes parameter as reference. I have following code
def func(arr):
print(arr)
if arr == [] :
return
for i in range(len(arr)):
arr[i] *= 2
func(arr[1:])
r = [1,1,1,1]
func(r)
print(r)
I would expect the output to be [2,4,8,16].
Why it outputs [2,2,2,2] as if the reference only works for one level of recursion?
maybe 'arr[1:]' always creates a new object? If that is the case, is there any way to make arr[1:] work?
You've asked a couple different questions, let's go through them one by one.
As far as I know, python passes parameter as reference.
The correct term for how python passes it's arguments is "pass py assignment". It means that parameters inside the function behave similar to how they would if they had been directly assigned with an = sign. (So, mutations will reflect across all references to the object. So far so good)
Sidenote (skip if this is confusing): For all intents and purposes, the distinction of "pass by assignment" is important because it abstracts away pass by value vs pass by reference, concepts that are not exposed in python directly. If you wish to know how the underlying mechanism works, it's actually a pass by value, but every value itself is a reference to an object (equivalent to a first level pointer in C speak). We can see why it's easier and important initially not to worry about this particular abstraction, and use "pass by assignment" as the more intuitive explanation.
Next,
maybe 'arr[1:]' always creates a new object?
Correct, slicing always creates a shallow copy of the list. docs
If that is the case, is there any way to make arr[1:] work?
Not directly, but we can use indexes instead to build a solution that works and gives us the output you desire. Just keep track of a starting index while doing the recursion, and increment it as you continue recursing.
def func(arr, start=0):
print(arr)
if arr[start:] == [] :
return
for i in range(start, len(arr)):
arr[i] *= 2
func(arr, start + 1)
r = [1,1,1,1]
func(r)
print(r)
Output:
[1, 1, 1, 1]
[2, 2, 2, 2]
[2, 4, 4, 4]
[2, 4, 8, 8]
[2, 4, 8, 16]
[2, 4, 8, 16]
You do slice arr[1:] and as the result it creates new list. That's why you got such result, in future I would not recommend you do such thing it's implicit and hard to debug. Try to return new value instead of changing it by reference when you work with functions
For example like this:
def multiplier(arr):
return [
value * (2 ** idx)
for idx, value in enumerate(arr, start=1)
]
result = multiplier([1, 1, 1, 1])
print(result) # [2, 4, 8, 16]
Slicing a list will create a new object (as you speculated), which would explain why the original list isn't updated after the first call.
Yes, arr[1:] create a new object. You can pass the index to indicate the starting index.
def func(arr, start_idx):
print(arr)
if arr == [] :
return
for i in range(start_idx, len(arr)):
arr[i] *= 2
func(arr, start_idx + 1)
r = [1,1,1,1]
func(r, 0)
print(r)
You can use this. The variable s represents start and e represents end, of the array.
def func(arr,s,e):
print(arr) #comment this line if u dont want the output steps
if s>=e:
return
for i in range(s,e):
arr[i] *= 2
func(arr,s+1,e)
r = [1,1,1,1]
func(r,0,len(r))
print(r)
Related
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
This question already has answers here:
Why does += behave unexpectedly on lists?
(9 answers)
Closed last month.
I had learned that n = n + v and n += v are the same. Until this;
def assign_value(n, v):
n += v
print(n)
l1 = [1, 2, 3]
l2 = [4, 5, 6]
assign_value(l1, l2)
print(l1)
The output will be:
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6]
Now when I use the expanded version:
def assign_value(n, v):
n = n + v
print(n)
l1 = [1, 2, 3]
l2 = [4, 5, 6]
assign_value(l1, l2)
print(l1)
The output will be:
[1, 2, 3, 4, 5, 6]
[1, 2, 3]
Using the += has a different result with the fully expanded operation. What is causing this?
Thats because in the first implementation you are editing the list n itself (and therefore the changes still apply when leaving the function), while on the other implementation you are creating a new temporary list with the same name, so when you leave the function the new list disappears and the variable n is linked to the original list.
the += operator works similarly to x=x+y for immutable objects (since they always create new objects), but for mutable objects such as lists they work differently. x=x+y creats a new object x while x+=y edits the current object.
It may seem counter-intuitive, but they are not always the same. In fact,
a = a + b means a = a.__add__(b), creating a new object
a += b means a = a.__iadd__(b), mutating the object
__iadd__, if absent, defaults to the __add__, but it also can (and it does, in the case of lists) mutate the original object in-place.
This works on how python treats objects and passes variables into functions.
Basically - in first example (with += )
You are passing n and v into function by "pass-by-assignment"
So n gets modified and it will be also modified out of function scope.
In second example - n is reassigned inside of the function to a new list. Which is not seen outside of the function.
In your 1st code. You changes list n itself see the below image..!
In your 2nd code. you just created a temporary list which is cleared when function call ends.. see the below images..!
In the next step when function ends the temporary list clear!!
Is it possible to pass a (moving) pointer to a list start into a function in Python?
I have a recursive function working on a section of a list. The list itself is not changed, only the pointer to a 'starting-point' into it. The problem I ran into was that long lists killed the code with memory overrun.
Here is the code:
def trim(l):
print("list len= ", len(l))
if len(l)!= 1:
trim(l[1:])
else:
print("done")
The above example is contrived, my actual code does different stuff than just trimming the list, but it also has a moving start-pointer. A list of 1 million integers blew out of memory on a 10G RAM machine.
Any ideas are welcome.
Couldn't you just pass the index instead of passing the whole new list?
So you would call trim(l, 0) and then check the index against the length of the list, and then call trim(l, 1) if needed.
def trim(l, idx):
print("list len = ", (len(l) - idx))
if idx < (len(x) - 1):
trim(l, idx + 1)
else:
print("done")
If you're writing a non-tail-call recursive function to iterate over a list, your problem is more likely to be a stack overflow, or out-of-memory error related to the stack size.
I recommend re-writing this with an integer pointer and a for-loop, as it seems that Python doesn't have tail-call optimisation.
Here's a guess at what you might be wanting to do:
x = [0,0,0,0,0,1,2,3,4]
def trim_leading_zero(l):
the_len = len(l)
start_i = 0
for i in xrange(the_len):
if l[i] != 0:
return l[i:]
>>> trim_leading_zero(x)
[1, 2, 3, 4]
It's not clear from your code what it's meant to actually do. If you're trying to actually return a sequence, then you may want to look at Generators, which don't require holding an entire sequence in memory.
When dealing with large data, use generators instead of regular iterators.
def trim(l):
print("list len= ", len(l))
pointer = 0
if len(l)!= 1:
yield l[pointer:]
pointer += 1
else:
print("done")
x = [1, 2, 3, 4, 5, 6, 7, 8, 9]
for i in trim(x):
print i
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
Generators will yield one item at a time and let you do whatever you need with it, avoiding create the whole list first before processing. If you want to get a list out of it, you can simply do list(trim(x)).
There are great explanations of yield and generators here - What does the yield keyword do
I'm having a problem with Python 2.7 that is driving me insane.
I'm passing an array to some functions and altough that variable is suposed to be local, in the end the value of the variable inside main is changed.
I'm a bit new to Python, but this goes against any common sense I got.
Any ideas of what I'm doing wrong?
def mutate(chromo):
# chooses random genes and mutates them randomly to 0 or 1
for gene in chromo:
for codon in gene:
for base in range(2):
codon[randint(0, len(codon)-1)] = randint(0, 1)
return chromo
def mate(chromo1, chromo2):
return mutate([choice(pair) for pair in zip(chromo1, chromo2)])
if __name__ == '__main__':
# top 3 is a multidimensional array with 3 levels (in here I put just 2 for simplicity)
top3 = [[1, 0], [0, 0], [1, 1]]
offspring = []
for item in top3:
offspring.append(mate(top3[0], item))
# after this, top3 is diferent from before the for cycle
UPDATE
Because Python passes by reference, I must make a real copy fo the arrays before using them, so the mate functions must be changed to:
import copy
def mate(chromo1, chromo2):
return mutate([choice(pair) for pair in zip(copy.deepcopy(chromo1), copy.deepcopy(chromo2))])
The problem you are having is stemming from the fact that arrays and dictionaries in python are passed by reference. This means that instead of a fresh copy being created by the def and used locally you are getting a pointer to your array in memory...
x = [1,2,3,4]
def mystery(someArray):
someArray.append(4)
print someArray
mystery(x)
[1, 2, 3, 4, 4]
print x
[1, 2, 3, 4, 4]
You manipulate chromo, which you pass by reference. Therefore the changes are destructive... the return is therefore kind of moot as well (codon is in gene and gene is in chromo). You'll need to make a (deep) copy of your chromos, I think.
try changing
offspring.append(mate(top3[0], item)) to
offspring.append(mate(top3[0][:], item[:]))
or use the list() function
I want to change a in the for-loop to [4,5,6].
This code just print: 1, 2, 3
a = [1,2,3]
for i in a:
global a
a = [4,5,6]
print i
I want the ouput 1, 4, 5, 6.
You'll need to clarify the question because there is no explanation of how you should derive the desired output 1, 4, 5, 6 when your input is [1, 2, 3]. The following produces the desired output, but it's completely ad-hoc and makes no sense:
i = 0
a = [1, 2, 3]
while i < len(a):
print(a[i])
if a[i] == 1:
a = [4, 5, 6]
i = 0 # edit - good catch larsmans
else:
i += 1
The main point is that you can't modify the parameters of a for loop while the loop is executing. From the python documentation:
It is not safe to modify the sequence being iterated over in the loop
(this can only happen for mutable sequence types, such as lists). If
you need to modify the list you are iterating over (for example, to
duplicate selected items) you must iterate over a copy.
Edit: if based on the comments you are trying to walk URLs, you need more complicated logic to do a depth-first or breadth-first walk than just replacing one list (the top-level links) with another list (links in the first page). In your example you completely lose track of pages 2 and 3 after diving into page 1.
The issue is that the assignment
a = [4,5,6]
just changes the variable a, not the underlying object. There are various ways you could deal with this; one would be to use a while loop like
a = [1,2,3]
i = 0
while i<len(a):
print a[i]
a = [4,5,6]
i += 1
prints
1
5
6
If you print id(a) at useful points in your code you'll realise why this doesn't work.
Even something like this does not work:
a = [1,2,3]
def change_a(new_val):
a = new_val
for i in a:
change_a([4,5,6])
print i
I don't think it is possible to do what you want. Break out of the current loop and start a new one with your new value of a.