Python: Local variables mysteriously update Global variables - python

I have a function where I work with a local variable, and then pass back the final variable after the function is complete. I want to keep a record of what this variable was before the function however the global variable is updated along with the local variable. Here is an abbreviated version of my code (its quite long)
def Turn(P,Llocal,T,oflag):
#The function here changes P, Llocal and T then passes those values back
return(P, Llocal, T, oflag)
#Later I call the function
#P and L are defined here, then I copy them to other variables to save
#the initial values
P=Pinitial
L=Linitial
P,L,T,oflag = Turn(P,L,T,oflag)
My problem is that L and Linitial are both updated exactly when Llocal is updated, but I want Linitial to not change. P doesn't change so I'm confused about what is happening here. Help? Thanks!
The whole code for brave people is here: https://docs.google.com/document/d/1e6VJnZgVqlYGgYb6X0cCIF-7-npShM7RXL9nXd_pT-o/edit

The problem is that P and L are names that are bound to objects, not values themselves. When you pass them as parameters to a function, you're actually passing a copy of the binding to P and L. That means that, if P and L are mutable objects, any changes made to them will be visible outside of the function call.
You can use the copy module to save a copy of the value of a name.

Lists are mutable. If you pass a list to a function and that function modifies the list, then you will be able to see the modifications from any other names bound to the same list.
To fix the problem try changing this line:
L = Linitial
to this:
L = Linitial[:]
This slice makes a shallow copy of the list. If you add or remove items from the list stored in L it will not change the list Lintial.
If you want to make a deep copy, use copy.deepcopy.
The same thing does not happen with P because it is an integer. Integers are immutable.

In Python, a variable is just a reference to an object or value in the memory. For example, when you have a list x:
x = [1, 2, 3]
So, when you assign x to another variable, let's call it y, you are just creating a new reference (y) to the object referenced by x (the [1, 2, 3] list).
y = x
When you update x, you are actually updating the object pointed by x, i.e. the list [1, 2, 3]. As y references the same value, it appears to be updated too.
Keep in mind, variables are just references to objects.
If you really want to copy a list, you shoud do:
new_list = old_list[:]
Here's a nice explanation: http://henry.precheur.org/python/copy_list

Related

Python remove method removes element from both lists

When I run the code below:
a=[0,1,2,3]
t=a
t.remove(3)
print(a)
it gives me a result of [0,1,2] even though I didn't use the remove() method for list a. Why does it happen?
Because list(and dicts) are pass by reference in Python. You have to shallow copy the list if you don't want it to happen like this:
t=a[:]
or
t=a.copy()
Lists in python are mutable objects and hence when you assigned t = a,
you created list 't' which is reference to list 'a'. Both lists refer to the same memory location.
Original list
a=[0,1,2,3]
id(a) # gives unique number which points to memory location
>> 60528136
assigning to t
t = a
id(t)
>> 60528136
The result shows same number which signifies that both lists refer to the same memory location. So any modification on list 'a' would be reflected in list 't' and vice-versa.
To avoid this create a copy.
t = a[::]
or
t = a.copy()
id(t)
>>60571720
Check out the Python docs on Programming FAQ here.
Why did changing list ‘y’ also change list ‘x’?
There are two factors that produce this result:
Variables are simply names that refer to objects. Doing y = x doesn’t create a copy of the list – it creates a new variable y that refers to the same object x refers to. This means that there is only one object (the list), and both x and y refer to it.
Lists are mutable, which means that you can change their content.
How do I copy an object in Python?
In general, try copy.copy() or copy.deepcopy() for the general case. Not all objects can be copied, but most can.
Some objects can be copied more easily. Dictionaries have a copy() method:
newdict = olddict.copy()
Sequences can be copied by slicing:
new_l = l[:]

How can I preserve the value of my parameters through a function so it can be used multiple times with its initial value?

I have just started using SAGE which is pretty close to python as I understand it, and I have come accross this problem where I'll have as a parameter of a function a matrix which I wish to use multiple times in the function with its same original function but through the different parts of the function it changes values.
I have seen in a tutorial that declaring a variable in the function as
variable = list(parameter) doesnt affect the parameter or whatever is put in the parentheses. However I can't make it work..
Below is part of my program posing the problem (I can add the rest if necessary): I declare the variable determinant which has as value the result of the function my_Gauss_determinant with the variable auxmmatrix as parameter. Through the function my_Gauss_determinant the value of auxmmatrix changes but for some reason the value of mmatrix as well. How can avoid this and be able to re-use the parameter mmatrix with its original value?
def my_Cramer_solve(mmatrix,bb):
auxmmatrix=list(mmatrix)
determinant=my_Gauss_determinant(auxmmatrix)
if determinant==0:
print
k=len(auxmmatrix)
solution=[]
for l in range(k):
auxmmatrix1=my_replace_column(list(mmatrix),l,bb)
determinant1=my_Gauss_determinant(auxmmatrix1)
solution.append(determinant1/determinant0)
return solution
What you want is a copy of mmatrix. The reason list(other_list) works is because it iterates through every item in other_list to create a new list. But the mutable objects within the list aren't copied
>>> a = [{1,2}]
>>> b = list(a)
>>> b[0].add(7)
>>> a
[set([1,2,7])]
To make a complete copy, you can use copy.deepcopy to make copies of the elements within the list
>>> import copy
>>> a = [{1,2}]
>>> b = copy.deepcopy(a)
>>> b[0].add(7)
>>> a
[set([1,2])]
So if you only want to copy the list, but don't want to copy the elements within the list, you would do this
auxmmatrix = copy.copy(matrix)
determinant = my_Gauss_determinant(copy.copy(matrix))
If you want to copy the elements within the list as well, use copy.deepcopy
If m is a matrix, you can copy it into mm by doing
sage: mm = m[:, :]
or
sage: mm = matrix(m)
To understand the need to copy container structures such as lists and matrices, you could read the tutorial on objects and classes in Python and Sage.
The other Sage tutorials are recommended too!

Issues changing a global variable value in python

Suppose I have this function
>>>a=3
>>>def num(a):
a=5
return a
>>>num(a)
5
>>>a
3
Value of a doesnt change.
Now consider this code :
>>> index = [1]
>>> def change(a):
a.append(2)
return a
>>> change(index)
>>> index
>>> [1,2]
In this code the value of index changes.
Could somebody please explain what is happening in these two codes. As per first code, the value of index shouldnt change(ie should remain index=[1]).
You need to understand how python names work. There is a good explanation here, and you can click here for an animation of your case.
If you actually want to operate on a separate list in your function, you need to make a new one, for instance by using
a = a[:]
before anything else. Note that this will only make a new list, but the elements will still be the same.
The value of index doesn't change. index still points to the same object it did before. However, the state of the object index points to has changed. That's just how mutable state works.
Line 3 in the first block of code is assignment and in the second block is mutation, and that's why you are observing that behavior.
The issue you are encountering is:
a = 3
def num(a):
# `a` is a reference to the argument passed, here 3.
a = 5
# Changed the reference to point at 5, and return the reference.
return a
num(a)
The a in the num function is a diffrent object than the a defined globally.
It works in case of the list because a points at the list passed, and you modify the object being referenced to by the variable, not the reference variable itself.

Why does not the + operator change a list while .append() does?

I'm working through Udacity and Dave Evans introduced an exercise about list properties
list1 = [1,2,3,4]
list2 = [1,2,3,4]
list1=list1+[6]
print(list1)
list2.append(6)
print(list2)
list1 = [1,2,3,4]
list2 = [1,2,3,4]
def proc(mylist):
mylist = mylist + [6]
def proc2(mylist):
mylist.append(6)
# Can you explain the results given by the four print statements below? Remove
# the hashes # and run the code to check.
print (list1)
proc(list1)
print (list1)
print (list2)
proc2(list2)
print (list2)
The output is
[1, 2, 3, 4, 6]
[1, 2, 3, 4, 6]
[1, 2, 3, 4]
[1, 2, 3, 4]
[1, 2, 3, 4]
[1, 2, 3, 4, 6]
So in a function the adding a 6 to the set doesn't show but it does when not in a function?
So in a function the adding a 6 to the set doesn't show but it does when not in a function?
No, that is not what happens.
What happens is that, when you execute mylist = mylist + [6], you are effectively creating an entirely new list and putting it in the local mylist variable. This mylist variable will vanish after the execution of the function and the newly created list will vanish as well.
OTOH when you execute mylist.append(6) you do not create a new list. You get the list already in the mylist variable and add a new element to this same list. The result is that the list (which is pointed by list2 too) will be altered itself. The mylist variable will vanish again, but in tis case you altered the original list.
Let us see if a more visual explanation can help you :)
What happens when you call proc()
When you write list1 = [1, 2, 3, 4, 5] you are creating a new list object (at the right side of the equals sign) and creating a new variable, list1, which will point to this object.
Then, when you call proc(), you create another new variable, mylist, and since you pass list1 as parameter, mylist will point to the same object:
However, the operation mylist + [6] creates a whole new list object whose contents are the contents of the object pointed by mylist plus the content of the following list object - that is, [6]. Since you attribute this new object to mylist, our scenario changes a bit and mylist does not point to the same object pointed by list1 anymore:
What I have not said is that mylist is a local variable: it will disappear after the end of the proc() function. So, when the proc() execution ended, the mylist is gone:
Since no other variable points to the object generated by mylist + [6], it will disappear, too (since the garbage collector* will collect it):
Note that, in the end, the object pointed by list1 is not changed.
What happens when you call proc2()
Everything changes when you call proc2(). At first, it is the same thing: you create a list...
...and pass it as a parameter to a function, which will generate a local variable:
However, instead of using the + concatenation operator, which generates a new list, you apply the append() method to the existing list. The append() method does not create a new object; instead, it _changes the existing one:
After the end of the function, the local variable will disappear, but the original object pointed by it and by list1 will be already altered:
Since it is still pointed by list1, the original list is not destroyed.
EDIT: if you want to take a look at all this stuff happening before your eyes just go to this radically amazing simulator:
* If you do not know what is garbage collector... well, you will discover soon after understanding your own question.
Variables in python can always be thought of as references. When you call a function with an argument, you are passing in a reference to the actual data.
When you use the assignment operator (=), you're assigning that name to refer to an entirely new object. So, mylist = mylist + [6] creates a new list containing the old contents of mylist, as well as 6, and assigns the variable mylist to refer to the new list. list1 is still pointing to the old list, so nothing changes.
On the other hand, when you use .append, that is actually appending an element to the list that the variable refers to - it is not assigning anything new to the variable. So your second function modifies the list that list2 refers to.
Ordinarily, in the first case in function proc you would only be able to change the global list by assignment if you declared
global mylist
first and did not pass mylist as a parameter. However, in this case you'd get an error message that mylist is global and local:
name 'mylist' is local and global. What happens in proc is that a local list is created when the assignment takes place. Since local variables go away when the function ends, the effect of any changes to the local list doesn't propagate to the rest of the program when it is printed out subsequently.
But in the second function proc2 you are modifying the list by appending rather than assigning to it, so the global keyword is not required and changes to the list show elsewhere.
As well as the comprehensive answers already given, it's also worth being aware that if you want the same looking syntax as:
mylist = mylist + [6]
...but still want the list to be updated "in place", you can do:
mylist += [6]
Which, while it looks like it would do the same thing as the first version, is actually the same as:
mylist.extend([6])
(Note that extend takes the contents of an iterable and adds them one by one, whereas append takes whatever it is given and adds that as a single item. See append vs. extend for a full explanation.)
This, in one form or another, is a very common question. I took a whack at explaining Python parameter passing myself a couple days ago. Basically, one of these creates a new list and the other modifies the existing one. In the latter case, all variables that refer to the list "see" the change because it's still the same object.
In the third line, you did this
list1=list1+[6]
So, when you did the below after the above line,
print (list1)
which printed list1 that you created at the start and proc procedure which adds list1 + [6] which is creating a new list inside the function proc. Where as when you are appending [6], you are not creating a new list rather you are appending a list item to already existing list.
But, keep in mind. In line 7, you again created the list
list1 = [1,2,3,4]
Now you wanted to print the list1 by calling it explicitly, which will print out the list1 that you reinitialized again but not the previous one.

Difference between mutation, rebinding, copying value, and assignment operator [duplicate]

This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 6 months ago.
#!/usr/bin/env python3.2
def f1(a, l=[]):
l.append(a)
return(l)
print(f1(1))
print(f1(1))
print(f1(1))
def f2(a, b=1):
b = b + 1
return(a+b)
print(f2(1))
print(f2(1))
print(f2(1))
In f1 the argument l has a default value assignment, and it is only evaluated once, so the three print output 1, 2, and 3. Why f2 doesn't do the similar?
Conclusion:
To make what I learned easier to navigate for future readers of this thread, I summarize as the following:
I found this nice tutorial on the topic.
I made some simple example programs to compare the difference between mutation, rebinding, copying value, and assignment operator.
This is covered in detail in a relatively popular SO question, but I'll try to explain the issue in your particular context.
When your declare your function, the default parameters get evaluated at that moment. It does not refresh every time you call the function.
The reason why your functions behave differently is because you are treating them differently. In f1 you are mutating the object, while in f2 you are creating a new integer object and assigning it into b. You are not modifying b here, you are reassigning it. It is a different object now. In f1, you keep the same object around.
Consider an alternative function:
def f3(a, l= []):
l = l + [a]
return l
This behaves like f2 and doesn't keep appending to the default list. This is because it is creating a new l without ever modifying the object in the default parameter.
Common style in python is to assign the default parameter of None, then assign a new list. This gets around this whole ambiguity.
def f1(a, l = None):
if l is None:
l = []
l.append(a)
return l
Because in f2 the name b is rebound, whereas in f1 the object l is mutated.
This is a slightly tricky case. It makes sense when you have a good understanding of how Python treats names and objects. You should strive to develop this understanding as soon as possible if you're learning Python, because it is central to absolutely everything you do in Python.
Names in Python are things like a, f1, b. They exist only within certain scopes (i.e. you can't use b outside the function that uses it). At runtime a name refers to a value, but can at any time be rebound to a new value with assignment statements like:
a = 5
b = a
a = 7
Values are created at some point in your program, and can be referred to by names, but also by slots in lists or other data structures. In the above the name a is bound to the value 5, and later rebound to the value 7. This has no effect on the value 5, which is always the value 5 no matter how many names are currently bound to it.
The assignment to b on the other hand, makes binds the name b to the value referred to by a at that point in time. Rebinding the name a afterwards has no effect on the value 5, and so has no effect on the name b which is also bound to the value 5.
Assignment always works this way in Python. It never has any effect on values. (Except that some objects contain "names"; rebinding those names obviously effects the object containing the name, but it doesn't affect the values the name referred to before or after the change)
Whenever you see a name on the left side of an assignment statement, you're (re)binding the name. Whenever you see a name in any other context, you're retrieving the (current) value referred to by that name.
With that out of the way, we can see what's going on in your example.
When Python executes a function definition, it evaluates the expressions used for default arguments and remembers them somewhere sneaky off to the side. After this:
def f1(a, l=[]):
l.append(a)
return(l)
l is not anything, because l is only a name within the scope of the function f1, and we're not inside that function. However, the value [] is stored away somewhere.
When Python execution transfers into a call to f1, it binds all the argument names (a and l) to appropriate values - either the values passed in by the caller, or the default values created when the function was defined. So when Python beings executing the call f3(5), the name a will be bound to the value 5 and the name l will be bound to our default list.
When Python executes l.append(a), there's no assignment in sight, so we're referring to the current values of l and a. So if this is to have any effect on l at all, it can only do so by modifying the value that l refers to, and indeed it does. The append method of a list modifies the list by adding an item to the end. So after this our list value, which is still the same value stored to be the default argument of f1, has now had 5 (the current value of a) appended to it, and looks like [5].
Then we return l. But we've modified the default list, so it will affect any future calls. But also, we've returned the default list, so any other modifications to the value we returned will affect any future calls!
Now, consider f2:
def f2(a, b=1):
b = b + 1
return(a+b)
Here, as before, the value 1 is squirreled away somewhere to serve as the default value for b, and when we begin executing f2(5) call the name a will become bound to the argument 5, and the name b will become bound to the default value 1.
But then we execute the assignment statement. b appears on the left side of the assignment statement, so we're rebinding the name b. First Python works out b + 1, which is 6, then binds b to that value. Now b is bound to the value 6. But the default value for the function hasn't been affected: 1 is still 1!
Hopefully that's cleared things up. You really need to be able to think in terms of names which refer to values and can be rebound to point to different values, in order to understand Python.
It's probably also worth pointing out a tricky case. The rule I gave above (about assignment always binding names with no effect on the value, so if anything else affects a name it must do it by altering the value) are true of standard assignment, but not always of the "augmented" assignment operators like +=, -= and *=.
What these do unfortunately depends on what you use them on. In:
x += y
this normally behaves like:
x = x + y
i.e. it calculates a new value with and rebinds x to that value, with no effect on the old value. But if x is a list, then it actually modifies the value that x refers to! So be careful of that case.
In f1 you are storing the value in an array or better yet in Python a list where as in f2 your operating on the values passed. Thats my interpretation on it. I may be wrong
Other answers explain why this is happening, but I think there should be some discussion of what to do if you want to get new objects. Many classes have the method .copy() that allows you create copies. For instance, if we rewrite f1 as
def f1(a, l=[]):
new_l = l.copy()
new_l.append(a)
return(new_l)
then it will continue to return [1]no matter how many times we call it. There is also the library https://docs.python.org/3/library/copy.html for managing copies.
Also, if you're looping through the elements of a container and mutating them one by one, using comprehensions not only is more Pythonic, but can avoid the issue of mutating the original object. For instance, suppose we have the following code:
data = [1,2,3]
scaled_data = data
for i, value in enumerate(scaled_data):
scaled_data[i] = value/sum(data)
This will set scaled_data to [0.16666666666666666, 0.38709677419354843, 0.8441754916792739]; each time you set a value of scaled_data to the scaled version, you also change the value in data. If you instead have
data = [1,2,3]
scaled_data = [x/sum(data) for x in data]
this will set scaled_data to [0.16666666666666666, 0.3333333333333333, 0.5] because you're not mutating the original object but creating a new one.

Categories