I am confused with Python referencing. Consider the following example:
My task : To edit each element in the list
d = { 'm': [1,2,3] }
m = d['m']
m = m[1:] # m changes its reference to the new sliced list, edits m but not d (I wanted to change d)
Similarly:
d = { 'm': [1,2,3] }
m = d['m']
m = m[0] # As per python referencing, m should be pointing to d['m'] and should have edited d
In python everything goes by reference, then when is a new object created?
Do we always need copy and deepcopy from copy module to make object copies?
Please clarify.
In Python a variable is not a box that holds things, it is a name that points to an object. In your code:
d = { 'm': [1,2,3] } --> binds the name d to a dictionary
m = d['m'] --> binds the name m to a list
m = m[1:] --> binds the name m to another list
Your third line is not changing m itself, but what m is pointing to.
To edit the elements in the list what you can do is:
m = d['m']
for i, item in enumerate(m):
result = do_something_with(item)
m[i] = result
Ethan Furman did an excellent job of explaining how Python internals work, I won't repeat it.
Since m really does represent the list inside the dictionary, you can modify it. You just can't reassign it to something new, which is what happens when you use = to equate it to a new slice.
To slice off the first element of the list for example:
>>> m[0:1] = []
>>> d
{'m': [2, 3]}
In python everything goes by reference
In Python, everything is a reference, and the references get passed around by value.
If you want to use those terms. But those terms make things harder to understand.
Much simpler: in Python, a variable is a name for an object. = is used to change what object a name refers to. The left-hand side can refer to part of an existing object, in which case the whole object is changed by replacing that part. This is because the object, in turn, doesn't really contain its parts, but instead contains more names, which can be caused to start referring to different things.
then when is a new object created ?
Objects are created when they are created (by using the class constructor, or in the case of built-in types that have a literal representation, by typing out a literal). I don't understand how this is relevant to the rest of your question.
m = m[1:] # m changes its reference to the new sliced list
Yes, of course. Now m refers to the result of evaluating m[1:].
edits m but not d (I wanted to change d)
Yes, of course. Why would it change d? It wasn't some kind of magic, it was simply the result of evaluating d['m']. Exactly the same thing happens on both lines.
Let's look at a simpler example.
m = 1
m = 2
Does this cause 1 to become 2? No, of course not. Integers are immutable. But the same thing is happening: m is caused to name one thing, and then to name another thing.
Or, another way: if "references" were to work the way you expect, then the line m = m[1:] would be recursive. You're expecting it to mean "anywhere that you see m, treat it as if it meant m[1:]". Well, in that case, m[1:] would actually mean m[1:][1:], which would then mean m[1:][1:][1:], etc.
Related
I was making a code, and variables started to behave strangely and get assigned to things which I thought they shouldn't. So, I decided to reduce the situation to minimal complexity in order to solve my doubts, and this is what happened:
The following code:
a = [2]
def changeA(c):
d = c
d[0] = 10
return True
changeA(a)
print(a)
prints '[10]'. This doesn't make sense to me, since I never assigned the list "a" to be anything after the first assignment. Inside the function changeA, the local variable d is assigned to be the input of the function, and it seems to me that this assignment is happening both ways, and even changing the "outside". If so, why? If not, why is this happening?
I've also noticed that the code
a = [2]
def changeA(c):
d = list(c)
d[0] = 10
return True
changeA(a)
print(a)
behaves normally (i.e., as I would expect).
EDIT: This question is being considered a duplicate of this one. I don't think this is true, since it is also relevant here that the locality character of procedures inside a function is being violated.
Python variables are references to objects, and some objects are mutable. Numbers are not, neither are strings nor tuples, but lists, sets and dicts are.
Let us look at the following Python code
a = [2] # ok a is a reference to a mutable list
b = a # b is a reference to the exact same list
b[0] = 12 # changes the value of first element of the unique list
print(a) # will display [12]
In the first example, you simply pass the reference of c(which is a) to d.
So whatever you do to d will happen on a.
In the second example, you copy the value of c(which is a) and give it to a new variable d.
So the d now has the same value as c(which is a) but different reference.
Note: you can see the reference or id of a variable using the id() function.
a = [2]
print id(a)
def changeA(c):
d = c
pirnt id(d)
d[0] = 10
return True
changeA(a)
print(a)
a = [2]
print id(a)
def changeA(c):
d = list(c)
print id(d)
d[0] = 10
return True
changeA(a)
print(a)
Its because when you do:
d = list(c)
that creates a new object. But when you do
d = c
You are making a reference to that object.
if you did
d.append(5)
to the first example you would get
[10,5]
Same operation to the second one and the list isn't modified.
Deeper explanation in the following link: http://henry.precheur.org/python/copy_list
In Python, names refer to values, so you have 2 names pointing to the same value.
In version 1 you create a list a, pass it in to the function under the pseudonym c, create another pseudonym d for the very same list, and then change the first value in that list. a, c, and d all refer to the same list.
In version 2, you're using list(c) which, to Python, means "take the contents of this iterable thing named c and make a new, different, list from it named d". Now, there are two copies of your list floating around. One is referred to as a or c, and the other is d. Therefore, when you update d[0] you're operating a second copy of the list. a remains unchanged.
'd=c' means reference copy as stated before. What it means, is that now d will reference the same object as c. As you are doing a direct manipulation on the referenced object, the value of the object a was referencing to is changed as well.
When you do 'd = list(c)' what it means that a new list object is created, with the baked of c. However, d is not referencing the same object as a anymore. Hence, the changes within the function doesn't impact a.
I have just started using SAGE which is pretty close to python as I understand it, and I have come accross this problem where I'll have as a parameter of a function a matrix which I wish to use multiple times in the function with its same original function but through the different parts of the function it changes values.
I have seen in a tutorial that declaring a variable in the function as
variable = list(parameter) doesnt affect the parameter or whatever is put in the parentheses. However I can't make it work..
Below is part of my program posing the problem (I can add the rest if necessary): I declare the variable determinant which has as value the result of the function my_Gauss_determinant with the variable auxmmatrix as parameter. Through the function my_Gauss_determinant the value of auxmmatrix changes but for some reason the value of mmatrix as well. How can avoid this and be able to re-use the parameter mmatrix with its original value?
def my_Cramer_solve(mmatrix,bb):
auxmmatrix=list(mmatrix)
determinant=my_Gauss_determinant(auxmmatrix)
if determinant==0:
print
k=len(auxmmatrix)
solution=[]
for l in range(k):
auxmmatrix1=my_replace_column(list(mmatrix),l,bb)
determinant1=my_Gauss_determinant(auxmmatrix1)
solution.append(determinant1/determinant0)
return solution
What you want is a copy of mmatrix. The reason list(other_list) works is because it iterates through every item in other_list to create a new list. But the mutable objects within the list aren't copied
>>> a = [{1,2}]
>>> b = list(a)
>>> b[0].add(7)
>>> a
[set([1,2,7])]
To make a complete copy, you can use copy.deepcopy to make copies of the elements within the list
>>> import copy
>>> a = [{1,2}]
>>> b = copy.deepcopy(a)
>>> b[0].add(7)
>>> a
[set([1,2])]
So if you only want to copy the list, but don't want to copy the elements within the list, you would do this
auxmmatrix = copy.copy(matrix)
determinant = my_Gauss_determinant(copy.copy(matrix))
If you want to copy the elements within the list as well, use copy.deepcopy
If m is a matrix, you can copy it into mm by doing
sage: mm = m[:, :]
or
sage: mm = matrix(m)
To understand the need to copy container structures such as lists and matrices, you could read the tutorial on objects and classes in Python and Sage.
The other Sage tutorials are recommended too!
I understand that a namedtuple in python is immutable and the values of its attributes cant be reassigned directly
N = namedtuple("N",['ind','set','v'])
def solve()
items=[]
R = set(range(0,8))
for i in range(0,8):
items.append(N(i,R,8))
items[0].set.remove(1)
items[0].v+=1
Here last like where I am assigning a new value to attribute 'v' will not work. But removing the element '1' from the set attribute of items[0] works.
Why is that and will this be true if set attribute were of List type
Immutability does not get conferred on mutable objects inside the tuple. All immutability means is you can't change which particular objects are stored - ie, you can't reassign items[0].set. This restriction is the same regardless of the type of that variable - if it was a list, doing items[0].list = items[0].list + [1,2,3] would fail (can't reassign it to a new object), but doing items[0].list.extend([1,2,3]) would work.
Think about it this way: if you change your code to:
new_item = N(i,R,8)
then new_item.set is now an alias for R (Python doesn't copy objects when you reassign them). If tuples conferred immutability to mutable members, what would you expect R.remove(1) to do? Since it is the same set as new_item.set, any changes you make to one will be visible in the other. If the set had become immutable because it has become a member of a tuple, R.remove(1) would suddenly fail. All method calls in Python work or fail depending on the object only, not on the variable - R.remove(1) and new_item.set.remove(1) have to behave the same way.
This also means that:
R = set(range(0,8))
for i in range(0,8):
items.append(N(i,R,8))
probably has a subtle bug. R never gets reassigned here, and so every namedtuple in items gets the same set. You can confirm this by noticing that items[0].set is items[1].set is True. So, anytime you mutate any of them - or R - the modification would show up everywhere (they're all just different names for the same object).
This is a problem that usually comes up when you do something like
a = [[]] * 3
a[0].append(2)
and a will now be [[2], [2], [2]]. There are two ways around this general problem:
First, be very careful to create a new mutable object when you assign it, unless you do deliberately want an alias. In the nested lists example, the usual solution is to do a = [[] for _ in range(3)]. For your sets in tuples, move the line R = ... to inside the loop, so it gets reassigned to a new set for each namedtuple.
The second way around this is to use immutable types. Make R a frozenset, and the ability to add and remove elements goes away.
You mutate the set, not the tuple. And sets are mutable.
>>> s = set()
>>> t = (s,)
>>> l = [s]
>>> d = {42: s}
>>> t
(set([]),)
>>> l
[set([])]
>>> d
{42: set([])}
>>> s.add('foo')
>>> t
(set(['foo']),)
>>> l
[set(['foo'])]
>>> d
{42: set(['foo'])}
I have the following code in python:
a = "xxx" # a is a string
b = "yyy" # b is another string
for s in a, b:
t = s[:]
...
I dont understand the meaning of for line. I know a, b returns a tuple. But what about looping through a, b? And why you need t = s[:]. I know s[:] creates a copy of list. But if s is a string, why don't you write t = s to make a copy of the string s into t?
Thank you.
The meaning of the for loop is to iterate over the tuple (a, b). So the loop body will run twice, once with s equal to a and again equal to b.
t = s[:]
On the face of it, this creates a copy of the string s, and makes t a reference to that new string.
However strings are immutable, so for most purposes the original is as good as a copy. As an optimization, Python implementations are allowed to just re-use the original string. So the line is likely to be equivalent to:
t = s
That is to say, it will not make a copy. It will just make t refer to the same object s refers to.
This is my first question, so if I'm being a total dweeb posting this, let me know why and how I can avoid it in the future!
I have a bit of python code that should just take a list, and multiply the jth component by -1. This is the code in question.
def flip(spins,j):
z = spins
z[j] = z[j]*-1
return z
However, what I am noticing is that if I try to do something like
spin = [1,1,1]
test = flip(spin,1)
it will assign the proper value [1,-1,1] to 'test', but it will also change the value of 'spin' to [1,-1,1]. I know there must be something totally obvious I'm overlooking, but I've been staring at this for 2 hours and still can't see it.
Thanks for your help!
Inside your function, z and spins refer to the same list, which is also known by the global name of spin. If you modify one, those changes are visible through the other names as well. The variable z is superfluous.
If you want z to be a copy of spins then just do:
z = spins[:]
or:
z = list(spins)
In Python, all objects are referenced by pointers. All variable names like spin and test are just pointers. When you assign spin to z you are just making z point to the same object as spin and then mutating the object pointed to by z (which is the same as the object pointed to by spin).
All function calls in Python are pass-by-value where the values are direct references to objects. Thus, if you mutate a list object within a function, it will appear mutated even after you return from the function.
You have to create a new copy of the list if you don't want to accidentally mutate spin. Other answers have shown ways of copying a list. I personally like the [:] syntax that kindall showed in his answer.
The thing you are forgetting is that z and spins are essentially the same object. Both point to the same list and have the same elements.
For example:
a = [0]
b = a
b[0] = 1
print a
Result:
[1]
To copy the list instead, try something like this:
a = [0]
b = a[:]
b[0] = 1
print a
Result:
[0]
z = spins
puts two names on the same object. This also happens with num = old_num but since you can never change numbers in-place (they're immutable) something like number = number * -1 just gives number * -1 the label number and removes the old one.
When you do
z[j] = ...
you're changing z, not assigning, which is why this changes both of the labelled items. What you want to do is make a copy, which you can do like:
z = spins[:] # Traditional way
z = spins(spins) # Pragmatic way
z = spins.copy() # My favorite way
All of which are equally valid. Because this is a new item, changing it does not affect the original.
Another way to create a new list is to use a list comprehension
def flip(spins, j):
return [e * -1 if i == j else e for i, e in enumerate(spins)]
Multiply by -1 can also just be -e so
def flip(spins, j):
return [-e if i == j else e for i, e in enumerate(spins)]
Python treats mutable (list, dictionary, pandas dataframe and series) and immutable (int, tuple, float) data types differently as they get processed inside a function.
Immutable data type gets changed even if it's used inside a function. To avoid this, always make sure to do the processing on the copy of the data by calling .copy() method which creates a copy of the data in a new location.
.copy() works with all the above mentioned mutable data types.
This code will do the job for you:
def flip(spins,j):
z = spins.copy()
z[j] = z[j]*-1
return z