Basic python, pass by reference confusion [duplicate] - python

This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 3 years ago.
I am not understanding the difference between these two methods can someone explain? Why in one of them the reference object was changed but in the second one the reference remains unchanged? I come from Java, C# background if that helps. To me it seems like the reference should update in both of them. Thanks
def changeme( mylist ):
"This changes a passed list into this function"
mylist.append([1,2,3,4]);
print "Values inside the function: ", mylist
return
# Now you can call changeme function
mylist = [10,20,30];
changeme( mylist );
print "Values outside the function: ", mylist
Values inside the function: [10, 20, 30, [1, 2, 3, 4]]
Values outside the function: [10, 20, 30, [1, 2, 3, 4]]
def changeme( mylist ):
"This changes a passed list into this function"
mylist = [1,2,3,4]; # This would assig new reference in mylist
print "Values inside the function: ", mylist
return
# Now you can call changeme function
mylist = [10,20,30];
changeme( mylist );
print "Values outside the function: ", mylist
Values inside the function: [1, 2, 3, 4]
Values outside the function: [10, 20, 30]

mylist = [1,2,3,4] in the function changes the value of the parameter mylist, which is a copy of the reference to mylist outside of the function. The original reference does not change.
If you want to modify the list (rather than a reference to it), use mylist[:] = [1,2,3,4].

This is one of the most important things you will learn about Python (well, it was for me).
Every object is inherently unnamed but you can bind a variable name to it.
When you do:
x = [1, 2, 3]
two things happen:
the [1, 2, 3] object is created; and
the x name is bound to it.
That's why, when you change an object, all bound names seem to change:
>>> x = [1, 2, 3] ; y = x ; x ; y
[1, 2, 3]
[1, 2, 3]
>>> x.append(42) ; x ; y
[1, 2, 3, 42]
[1, 2, 3, 42]
You're not changing x or y in that case, you're changing the object behind those variables and, since both variables are bound to that object, both will be affected.
So, how does that affect your question. When you pass a variable to a function, the name in the function definition is simply bound to the same object as the one you passed in:
def modList(x): # 'x' and 'actual' now point to the SAME object.
x.append(42) # this will modify that object, no binding changes.
x = [4, 5, 6] # this will create NEW object and rebind 'x' to it.
actual = [1, 2, 3] # object is [1, 2, 3], 'actual' is bound to it.
modList(actual) # pass object reference to function.
So statements that change the object (like the first line of modList) will change it for all bindings whereas statements that rebind (like the second line) will lose access to the original object.
If you want to change the object with an assignment-looking statement, you can use array slicing to do so. This works because you're changing the elements in the object rather than rebinding x to a new object:
x[:] = [4, 5, 6]

Related

Why is it that modifying lists in functions changes the original list, but declaring them in a function creates a new object?

I'm new to writing code, and I understand the behavior of lists to an extent. Whenever a list is modified within a function's scope, the one in global scope changes too.
For example,
def modify_list(lst):
lst.append(5)
lst = [1, 2, 3, 4]
#This output is [1, 2, 3, 4]
print(lst)
modify_list(lst)
#This output is [1, 2, 3, 4, 5] because of the function.
print(lst)
I don't understand why this example won't work:
def modify_list(lst):
lst = [1, 2, 3, 4, 5]
lst = [1, 2, 3, 4]
#Output is [1, 2, 3, 4]
print(lst)
modify_list(lst)
#Output is [1, 2, 3, 4]
print(lst)
Why doesn't lst get modified in the second example? Is it because I'm creating a new object within the function's scope? Using the global keyword works instead of passing a parameter, but I want to avoid using global unless absolutely necessary.
I'm using this in an initialization function and want to revert the list back to its original state whenever the function is called. Again, using global works, I'm just wondering why this doesn't work.
Thanks! (Sorry if I'm not good at explaining things well)
The id function comes in handy here. Basically, the id function returns an integer that is guaranteed to be unique and constant for the lifetime of whatever object it was called on. In fact, in CPython (regular Python), id returns the address of the object in memory.
So if you run your code, printing the id of lst before and after running your modify_list, you'll find that the id changes when you assign to lst but not when you append.
Calling append on lst won't change the id of lst because lists in Python are mutable, and appending simply mutates the list. But, when you assign [1, 2, 3, 4, 5] to lst, you are creating a brand-new list object and assigning it to lst. This is not a mutation, and doesn't change the original in anyway. In general in Python, you can mutate arguments within a function to modify the original copy, but assigning a new object to it is not a mutation and won't change the original copy.
In
def modify_list(lst):
lst = [1, 2, 3, 4, 5]
you make local variable lst point to a completely different object than the one you passed as the argument in modify_list(lst) call. Maybe this article will help you understand: https://medium.com/school-of-code/passing-by-assignment-in-python-7c829a2df10a

Which python data types act like pointers?

For example, the dictionary below updates for changes in the list but not for the int.
What is the name of the property of the data type that tells me if it will act like the int or like the list in this example?
How do I set this property when building classes?
Is there a way to make the "i" variable in my example behave like the "l" variable?
i = 1
l = [1]
d = {'i':i,'l':l}
i = 0
l[0] = 0
print(d) # {'i': 1, 'l': [0]}
Python passes things around by assignment, so basically Python always passes a reference to the object, but the reference is passed by value.
An example where the value of the reference is not changed:
def foo(list_):
list_.append(0)
l = [3, 2, 1]
foo(l)
# l = [3, 2, 1, 0]
Now one where it is changed:
def foo(list_):
list_ = [1, 2, 3]
l = [3, 2, 1]
foo(l)
# l = [1, 2, 3]
You could also get a copy of the list:
def foo(list_):
list_copy = list_.copy()
list_copy.append(0)
l = [3, 2, 1]
foo(l)
# l = [3, 2, 1]
In your example you changed the value of the reference of the i variable inside the dict d, but not of the list l, you used the reference and changed an item inside the list. This happened because lists are mutable, Python integers are not, you made two different operations. You won't be able to change what is inside the dict d using the variable i after the assignment, you would need to do d['i'] = 0 for that.

Appending to a list comprehension in Python returns None

This is a question out of curiosity rather than trying to use it for a practical purpose.
Consider I have the following simple example where I generate a list through list comprehension:
>>> a = [1, 2, 3]
>>> b = [2 * i for i in a]
>>> b
[2, 4, 6]
>>> b.append(a)
>>> b
[2, 4, 6, [1, 2, 3]]
However if I try and do this all in one action
>>> a = [1, 2, 3]
>>> b = [2 * i for i in a].append(a)
>>> b == None
True
The result returns None. Is there any reason why this is the case?
I would have thought that an action like this would either return an answer like in the first example or throw an error.
For reference I'm using Python 3.6.5
append only works on variables, not list literals, since it updates the list object itself and does not return the resulting list.
As #Tomalak mentioned noted, running a similar operation on a simple list also returns None
>>> [1, 2, 3].append(4) == None
True
You can use concatination + instead of append in list comprehension
In [1]: a = [1, 2, 3]
In [2]: b = [2 * i for i in a] + [a]
In [3]: b
Out[3]: [2, 4, 6, [1, 2, 3]]
#ScottMcC, methods defined on mutable objects like list, dictionary mostly perform operations on calling object and doesn't return anything.
In case of immutable object like string you may see, methods return the modified form(a different object) of the calling object. In case of list, it's different.
You can't expect the below operations on list kind of mutable objects.
s = "hello DJANGO"
s2 = s.upper()
s3 = s.lower()
print(s) # hello DJANGO
print(s2) # HELLO DJANGO
print(s3) # hello django
Now, have a look at the below examples.
list is mutable object.
Calling sort() method on list directly modified the calling object and doesn't return anything (That's why None).
Calling sorted() function doesn't alter the passing list. It creates a separate sorted list based on the passed list. As it is not a method defined on list object, it returns the new sorted list.
append() method appends item on the calling list and doesn't return anything. Once you call it, you are done with updating (appending an item) the list.
# sort() method defined on list updates the calling list
# As it updates current list, it doesn't return anything. That's why None.
a = [5, 8, 1, 2, 7]
n = a.sort()
print (a)
print(n)
print ()
# sorted() function returns a new sorted list
# It doesn't update the calling list a2
a2 = [5, 8, 1, 2, 7];
n = sorted(a2);
print (a2)
print(n)
print()
# append() is method defined on list, it updates calling list so it doesn't return anything (None)
l = []
n = l.append(34)
print(l)
print (n)
Output
[1, 2, 5, 7, 8]
None
[5, 8, 1, 2, 7]
[1, 2, 5, 7, 8]
[34]
None

Python Variable assignment in a for loop

I understand that in Python regular c++ style variable assignment is replaced by references to stuff ie
a=[1,2,3]
b=a
a.append(4)
print(b) #gives [1,2,3,4]
print(a) #gives [1,2,3,4]
but I'm still confused why an analogous situation with basic types eg. integers works differently?
a=1
b=a
a+=1
print(b) # gives 1
print(a) # gives 2
But wait, it gets even more confusing when we consider loops!
li=[1,2,3]
for x in li:
x+=1
print(li) #gives [1,2,3]
Which is what I expected, but what happens if we do:
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #gives [1,2,3]
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there. The only thing I can come up short of using
for x in range(len(li)):
Do stuff to li[x]
is packaging the integers in one element list. But there must be a better way.
Well, you need to think of mutable and immutable type.
For a list, it's mutable.
For a integer, it's immutable, which means you will refer to a new object if you change it. When a+=1 is executed, a will be assigned a new object, but b is still refer to the same one.
a=[1,2,3]
b=a
a.append(4)
print(b) #[1,2,3,4]
print(a) #[1,2,3,4]
Here you are modifying the list. The list content changes, but the list identity remains.
a=1
b=a
a+=1
This, however, is a reassignment. You assign a different object to a.
Note that if you did a += [4] in the 1st example, you would have seen the same result. This comes from the fact that a += something is the same as a = a.__iadd__(something), with a fallback to a = a.__add__(something) if __iadd__() doesn't exist.
The difference is that __iadd__() tries to do its job "inplace", by modifying the object it works on and returning it. So a refers to the same as before. This only works with mutable objects such as lists.
On immutable objects such as ints __add__() is called. It returns a different object, which leads to a pointing to another object than before. There is no other choice, as ints are immutable.
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #[1,2,3]
Here x += 1 means the same as x = x + 1. It changes where x refers to, but not the list contents.
Maybe my question should be how to loop over a list of integers and change them without >map() as i need a if statement in there.
for i, x in enumerate(li):
li[i] = x + 1
assigns to every list position the old value + 1.
The important thing here are the variable names. They really are just keys to a dictionary. They are resolved at runtime, depending on the current scope.
Let's have a look what names you access in your code. The locals function helps us: It shows the names in the local scope (and their value). Here's your code, with some debugging output:
a = [1, 2, 3] # a is bound
print(locals())
for x in a: # a is read, and for each iteration x is bound
x = x + 3 # x is read, the value increased and then bound to x again
print(locals())
print(locals())
print(x)
(Note I expanded x += 3 to x = x + 3 to increase visibility for the name accesses - read and write.)
First, you bind the list [1, 2, 3]to the name a. Then, you iterate over the list. During each iteration, the value is bound to the name x in the current scope. Your assignment then assigns another value to x.
Here's the output
{'a': [1, 2, 3]}
{'a': [1, 2, 3], 'x': 4}
{'a': [1, 2, 3], 'x': 5}
{'a': [1, 2, 3], 'x': 6}
{'a': [1, 2, 3], 'x': 6}
6
At no point you're accessing a, the list, and thus will never modify it.
To fix your problem, I'd use the enumerate function to get the index along with the value and then access the list using the name a to change it.
for idx, x in enumerate(a):
a[idx] = x + 3
print(a)
Output:
[4, 5, 6]
Note you might want to wrap those examples in a function, to avoid the cluttered global namespace.
For more about scopes, read the chapter in the Python tutorial. To further investigate that, use the globals function to see the names of the global namespace. (Not to be confused with the global keyword, note the missing 's'.)
Have fun!
For a C++-head it easiest tho think that every Python object is a pointer. When you write a = [1, 2, 3] you essentially write List * a = new List(1, 2, 3). When you write a = b, you essentially write List * b = a.
But when you take out actual items from the lists, these items happen to be numbers. Numbers are immutable; holding a pointer to an immutable object is about as good as holding this object by value.
So your for x in a: x += 1 is essentially
for (int x, it = a.iterator(); it->hasMore(); x=it.next()) {
x+=1; // the generated sum is silently discarded
}
which obviously has no effect.
If list elements were mutable objects you could mutate them exactly the way you wrote. See:
a = [[1], [2], [3]] # list of lists
for x in a: # x iterates over each sub-list
x.append(10)
print a # prints [[1, 10], [2, 10], [3, 10]]
But unless you have a compelling reason (e.g. a list of millions of objects under heavy memory load) you are better off making a copy of the list, applying a transformation and optionally a filter. This is easily done with a list comprehension:
a = [1, 2, 3, 0]
b = [n + 1 for n in a] # [2, 3, 4, 1]
c = [n * 10 for n in a if n < 3] # [10, 20, 0]
Either that, or you can write an explicit loop that creates another list:
source = [1, 2, 3]
target = []
for n in source:
n1 = <many lines of code involving n>
target.append(n1)
Your question has multiple parts, so it's going to be hard for one answer to cover all of them. glglgl has done a great job on most of it, but your final question is still unexplained:
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there
"I need an if statement in there" doesn't mean you can't use map.
First, if you want the if to select which values you want to keep, map has a good friend named filter that does exactly that. For example, to keep only the odd numbers, but add one to each of them, you could do this:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> for x in a:
... if x%2:
... b.append(x+1)
Or just this:
>>> b = map(lambda x: x+1, filter(lambda x: x%2, a))
If, on the other hand, you want the if to control the expression itself—e.g., to add 1 to the odd numbers but leave the even ones alone, you can use an if expression the same way you'd use an if statement:
>>> for x in a:
... if x%2:
... b.append(x+1)
... else:
... b.append(x)
>>> b = map(lambda x: x+1 if x%2 else x, a)
Second, comprehensions are basically equivalent to map and filter, but with expressions instead of functions. If your expression would just be "call this function", then use map or filter. If your function would just be a lambda to "evaluate this expression", then use a comprehension. The above two examples get more readable this way:
>>> b = [x+1 for x in a if x%2]
>>> b = [x+1 if x%2 else x for x in a]
You can do something like this: li = [x+1 for x in li]

Python list reference in a function.

I'm having trouble understanding the odd behaviour in python functions if i pass in a list.
I made the following functions:
def func(x):
y = [4, 5, 6]
x = y
def funcsecond(x):
y = [4, 5, 6]
x[1] = y[1]
x = [1, 2, 3]
When i call func(x) and then print out x , it prints out [1, 2, 3], just the way x was before, it doesnt assign the list y to x. However, if i call funcsecond(x), it assigns 5 to the second position of x. Why is that so? When i assign the whole list, it doesn't do anything, but when i assign only one element, it changes the list i originally called it with.
Thank you very much and i hope you understand what im intending to say , i'm having a hard time expressing myself in English.
The former rebinds the name, the latter mutates the object. Changes to the name only exist in local scope, whereas a mutated object remains mutated after the scope is exited.
this is happening beacuse x points to a object which is mutable.
def func(x): # here x is a local variable which refers to the object[1,2,3]
y = [4, 5, 6]
x = y #now the local variable x refers to the object [4,5,6]
def funcsecond(x): # here x is a local variable which refers to the object[1,2,3]
y = [4, 5, 6]
x[1] = y[1] # it means [1,2,3][1]=5 , means you changed the object x was pointing to
x = [1, 2, 3]

Categories