I have the following piece of code in Python 2.7
abc=[[1,2,3],[0,1,2]]
def make_change(some_list):
for i in range(len(some_list)):
if some_list[i][0]==0:
some_list[i]=[]
return some_list
new_list=make_change(abc)
print abc
print new_list
My understanding was that it should produce the following output.
[[1,2,3],[0,1,2]]
[[1,2,3],[]]
But the python actually produces
[[1,2,3],[]]
[[1,2,3],[]]
Am I missing something?
You can prevent this issue by copying the list while passing it to the function:
abc=[[1,2,3],[0,1,2]]
def make_change(some_list):
for i in range(len(some_list)):
if some_list[i][0]==0:
some_list[i]=[]
return some_list
new_list=make_change(abc[:])
print abc
print new_list
The changed part:
new_list=make_change(abc[:])
The reason this happens is Python passes the list by reference, so changes will be made to the original as well. Using [:] creates a shallow copy, which is enough to prevent this.
Do not change the list you pass to a function unless you specifically want the function to change a list that's passed to it. Make a copy of the list (or any other mutable object) and work on the copy. If you're using compound objects (objects with objects in them) use copy.deepcopy to make sure everything is a copy.
Since functions exist to encapsulate the weird stuff you have to do, doing weird stuff to the objects you're passing to a function rarely makes sense to me. The other answer has you pass a slice copy of the list to the function. Why not make the whole thing more readable by passing your mutable to the function and having the function create the copy. Better encapsulation = less annoying code.
abc=[[1,2,3],[0,1,2]]
def make_change(some_list):
from copy import deepcopy
new_list = deepcopy(some_list)
for i in range(len(new_list)):
if new_list[i][0]==0:
new_list[i]=[]
return new_list
new_list=make_change(abc)
print abc
print new_list
List Comprehensions to the rescue!
abc=[[1,2,3],[0,1,2]]
def make_change(some_list):
return [[] if a[0]==0 else a for a in some_list]
make_change(abc)
[[1, 2, 3], []]
Related
My understanding is that:
def copy_2d(p):
return list(p)
Would make a full copy of p and return it as a result. list( p) seems to do this when I try it in the repl. However it seems like calling the above method like this:
b = [[1,2,3],[3,4,5]]
a = copy_2d(b)
a[0][0] = 0
if (b[0][0] == 0): print "Huh??"
It prints "Huh??", that is, it appears that b is just a reference to a. I double checked but I might be blind. Can someone clarify please?
Your current code for copy_2d returns a shallow copy of the list you pass as its argument. That is, you're creating a new outer list, but the inner values (which may be lists themselves) are not copied. The new list references the same inner lists, so when you mutate one of them, you'll see the same change in the shallow copy as you do in the original list.
You can fix the issue by copying the inner lists as well as creating a new outer list. Try:
def copy_2d(p):
return map(list, p) # warning: this only works as intended in Python 2
In Python 2, this works because map returns a list. In Python 3, the map function returns an iterator, so a list comprehension like [list(inner) for inner in p] would be better.
Of course, if you don't need to write your own code to solve this problem, you should just use copy.deepcopy from the standard library.
import copy
def copy_2d(p):
return copy.deepcopy(p)
or
def copy_2d(p):
return [list(p2) for p2 in p]
What you did copied the array with all the values inside. But inside you had objects with were arrays. Your function did not copied inside lists but their references.
The second solution still copies references, but one layer below.
Shallow copy made. Same logic as here?what-does-the-list-function-do-in-python
"list() converts the iterable passed to it to a list. If the iterable is already a list then a shallow copy is returned, i.e only the outermost container is new rest of the objects are still the same."
This question already has answers here:
Correct Style for Python functions that mutate the argument
(4 answers)
Closed 6 years ago.
I want to do some list modification in a function and then continue to use the modified list after calling the function, which is the better approach to do this:
def modify(alist):
alist.append(4)
alist = [1,2,3]
modify(alist)
alist.append(5)
Or this:
def modify(alist):
alist.append(4)
return alist
alist = [1,2,3]
alist = modify(alist)
alist.append(5)
Is the first some kind of bad tone?
Ordinarily a function should return the results it generates. However when you pass a list, you either need to make a copy of the list or accept the fact that it will be modified; it's redundant to return the modified list. It also leads to a problem if you ever provide a default argument, since the default will also be modified.
I generally prefer to make arguments read-only unless it's obvious that they will be modified inplace.
My recommendation:
def modify(alist=[]):
alist = alist[:] # make a copy
alist.append(4)
return alist
Since you're modifying the list inplace, it would not make much sense to return the same list you have modified.
Your inplace mutation propagates from the reference to the list, so better to leave out the return statement and raise an exception if an error occurs somewhere in the function, but not return the modified object.
There should be one-- and preferably only one --obvious way to do it.
It's safe to return the list from a function.
Formally, both approaches fullfil the task you want - to modify a list.
But, the second approach with function return some value which is stored in your variable is safer, and such code is much-much easier to maintain and develop, especially if you have many lists to modify, for example. Or in general you have a code that solves some difficult task, for which you need a lot of functions and variables. "Safe" means you don't have to worry about variable name conflict inside you code - everything you create inside your function namespace, stays local (except the case when you create a class attribute). So, it's generally considered a better practice. Good luck!
I am having a hard time understanding whats going on in the my code. So if I have the following line:
d = {}
d.setdefault("key",[]).append("item")
This returns
{'key': ['item']}
So I get what setdefault does. It checks for "key" in the d, a dictionary, and if it doesn't exist it creates it otherwise if it does exist then it returns the value. This returns a copy which can be manipulated and will be updated in the original dictionary. This is a new idea to me. Does this mean that setdefault returns a deep copy, as opposed to a shallow copy? Trying to get wrap my head around this shallow copy vs. deep copy.
No Python operation does implicit copying. Ever. Implicit copying is evil, as far as Python is concerned.
It's literals that create objects. Every time setdefault is called, it evaluates both its arguments. When it evaluates its second argument ([]), a new list is created. It's completely the same as a = [].
If you write el = [] and then try .setdefaulting el into some dict more than one time, you'll see that no copies are being made.
it is equivelent to
item = d.get(key,default)
d[key] = item
d[key].action #in this case append
From the holy docs:
setdefault(key[, default])
If key is in the dictionary, return its
value. If not, insert key with a value of default and return default.
default defaults to None.
The behaviour is easily explicable once you drop the idea that it is a copy. It is not; it is the actual object.
These work:
>>> print [1,2,3] + [4,5]
[1,2,3,4,5]
>>> 'abc'.upper()
'ABC'
This doesn't:
>>> print [1,2,3].extend([4,5])
None
Why? You can use string methods on bare strings, so why can't you use methods on bare sequence types such as lists? Even this doesn't work:
>>> print list([1,2,3]).extend([4,5])
None
N.B. For a colloquial meaning of 'work'. Of course there'll be a good reason why my expected behaviour is incorrect. I'm just curious what it is.
P.S. I've accepted soulcheck's answer below, and he is right, but on investigating how the addition operator is implemented I just found that the following works:
>>> [1,2,3].__add__([4,5])
[1, 2, 3, 4, 5]
But presumably the add method doesn't modify the underlying object and creates a new one to return, like string methods.
extend works, just doesn't return any value as it's modifying the object it works on. That's a python convention to not return any value from methods that modify the object they're called on.
Try:
a = [1,2,3]
a.extend([4,5])
print a
edit:
As for why one reason for this could be that it's sometimes ambigous what the method should return. Should list.extend return the extended list? A boolean? New list size?
Also it's an implementation of command-query separation, ie. if it returned something it would be a query and command in one and cqs says it's evil (although sometimes handy - see list.pop()).
So generally you shouldn't return anything from mutators, but as with almost everything in python it's just a guideline.
list.__add__() is the method called when you do + on lists - it returns a new list.
This is due to the design of the interface. Extend does extend the list but does not return any value.
l = [1, 2, 3]
l.extend([4, 5])
print l
I know that "variable assignment" in python is in fact a binding / re-bindign of a name (the variable) to an object.
This brings the question: is it possible to have proper assignment in python, eg make an object equal to another object?
I guess there is no need for that in python:
Inmutable objects cannot be 'assigned to' since they can't be changed
Mutable objects could potentially be assigned to, since they can change, and this could be useful, since you may want to manipulate a copy of dictionary separately from the original one. However, in these cases the python philosophy is to offer a cloning method on the mutable object, so you can bind a copy rather than the original.
So I guess the answer is that there is no assignment in python, the best way to mimic it would be binding to a cloned object
I simply wanted to share the question in case I'm missing something important here
Thanks
EDIT:
Both Lie Ryan and Sven Marnach answers are good, I guess the overall answer is a mix of both:
For user defined types, use the idiom:
a.dict = dict(b.dict)
(I guess this has problems as well if the assigned class has redefined attribute access methods, but lets not be fussy :))
For mutable built-ins (lists and dicts) use the cloning / copying methods they provide (eg slices, update)
finally inmutable built-ins can't be changed so can't be assigned
I'll choose Lie Ryan because it's an elegant idiom that I hadn't thought of.
Thanks!
I think you are right with your characterization of assignment in Python -- I just would like to add a different method of cloning and ways of assignment in special cases.
"Copy-constructing" a mutable built-in Python object will yield a (shallow) copy of that object:
l = [2, 3]
m = list(l)
l is m
--> False
[Edit: As pointed out by Paul McGuire in the comments, the behaviour of a "copy contructor" (forgive me the C++ terminology) for a immutable built-in Python object is implementation dependent -- you might get a copy or just the same object. But because the object is immutable anyway, you shouldn't care.]
The copy constructor could be called generically by y = type(x)(x), but this seems a bit cryptic. And of course, there is the copy module which allows for shallow and deep copies.
Some Python objects allow assignment. For example, you can assign to a list without creating a new object:
l = [2, 3]
m = l
l[:] = [3, 4, 5]
m
--> [3, 4, 5]
For dictionaries, you could use the clear() method followed by update(otherdict) to assign to a dictionary without creating a new object. For a set s, you can use
s.clear()
s |= otherset
This brings the question: is it
possible to have proper assignment in
python, eg make an object equal to
another object?
Yes you can:
a.__dict__ = dict(b.__dict__)
will do the default assignment semantic in C/C++ (i.e. do a shallow assignment).
The problem with such generalized assignment is that it never works for everybody. In C++, you can override the assignment operator since you always have to pick whether you want a fully shallow assignment, fully deep assignment, or any shade between fully deep copy and fully shallow copy.
I don't think you are missing anything.
I like to picture variables in python as the name written on 'labels' that are attached to boxes but can change its placement by assignment, whereas in other languages, assignment changes the box's contents (and the assignment operator can be overloaded).
Beginners can write quite complex applications without being aware of that, but they are usually messy programs.