What's the difference between
lst = range(100)
and
lst[:] = range(100)
Before that assignment the lst variable was already assigned to a list:
lst = [1, 2, 3]
lst = range(100)
or
lst = [1, 2, 3]
lst[:] = range(100)
When you do
lst = anything
You're pointing the name lst at an object. It doesn't change the old object lst used to point to in any way, though if nothing else pointed to that object its reference count will drop to zero and it will get deleted.
When you do
lst[:] = whatever
You're iterating over whatever, creating an intermediate tuple, and assigning each item of the tuple to an index in the already existing lst object. That means if multiple names point to the same object, you will see the change reflected when you reference any of the names, just as if you use append or extend or any of the other in-place operations.
An example of the difference:
>>> lst = range(1, 4)
>>> id(lst)
74339392
>>> lst = [1, 2, 3]
>>> id(lst) # different; you pointed lst at a new object
73087936
>>> lst[:] = range(1, 4)
>>> id(lst) # the same, you iterated over the list returned by range
73087936
>>> lst = xrange(1, 4)
>>> lst
xrange(1, 4) # not a list, an xrange object
>>> id(lst) # and different
73955976
>>> lst = [1, 2, 3]
>>> id(lst) # again different
73105320
>>> lst[:] = xrange(1, 4) # this gets read temporarily into a tuple
>>> id(lst) # the same, because you iterated over the xrange
73105320
>>> lst # and still a list
[1, 2, 3]
When it comes to speed, slice assignment is slower. See Python Slice Assignment Memory Usage for more information about its memory usage.
The first one redefines the built-in name list to point to some list.
The second fails with TypeError: 'type' object does not support item assignment.
list[:] will only work if there is already an object named list that allows slice assignment.
Also, you shouldn't name variables list because there is a built-in named list which is the list type itself.
list[:] specifies a range within the list, in this case it defines the complete range of the list, i.e. the whole list and changes them. list=range(100), on the other hand, kind of wipes out the original contents of list and sets the new contents.
But try the following:
a=[1,2,3,4]
a[0:2]=[5,6]
a # prints [5,6,3,4]
You see, we changed the first two elements with the assignment. This means, using this notation, you can change several elements in the list once.
[:] is also useful to make a deep copy of the list.
def x(l):
f=l[:]
g=l
l.append(8)
print "l", l
print "g", g
print "f", f
l = range(3)
print l
#[0, 1, 2]
x(l)
#l [0, 1, 2, 8]
#g [0, 1, 2, 8]
#f [0, 1, 2]
print l
#[0, 1, 2, 8]
Modification to l is get reflected in g (because, both point to same list, in fact, both g and l are just names in python), not in f(because, it's a copy of l)
But, in your case, It doesn't make any difference. (Though, I'm not eligible to comment on any memory usage of both methods.)
Edit
h = range(3)
id(h) #141312204
h[:]=range(3)
id(h) #141312204
h=range(3)
id(h) #141312588
list[:] = range(100) updates the list
list = range(100) creates new list.
#agf: thanks for pointing my error
list[:] = range(100)
won't work on uninitialized variable, as it is modifying it. The [:] specifies the whole list/touple.
Related
>>> a = [1,2,3]
>>> a.pop()
3
>>> a
[1, 2]
>>> a = [1,2,3]
>>> a = a[:-1]
>>> a
[1, 2]
>>>
Is there any difference between the above methods to remove the last element from a list?
Yes. pop is O(1) and mutates the original list, while slice is O(n) and creates a copy of the list. Less formally, the pop method is an operation on the element at the end of the list and is defined in CPython as a call to list_resize(self, Py_SIZE(self) - 1);. This doesn't traverse the entire structure.
On the other hand, list_slice allocates a new list and loops over the entries in the old list ranging from the beginning to the end - 1, copying references to each item to the new list.
If what you're trying to do is remove the last element of the list, use pop or del a[-1].
pop do not change the id, just pop one item of list.
[:-1] is slice operation, which create a new list from old list.
>>> a = [1,2,3]
print(id(a))
>>> a.pop()
3
print(id(a))
>>> a
[1, 2]
>>> a = [1,2,3]
>>> a = a[:-1]
>>> a
print(id(a))
[1, 2]
>>>
id output (the number is not important, same or not same is key point):
4470627464
4470627464
4474450952
the pop method returns the last item from the list that it removes.
for example:
a = [1,2,3,4]
b = a.pop()
print(b) # 4
Also, using slicing, you are making a copy of the old list, whereas with using pop the list reference remains the same.
There is an basic difference thats occurs using in functions . Using[:-1] unchanged the original list but pop() can do.
a = [1,2,3]
b = [1,2,3]
def functionb(list):
list = list[:-1]
return list
def withpop(list):
return list.pop()
functionb(b)
withpop(a)
print b
print a
Will printed:
[1, 2, 3]
[1, 2]
Second is execution time . pop() is faster than [:-1] Because when you use [:-1]you have to overwrite to list.Lets say you have thousands values in index so it will be slowly than pop()
The way you have presented them, there's no outward difference. The pop instruction gives the interpreter an easier time of optimizing the instruction, as it can merely decrement the length attribute of the list. The -1 assignment will construct a new list, assign that to a, and then leave the old one for garbage collection.
There is a huge difference in aliasing: if you assigned something else to that list, you will get side effects with pop. For instance:
>>> a = [1, 2, 3, 4]
>>> b = a
>>> b
[1, 2, 3, 4]
>>> a.pop()
4
>>> b
[1, 2, 3]
>>> a = a[:-1]
>>> b
[1, 2, 3]
>>> a
[1, 2]
Yes there is difference.
when you use a.pop() you remove from list too
when you use a[:-1] object list not change
check with len(a)
>>> a = [1,2,3]
>>> a
[1, 2, 3]
>>> a[:-1]
[1, 2]
>>> len(a[:-1])
2
>>> a.pop()
3
>>> a
[1, 2]
>>> len(a)
2
>>>
from this answer https://stackoverflow.com/a/1207461/8074522
I'm trying to find the difference between using
somelist = [1,2,3,4,5,6,7,8,9]
somelist = [x for x in somelist if x>4]
and
somelist[:] = [x for x in somelist if x>4]
in his answer, he is saying
Or, by assigning to the slice somelist[:], you can mutate the existing
list to contain only the items you want:
"to contain only" < I can use assign to the somelist without slicing.
So what's the need of somelist[:] here?
Instruction like somelist = [1,2,3,4,5,6,7,8,9] creates a new list and
assigns it (the whole list) to your variable (somelist).
The second case, somelist = [x for x in somelist if x>4] is just filtering.
The for loop assigns each element from this list to a temporary variable x.
Then (inside this loop) if checks whether this (current) element meets
some condition, in your case, whether it is greater than 4.
Only if it is, this element is added to the new array, which finally will be
assigned to somelist.
And now what is the difference between plain assignment (just to somelist)
and the slice assignment ( to somelist[:]).
Imagine that initially there was otherList and you created your somelist
using plain assignment: somelist = otherList.
Actually, no second list has then been created, but somelist may be regarded as
another "pointer to" that first list.
In other words, both otherList and somelist point to the same internal
object (in this case, array).
Then, later in your code, if you used somelist[:] = ... you write new content
to the same internal object, so this new content is visible also using otherList
(e.g. in other part of your code).
But if you used plain assignment (somelist = ...), then:
There is created a new internal object (array) and somelist points
just to it.
The old content is still visible as otherList.
So as jonrsharpe pointed out, the decision which option to use depends
on what your application really needs.
In the simplest case, if you have no other variable pointing to the "old"
array, you can use plain assignment (IMHO a more natural option) and the
old array (now with no reference from any other variable) will be garbage
collected.
#jonsharpe gave a terse answer in the comment, but to illustrate this clearly:
Assign a new list. Loses the reference to the old list object and creates a new one:
>>> lst1 = [[1, 2, 3],[4, 5, 6]]
>>> lst2 = lst1
>>> lst1 = [[7,8,9]]
>>> lst1 is lst2
False
>>> lst1
[[7, 8, 9]]
>>> lst2
[[1, 2, 3], [4, 5, 6]]
Assign to slice. Replaces the list (or a part of the list, depending on what the slice is) in-place, keeping the same object.
>>> lst1 = [[1, 2, 3],[4, 5, 6]]
>>> lst2 = lst1
>>> lst1[:] = [[7,8,9]]
>>> lst1 is lst2
True
>>> lst1
[[7, 8, 9]]
>>> lst2
[[7, 8, 9]]
Thank you for your valuable time, I have just started learning Python. I came across Mutable and Immutable objects.
As far as I know mutable objects can be changed after their creation.
a = [1,2,3]
print(id(a))
45809352
a = [3,2,1]
print(id(a))
52402312
Then why id of the same list "a" gets changed when its values are changed.
your interpretation is incorrect.
When you assign a new list to a, you change its reference.
On the other hand you could do:
a[:] = [3,2,1]
and then the reference would not change.
mutable means that the content of the object is changed. for example a.append(4) actually make a equal to [1, 2, 3, 4], while on the contrary, appending to a string (which is immutable) does not change it, it creates a new one.
However, when you re-assign, you create a new object and assign it to a, you don't alter the existing content of a. The previous content is lost (unless refered-to by some other variable)
If you change a list, its id doesn't change. But you may do things that instead create a new list, and then it will also have a new id.
E.g.,
>>> l=[]
>>> id(l)
140228658969920
>>> l.append(3) # Changes l
>>> l
[3]
>>> id(l)
140228658969920 # Same ID
>>> l = l + [4] # Computes a new list that is the result of l + [4], assigns that
>>> l
[3, 4]
>>> id(l)
140228658977608 # ID changed
When you do
a = [3, 2, 1]
You unlink the list of [1, 2, 3] from variable a.
Create a new list [3, 2, 1] then assign it to a variable.
Being immutable doesn't mean you assign a new object, it means your original object can be changed "in place" for example via .append()
>>> my_list = [1,2,3]
>>> id(my_list)
140532335329544
>>> my_list.append(5)
>>> id(my_list)
140532335329544
>>> my_list[3] = 4
>>> my_list
[1, 2, 3, 4]
>>> id(my_list)
140532335329544
I understand that in Python regular c++ style variable assignment is replaced by references to stuff ie
a=[1,2,3]
b=a
a.append(4)
print(b) #gives [1,2,3,4]
print(a) #gives [1,2,3,4]
but I'm still confused why an analogous situation with basic types eg. integers works differently?
a=1
b=a
a+=1
print(b) # gives 1
print(a) # gives 2
But wait, it gets even more confusing when we consider loops!
li=[1,2,3]
for x in li:
x+=1
print(li) #gives [1,2,3]
Which is what I expected, but what happens if we do:
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #gives [1,2,3]
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there. The only thing I can come up short of using
for x in range(len(li)):
Do stuff to li[x]
is packaging the integers in one element list. But there must be a better way.
Well, you need to think of mutable and immutable type.
For a list, it's mutable.
For a integer, it's immutable, which means you will refer to a new object if you change it. When a+=1 is executed, a will be assigned a new object, but b is still refer to the same one.
a=[1,2,3]
b=a
a.append(4)
print(b) #[1,2,3,4]
print(a) #[1,2,3,4]
Here you are modifying the list. The list content changes, but the list identity remains.
a=1
b=a
a+=1
This, however, is a reassignment. You assign a different object to a.
Note that if you did a += [4] in the 1st example, you would have seen the same result. This comes from the fact that a += something is the same as a = a.__iadd__(something), with a fallback to a = a.__add__(something) if __iadd__() doesn't exist.
The difference is that __iadd__() tries to do its job "inplace", by modifying the object it works on and returning it. So a refers to the same as before. This only works with mutable objects such as lists.
On immutable objects such as ints __add__() is called. It returns a different object, which leads to a pointing to another object than before. There is no other choice, as ints are immutable.
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #[1,2,3]
Here x += 1 means the same as x = x + 1. It changes where x refers to, but not the list contents.
Maybe my question should be how to loop over a list of integers and change them without >map() as i need a if statement in there.
for i, x in enumerate(li):
li[i] = x + 1
assigns to every list position the old value + 1.
The important thing here are the variable names. They really are just keys to a dictionary. They are resolved at runtime, depending on the current scope.
Let's have a look what names you access in your code. The locals function helps us: It shows the names in the local scope (and their value). Here's your code, with some debugging output:
a = [1, 2, 3] # a is bound
print(locals())
for x in a: # a is read, and for each iteration x is bound
x = x + 3 # x is read, the value increased and then bound to x again
print(locals())
print(locals())
print(x)
(Note I expanded x += 3 to x = x + 3 to increase visibility for the name accesses - read and write.)
First, you bind the list [1, 2, 3]to the name a. Then, you iterate over the list. During each iteration, the value is bound to the name x in the current scope. Your assignment then assigns another value to x.
Here's the output
{'a': [1, 2, 3]}
{'a': [1, 2, 3], 'x': 4}
{'a': [1, 2, 3], 'x': 5}
{'a': [1, 2, 3], 'x': 6}
{'a': [1, 2, 3], 'x': 6}
6
At no point you're accessing a, the list, and thus will never modify it.
To fix your problem, I'd use the enumerate function to get the index along with the value and then access the list using the name a to change it.
for idx, x in enumerate(a):
a[idx] = x + 3
print(a)
Output:
[4, 5, 6]
Note you might want to wrap those examples in a function, to avoid the cluttered global namespace.
For more about scopes, read the chapter in the Python tutorial. To further investigate that, use the globals function to see the names of the global namespace. (Not to be confused with the global keyword, note the missing 's'.)
Have fun!
For a C++-head it easiest tho think that every Python object is a pointer. When you write a = [1, 2, 3] you essentially write List * a = new List(1, 2, 3). When you write a = b, you essentially write List * b = a.
But when you take out actual items from the lists, these items happen to be numbers. Numbers are immutable; holding a pointer to an immutable object is about as good as holding this object by value.
So your for x in a: x += 1 is essentially
for (int x, it = a.iterator(); it->hasMore(); x=it.next()) {
x+=1; // the generated sum is silently discarded
}
which obviously has no effect.
If list elements were mutable objects you could mutate them exactly the way you wrote. See:
a = [[1], [2], [3]] # list of lists
for x in a: # x iterates over each sub-list
x.append(10)
print a # prints [[1, 10], [2, 10], [3, 10]]
But unless you have a compelling reason (e.g. a list of millions of objects under heavy memory load) you are better off making a copy of the list, applying a transformation and optionally a filter. This is easily done with a list comprehension:
a = [1, 2, 3, 0]
b = [n + 1 for n in a] # [2, 3, 4, 1]
c = [n * 10 for n in a if n < 3] # [10, 20, 0]
Either that, or you can write an explicit loop that creates another list:
source = [1, 2, 3]
target = []
for n in source:
n1 = <many lines of code involving n>
target.append(n1)
Your question has multiple parts, so it's going to be hard for one answer to cover all of them. glglgl has done a great job on most of it, but your final question is still unexplained:
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there
"I need an if statement in there" doesn't mean you can't use map.
First, if you want the if to select which values you want to keep, map has a good friend named filter that does exactly that. For example, to keep only the odd numbers, but add one to each of them, you could do this:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> for x in a:
... if x%2:
... b.append(x+1)
Or just this:
>>> b = map(lambda x: x+1, filter(lambda x: x%2, a))
If, on the other hand, you want the if to control the expression itself—e.g., to add 1 to the odd numbers but leave the even ones alone, you can use an if expression the same way you'd use an if statement:
>>> for x in a:
... if x%2:
... b.append(x+1)
... else:
... b.append(x)
>>> b = map(lambda x: x+1 if x%2 else x, a)
Second, comprehensions are basically equivalent to map and filter, but with expressions instead of functions. If your expression would just be "call this function", then use map or filter. If your function would just be a lambda to "evaluate this expression", then use a comprehension. The above two examples get more readable this way:
>>> b = [x+1 for x in a if x%2]
>>> b = [x+1 if x%2 else x for x in a]
You can do something like this: li = [x+1 for x in li]
Simple example:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print myList
prints
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]
while:
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
prints
[1, 2, 3, 4, 5]
[2, 3, 4, 5, 6]
Conclusion:
Lists can be modified in place using global list access Lists can
List items can NOT be modified in place using the iterator object
All example code I can find uses the global list accessors to modify the list inplace.
Is it so evil to modify a list iterator?
The reason obj += 1 does not do what you expect is that this statement does not modify obj in-place. Instead, it computes the new value, and rebinds the variable obj to point to the new value. This means that the contents of the list remain unchanged.
In general it is possible to modify the list while iterating over it using for obj in myList. For example:
myList = [[1], [2], [3], [4], [5]]
for obj in myList:
obj[0] += 1
print(myList)
This prints out:
[[2], [3], [4], [5], [6]]
The difference between this and your first example is that here, the list contains mutable objects, and the code modifies those objects in-place.
Note that one could also write the loop using a list comprehension:
myList = [val+1 for val in myList]
I think you've misunderstood what an "iterator object" is. A for loop is not an iterator object. For all intents and purposes, a for loop like this:
myList = [0, 1, 2, 3, 4]
for x in myList:
print x
does this (but more efficiently and less verbosely):
i = 0
while i < len(myList)
x = myList[i]
print x
i += 1
So you see, any changes made to x are lost as soon as the next loop starts, because the value of x is overwritten by the value of the next item in the list.
As others have observed, it is possible to alter the value of a list while iterating over it. (But don't change its length! That's where you get into trouble.) One elegant way to do so is as follows:
for i, x in enumerate(myList):
myList[i] = some_func(x)
Update: It's also important to understand that no copying goes on in a for loop. In the above example, i and x -- like all variables in Python -- are more like pointers in C/C++. As the for loop progresses, obj points at myList[0], myList[1], etc, in turn. And like a C/C++ pointer, the properties of the object pointed to are not changed when the pointer is changed. But also like a C pointer, you can directly modify the thing pointed at, because it's not a copy. In C, this is done by dereferencing the pointer; in Python, this is done by using a mutable object. That's why NPE's answer works. If i and x were even shallow copies, it wouldn't be possible to do what he does.
The reason you can't directly change ints the way you can change lists (as in NPE's answer), is that ints aren't mutable. Once a 5 object is created, nothing can change its value. That's why passing around a pointer to 5 is safe in Python -- no side-effects can occur, because the thing pointed to is immutable.
in for obj in myList:, in every iteration, obj is a (shallow) copy of the element in myList. So the change on the obj does nothing to myList's elements.
It's different with the Perl for my $obj (#myList) {}
You are confused. Consider your first snippet:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print a
obj is not some kind of magical pointer into the list. It is a variable which holds a reference to an object which happens to also be in myList. obj += 1 has the effect of increasing the value stored in obj. Your code then does nothing with that value.
To be clear: There are no copies in this code example. obj is a variable, which holds an object in the list. That is all.
In the first example the integer is copied into obj which is increased by 1.
The list is not changed.
If you would use a class instance and perform operations on it, it would be changed.
Modification in list is allowed. Your code examples arbove are pretty garbled...
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
This works.