Related
I'm at the very beginning of learning Python 3. Getting to know the language basics. There is a method to the list data type:
list.append(x)
and in the tutorial it is said to be equivalent to this expression:
a[len(a):] = [x]
Can someone please explain this expression? I can't grasp the len(a): part. It's a slice right? From the last item to the last? Can't make sense of it.
I'm aware this is very newbie, sorry. I'm determined to learn Python for Blender scripting and the Game Engine, and want to understand well all the constructs.
Think back to how slices work: a[beginning:end].
If you do not supply one of them, then you get all the list from beginning or all the way to end.
What that means is if I ask for a[2:], I will get the list from the index 2 all the way to the end of the list and len(a) is an index right after the last element of the array... so a[len(a):] is basically an empty array positioned right after the last element of the array.
Say you have a = [0,1,2], and you do a[3:] = [3,4,5], what you're telling Python is that right after [0,1,2 and right before ], there should be 3,4,5.
Thus a will become [0,1,2,3,4,5] and after that step a[3:] will indeed be equal to [3,4,5] just as you declared.
Edit: as chepner commented, any index greater than or equal to len(a) will work just as well. For instance, a = [0,1,2] and a[42:] = [3,4,5] will also result in a becoming [0,1,2,3,4,5].
One could generally state that l[len(l):] = [1] is similar to append, and that is what is stated in the docs, but, that is a special case that holds true only when the right hand side has a single element.
In the more general case it is safer to state that it is equivalent to extend for the following reasons:
Append takes an object and appends that to the end; with slice assignment you extend a list with the given iterable on the right hand side:
l[len(l):] = [1, 2, 3]
is equivalent to:
l.extend([1, 2, 3])
The same argument to append would cause [1, 2, 3] to be appended as an object at the end of l. In this scenario len(l) is simply used in order for the extending of the list to be performed at the end of l.
Some examples to illustrate their difference:
l = [1, 2]
l[len(l):] = [1, 2] # l becomes [1, 2, 1, 2]
l.extend([1, 2]) # l becomes [1, 2, 1, 2, 1, 2]
l.append([1, 2]) # l becomes [1, 2, 1, 2, 1, 2, [1, 2]]
As you note, l.append(<iterable>) doesn't actually append each value in the iterable, it appends the iterable itself.
As I am new to programming in Python. I am trying to remove particular elements from array using for loop which looks like
a=[2,3,1,4,1,1,1,5]
n=a.count(1)
for i in range (len(a)-n):
if (a[i]==1):
del a[i]
else:
a[i]=a[i]
print (a)
I want to remove 1 from array a. But, I am getting result as:
[2, 3, 4, 1, 1, 5].
That is 1 still exists in my new array. Can somebody please answer my problem?
try like this:
a = [2,3,1,4,1,1,1,5]
a = [x for x in a if x!=1] # this is called list comprehension
note Never modify list while iterating
Use a while loop and the remove method:
a = [2, 3, 1, 4, 1, 1, 1, 5]
while 1 in a:
a.remove(1)
print a
The real answer to your question (which none of the other answers addresses) is that every time you remove an item, the index i moves past it.
in your case:
a = [2,3,1,4,1,1,1,5]
after deleting the 5th item in the original list, the pointer moves to the 6th item, and the new 5th item (the second 1 in the sequence of three 1s) is skipped.
Regarding the comment never modify a list in a loop, try to implement an in-place algorithm like Fisher-Yates without modifying the list. Never say never. Know what you're doing.
The OP changes the list in-place, not creating a new list.
There are two methods, the second is safe, the first might be faster.
a = [2, 3, 1, 4, 1, 1, 1, 5]
toremove = 1
for i in range(len(a)-1, -1, -1):
if a[i] == toremove:
del a[i]
and
a = [2, 3, 1, 4, 1, 1, 1, 5]
toremove = 1
for i in range(a.count(toremove)):
a.remove(toremove)
The second removes the element however many times it exists (before the loop). Since we are not iterating on the list, it is safe to use the remove method.
Both fragments should be O(n) (but haven't done the calculations).
You can copy a and then remove but you cannot iterate over and delete elements from the same list, if your list starts with n elements python will have n pointers to each element so removing elements from the list as your are iterating over it will cause elements to be missed.python has no way of knowing you have removed elements from the list:
a = [2,3,1,4,1,1,1,5]
for ele in a[:]:
if ele == 1:
a.remove(1)
print(a)
[2, 3, 4, 5]
You can also use reversed which returns and iterator avoiding creating a whole copy of the list at once:
a = [2,3,1,4,1,1,1,5]
for ele in reversed(a):
if ele == 1:
a.remove(1)
print(a)
[2, 3, 4, 5]
Or using a list comprehension with the [:] syntax so we actually update the original object:
a[:] = (ele for ele in a if ele != 1)
All the above are linear operations using a single pass over a.
Actually as the del statement will remove elements from your list , and as the list that you bound in your loop doesn't been update after the first deleting you remove incorrect elements from your list , so if you want to use del you need to make the list name in your loop to reference to new list , that you can use a function for this aim , but as a more python way you can just use a list comprehension:
>>> a=[2,3,1,4,1,1,1,5]
>>> a=[i for i in a if i !=1]
>>> a
[2, 3, 4, 5]
Or you can use filter :
>>> a=[2,3,1,4,1,1,1,5]
>>> a=filter(lambda x: x !=1,a)
>>> a
[2, 3, 4, 5]
I understand that in Python regular c++ style variable assignment is replaced by references to stuff ie
a=[1,2,3]
b=a
a.append(4)
print(b) #gives [1,2,3,4]
print(a) #gives [1,2,3,4]
but I'm still confused why an analogous situation with basic types eg. integers works differently?
a=1
b=a
a+=1
print(b) # gives 1
print(a) # gives 2
But wait, it gets even more confusing when we consider loops!
li=[1,2,3]
for x in li:
x+=1
print(li) #gives [1,2,3]
Which is what I expected, but what happens if we do:
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #gives [1,2,3]
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there. The only thing I can come up short of using
for x in range(len(li)):
Do stuff to li[x]
is packaging the integers in one element list. But there must be a better way.
Well, you need to think of mutable and immutable type.
For a list, it's mutable.
For a integer, it's immutable, which means you will refer to a new object if you change it. When a+=1 is executed, a will be assigned a new object, but b is still refer to the same one.
a=[1,2,3]
b=a
a.append(4)
print(b) #[1,2,3,4]
print(a) #[1,2,3,4]
Here you are modifying the list. The list content changes, but the list identity remains.
a=1
b=a
a+=1
This, however, is a reassignment. You assign a different object to a.
Note that if you did a += [4] in the 1st example, you would have seen the same result. This comes from the fact that a += something is the same as a = a.__iadd__(something), with a fallback to a = a.__add__(something) if __iadd__() doesn't exist.
The difference is that __iadd__() tries to do its job "inplace", by modifying the object it works on and returning it. So a refers to the same as before. This only works with mutable objects such as lists.
On immutable objects such as ints __add__() is called. It returns a different object, which leads to a pointing to another object than before. There is no other choice, as ints are immutable.
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #[1,2,3]
Here x += 1 means the same as x = x + 1. It changes where x refers to, but not the list contents.
Maybe my question should be how to loop over a list of integers and change them without >map() as i need a if statement in there.
for i, x in enumerate(li):
li[i] = x + 1
assigns to every list position the old value + 1.
The important thing here are the variable names. They really are just keys to a dictionary. They are resolved at runtime, depending on the current scope.
Let's have a look what names you access in your code. The locals function helps us: It shows the names in the local scope (and their value). Here's your code, with some debugging output:
a = [1, 2, 3] # a is bound
print(locals())
for x in a: # a is read, and for each iteration x is bound
x = x + 3 # x is read, the value increased and then bound to x again
print(locals())
print(locals())
print(x)
(Note I expanded x += 3 to x = x + 3 to increase visibility for the name accesses - read and write.)
First, you bind the list [1, 2, 3]to the name a. Then, you iterate over the list. During each iteration, the value is bound to the name x in the current scope. Your assignment then assigns another value to x.
Here's the output
{'a': [1, 2, 3]}
{'a': [1, 2, 3], 'x': 4}
{'a': [1, 2, 3], 'x': 5}
{'a': [1, 2, 3], 'x': 6}
{'a': [1, 2, 3], 'x': 6}
6
At no point you're accessing a, the list, and thus will never modify it.
To fix your problem, I'd use the enumerate function to get the index along with the value and then access the list using the name a to change it.
for idx, x in enumerate(a):
a[idx] = x + 3
print(a)
Output:
[4, 5, 6]
Note you might want to wrap those examples in a function, to avoid the cluttered global namespace.
For more about scopes, read the chapter in the Python tutorial. To further investigate that, use the globals function to see the names of the global namespace. (Not to be confused with the global keyword, note the missing 's'.)
Have fun!
For a C++-head it easiest tho think that every Python object is a pointer. When you write a = [1, 2, 3] you essentially write List * a = new List(1, 2, 3). When you write a = b, you essentially write List * b = a.
But when you take out actual items from the lists, these items happen to be numbers. Numbers are immutable; holding a pointer to an immutable object is about as good as holding this object by value.
So your for x in a: x += 1 is essentially
for (int x, it = a.iterator(); it->hasMore(); x=it.next()) {
x+=1; // the generated sum is silently discarded
}
which obviously has no effect.
If list elements were mutable objects you could mutate them exactly the way you wrote. See:
a = [[1], [2], [3]] # list of lists
for x in a: # x iterates over each sub-list
x.append(10)
print a # prints [[1, 10], [2, 10], [3, 10]]
But unless you have a compelling reason (e.g. a list of millions of objects under heavy memory load) you are better off making a copy of the list, applying a transformation and optionally a filter. This is easily done with a list comprehension:
a = [1, 2, 3, 0]
b = [n + 1 for n in a] # [2, 3, 4, 1]
c = [n * 10 for n in a if n < 3] # [10, 20, 0]
Either that, or you can write an explicit loop that creates another list:
source = [1, 2, 3]
target = []
for n in source:
n1 = <many lines of code involving n>
target.append(n1)
Your question has multiple parts, so it's going to be hard for one answer to cover all of them. glglgl has done a great job on most of it, but your final question is still unexplained:
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there
"I need an if statement in there" doesn't mean you can't use map.
First, if you want the if to select which values you want to keep, map has a good friend named filter that does exactly that. For example, to keep only the odd numbers, but add one to each of them, you could do this:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> for x in a:
... if x%2:
... b.append(x+1)
Or just this:
>>> b = map(lambda x: x+1, filter(lambda x: x%2, a))
If, on the other hand, you want the if to control the expression itself—e.g., to add 1 to the odd numbers but leave the even ones alone, you can use an if expression the same way you'd use an if statement:
>>> for x in a:
... if x%2:
... b.append(x+1)
... else:
... b.append(x)
>>> b = map(lambda x: x+1 if x%2 else x, a)
Second, comprehensions are basically equivalent to map and filter, but with expressions instead of functions. If your expression would just be "call this function", then use map or filter. If your function would just be a lambda to "evaluate this expression", then use a comprehension. The above two examples get more readable this way:
>>> b = [x+1 for x in a if x%2]
>>> b = [x+1 if x%2 else x for x in a]
You can do something like this: li = [x+1 for x in li]
Tried deleting items in a list, no success.
>>> r = [1,2,3,4,5]
>>> for i in r:
if i<3:
del i
>>> print r
[1, 2, 3, 4, 5]
I even tried filtering it,
>>> def f(i):
True if i>2 else False
>>> print list(filter(f,r))
[]
I do not understand why the first one is not working. And I dont understand the result at all, when I use filter(function,iterable).
EDIT:
Seeing Paulo's comment below, now I do not understand why this works.
>>> for i in r:
if i<3:
r.remove(i)
>>> print r
[3, 4, 5]
Shouldn't the iterator problem be still there, and shouldn't the code end up removing only the first element (r[0])
Use a list comprehension instead:
[i for i in r if i >= 3]
and retain instead of delete.
Your filter never returned the test; so you always return None instead and that's false in a boolean context. The following works just fine:
def f(i):
return i > 2
Your initial attempt failed because del i unbinds i, but the list remains unaffected. Only the local name i is cleared.
If you want to delete an item from a list, you need to delete the index:
del r[0]
deletes the first element from the list.
Even if you did manage to delete indices the loop would have held some suprises:
>>> for i, element in enumerate(r):
... if element < 3:
... del r[i]
...
>>> r
[2, 3, 4, 5]
This version fails because the list iterator used by the for loop doesn't know you deleted elements from the list; deleting the value at index 0 shifts up the rest of the list, but the loop iterator looks at item 1 regardless:
first iteration, r = [1, 2, 3, 4, 5], iterator index 0 -> element = 1
second iteration, r = [2, 3, 4, 5], iterator index 1 -> element = 3
I do not understand why the first one is not working.
It is not working because the statement del i undefines the variable i - that is, it deletes it from the scope (global or local) which contains it.
And I dont understand the result at all, when I use filter(function,iterable)
Your function, f does not contain a return statement. Accordingly, it always returns None, which has the boolean equivalent value of False. Thus, filter excludes all values.
What you should probably be doing is filtering using a comprehension, and replacing the list, like so:
r = [i for i in r if i >= 3]
Or, if you really do want to delete part of the original list and modify it, use del on a slice of the list:
del r[:3]
Seeing Paulo's comment below, now I do not understand why [using remove] works.
Because remove(r) searches for the value r in the list, and deletes the first instance of it. Accordingly, repeated modification of the list does not affect the iteration that happens inside remove. However, note that it is still susceptible to the same error, if removal of an item leads to an item being skipped in iteration of the list.
Simple example:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print myList
prints
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]
while:
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
prints
[1, 2, 3, 4, 5]
[2, 3, 4, 5, 6]
Conclusion:
Lists can be modified in place using global list access Lists can
List items can NOT be modified in place using the iterator object
All example code I can find uses the global list accessors to modify the list inplace.
Is it so evil to modify a list iterator?
The reason obj += 1 does not do what you expect is that this statement does not modify obj in-place. Instead, it computes the new value, and rebinds the variable obj to point to the new value. This means that the contents of the list remain unchanged.
In general it is possible to modify the list while iterating over it using for obj in myList. For example:
myList = [[1], [2], [3], [4], [5]]
for obj in myList:
obj[0] += 1
print(myList)
This prints out:
[[2], [3], [4], [5], [6]]
The difference between this and your first example is that here, the list contains mutable objects, and the code modifies those objects in-place.
Note that one could also write the loop using a list comprehension:
myList = [val+1 for val in myList]
I think you've misunderstood what an "iterator object" is. A for loop is not an iterator object. For all intents and purposes, a for loop like this:
myList = [0, 1, 2, 3, 4]
for x in myList:
print x
does this (but more efficiently and less verbosely):
i = 0
while i < len(myList)
x = myList[i]
print x
i += 1
So you see, any changes made to x are lost as soon as the next loop starts, because the value of x is overwritten by the value of the next item in the list.
As others have observed, it is possible to alter the value of a list while iterating over it. (But don't change its length! That's where you get into trouble.) One elegant way to do so is as follows:
for i, x in enumerate(myList):
myList[i] = some_func(x)
Update: It's also important to understand that no copying goes on in a for loop. In the above example, i and x -- like all variables in Python -- are more like pointers in C/C++. As the for loop progresses, obj points at myList[0], myList[1], etc, in turn. And like a C/C++ pointer, the properties of the object pointed to are not changed when the pointer is changed. But also like a C pointer, you can directly modify the thing pointed at, because it's not a copy. In C, this is done by dereferencing the pointer; in Python, this is done by using a mutable object. That's why NPE's answer works. If i and x were even shallow copies, it wouldn't be possible to do what he does.
The reason you can't directly change ints the way you can change lists (as in NPE's answer), is that ints aren't mutable. Once a 5 object is created, nothing can change its value. That's why passing around a pointer to 5 is safe in Python -- no side-effects can occur, because the thing pointed to is immutable.
in for obj in myList:, in every iteration, obj is a (shallow) copy of the element in myList. So the change on the obj does nothing to myList's elements.
It's different with the Perl for my $obj (#myList) {}
You are confused. Consider your first snippet:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print a
obj is not some kind of magical pointer into the list. It is a variable which holds a reference to an object which happens to also be in myList. obj += 1 has the effect of increasing the value stored in obj. Your code then does nothing with that value.
To be clear: There are no copies in this code example. obj is a variable, which holds an object in the list. That is all.
In the first example the integer is copied into obj which is increased by 1.
The list is not changed.
If you would use a class instance and perform operations on it, it would be changed.
Modification in list is allowed. Your code examples arbove are pretty garbled...
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
This works.