Python: identical items in list are treated as the same - python

I have a nested list in python. Each item the second list is also a nested list. My goal is to duplicate one list, insert it right at the same index, and then modify each one. So, example of start condition:
myList = [[first_list], [[element_1], [element_2, element_3], [duplicate_me]]]
Duplication/insertion at myList[1][2]:
myList = [[first_list], [[element_1], [element_2, element_3], [duplicate_me], [duplicate_me]]]
This all works fine. However, when I run the append code:
myList[1][2].append(new_element)
It appends the new element to both duplicates, like this:
myList = [[first_list], [[element_1], [element_2, element_3], [duplicate_me, new_element], [duplicate_me, new_element]]]
Is there something bizarre going on in the way the elements are called or indexed? I see a potential workaround (calling the item to be duplicated to a working variable, modifying it there, and then inserting it at the same point), but that seems needlessly complex.
Thanks!

myList[1][2] and myList[1][3] don't just have the same values, they are actually the same list. You are looking at the same area in memory when you read both of them. So, when you change one, you are changing the other, because both are actually the exact same thing! Instead doing what you do to duplicate the list, you should make a copy of it.
Here's an example of the problem from a python shell:
>>> mylist = [1, 2, 3]
>>> newlist = mylist
>>> newlist.append(4)
>>> newlist
[1, 2, 3, 4]
>>> mylist
[1, 2, 3, 4]
A common way to fix this is to use one of these tricks:
>>> mylist = [1, 2, 3]
>>> newlist = mylist[:] # OR :
>>> newlist = [x for x in mylist]
>>> newlist.append(4)
>>> newlist
[1, 2, 3, 4]
>>> mylist
[1, 2, 3]
The second and third lines will create a copy of the list. I tend to combine the two like this, if I need to copy a 2D list:
>>> newlist = [x[:] for x in mylist]
Use one of these to create your duplicate, and all will be well again.

You are most likely not duplicating the target list (*[duplicate_me]*), but rather simply appending a duplicated reference to the same list into myList.
You need to copy the list before adding it to myList. One simple way is to call the list constructor passing in the original [duplicate_me]

Related

How to get a flat list while avoiding to make a nested list in the first place?

My goal
My question is about a list comprehension that does not puts elements in the resulting list as they are (which would results in a nested list), but extends the results into a flat list. So my question is not about flattening a nested list, but how to get a flat list while avoiding to make a nested list in the first place.
Example
Consider a have class instances with attributes that contains a list of integers:
class Foo:
def __init__(self, l):
self.l = l
foo_0 = Foo([1, 2, 3])
foo_1 = Foo([4, 5])
list_of_foos = [foo_0, foo_1]
Now I want to have a list of all integers in all instances of Foo. My best solution using extend is:
result = []
for f in list_of_foos:
result.extend(f.l)
As desired, result is now [1, 2, 3, 4, 5].
Is there something better? For example list comprehensions?
Since I expect list comprehension to be faster, I'm looking for pythonic way get the desired result with a list comprehension. My best approach is to get a list of lists ('nested list') and flatten this list again - which seems quirky:
result = [item for sublist in [f.l for f in list_of_foos] for item in sublist]
What functionaly I'm looking for
result = some_module.list_extends(f.l for f in list_of_foos)
Questions and Answers I read before
I was quite sure there is an answer to this problem, but during my search, I only found list.extend and list comprehension where the reason why a nested list occurs is different; and python list comprehensions; compressing a list of lists? where the answers are about avoiding the nested list, or how to flatten it.
You can use multiple fors in a single comprehension:
result = [
n
for foo in list_of_foos
for n in foo.l
]
Note that the order of fors is from the outside in -- same as if you wrote a nested for-loop:
for foo in list_of_foos:
for n in foo.l:
print(n)
If you want to combine multiple lists, as if they were all one list, I'd immediately think of itertools.chain. However, you have to access an attribute on each item, so we're also going to need operator.attrgetter. To get those together, I used map and itertools.chain.from_iterable()
https://docs.python.org/3/library/itertools.html#itertools.chain.from_iterable
from itertools import chain
from operator import attrgetter
class Foo:
def __init__(self, l):
self.l = l
foo_0 = Foo([1, 2, 3])
foo_1 = Foo([4, 5])
list_of_foos = [foo_0, foo_1]
for item in chain.from_iterable(map(attrgetter('l'), list_of_foos)):
print(item)
That demonstrates iterating through iterators with chain, as if they were one. If you don't specifically need to keep the list around, don't. But in case you do, here is the comprehension:
final = [item for item in chain.from_iterable(map(attrgetter('l'), list_of_foos))]
print(final)
[1, 2, 3, 4, 5]
In a list, you can make good use to + operator to concatenate two or more list together. It acts like an extend function to your list.
foo_0.l + foo_1.l
Out[7]: [1, 2, 3, 4, 5]
or you can use sum to perform this operation
sum([foo_0.l, foo_1.l], [])
Out[15]: [1, 2, 3, 4, 5]
In fact, it's in one of the post you have read ;)

Python list comprehension and do action

Is it possible to do an action with the item in a list comprehension?
Example:
list = [1, 2, 3]
list_filtered = [ i for i in list if i == 3 AND DO SOMETHING WITH I]
print (list_filtered)
For example if I want to remove the '3' how would I go about it? Logic says that it's something like:
list = [1, 2, 3]
list_filtered = [ i for i in list if i == 3 && list.remove(i) ]
print (list_filtered)
I can't seem to make Python perform an action with that 'i' with any syntax that I tried. Can someone please elaborate?
EDIT: Sorry, the explanation might not be clear enough. I know how to iterate and create the new list. I want to create "list_filtered" and remove that value from "list" if it fits the "IF" statement.
Practically I need the following:
list = [1, 2, 3]
list_filtered = [ i for i in list if i == 3 && list.remove(i) ]
print (list_filtered)
# output >> [3]
print (list)
# output >> [1, 2]
I hope the above makes it more clear. Also, please note that my question is if this can be done in the list comprehension specifically. I know how to do it with additional code. :)
EDIT2: Apparently what I wanted to do isn't possible and also isn't advisable (which is the reason it isn't possible). It seemed like a logical thing that I just didn't know how to do. Thanks guys :)
If you're just trying to remove 3 you could do:
list_filtered=[i for i in list if i != 3]
print(list_filtered) # [1,2]
This will remove all values that are not equal to 3.
Alternatively if you wanted to do something like increment all the items in the list you would do:
[i+1 for i in list]
>>> [2,3,4]
Using a function on every item of the list would look like:
[float(i) for i in list]
>>> [1.0, 2.0, 3.0]
You can do ternary statements:
[i if i<3 else None for i in list]
>>>[1, 2, None]
and a whole lot more...
Here is more documentation on list comprehensions.
Given your new updates, I would try something like:
list_filtered=[list.pop(list.index(3))]
Then list_filtered would be [3] and list would be [1,2] as your specified.
Your miss understand the purpose of list comprehension. List comprehension should be used to create a list, not to use its side effects. Further more, as Leijot has already mentioned, you should never modify a container while iterating over it.
If you want to filter out certain elements in a list using list comprehension, use an if statement:
>>> l = [1, 2, 3]
>>> l_filtered = [i for i in l if i != 3]
>>> l_filtered
[1, 2]
>>>
Alternatively, you can use the builtin function filter():
>>> l = [1, 2, 3]
>>> l_filtered = list(filter(lambda x: x != 3, l))
>>> l_filtered
[1, 2]
>>>
Edit: Based on your most recent edit, what your asking is absolutely possible. Just use two different list comprehensions:
>>> l = [1, 2, 3]
>>> l_filtered = [i for i in l if i == 3]
>>> l_filtered
[3]
>>> l = [i for i in l if i != 3] # reassigning the value of l to a new list
>>> l
[1, 2]
>>>
Maybe you want to provide a different example of what you're trying to do since you shouldn't remove elements from a list while you're iterating over it. Since list comprehensions create another list anyway, you could accomplish the same thing as your example by building a list that omits certain elements. If your goal is to create a list that excludes the value 3, to slightly modify your example:
l = [1, 2, 3]
list_filtered = [ i for i in l if i != 3 ]
l remains unaltered, but list_filtered now contains [1, 2].

How to delete a fixed number of items from end of list in python

If you have a list
myList = [1,2,3,4,5,6,7,8,9,0]
Is there a pythonic way to delete a defined number of items from the end of a list.
EG (pseudocode):
removeFromend(myList, 3)
print myList
>>>[1,2,3,4,5,6,7]
You can use list slicing, which I think is the most pythonic way of doing it:
end_trimm = 3
myList = myList[:-end_trimm]
If you want to mutate the list, setting a slice to an empty list is equivalent to deleting those indices.
myList = [1,2,3,4,5,6,7,8,9,0]
myList[-3:] = []
myList
Out[16]: [1, 2, 3, 4, 5, 6, 7]
This works in cases where you can't simply rebind myList to a new list, e.g. you pass your list to a function and want that function to mutate your list.
deling the slice is the direct approach
>>> myList = [1,2,3,4,5,6,7,8,9,0]
>>> del myList[-3:]
>>> myList
[1, 2, 3, 4, 5, 6, 7]
>>>
The -3 means the slice starts 3 from the end, So the general form is del myList[-n:]

Why is list.remove only removing every second item? [duplicate]

This question already has answers here:
Strange result when removing item from a list while iterating over it
(8 answers)
Closed 7 months ago.
In my Python 2.7.2 IDLE interpreter:
>>> mylist = [1, 2, 3, 4, 5]
>>> for item in mylist:
mylist.remove(item)
>>> mylist
[2, 4]
Why?
It's because when you iterate over a list, python keeps track of the index in the list. Consider the following code instead:
for i in range(len(mylist)):
if i >= len(mylist):
break
item = mylist[i]
mylist.remove(item)
If we track this (which is essentially what python is doing in your code), then we see that when we remove an item in the list, the number to the right shifts one position to the left to fill the void left when we removed the item. The right item is now at index i and so it will never actually get seen in the iteration because the next thing that happens is we increment i for the next iteration of the for loop.
Now for something a little clever. If instead we iterate over the list backward, we'll clear out the list:
for item in reversed(mylist):
mylist.remove(item)
The reason here is that we're taking an item off the end of the list at each iteration of the for loop. Since we're always taking items off the end, nothing needs to shift (assuming uniqueness in the list -- If the list isn't unique, the result is the same, but the argument gets a bit more complicated).
Of course, If you're looking to remove all the items from a list, you can do that really easily:
del mylist[:]
or even with slice assignment:
mylist[:] = []
(I mention the latter because it can be useful to replace segments of a list with other items which don't even need to be the same length).
That's because you're modifying the list while iterating over it, iterate over a shallow copy instead:
>>> mylist = [1, 2, 3, 4, 5]
>>> for item in mylist[:]: #use mylist[:] or list(mylist)
mylist.remove(item)
...
>>> mylist
[]
You are modifying your list while you are looping through it, which is very bad practice.
Problem is that you are altering the list while iterating on it. Use a list comprehension instead:
mylist = [1, 2, 3, 4, 5]
mylist = [x for x in mylist if condition(x)]
>>> mylist = [1, 2, 3, 4, 5]
>>> for item in mylist:
mylist.remove(item)
in this loop :
1st time it will take 0th element (1) as an item and remove it from mylist.so after that mylist will be [2,3,4,5] but the pointer will be in the 1th element of the new mylist ([2,3,4,5]) that is 3. and it will remove 3.so 2 will not be remove from the list.
thats why after the full operation [2,4] will be left.
using for loop:-
>>>for i in range(len(mylist)):
mylist.remove(mylist[0])
i-=1
>>>mylist
[]
you can do this using while loop:
>>>mylist = [1, 2, 3, 4, 5]
>>>while (len(mylist)>0):
mylist.remove(mylist[0])
print mylist
Use a deep copy of mylist
mylist = [1, 2, 3, 4, 5]
l = [k for k in mylist] # deep copy
for i in range(len(mylist)):
mylist.remove(l[i])
print
print mylist

iterating through a list removing items, some items are not removed

I'm trying to transfer the contents of one list to another, but it's not working and I don't know why not. My code looks like this:
list1 = [1, 2, 3, 4, 5, 6]
list2 = []
for item in list1:
list2.append(item)
list1.remove(item)
But if I run it my output looks like this:
>>> list1
[2, 4, 6]
>>> list2
[1, 3, 5]
My question is threefold, I guess: Why is this happening, how do I make it work, and am I overlooking an incredibly simple solution like a 'move' statement or something?
The reason is that you're (appending and) removing from the first list whereby it gets smaller. So the iterator stops before the whole list could be walked through.
To achieve what you want, do this:
list1 = [1, 2, 3, 4, 5, 6]
list2 = []
# You couldn't just make 'list1_copy = list1',
# because this would just copy (share) the reference.
# (i.e. when you change list1_copy, list1 will also change)
# this will make a (new) copy of list1
# so you can happily iterate over it ( without anything getting lost :)
list1_copy = list1[:]
for item in list1_copy:
list2.append(item)
list1.remove(item)
The list1[start:end:step] is the slicing syntax: when you leave start empty it defaults to 0, when you leave end empty it is the highest possible value. So list1[:] means everything in it. (thanks to Wallacoloo)
Like some dudes said, you could also use the extend-method of the list-object to just copy the one list to another, if this was your intention. (But I choosed the way above, because this is near to your approach.)
As you are new to python, I have something for you: Dive Into Python 3 - it's free and easy. - Have fun!
You're deleting items from list1 while you're iterating over it.
That's asking for trouble.
Try this:
>>> list1 = [1,2,3,4,5,6]
>>> list2 = []
>>> list2 = list1[:] # we copy every element from list1 using a slice
>>> del list1[:] # we delete every element from list1
Essential debugging skill: Add print statements. (or print functions in Python 3)
>>> list1= [1, 2, 3, 4, 5, 6]
>>> for item in list1:
... print item
... list1.remove(item)
... print list1
...
1
[2, 3, 4, 5, 6]
3
[2, 4, 5, 6]
5
[2, 4, 6]
Notice that Python is trying to step through the positions of the list, but you keep removing items from the list, making the positions become meaningless.
Python picks the item at position 0 from the list.
You then remove the item, changing the list.
Python then picks the item at position 1 from the list (appearing to skip an item)
You then remove that item, changing the list.
Python then picks the item at position 2 from the list (appearing to skip an item)
You then remove the item, changing the list.
Python would then like to pick the item at position 3, but there's no such item. So the loop stops.
You shouldn't modify a list while you iterate over it. This causes the iterator to point at the wrong item. After the first item is handled, the iterator is pointing at index = 1, but because you've removed an item, the next item is now at index zero, so it will be skipped. This is why you are only handling every other item.
Try:
list2.extend(list1) # This appends all items from list1 to list2.
del list1[:] # From ChristopheD's post.
Exactly as ChristopheD says.
Could do this:
list1 = [1, 2, 3, 4, 5, 6]
list2 = []
for item in list1:
list2.append(item)
list1 = []
That'll clear list1.
Edit He/she has updated their post. I'll leave this up as a slight alternative.
Try this instead:
list1 = [1, 2, 3, 4, 5, 6]
list2 = []
list2.extend(list1)
list1[:] = []
There is also the humble copy function (or deepcopy if you have complex objects and not just integers in your list):
from copy import copy
list2 = copy(list1)
You might get a more appropriate answer if you explain what you're trying to accomplish (unless you're just learning about python lists).
"Variables" in Python are just names/references, so the destructive copy you seem to want to do seems kind of strange. If you want list2 to have the same values as list1 you can just do:
list2 = list1 # now they are both referring to the same list
And if after that you want to use list1 for something else, you can just do:
list1 = ['A', 'B', 'C']
Using a list comprehension:
list2 = [item for item in list1]
Bind the name list2 to the the same object as list1:
list2 = list1
(Note that if you modify the contents of list1, list2 will be change accordingly.)
Create a copy of list1 and bind it to the name list2:
list2 = list1[:]
In this case list1 and list2 are different objects.
As you can see in other answers, you are trying to modify the list while iterating over it. This doesn't work. There are many ways to copy one list to another. I did some tests to see how fast each approach is:
>>> timeit.Timer('list2 = list1[:]', 'list1 = range(10**3)').timeit(10**6)
3.9134418964385986
>>> timeit.Timer('list2 = []; list2.extend(list1)', 'list1 = range(10**3)').timeit(10**6)
4.9082601070404053
>>> timeit.Timer('list2 = copy.copy(list1)', 'import copy; list1 = range(10**3)').timeit(10**6)
7.5023419857025146
>>> timeit.Timer('list2 = [i for i in list1]', 'list1 = range(10**3)').timeit(10**6)
95.697894811630249
The slice syntax is the fastest. It's much faster than using a list comprehension.
To clear a list, you can use:
del list1[:]
#potatocubed: Many of the answers given solve the trivial example you gave ("move list1 to list2"), but don't really explain the "why" of the deeper problem, which is modifying a list as you iterate over it. Study S.Lott's answer...

Categories