I've a list of lists and I want to update each list at iteration.
I initialized my list as follows:
my_list = [[0]*n]*n
When I want to update the inner lists, by something like:
for i in range(something):
for j in range(anotherthing):
my_list[i][j] = something
What happens is that the whole list is updated rather than the ith list only at each iteration, e.g. [[1,2], [1,2]].
What I want is at the first iteration to be [[1,2], [0,0]] as I initialized it, and in to be [[1,2], [values]]. What am I doing wrong?
The list multiplication operator * does not create copies, but creates multiple references to the same data. Instead of using * consider using comprehensions for initializing your list:
my_list = [[0 for i in range(n)] for j in range(n)]
Use this to initialize your list of lists and it will work fine.
x = [[0]*n for i in range(n)]
The original code creates a list of sublists too but every single one of the sublists reference the same object.
Note: I am using Python3. If you are using 2, you might need to use xrange() instead of range().
Related
Background (not the subject of the question):
I am trying to solve the problem of generating all paths of length 3 from a list of all paths of length two in a list (p2 in the code below).
To that end I want to store all pairs of indices (i,j) for which p2[i][1]==p2[j][0] and `p2[i][2]==p2[j][1] (a path of length 2 is defined by a sequence of 3 vertices)
Question:
I tried store the indices in 2D lists whose entries are initially empty list that are appended newly encountered indices, but what is generated is a 2D list whose entries are lists that contain all indices.
Can someone explain what is actually going on behind the curtains and how to correctly add the indices to the innermost lists?
# -*- coding: utf-8 -*-
"""
Created on Sun May 29 10:10:40 2022
#author: Manfred Weis
"""
# generare all unsorted 3-tuples
from itertools import permutations
n=4
p2 = list(permutations(range(n),3))
print('p2:')
print(p2)
print()
N = len(p2)
Rows = [[[]]*n]*n # create 3D list
Cols = [[[]]*n]*n # create 3D list
for i in range(N):
print(str(i)+': Rows['+str(p2[i][1])+']['+str(p2[i][2])+']')
print(str(i)+': Cols['+str(p2[i][0])+']['+str(p2[i][1])+']')
(Rows[p2[i][1]][p2[i][2]]).append(i) # add to the most nested list
(Cols[p2[i][0]][p2[i][1]]).append(i) # add to the most nested list
for i in range(n):
for j in range(n):
print(str(i)+','+str(j)+':')
print(str(Rows[i][j]))
print(str(Cols[i][j]))
print()
You are having a shallow copy problem when you initialize your Rows and Cols 3D lists. Instead of using multiplication, you need to initialize them like so:
for k in range(n):
Rows.append([])
Cols.append([])
for l in range(n):
Rows[k].append([])
Cols[k].append([])
The problem with doing multiplication in order to initialize the 3D list of lists is that all n lists inside are all pointing to the same list, so when you append to one of them, you append to all of them, since they are technically only shallow copies of each other. Therefore, when you print them at the end, all the lists have all the numbers added to them, since they all point to the same place in memory. By initializing the 3D list using the method above, you are creating separate lists in separate locations in memory and you are inserting each of them into your Rows and Cols 3D lists.
Check out what happens when you do the following:
x = [[]]*2
x[0].append(5)
Output:
[[5], [5]]
But if you do the following, you solve the problem:
x = [[],[]]
x[0].append(5)
Output:
[[5], []]
Here are some other people's explanations which are better than mine:
List of lists changes reflected across sublists unexpectedly
Edit:
Here is a better more pythonic way to initialize Rows and Cols:
Rows = [[ [] for j in range(n)] for i in range(n)]
Cols = [[ [] for j in range(n)] for i in range(n)]
I would like to know if there's any way to remove an index in a list within a list. For example, given [[1,2,3],[4,5,6],[7,8,9]], I would like to remove the the first element of each list so that it becomes [[2,3],[5,6],[8,9]]. I can do a for loop to slowly remove them but I was wondering if there's a more efficient manner to do so?
You can do it with iterating over the list, and then call the pop() function on the inner list. This will remove an element at the specified index.
test_list = [[1,2,3],[4,5,6],[7,8,9]]
for i in test_list:
i.pop(0)
print(test_list)
output: [[2, 3], [5, 6], [8, 9]]
I would say you have two options.
You can use pop():
for item in list:
item.pop(0)
Or recreate a new list :
list2 = [item[1:] for item in list]
For small lists as in your example, doing something like list.pop(0) is OK:
nested = [[1,2,3], [4,5,6], [7,8,9]]
for inner in nested:
inner.pop(0) # Remove first element of each inner list
However, as lists are basically implemented as arrays (of pointers), removing the first element means that all other elements has to be shifted back by one, resulting in a lot of (pointer) copying if your lists are large. Also, using inner[1:] rather than inner.pop(0) does not solve the problem, as inner[1:] creates a new list rather than returning a view.
One way out is to make the redundant elements appear as the last element instead of the first, so that you can do inner.pop() instead, which removes the last element. No further shifting/copying is then required.
Another solution is to switch out your data structure from a list to a deque ("double-ended queue"). This has a popleft() method, which is equivalent to pop(0) but fast, as deques support fast popping from both ends:
import collections
nested = [[1,2,3], [4,5,6], [7,8,9]]
nested = [collections.deque(inner) for inner in nested] # list of deques
for inner in nested:
inner.popleft() # Remove first element of each inner list, fast!
for y in range(len(List)):
print("length",len(List))
print ("y",y)
print("List",List[y])
if (List[y])%(dividing_prime)==0:
print(List[y])
counter=counter+1
List[y]=0
List.remove(0)
You are modifying a list while iterating over it. Because of that, the list is changing in size while you are iterating cause an index out of bounds.
What you could do is create another list from the elements you don't want to remove.
new_list = [x for x in mylist if not remove(x)]
If you want to keep the same list and not create a new copy then you could use the slice operator.
mylist[:] = [x for x in mylist if not remove(x)]
The itertools library also provides a tool to do this called filterfalse
https://docs.python.org/3/library/itertools.html#itertools.filterfalse
The code below seems to be iterating through a 2d list using another list which conceptually doesn't make much sense to me.
What would be the in range equivalent to the code below, using lens as I'm finding it quite difficult to understand.
I've changed the variable names as I'm working on coursework but if its too abstract I can add in the original variable names.
#list2 is a 2d list
#list1 is a normal list
for list1 in list2
for k in range(n) #n and k are constants
#any if statement
A "2D" list is just a list where each element is itself a list. To access each element of the lists inside the "main" list, do
for list1 in list2:
for element in list1:
print(element)
If you want a version using range:
L2 = len(list2)
for i in range(L2):
list1 = list2[i]
L1 = len(list1)
for j in range(L1):
element = list1[j]
print(element)
As should be clear from the above, using range in a for loop is very rarely a good idea, as the code is much less readable.
I've got a list in which some items shall be moved into a separate list (by a comparator function). Those elements are pure dicts. The question is how should I iterate over such list.
When iterating the simplest way, for element in mylist, then I don't know the index of the element. There's no .iteritems() methods for lists, which could be useful here. So I've tried to use for index in range(len(mylist)):, which [1] seems over-complicated as for python and [2] does not satisfy me, since range(len()) is calculated once in the beginning and if I remove an element from the list during iteration, I'll get IndexError: list index out of range.
Finally, my question is - how should I iterate over a python list, to be able to remove elements from the list (using a comparator function and put them in another list)?
You can use enumerate function and make a temporary copy of the list:
for i, value in enumerate(old_list[:]):
# i == index
# value == dictionary
# you can safely remove from old_list because we are iterating over copy
Creating a new list really isn't much of a problem compared to removing items from the old one. Similarly, iterating twice is a very minor performance hit, probably swamped by other factors. Unless you have a very good reason to do otherwise, backed by profiling your code, I'd recommend iterating twice and building two new lists:
from itertools import ifilter, ifilterfalse
l1 = list(ifilter(condition, l))
l2 = list(ifilterfalse(condition, l))
You can slice-assign the contents of one of the new lists into the original if you want:
l[:] = l1
If you're absolutely sure you want a 1-pass solution, and you're absolutely sure you want to modify the original list in place instead of creating a copy, the following avoids quadratic performance hits from popping from the middle of a list:
j = 0
l2 = []
for i in range(len(l)):
if condition(l[i]):
l[j] = l[i]
j += 1
else:
l2.append(l[i])
del l[j:]
We move each element of the list directly to its final position without wasting time shifting elements that don't really need to be shifted. We could use for item in l if we wanted, and it'd probably be a bit faster, but when the algorithm involves modifying the thing we're iterating over, I prefer the explicit index.
I prefer not to touch the original list and do as #Martol1ni, but one way to do it in place and not be affected by the removal of elements would be to iterate backwards:
for i in reversed(range(len()):
# do the filtering...
That will affect only the indices of elements that you have tested/removed already
Try the filter command, and you can override the original list with it too if you don't need it.
def cmp(i): #Comparator function returning a boolean for a given item
...
# mylist is the initial list
mylist = filter(cmp, mylist)
mylist is now a generator of suitable items. You can use list(mylist) if you need to use it more than once.
Haven't tried this yet but.. i'll give it a quick shot:
new_list = [old.pop(i) for i, x in reversed(list(enumerate(old))) if comparator(x)]
You can do this, might be one line too much though.
new_list1 = [x for x in old_list if your_comparator(x)]
new_list2 = [x for x in old_list if x not in new_list1]