issue with iterating in a nested list - python

i am a super beginner in python (took an introductory to python course 4 years ago!) and I am trying to append the numbers 0-3 each in a list inside a list, so:
[[0], [1], [2], [3]]
i have written the code below which should work but it doesn't and i can't figure out why:
l = 4
data = [[None]] * l
for k in range(l):
data[k].append(k)
print(data)
which produces this in the shell (don't worry about the None element in there I know how to get rid of those):
[[None, 0, 1, 2, 3], [None, 0, 1, 2, 3], [None, 0, 1, 2, 3], [None, 0, 1, 2, 3]]
i have been working on this for over an hour but i can't figure out what is wrong and how to fix it. any halp would be highly appriciated

Your data list consists of four references to the same inner list, which is initialized to [None]. As you append to this inner list in your loop, all four references to it are affected.
If you create a new list for each element, you won't have this issue:
l = 4
data = []
for k in range(l):
data.append([k])
print(data)
In the above code, the expression [k] creates a new list that contains only the int value k.
Simpler yet:
l = 4
data = [[k] for k in range(l)]
print(data)

l = 4
data = []
for k in range(l):
data.append([k])
print(data)
None will also stay in the list. You are not removing it. Also, it creates a reference to the same list. so, appending to it will modify all the inner lists.
Instead of appending to an inner list, you can append a list to the list with only one element in it.
Or you can use a list comprehension:
print([[k] for k in range(4)])

Related

how to avoid List assignment index out of range [duplicate]

How do I create an empty list that can hold 10 elements?
After that, I want to assign values in that list. For example:
xs = list()
for i in range(0, 9):
xs[i] = i
However, that gives IndexError: list assignment index out of range. Why?
Editor's note:
In Python, lists do not have a set capacity, but it is not possible to assign to elements that aren't already present. Answers here show code that creates a list with 10 "dummy" elements to replace later. However, most beginners encountering this problem really just want to build a list by adding elements to it. That should be done using the .append method, although there will often be problem-specific ways to create the list more directly. Please see Why does this iterative list-growing code give IndexError: list assignment index out of range? How can I repeatedly add elements to a list? for details.
You cannot assign to a list like xs[i] = value, unless the list already is initialized with at least i+1 elements. Instead, use xs.append(value) to add elements to the end of the list. (Though you could use the assignment notation if you were using a dictionary instead of a list.)
Creating an empty list:
>>> xs = [None] * 10
>>> xs
[None, None, None, None, None, None, None, None, None, None]
Assigning a value to an existing element of the above list:
>>> xs[1] = 5
>>> xs
[None, 5, None, None, None, None, None, None, None, None]
Keep in mind that something like xs[15] = 5 would still fail, as our list has only 10 elements.
range(x) creates a list from [0, 1, 2, ... x-1]
# 2.X only. Use list(range(10)) in 3.X.
>>> xs = range(10)
>>> xs
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Using a function to create a list:
>>> def display():
... xs = []
... for i in range(9): # This is just to tell you how to create a list.
... xs.append(i)
... return xs
...
>>> print display()
[0, 1, 2, 3, 4, 5, 6, 7, 8]
List comprehension (Using the squares because for range you don't need to do all this, you can just return range(0,9) ):
>>> def display():
... return [x**2 for x in range(9)]
...
>>> print display()
[0, 1, 4, 9, 16, 25, 36, 49, 64]
Try this instead:
lst = [None] * 10
The above will create a list of size 10, where each position is initialized to None. After that, you can add elements to it:
lst = [None] * 10
for i in range(10):
lst[i] = i
Admittedly, that's not the Pythonic way to do things. Better do this:
lst = []
for i in range(10):
lst.append(i)
Or even simpler, in Python 2.x you can do this to initialize a list with values from 0 to 9:
lst = range(10)
And in Python 3.x:
lst = list(range(10))
varunl's currently accepted answer
>>> l = [None] * 10
>>> l
[None, None, None, None, None, None, None, None, None, None]
Works well for non-reference types like numbers. Unfortunately if you want to create a list-of-lists you will run into referencing errors. Example in Python 2.7.6:
>>> a = [[]]*10
>>> a
[[], [], [], [], [], [], [], [], [], []]
>>> a[0].append(0)
>>> a
[[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
>>>
As you can see, each element is pointing to the same list object. To get around this, you can create a method that will initialize each position to a different object reference.
def init_list_of_objects(size):
list_of_objects = list()
for i in range(0,size):
list_of_objects.append( list() ) #different object reference each time
return list_of_objects
>>> a = init_list_of_objects(10)
>>> a
[[], [], [], [], [], [], [], [], [], []]
>>> a[0].append(0)
>>> a
[[0], [], [], [], [], [], [], [], [], []]
>>>
There is likely a default, built-in python way of doing this (instead of writing a function), but I'm not sure what it is. Would be happy to be corrected!
Edit: It's [ [] for _ in range(10)]
Example :
>>> [ [random.random() for _ in range(2) ] for _ in range(5)]
>>> [[0.7528051908943816, 0.4325669600055032], [0.510983236521753, 0.7789949902294716], [0.09475179523690558, 0.30216475640534635], [0.3996890132468158, 0.6374322093017013], [0.3374204010027543, 0.4514925173253973]]
There are two "quick" methods:
x = length_of_your_list
a = [None]*x
# or
a = [None for _ in xrange(x)]
It appears that [None]*x is faster:
>>> from timeit import timeit
>>> timeit("[None]*100",number=10000)
0.023542165756225586
>>> timeit("[None for _ in xrange(100)]",number=10000)
0.07616496086120605
But if you are ok with a range (e.g. [0,1,2,3,...,x-1]), then range(x) might be fastest:
>>> timeit("range(100)",number=10000)
0.012513160705566406
You can .append(element) to the list, e.g.:
s1.append(i)
What you are currently trying to do is access an element (s1[i]) that does not exist.
I'm surprised nobody suggest this simple approach to creating a list of empty lists. This is an old thread, but just adding this for completeness. This will create a list of 10 empty lists
x = [[] for i in range(10)]
The accepted answer has some gotchas. For example:
>>> a = [{}] * 3
>>> a
[{}, {}, {}]
>>> a[0]['hello'] = 5
>>> a
[{'hello': 5}, {'hello': 5}, {'hello': 5}]
>>>
So each dictionary refers to the same object. Same holds true if you initialize with arrays or objects.
You could do this instead:
>>> b = [{} for i in range(0, 3)]
>>> b
[{}, {}, {}]
>>> b[0]['hello'] = 6
>>> b
[{'hello': 6}, {}, {}]
>>>
How do I create an empty list that can hold 10 elements?
All lists can hold as many elements as you like, subject only to the limit of available memory. The only "size" of a list that matters is the number of elements currently in it.
However, that gives IndexError: list assignment index out of range. Why?
The first time through the loop, i is equal to 0. Thus, we attempt xs[0] = 0. This does not work because there are currently 0 elements in the list, so 0 is not a valid index.
We cannot use indexing to write list elements that don't already exist - we can only overwrite existing ones. Instead, we should use the .append method:
xs = list();
for i in range(0, 9):
xs.append(i)
The next problem you will note is that your list will actually have only 9 elements, because the end point is skipped by the range function. (As side notes: [] works just as well as list(), the semicolon is unnecessary, and only one parameter is needed for range if you're starting from 0.) Addressing those issues gives:
xs = []
for i in range(10):
xs.append(i)
However, this is still missing the mark - range is not some magical keyword that's part of the language the way for (or, say, def) is.
In 2.x, range is a function, which directly returns the list that we already wanted:
xs = range(10) # 2.x specific!
# In 3.x, we don't get a list; we can do a lot of things with the
# result, but we can't e.g. append or replace elements.
In 3.x, range is a cleverly designed class, and range(10) creates an instance. To get the desired list, we can simply feed it to the list constructor:
xs = list(range(10)) # correct in 3.x, redundant in 2.x
One simple way to create a 2D matrix of size n using nested list comprehensions:
m = [[None for _ in range(n)] for _ in range(n)]
I'm a bit surprised that the easiest way to create an initialised list is not in any of these answers. Just use a generator in the list function:
list(range(9))
Another option is to use numpy for fixed size arrays (of pointers):
> pip install numpy
import numpy as np
a = np.empty(10, dtype=np.object)
a[1] = 2
a[5] = "john"
a[3] = []
If you just want numbers, you can do with numpy:
a = np.arange(10)
Here's my code for 2D list in python which would read no. of rows from the input :
empty = []
row = int(input())
for i in range(row):
temp = list(map(int, input().split()))
empty.append(temp)
for i in empty:
for j in i:
print(j, end=' ')
print('')
A list is always "iterable" and you can always add new elements to it:
insert: list.insert(indexPosition, value)
append: list.append(value)
extend: list.extend(value)
In your case, you had instantiated an empty list of length 0. Therefore, when you try to add any value to the list using the list index (i), it is referring to a location that does not exist. Therefore, you were getting the error "IndexError: list assignment index out of range".
You can try this instead:
s1 = list();
for i in range(0,9):
s1.append(i)
print (s1)
To create a list of size 10(let's say), you can first create an empty array, like np.empty(10) and then convert it to list using arrayName.tolist(). Alternately, you can chain them as well.
**`np.empty(10).tolist()`**
I came across this SO question while searching for a similar problem. I had to build a 2D array and then replace some elements of each list (in 2D array) with elements from a dict.
I then came across this SO question which helped me, maybe this will help other beginners to get around.
The key trick was to initialize the 2D array as an numpy array and then using array[i,j] instead of array[i][j].
For reference this is the piece of code where I had to use this :
nd_array = []
for i in range(30):
nd_array.append(np.zeros(shape = (32,1)))
new_array = []
for i in range(len(lines)):
new_array.append(nd_array)
new_array = np.asarray(new_array)
for i in range(len(lines)):
splits = lines[i].split(' ')
for j in range(len(splits)):
#print(new_array[i][j])
new_array[i,j] = final_embeddings[dictionary[str(splits[j])]-1].reshape(32,1)
Now I know we can use list comprehension but for simplicity sake I am using a nested for loop. Hope this helps others who come across this post.
Not technically a list but similar to a list in terms of functionality and it's a fixed length
from collections import deque
my_deque_size_10 = deque(maxlen=10)
If it's full, ie got 10 items then adding another item results in item #index 0 being discarded. FIFO..but you can also append in either direction.
Used in say
a rolling average of stats
piping a list through it aka sliding a window over a list until you get a match against another deque object.
If you need a list then when full just use list(deque object)
s1 = []
for i in range(11):
s1.append(i)
print s1
To create a list, just use these brackets: "[]"
To add something to a list, use list.append()
Make it more reusable as a function.
def createEmptyList(length,fill=None):
'''
return a (empty) list of a given length
Example:
print createEmptyList(3,-1)
>> [-1, -1, -1]
print createEmptyList(4)
>> [None, None, None, None]
'''
return [fill] * length
This code generates an array that contains 10 random numbers.
import random
numrand=[]
for i in range(0,10):
a = random.randint(1,50)
numrand.append(a)
print(a,i)
print(numrand)

Mysterious duplicate arrays in Python

I am trying to understand why the first for loop does not return what the second for loop is returning. Can anyone explain this clearly to me? My searching did not unearth any results since I don't know what this problem is called.
arr = [1,2,3]
tempArr = []
res = []
for num in range(0,3):
tempArr.append(arr[num])
res.append(tempArr)
print(tempArr, res)
print()
tempArr = []
res = []
for num in range(0,3):
tempArr.append(arr[num])
res.append(list(tempArr))
print(tempArr, res)
Returns:
[1] [[1]]
[1, 2] [[1, 2], [1, 2]]
[1, 2, 3] [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[1] [[1]]
[1, 2] [[1], [1, 2]]
[1, 2, 3] [[1], [1, 2], [1, 2, 3]]
Lists in Python are mutable sequences. This means that you can modify the elements of a list once it is created.
You are creating a list and assigning it to the variable 'tempArr'. Then in the first loop you are appending 'tempArr' to the 'res' list.
Note that because lists are mutable, you are never changing what 'tempArr' is, you only modify its content, so you are adding three times the same list to 'res'. When you modify the 'tempArr' list, you modify all the lists inside 'res' (again, because they are the same list)
An explicit example you can play with is the following:
# Create a new list
tempArr = []
# Create a list containing the initial one (which is empty)
res = [tempArr]
# Now, lets modify the 'tempArr'
tempArr.append("This wasn't here before")
# Make your prediction on what will happen now:
print(res)
Another thing we can do is play with the 'is' comparison, in your code:
arr = [1,2,3]
tempArr = []
res = []
for num in range(0,3):
tempArr.append(arr[num])
res.append(tempArr)
# Check if the element we have appended into 'res' is the 'tempArr' list. The result is always True.
print(tempArr is res[-1])
print(tempArr, res)
# Let's compare the three elements of the 'res' list and check if they are the same list, we see that they are.
print(res[0] is res[1] is res[2])
On the other hand, in the second loop, you are not appending the 'tempArr' list into 'res', you are first creating a new list by calling "list(tempArr)", and then appending this new list into 'res'. Now we can play a little bit with this:
tempArr = []
res = []
for num in range(0,3):
tempArr.append(arr[num])
res.append(list(tempArr))
# Check if the element we have appended into 'res' is the 'tempArr' list.
print(tempArr is res[-1])
# This time the result is False. So we are not appending 'tempArr', but a new list with the same elements.
print(tempArr, res)
# Let's compare again the three elements of the 'res' list and see if they are the same elements:
print(res[0] is res[1] is res[2])
As a consequence, because in the second loop we are creating new lists, when you modify the initial one, this doesn't affect the elements inside the 'res' list.
list(...) makes a copy. Without making a copy, the same reference is appended, thus subsequent changes show up later in earlier elements as well.
To see the references, do a print(list(map(id, res))). If the printed number is the same, then it's the same list object.
Because we use list() to make shallow copy
This means list(tempArr) will now be a new and independent object with the same contents as tempArr

Assigning elements to a lists within a list

I am running Python3.6 and am working with lists which contain other lists within it.
list_array = [[1,0,1,0,2,2],
[1,1,2,0,1,2],
[2,2,2,1,0,1]]
I would like to modify the list called list_array be deleting all the entries with value 2 within the sub lists.
The code I used for this is
for k in list_array:
k = [x for x in k if x!=2]
However, this code doesn't modify list_array.
Why isn't it possible to replace the elements in the lists within list_array this way?
You are creating a new list instead of assigning to the old one. You can fix this by adding an assignment using k[:] =, like this:
for k in list_array:
k[:] = [x for x in k if x!=2]
Your code creates a new list every time, and erase the previous one.
At the last iteration you should get this:
k = [1, 0, 1]
Instead, a list comprehension works fine:
list_array = [[x for x in sublist if x != 2] for sublist in list_array]
Output:
[[1, 0, 1, 0], [1, 1, 0, 1], [1, 0, 1]]
If you want to write it with an explicit for loop, it could be done like this:
new_list_array = list()
for sublist in list_array:
new_list_array.append([x for x in sublist if x != 2])
You are not modifying the old list you can modify it like that
list_array = [[1,0,1,0,2,2],[1,1,2,0,1,2],[2,2,2,1,0,1]]
for k in list_array:
k[:]= [x for x in k if x!=2]
print(list_array)

Python: why appending to a list results in same values

This is my code:
a = []
res = []
for i in range(0, 3):
a.append(i)
res.append(a)
print(res)
The result is:
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
But I want the result to be:
[[0], [0, 1], [0, 1, 2]]
I know the solution is using the shallow copy: res.append(a[:]). But can somebody tell me why?
You appended the same thing (a) to res three times, so it appears 3 times. The fact that you changed the contents of a between each call to append doesn't matter. If each call to append was given its own copy of a, then you'd get the result you expect.
when you append the object "a", python actually appends a pointer that points to the original list "a". In the next iteration of the loop you change the original object, so all of the pointers to that object show the latest state of the object.
you can add print the lists in every iteration to see that in action.
What you want is to create a copy of "a", that would remain unchanged, and append the copy to "res". Like you said, using the syntax a[:] would do the job.
When you append a to the res array, you are appending a pointer to the a variable. Not the 'value' of the a variable. So when you are done, the res array has the 'value' - [a, a, a].
When you shallow copy, you are copying the 'value' of the a variable at that stage of the loop into the res array, giving you a 'value' of - [[0],[0,1],[0,1,2]].
a = [] # (1) list created here
res = []
for i in range(0, 3):
a.append(i) # (2) list modified here
res.append(a) # (3) res is modified here
print(res)
What your code is saying is this:
At (1) a list is created and a refers to this list.
At (2) you modify the list from (1), but the list itself remains at the same memory location and a still refers to this list.
At (3) you just make a copy of a reference and add it to res, and still neither a nor the list at (1) change.
The end results is that res gets 3 copies of the reference to the list at (1).
Here is a side effect:
a[1] = 42
print(res)
Output:
[[0, 42, 2], [0, 42, 2], [0, 42, 2]]
You say that you know that this is the code you are after:
a = []
res = []
for i in range(0, 3):
a.append(i)
res.append(a[:]) # (4) new list created
print(res)
At (4) a new list is created whose contents is the same as the list that a refers to. This new list is not referred to by a, but instead, one of the elements of res has a reference to this new list.
Firstly this means that res holds references to lists, which is why they hang around long enough to be printed. Secondly a still refers to the original list.
Here is a side effect:
a[1] = 42
print(res)
Output:
[0, 42, 2] [[0], [0, 1], [0, 1, 2]]
However, this is not the end of the story if you examine this code:
a = []
res = []
for i in range(0, 3):
a.append([i]) # (5) Create a new list with one element
res.append(a[:]) # (6) Shallow copy as above
print(res)
a[1].append(42)
print(a, res)
Output:
[[[0]], [[0], [1]], [[0], [1], [2]]]
[[0], [1, 42], [2]] [[[0]], [[0], [1, 42]], [[0], [1, 42], [2]]]
This occurs because at (6) there was only a shallow copy made.

Using for loop in Python 3.4 to remove particular element from array

As I am new to programming in Python. I am trying to remove particular elements from array using for loop which looks like
a=[2,3,1,4,1,1,1,5]
n=a.count(1)
for i in range (len(a)-n):
if (a[i]==1):
del a[i]
else:
a[i]=a[i]
print (a)
I want to remove 1 from array a. But, I am getting result as:
[2, 3, 4, 1, 1, 5].
That is 1 still exists in my new array. Can somebody please answer my problem?
try like this:
a = [2,3,1,4,1,1,1,5]
a = [x for x in a if x!=1] # this is called list comprehension
note Never modify list while iterating
Use a while loop and the remove method:
a = [2, 3, 1, 4, 1, 1, 1, 5]
while 1 in a:
a.remove(1)
print a
The real answer to your question (which none of the other answers addresses) is that every time you remove an item, the index i moves past it.
in your case:
a = [2,3,1,4,1,1,1,5]
after deleting the 5th item in the original list, the pointer moves to the 6th item, and the new 5th item (the second 1 in the sequence of three 1s) is skipped.
Regarding the comment never modify a list in a loop, try to implement an in-place algorithm like Fisher-Yates without modifying the list. Never say never. Know what you're doing.
The OP changes the list in-place, not creating a new list.
There are two methods, the second is safe, the first might be faster.
a = [2, 3, 1, 4, 1, 1, 1, 5]
toremove = 1
for i in range(len(a)-1, -1, -1):
if a[i] == toremove:
del a[i]
and
a = [2, 3, 1, 4, 1, 1, 1, 5]
toremove = 1
for i in range(a.count(toremove)):
a.remove(toremove)
The second removes the element however many times it exists (before the loop). Since we are not iterating on the list, it is safe to use the remove method.
Both fragments should be O(n) (but haven't done the calculations).
You can copy a and then remove but you cannot iterate over and delete elements from the same list, if your list starts with n elements python will have n pointers to each element so removing elements from the list as your are iterating over it will cause elements to be missed.python has no way of knowing you have removed elements from the list:
a = [2,3,1,4,1,1,1,5]
for ele in a[:]:
if ele == 1:
a.remove(1)
print(a)
[2, 3, 4, 5]
You can also use reversed which returns and iterator avoiding creating a whole copy of the list at once:
a = [2,3,1,4,1,1,1,5]
for ele in reversed(a):
if ele == 1:
a.remove(1)
print(a)
[2, 3, 4, 5]
Or using a list comprehension with the [:] syntax so we actually update the original object:
a[:] = (ele for ele in a if ele != 1)
All the above are linear operations using a single pass over a.
Actually as the del statement will remove elements from your list , and as the list that you bound in your loop doesn't been update after the first deleting you remove incorrect elements from your list , so if you want to use del you need to make the list name in your loop to reference to new list , that you can use a function for this aim , but as a more python way you can just use a list comprehension:
>>> a=[2,3,1,4,1,1,1,5]
>>> a=[i for i in a if i !=1]
>>> a
[2, 3, 4, 5]
Or you can use filter :
>>> a=[2,3,1,4,1,1,1,5]
>>> a=filter(lambda x: x !=1,a)
>>> a
[2, 3, 4, 5]

Categories