I am trying to remove items from a nested list in Python. I have a nested list as follows:
families = [[0, 1, 2],[0, 1, 2, 3],[0, 1, 2, 3, 4],[1, 2, 3, 4, 5],[2, 3, 4, 5, 6]]
I want to remove the entries in each sublist that coorespond to the indexed position of the sublist in the master list. So, for example, I need to remove 0 from the first sublist, 1 from second sublist, etc. I am trying to use a list comrehension do do this. This is what I have tried:
familiesNew = [ [ families[i][j] for j in families[i] if i !=j ] for i in range(len(families)) ]
This works for range(len(families)) up to 3, however beyond that I get IndexError: list index out of range. I am not sure why. Can somebody give me an idea of how to do this. Preferably a one-liner (list comprehension).
Thanks.
You almost got it right. Just replace families[i][j] with j and it works:
>>> [ [ j for j in families[i] if i !=j ] for i in range(len(families)) ]
[[1, 2], [0, 2, 3], [0, 1, 3, 4], [1, 2, 4, 5], [2, 3, 5, 6]]
It can be written a bit cleaner using the enumerate function:
>>> [[f for f in family if f != i] for i, family in enumerate(families)]
[[1, 2], [0, 2, 3], [0, 1, 3, 4], [1, 2, 4, 5], [2, 3, 5, 6]]
Or even using remove if you don't mind changing the original list:
>>> for i, family in enumerate(families): family.remove(i)
Edited question, removing my answer which was solving the wrong problem. Also, added additional answer by #Ashwini:
For comparison's sake:
root# python -m timeit 'families = [[0, 1, 2],[0, 1, 2, 3],[0, 1, 2, 3, 4],[1, 2, 3, 4, 5],[2, 3, 4, 5, 6]]' '[x.remove(ind) for ind,x in enumerate(families) ]'
100000 loops, best of 3: 3.42 usec per loop
root# python -m timeit -s 'families = [[0, 1, 2],[0, 1, 2, 3],[0, 1, 2, 3, 4],[1, 2, 3, 4, 5],[2, 3, 4, 5, 6]]' '[[f for f in family if f != i] for i, family in enumerate(families)]'
100000 loops, best of 3: 4.87 usec per loop
root# python -m timeit -s 'families = [[0, 1, 2],[0, 1, 2, 3],[0, 1, 2, 3, 4],[1, 2, 3, 4, 5],[2, 3, 4, 5, 6]]' '[ filter(lambda x:x!=i,j) for i,j in enumerate(families) ]'
100000 loops, best of 3: 7.99 usec per loop
These are micro-second, so I think whatever you want to do is fine unless you are going to be doing this a lot of times.
Does this do what you want?
familiesNew=[ filter(lambda x:x!=i,j) for i,j in enumerate(families) ]
EDIT
Also note, the reason yours failed is because at the third element of the outer list ([1, 2, 3, 4, 5]) you're trying to get the fifth element in your for loop (for j in families[i] == for j in [1,2,3,4,5]), but families[i] has a length of 5, meaning the largest index is 4. Sorry if that explanation is a little unclear...perhaps the following will help clear it up a little:
families = [[0, 1, 2],[0, 1, 2, 3],[0, 1, 2, 3, 4],[1, 2, 3, 4, 5],[2, 3, 4, 5, 6]]
def f(i,j):
print i,j,families[i]
return families[i][j]
#THIS DOES NOT WORK -- but it will tell you where it failed.
familiesNew = [ [ f(i,j) for j in families[i] if i !=j ] for i in range(len(families)) ]
If you want to modify the original list then try this:
>>>[x.remove(ind) for ind,x in enumerate(families) ]
>>>families
[[1, 2], [0, 2, 3], [0, 1, 3, 4], [1, 2, 4, 5], [2, 3, 5, 6]]
Related
I'm trying to write a smart snippet of code that does the following:
Given a list and an integer-valued parameter k:
k = 2
myList = [1, 2, 3, 4, 5]
I would like to find a way of slicing my list such that I can later construct the following dictionary:
{1: [5, 4, 2, 3],
2: [1, 5, 3, 4],
3: [2, 1, 4, 5],
4: [3, 2, 5, 1],
5: [4, 3, 1, 2]}
i.e, I need to slice my list and extract my last k and next k elements (the order of the elements in the list after slicing does not matter), given an index.
For example, if my index is 0, then I would expect [5, 4, 2, 3].
My problem is similar to this question. However not exactly the same.
I'd appreciate any help, hint, or reference to any source.
You could do:
k = 2
myList = [1, 2, 3, 4, 5]
offset = len(myList)
padded_list = myList * 3
result = {myList[i]: padded_list[j - k: j][::-1] + padded_list[j + 1: j + k + 1] for i, j in
zip(range(offset), range(offset, 2 * offset))}
print(result)
Output
{1: [5, 4, 2, 3], 2: [1, 5, 3, 4], 3: [2, 1, 4, 5], 4: [3, 2, 5, 1], 5: [4, 3, 1, 2]}
The idea is to pad the list with itself before and after, and the iterate in the middle section. This should work without problem while k < len(myList).
I think simple python list slicing should suffice. Basically I sliced the list two times and concatenated the resulting two lists.
>>> l = [1, 2, 3, 4, 5]
>>> k = 2
>>> l[:k-1] + l[k:]
[1, 3, 4, 5]
Good luck!
I have tried the code below: the purpose is to generate a dictionary where each key has a list as a value. The first iteration goes well and generates the item as I want it, but the second loop, the nested for loop, doesn't generate the list as expected.
Please help me with this simple code. There must be something wrong with it, the code is as below:
schop = [1, 3, 1, 5, 6, 2, 1, 4, 3, 5, 6, 6, 2, 2, 3, 4, 4, 5]
mop = [1, 1, 2, 1, 1, 1, 3, 1, 2, 2, 2, 3, 2, 3, 3, 2, 3, 3]
mlist = ["1","2","3"]
wmlist=zip(mop,schop)
title ={}
for m in mlist:
m = int(m)
k=[]
for a,b in wmlist:
if a == m:
k.append(b)
title[m]=k
print(title)
The results are like:
title: {1: [1, 3, 5, 6, 2, 4], 2: [], 3: []}
Why do the second key and the third key have an empty list?
Thanks!
Your code would have worked as you expect in Python 2, where zip creates a list of tuples.
In Python 3, zip is an iterator. Once you iterate over it, it gets exhausted, so your second and third for loops won't have anything left to iterate over.
The simplest solution here would be to create a list from the iterator:
wmlist = list(zip(mop,schop))
i think that the best thing that you have to consider is the version of python that you have installed.
This is the output that i obtained with your code in python2:
{1: [1, 3, 5, 6, 2, 4], 2: [1, 3, 5, 6, 2, 4], 3: [1, 6, 2, 3, 4, 5]}
But with Python3 this is the answer that i obtained:
{1: [1, 3, 5, 6, 2, 4], 2: [], 3: []}
If you are sure that you have the properly vesion, only you have to consider the indentation that you have in your code. Good luck!!
I have a list with mixed sequences like
[1,2,3,4,5,2,3,4,1,2]
I want to know how I can use itertools to split the list into increasing sequences cutting the list at decreasing points. For instance the above would output
[[1, 2, 3, 4, 5], [2, 3, 4], [1, 2]]
this has been obtained by noting that the sequence decreases at 2 so we cut the first bit there and another decrease is at one cutting again there.
Another example is with the sequence
[3,2,1]
the output should be
[[3], [2], [1]]
In the event that the given sequence is increasing we return the same sequence. For example
[1,2,3]
returns the same result. i.e
[[1, 2, 3]]
For a repeating list like
[ 1, 2,2,2, 1, 2, 3, 3, 1,1,1, 2, 3, 4, 1, 2, 3, 4, 5, 6]
the output should be
[[1, 2, 2, 2], [1, 2, 3, 3], [1, 1, 1, 2, 3, 4], [1, 2, 3, 4, 5, 6]]
What I did to achieve this is define the following function
def splitter (L):
result = []
tmp = 0
initialPoint=0
for i in range(len(L)):
if (L[i] < tmp):
tmpp = L[initialPoint:i]
result.append(tmpp)
initialPoint=i
tmp = L[i]
result.append(L[initialPoint:])
return result
The function is working 100% but what I need is to do the same with itertools so that I can improve efficiency of my code. Is there a way to do this with itertools package to avoid the explicit looping?
With numpy, you can use numpy.split, this requires the index as split positions; since you want to split where the value decreases, you can use numpy.diff to calculate the difference and check where the difference is smaller than zero and use numpy.where to retrieve corresponding indices, an example with the last case in the question:
import numpy as np
lst = [ 1, 2,2,2, 1, 2, 3, 3, 1,1,1, 2, 3, 4, 1, 2, 3, 4, 5, 6]
np.split(lst, np.where(np.diff(lst) < 0)[0] + 1)
# [array([1, 2, 2, 2]),
# array([1, 2, 3, 3]),
# array([1, 1, 1, 2, 3, 4]),
# array([1, 2, 3, 4, 5, 6])]
Psidom already has you covered with a good answer, but another NumPy solution would be to use scipy.signal.argrelmax to acquire the local maxima, then np.split.
from scipy.signal import argrelmax
arr = np.random.randint(1000, size=10**6)
splits = np.split(arr, argrelmax(arr)[0]+1)
Assume your original input array:
a = [1, 2, 3, 4, 5, 2, 3, 4, 1, 2]
First find the places where the splits shall occur:
p = [ i+1 for i, (x, y) in enumerate(zip(a, a[1:])) if x > y ]
Then create slices for each such split:
print [ a[m:n] for m, n in zip([ 0 ] + p, p + [ None ]) ]
This will print this:
[[1, 2, 3, 4, 5], [2, 3, 4], [1, 2]]
I propose to use more speaking names than p, n, m, etc. ;-)
I had thought that if you ran perhaps print mdarray[::][1], you would print the first sub-element of every element in the array. Where did I go wrong with this?
I especially need this for a p.plot(x,y[::][1]) where I definitely do not want to use a for loop, as it is horribly slow, unless I'm getting things confused.
What am I getting wrong? Thanks!
EDIT
I still don't know where I got the [::] thing but I solved my problem with either
p.plot(x,c[:,1],color='g',label="Closing value")
or
p.plot(x,[i[1] for i in c],color='g',label="Closing value")
There doesn't seem to be any appreciable difference in time, so I guess I'll use the second because it looks more pythonic/readable to me. Or am I missing something?
Thanks for all of the help!
If mdarray is a numpy array you can access first column of it with mdarray[:,0]
In [8]: mdarray = np.array([[1, 2, 4], [4, 5, 6], [7, 8, 9]])
In [9]: mdarray
Out[9]:
array([[1, 2, 4],
[4, 5, 6],
[7, 8, 9]])
In [10]: mdarray[:,0]
Out[10]: array([1, 4, 7])
UPD
Quick and dirty test
In [28]: mdarray = np.zeros((10000,10000))
In [29]: %timeit -n1000 [x[0] for x in mdarray]
1000 loops, best of 3: 2.7 ms per loop
In [30]: %timeit -n1000 mdarray[:,0]
1000 loops, best of 3: 567 ns per loop
What you did:
You used mdarray[::]. That makes a (shallow) copy of mdarray. Then you accessed the second element of it with [1]. [0] would be the first.
What you can do is a list comprehension:
[item[0] for item in mdarray]
This will return a list of the first elements of the lists in mdarray.
Talking about loops: A (one time) loop is rather effective to access something. Internally all the magic functions (like the comprehension above) are iterating over the data.
How about:
>>> Matrix = [[x for x in range(5)] for x in range(5)]
>>> Matrix
[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]
>>> [item[0] for item in Matrix]
[0, 0, 0, 0, 0]
As for ::, you can read more about it here, It will return the same list.
Not sure whether you use array or list, but for Python's lists:
Python 2:
>>> mdarray = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> zip(*mdarray)[0]
(1, 4, 7)
Python 3:
>>> mdarray = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> list(zip(*mdarray))[0]
(1, 4, 7)
Or for the special case of index 0:
>>> mdarray = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> next(zip(*mdarray))
(1, 4, 7)
Was just wondering what's the most efficient way of generating all the circular shifts of a list in Python. In either direction. For example, given a list [1, 2, 3, 4], I want to generate either:
[[1, 2, 3, 4],
[4, 1, 2, 3],
[3, 4, 1, 2],
[2, 3, 4, 1]]
where the next permutation is generated by moving the last element to the front, or:
[[1, 2, 3, 4],
[2, 3, 4, 1],
[3, 4, 1, 2],
[4, 1, 2, 3]]
where the next permutation is generated by moving the first element to the back.
The second case is slightly more interesting to me because it results in a reduced Latin square (the first case also gives a Latin square, just not reduced), which is what I'm trying to use to do experimental block design. It actually isn't that different from the first case since they're just re-orderings of each other, but order does still matter.
The current implementation I have for the first case is:
def gen_latin_square(mylist):
tmplist = mylist[:]
latin_square = []
for i in range(len(mylist)):
latin_square.append(tmplist[:])
tmplist = [tmplist.pop()] + tmplist
return latin_square
For the second case its:
def gen_latin_square(mylist):
tmplist = mylist[:]
latin_square = []
for i in range(len(mylist)):
latin_square.append(tmplist[:])
tmplist = tmplist[1:] + [tmplist[0]]
return latin_square
The first case seems like it should be reasonably efficient to me, since it uses pop(), but you can't do that in the second case, so I'd like to hear ideas about how to do this more efficiently. Maybe there's something in itertools that will help? Or maybe a double-ended queue for the second case?
You can use collections.deque:
from collections import deque
g = deque([1, 2, 3, 4])
for i in range(len(g)):
print list(g) #or do anything with permutation
g.rotate(1) #for right rotation
#or g.rotate(-1) for left rotation
It prints:
[1, 2, 3, 4]
[4, 1, 2, 3]
[3, 4, 1, 2]
[2, 3, 4, 1]
To change it for left rotation just replace g.rotate(1) with g.rotate(-1).
For the first part, the most concise way probably is
a = [1, 2, 3, 4]
n = len(a)
[[a[i - j] for i in range(n)] for j in range(n)]
# [[1, 2, 3, 4], [4, 1, 2, 3], [3, 4, 1, 2], [2, 3, 4, 1]]
and for the second part
[[a[i - j] for i in range(n)] for j in range(n, 0, -1)]
# [[1, 2, 3, 4], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 3]]
These should also be much more efficient than your code, though I did not do any timings.
variation on slicing "conservation law" a = a[:i] + a[i:]
ns = list(range(5))
ns
Out[34]: [0, 1, 2, 3, 4]
[ns[i:] + ns[:i] for i in range(len(ns))]
Out[36]:
[[0, 1, 2, 3, 4],
[1, 2, 3, 4, 0],
[2, 3, 4, 0, 1],
[3, 4, 0, 1, 2],
[4, 0, 1, 2, 3]]
[ns[-i:] + ns[:-i] for i in range(len(ns))]
Out[38]:
[[0, 1, 2, 3, 4],
[4, 0, 1, 2, 3],
[3, 4, 0, 1, 2],
[2, 3, 4, 0, 1],
[1, 2, 3, 4, 0]]
more_itertools is a third-party library that offers a tool for cyclic permutations:
import more_itertools as mit
mit.circular_shifts(range(1, 5))
# [(1, 2, 3, 4), (2, 3, 4, 1), (3, 4, 1, 2), (4, 1, 2, 3)]
See also Wikipedia:
A circular shift is a special kind of cyclic permutation, which in turn is a special kind of permutation.
The answer by #Bruno Lenzi does not seem to work:
In [10]: from itertools import cycle
In [11]: x = cycle('ABCD')
In [12]: print [[x.next() for _ in range(4)] for _ in range(4)]
[['A', 'B', 'C', 'D'], ['A', 'B', 'C', 'D'], ['A', 'B', 'C', 'D'], ['A', 'B', 'C', 'D']]
I give a correct version below, however the solution by #f5r5e5d is faster.
In [45]: def use_cycle(a):
x=cycle(a)
for _ in a:
x.next()
print [x.next() for _ in a]
....:
In [46]: use_cycle([1,2,3,4])
[2, 3, 4, 1]
[3, 4, 1, 2]
[4, 1, 2, 3]
[1, 2, 3, 4]
In [50]: def use_slice(a):
print [ a[n:] + a[:n] for n in range(len(a)) ]
....:
In [51]: use_slice([1,2,3,4])
[[1, 2, 3, 4], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 3]]
In [54]: timeit.timeit('use_cycle([1,2,3,4])','from __main__ import use_cycle',number=100000)
Out[54]: 0.4884989261627197
In [55]: timeit.timeit('use_slice([1,2,3,4])','from __main__ import use_slice',number=100000)
Out[55]: 0.3103291988372803
In [58]: timeit.timeit('use_cycle([1,2,3,4]*100)','from __main__ import use_cycle',number=100)
Out[58]: 2.4427831172943115
In [59]: timeit.timeit('use_slice([1,2,3,4]*100)','from __main__ import use_slice',number=100)
Out[59]: 0.12029695510864258
I removed the print statement in use_cycle and use_slice for timing purposes.
Using itertools to avoid indexing:
x = itertools.cycle(a)
[[x.next() for i in a] for j in a]
This will be my solution.
#given list
a = [1,2,3,4]
#looping through list
for i in xrange(len(a)):
#inserting last element at the starting
a.insert(0,a[len(a)-1])
#removing the last element
a = a[:len(a)-1]
#printing if you want to
print a
This will output the following:
[4, 1, 2, 3]
[3, 4, 1, 2]
[2, 3, 4, 1]
[1, 2, 3, 4]
You can also use pop instead of using list slicing but the problem with pop is that it will return something.
Also the above code will work for any length of list. I have not checked for performance of the code. I am assuming that it will work better.
You should have a look at Python docs for getting a good understanding of List slicing.