Related
This question already has answers here:
Zip lists in Python
(10 answers)
Closed 2 years ago.
My input is
tbl_ports = [[1,2,3,4], [5,6,7,8], [9,10,11,12]]
And my expected output is
[[1,5,9], [2,6,10], [3,7,11], [4,8,12]]
My limit was to do the following to create the output reorder_list
reorder_list = []
for i in range(len(tbl_ports)):
for col in tbl_ports:
reorder_list.append(col[i])
reorder_list=[1, 5, 9, 2, 6, 10, 3, 7, 11]
How can I contain them in a list of 3 elements?
To fix the code that you already have, you need to create a new list every time a row is complete, such as:
reorder_list = []
for i in range(len(tbl_ports)):
reorder_list.append([])
for col in tbl_ports:
reorder_list[-1].append(col[i])
Which would yield the following result:
[[1, 5, 9], [2, 6, 10], [3, 7, 11]]
You can also use a more pythonic method of solving the problem,
list(zip(*tbl_port))
Which would yield a list of tuples:
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
If you want a list of lists, then you can simply just use list comprehension:
[list(e) for e in zip(*tbl_port)]
Edit:
for an explanation of why zip(*list) works, you need to know what zip does.
zip is a function in python that takes in multiple lists, and outputs a generator of lists for each element in every corresponding list. So zip([1, 2, 3], [4, 5, 6]) would yield [(1, 4), (2, 5), (3, 6)].
the * basically expands the input into multiple positional arguments, where function(*[1, 2, 3, 4]) is equivalent to function(1, 2, 3, 4)
So the code passed in the input array as a list of arguments to the zip function, which then outputs the result in the order that you want.
The only remaining problem is that zip generates a generator instead of an actual list.
To solve that problem, simply call the list function on a generator to convert it into a list, or pass it in a list comprehension to yield the desired result.
This is exactly what the zip() function is for.
list(zip([1,2,3,4],'abcd'))
You can use the unpack syntax " * " to make python unpack your lists to the zip function.
tbl_ports = [[1,2,3,4], [5,6,7,8], [9,10,11,12]]
reorder_list = list(zip(*tbl_ports))
I have a question that I haven't quite found a good solution to. I'm looking for a better way to append function output to two or more lists, without using temp variables. Example below:
def f():
return 5,6
a,b = [], []
for i in range(10):
tmp_a, tmp_b = f()
a.append(tmp_a)
b.append(temp_b)
I've tried playing around with something like zip(*f()), but haven't quite found a solution that way.
Any way to remove those temp vars would be super helpful though, thanks!
Edit for additional info:
In this situation, the number of outputs from the function will always equal the number of lists that are being appended to. The main reason I'm looking to get rid of temps is for the case where there are maybe 8-10 function outputs, and having that many temp variables would get messy (though I don't really even like having two).
def f():
return 5,6
a,b = zip(*[f() for i in range(10)])
# this will create two tuples of elements 5 and 6 you can change
# them to list by type casting it like list(a), list(b)
First solution: we make a list of all results, then transpose it
def f(i):
return i, 2*i
# First make a list of all your results
l = [f(i) for i in range(5)]
# [(0, 0), (1, 2), (2, 4), (3, 6), (4, 8)]
# then transpose it using zip
a, b = zip(*l)
print(list(a))
print(list(b))
# [0, 1, 2, 3, 4]
# [0, 2, 4, 6, 8]
Or, all in one line:
a, b = zip(*[f(i) for i in range(5)])
A different solution, building the lists at each iteration, so that you can use them while they're being built:
def f(i):
return 2*i, i**2, i**3
doubles = []
squares = []
cubes = []
results = [doubles, squares, cubes]
for i in range(1, 4):
list(map(lambda res, val: res.append(val), results, f(i)))
print(results)
# [[2], [1], [1]]
# [[2, 4], [1, 4], [1, 8]]
# [[2, 4, 6], [1, 4, 9], [1, 8, 27]]
print(cubes)
# [1, 8, 27]
Note about list(map(...)): in Python3, map returns a generator, so we must use it if we want the lambda to be executed.list does it.
For your specific case, the zip answers are great.
Using itertools.cycle and itertools.chain is a different approach from the existing answers that might come in handy if you have a lot of pre-existing lists that you want to append to in a round-robin fashion. It also works when your function returns more values than you have lists.
>>> from itertools import cycle, chain
>>> a, b = [], [] # new, empty lists only for demo purposes
>>> for l, v in zip(cycle([a, b]), (chain(*(f() for i in range(10))))):
... l.append(v)
...
>>> a
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]
>>> b
[6, 6, 6, 6, 6, 6, 6, 6, 6, 6]
I'd do
tmp = f()
a.append(tmp[0])
b.append(tmp[1])
Not sure how pythonic it is for you though.
For some context, I'm trying to enumerate the number of unique situations that can occur when calculating the Banzhaf power indices for four players, when there is no dictator and there are either four or five winning coalitions.
I am using the following code to generate a set of lists that I want to iterate over.
from itertools import chain, combinations
def powerset(iterable):
s = list(iterable)
return chain.from_iterable(map(list, combinations(s, r)) for r in range(2, len(s)+1))
def superpowerset(iterable):
s = powerset(iterable)
return chain.from_iterable(map(list, combinations(s, r)) for r in range(4, 6))
set_of_lists = superpowerset([1,2,3,4])
However, two lists in this set shouldn't be considered unique if they are equivalent under remapping.
Using the following list as an example:
[[1, 2], [1, 3], [2, 3], [1, 2, 4]]
If each element 2 is renamed to 3 and vice-versa, we would get:
[[1, 3], [1, 2], [3, 2], [1, 3, 4]]
The order in each sub-list is unimportant, and the order of the sub-lists is also un-important. Thus, the swapped list can be rewritten as:
[[1, 2], [1, 3], [2, 3], [1, 3, 4]]
There are 4 values, so there are P(4,4)=24 possible remappings that could occur (including the trivial mapping).
Is there any way to check this easily? Or, even better, is there are way to avoid generating these lists to begin with?
I'm not even sure how I would go about transforming the first list into the second list (but could brute force it from there). Also, I'm not restricted to data type (to a certain extent) and using frozenset would be fine.
Edit: The solution offered by tobias_k answers the "checking" question but, as noted in the comments, I think I have the wrong approach to this problem.
This is probably no complete solution yet, but it might show you a direction to investigate further.
You could map each element to some characteristics concerning the "topology", how it is "connected" with other elements. You have to be careful not to take the ordering in the sets into account, or -- obviously -- the element itself. You could, for example, consider how often the element appears, in what sized groups it appears, and something like this. Combine those metrics to a key function, sort the elements by that key, and assign them new names in that order.
def normalize(lists):
items = set(x for y in lists for x in y)
counter = itertools.count()
sorter = lambda x: sorted(len(y) for y in lists if x in y)
mapping = {k: next(counter) for k in sorted(items, key=sorter)}
return tuple(sorted(tuple(sorted(mapping[x] for x in y)) for y in lists))
This maps your two example lists to the same "normalized" list:
>>> normalize([[1, 2], [1, 3], [2, 3], [1, 2, 4]])
((0, 1), (0, 2), (1, 2), (1, 2, 3))
>>> normalize([[1, 3], [1, 2], [3, 2], [1, 3, 4]])
((0, 1), (0, 2), (1, 2), (1, 2, 3))
When applied to all the lists, it gets the count down from 330 to 36. I don't know if this is minimal, but it looks like a good start.
>>> normalized = set(map(normalize, set_of_lists))
>>> len(normalized)
36
This question already has answers here:
Matrix Transpose in Python [duplicate]
(19 answers)
Closed 9 years ago.
I have a simple list of lists called square:
square = [[1,2,3],[4,5,6],[7,8,9]]
and my goal is to iterate through each of the nine elements and get all the values in that element's row and column simultaneously. In this example, the first iteration (for element 1) should return [1,2,3] and [1,4,7], the second (for element 2) would give me [1,2,3] and [2,5,8], etc. The following code works:
for r in range(3):
for c in range(3):
row = square[r]
col = [square[c1][c] for c1 in range(3)]
but is there another method using base Python to do this?
I can transpose and iterate over the list using for c in zip(*square):, which works if I just need the columns once, but is there no way to use array slicing to index the columns without storing the transpose of the matrix as well as the matrix itself?
Unfortunately, libraries like numpy aren't an option at the moment because they'll need to go through our code review process first, which takes a long time (on the order of six months...). I didn't write the policy or design the procedures, but it's the policy I have to work with at this institution. I already filed the request for numpy, but in the meantime, I'm just using base Python.
using numpy
>>> import numpy as np
>>> square = np.array([[1,2,3],[4,5,6],[7,8,9]])
>>> square[0]
array([1, 2, 3])
>>> square[..., 0]
array([1, 4, 7])
>>> square[:, 0]
array([1, 4, 7])
Using just Python
>>> square = [[1,2,3],[4,5,6],[7,8,9]]
>>> square[0]
[1, 2, 3]
>>> zip(*square)[0]
(1, 4, 7)
If you want to do this with standard Python, you might want to try transposing the list. You can then index as normal. You can do this with zip(*square) followed by map(list,) to turn it back into a list:
square = [[1,2,3],[4,5,6],[7,8,9]]
square_transposed = map(list, zip(*square))
Then square[1] will get you [2,5,8].
If you're wondering what the * operator is doing in this context, it unpacks the list to a series of position arguments. So zip sees [1,2,3] [4,5,6] [7,8,9] and zips them together as [(1, 4, 7), (2, 5, 8), (3, 6, 9)]. Then the final map turns the tuples to lists.
I have a numerical list:
myList = [1, 2, 3, 100, 5]
Now if I sort this list to obtain [1, 2, 3, 5, 100].
What I want is the indices of the elements from the
original list in the sorted order i.e. [0, 1, 2, 4, 3]
--- ala MATLAB's sort function that returns both
values and indices.
If you are using numpy, you have the argsort() function available:
>>> import numpy
>>> numpy.argsort(myList)
array([0, 1, 2, 4, 3])
http://docs.scipy.org/doc/numpy/reference/generated/numpy.argsort.html
This returns the arguments that would sort the array or list.
Something like next:
>>> myList = [1, 2, 3, 100, 5]
>>> [i[0] for i in sorted(enumerate(myList), key=lambda x:x[1])]
[0, 1, 2, 4, 3]
enumerate(myList) gives you a list containing tuples of (index, value):
[(0, 1), (1, 2), (2, 3), (3, 100), (4, 5)]
You sort the list by passing it to sorted and specifying a function to extract the sort key (the second element of each tuple; that's what the lambda is for. Finally, the original index of each sorted element is extracted using the [i[0] for i in ...] list comprehension.
myList = [1, 2, 3, 100, 5]
sorted(range(len(myList)),key=myList.__getitem__)
[0, 1, 2, 4, 3]
I did a quick performance check on these with perfplot (a project of mine) and found that it's hard to recommend anything else but
np.argsort(x)
(note the log scale):
Code to reproduce the plot:
import perfplot
import numpy as np
def sorted_enumerate(seq):
return [i for (v, i) in sorted((v, i) for (i, v) in enumerate(seq))]
def sorted_enumerate_key(seq):
return [x for x, y in sorted(enumerate(seq), key=lambda x: x[1])]
def sorted_range(seq):
return sorted(range(len(seq)), key=seq.__getitem__)
b = perfplot.bench(
setup=np.random.rand,
kernels=[sorted_enumerate, sorted_enumerate_key, sorted_range, np.argsort],
n_range=[2 ** k for k in range(15)],
xlabel="len(x)",
)
b.save("out.png")
The answers with enumerate are nice, but I personally don't like the lambda used to sort by the value. The following just reverses the index and the value, and sorts that. So it'll first sort by value, then by index.
sorted((e,i) for i,e in enumerate(myList))
Updated answer with enumerate and itemgetter:
sorted(enumerate(a), key=lambda x: x[1])
# [(0, 1), (1, 2), (2, 3), (4, 5), (3, 100)]
Zip the lists together: The first element in the tuple will the index, the second is the value (then sort it using the second value of the tuple x[1], x is the tuple)
Or using itemgetter from the operatormodule`:
from operator import itemgetter
sorted(enumerate(a), key=itemgetter(1))
Essentially you need to do an argsort, what implementation you need depends if you want to use external libraries (e.g. NumPy) or if you want to stay pure-Python without dependencies.
The question you need to ask yourself is: Do you want the
indices that would sort the array/list
indices that the elements would have in the sorted array/list
Unfortunately the example in the question doesn't make it clear what is desired because both will give the same result:
>>> arr = np.array([1, 2, 3, 100, 5])
>>> np.argsort(np.argsort(arr))
array([0, 1, 2, 4, 3], dtype=int64)
>>> np.argsort(arr)
array([0, 1, 2, 4, 3], dtype=int64)
Choosing the argsort implementation
If you have NumPy at your disposal you can simply use the function numpy.argsort or method numpy.ndarray.argsort.
An implementation without NumPy was mentioned in some other answers already, so I'll just recap the fastest solution according to the benchmark answer here
def argsort(l):
return sorted(range(len(l)), key=l.__getitem__)
Getting the indices that would sort the array/list
To get the indices that would sort the array/list you can simply call argsort on the array or list. I'm using the NumPy versions here but the Python implementation should give the same results
>>> arr = np.array([3, 1, 2, 4])
>>> np.argsort(arr)
array([1, 2, 0, 3], dtype=int64)
The result contains the indices that are needed to get the sorted array.
Since the sorted array would be [1, 2, 3, 4] the argsorted array contains the indices of these elements in the original.
The smallest value is 1 and it is at index 1 in the original so the first element of the result is 1.
The 2 is at index 2 in the original so the second element of the result is 2.
The 3 is at index 0 in the original so the third element of the result is 0.
The largest value 4 and it is at index 3 in the original so the last element of the result is 3.
Getting the indices that the elements would have in the sorted array/list
In this case you would need to apply argsort twice:
>>> arr = np.array([3, 1, 2, 4])
>>> np.argsort(np.argsort(arr))
array([2, 0, 1, 3], dtype=int64)
In this case :
the first element of the original is 3, which is the third largest value so it would have index 2 in the sorted array/list so the first element is 2.
the second element of the original is 1, which is the smallest value so it would have index 0 in the sorted array/list so the second element is 0.
the third element of the original is 2, which is the second-smallest value so it would have index 1 in the sorted array/list so the third element is 1.
the fourth element of the original is 4 which is the largest value so it would have index 3 in the sorted array/list so the last element is 3.
If you do not want to use numpy,
sorted(range(len(seq)), key=seq.__getitem__)
is fastest, as demonstrated here.
The other answers are WRONG.
Running argsort once is not the solution.
For example, the following code:
import numpy as np
x = [3,1,2]
np.argsort(x)
yields array([1, 2, 0], dtype=int64) which is not what we want.
The answer should be to run argsort twice:
import numpy as np
x = [3,1,2]
np.argsort(np.argsort(x))
gives array([2, 0, 1], dtype=int64) as expected.
Most easiest way you can use Numpy Packages for that purpose:
import numpy
s = numpy.array([2, 3, 1, 4, 5])
sort_index = numpy.argsort(s)
print(sort_index)
But If you want that you code should use baisc python code:
s = [2, 3, 1, 4, 5]
li=[]
for i in range(len(s)):
li.append([s[i],i])
li.sort()
sort_index = []
for x in li:
sort_index.append(x[1])
print(sort_index)
We will create another array of indexes from 0 to n-1
Then zip this to the original array and then sort it on the basis of the original values
ar = [1,2,3,4,5]
new_ar = list(zip(ar,[i for i in range(len(ar))]))
new_ar.sort()
`
s = [2, 3, 1, 4, 5]
print([sorted(s, reverse=False).index(val) for val in s])
For a list with duplicate elements, it will return the rank without ties, e.g.
s = [2, 2, 1, 4, 5]
print([sorted(s, reverse=False).index(val) for val in s])
returns
[1, 1, 0, 3, 4]
Import numpy as np
FOR INDEX
S=[11,2,44,55,66,0,10,3,33]
r=np.argsort(S)
[output]=array([5, 1, 7, 6, 0, 8, 2, 3, 4])
argsort Returns the indices of S in sorted order
FOR VALUE
np.sort(S)
[output]=array([ 0, 2, 3, 10, 11, 33, 44, 55, 66])
Code:
s = [2, 3, 1, 4, 5]
li = []
for i in range(len(s)):
li.append([s[i], i])
li.sort()
sort_index = []
for x in li:
sort_index.append(x[1])
print(sort_index)
Try this, It worked for me cheers!
firstly convert your list to this:
myList = [1, 2, 3, 100, 5]
add a index to your list's item
myList = [[0, 1], [1, 2], [2, 3], [3, 100], [4, 5]]
next :
sorted(myList, key=lambda k:k[1])
result:
[[0, 1], [1, 2], [2, 3], [4, 5], [3, 100]]
A variant on RustyRob's answer (which is already the most performant pure Python solution) that may be superior when the collection you're sorting either:
Isn't a sequence (e.g. it's a set, and there's a legitimate reason to want the indices corresponding to how far an iterator must be advanced to reach the item), or
Is a sequence without O(1) indexing (among Python's included batteries, collections.deque is a notable example of this)
Case #1 is unlikely to be useful, but case #2 is more likely to be meaningful. In either case, you have two choices:
Convert to a list/tuple and use the converted version, or
Use a trick to assign keys based on iteration order
This answer provides the solution to #2. Note that it's not guaranteed to work by the language standard; the language says each key will be computed once, but not the order they will be computed in. On every version of CPython, the reference interpreter, to date, it's precomputed in order from beginning to end, so this works, but be aware it's not guaranteed. In any event, the code is:
sizediterable = ...
sorted_indices = sorted(range(len(sizediterable)), key=lambda _, it=iter(sizediterable): next(it))
All that does is provide a key function that ignores the value it's given (an index) and instead provides the next item from an iterator preconstructed from the original container (cached as a defaulted argument to allow it to function as a one-liner). As a result, for something like a large collections.deque, where using its .__getitem__ involves O(n) work (and therefore computing all the keys would involve O(n²) work), sequential iteration remains O(1), so generating the keys remains just O(n).
If you need something guaranteed to work by the language standard, using built-in types, Roman's solution will have the same algorithmic efficiency as this solution (as neither of them rely on the algorithmic efficiency of indexing the original container).
To be clear, for the suggested use case with collections.deque, the deque would have to be quite large for this to matter; deques have a fairly large constant divisor for indexing, so only truly huge ones would have an issue. Of course, by the same token, the cost of sorting is pretty minimal if the inputs are small/cheap to compare, so if your inputs are large enough that efficient sorting matters, they're large enough for efficient indexing to matter too.