I am struggling with generating circular permutations, or solving the "Necklace Problem" with Python. I am basically looking to generate all circular permutations of a list as efficiently as possible.
Basically, assume we have a list [1,2,3,4], I want to generate all unique permutations in a circular fashion. So:
[1,2,3,4], [1,3,2,4]
Would be considered different, whereas:
[1,2,3,4], [4,1,2,3] would be considered the same, as I am looking only for unique permutations of a circular list.
I have tried generating all permutations with itertools but this is very inefficient when using larger length lists.
As another example, consider a running loop songs, characterised by [1,2,3,4,5] playing on repeat. I am trying to come up with an algorithm to generate only unique orders. Obviously [1,2,3,4,5] and [4,5,1,2,3] would result in the same order.
Struggling and would appreciate some help to solve this problem!
The following code produces all permutations of the last n-1 numbers, prepending each permutation with the first element of the original list.
from itertools import permutations
circular_perms = [my_list[:1]+list(perm) for perm in permutations(my_list[1:])]
Where my_list is your initial list of values to generation all circular permutations of.
Think of a "fixed point" on the necklace.
arrange for the lowest numbered bead, 1, to be rotated to this point
Then the only orders of the beads is all the permutations of items that start with 1
Because the itertool.permutations generator in Python has the first item in the input list varying the least in successive output, the following code just outputs all the solutions as seen when rotated to the fixed point above.
from itertools import permutations, islice
from math import factorial
items = [1, 2, 3 ,4]
ilength = len(items)
base_necklace_orders = list(islice(permutations(items), factorial(ilength) // ilength))
print(base_necklace_orders)
# [(1, 2, 3, 4), (1, 2, 4, 3), (1, 3, 2, 4), (1, 3, 4, 2), (1, 4, 2, 3), (1, 4, 3, 2)]
There are two Python libraries (that I know of) which will generate combinatoric necklaces. They are Sage and SymPy.
The complexity for making permutation is about O(n*n!), so for large number or list it would inefficient to generate all possible permutations, You can use backtracking to generating list permutation, I'll share a link may be this would help.
The solution is based on the backtracking
def permute(a, l, r):
if l == r:
print(a)
else:
for i in range(l, r + 1):
a[l], a[i] = a[i], a[l]
permute(a, l + 1, r)
a[l], a[i] = a[i], a[l]
data = [1,2,3,4,5]
n = len(data)
a = list(data)
permute(a, 0, n - 1)
When you havea list of N elements, one of the circular permutations of the list is uniquely given by its first element. Than means that you will have exactly N circular permutation (including the original list), and you can pass from one to another by removing fist element and adding it at the end of the list.
You can easily build a generator for all the circular permutations of a list:
def circ_perm(lst):
cpy = lst[:] # take a copy because a list is a mutable object
yield cpy
for i in range(len(lst) - 1):
cpy = cpy[1:] + [cpy[0]]
yield cpy
Demo:
>>> list(circ_perm([1,2,3,4]))
[[1, 2, 3, 4], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 3]]
If what you want is the unique permutations when two permutations that are a ciruclar permution of the other one, you can still use the fact that a circular permutation is given by its first element and fix the first element and find all the permutations of what remains:
def uniq_perm(lst):
gen = itertools.permutations(lst[1:])
for end in gen:
yield [lst[0]] + list(end)
Demo:
>>> list(uniq_perm([1,2,3,4]))
[[1, 2, 3, 4], [1, 2, 4, 3], [1, 3, 2, 4], [1, 3, 4, 2], [1, 4, 2, 3], [1, 4, 3, 2]]
Related
For example, there is a list, [1,2,3] with constraint such that each elements of the shuffled list should not overlap with the location of the previous elements.
To make the explanation clear, let's say the previous list was [1,2,3], but after shuffling it becomes [1,3,2]. For this case, since 1 is the first element of the both lists, this does not satisfy the constraint. The outputs that satisfy that constraint would be, [2,3,1] and [3,1,2].
Is there a way to make this constraint before shuffling the list?
Thanks in advance.
You can accomplish this by rotating the list. For example, the rotations of [1, 2, 3] are [3, 1, 2] and [2, 3, 1].
Note that for lengths > 3, the rotations will be a subset of all possible shuffles that fit your constraints, but if I understand correctly, that works for you. For example, with input [1, 2, 3, 4], [4, 3, 2, 1] is a valid output, but not a rotation.
collections.deque makes it easy to rotate, plus it's optimized for rotation.
random.randrange() can be used to randomize the rotation count.
import random
import collections
def random_rotation(lst):
"""
Rotate a list by a random amount.
Similar to "random.shuffle()", but ensures that all elements will move.
"""
n = random.randrange(1, len(lst))
d = collections.deque(lst)
d.rotate(n)
lst[:] = d # Overwrite
L = [1, 2, 3, 4]
random_rotation(L)
print(L)
Example output:
[2, 3, 4, 1]
Based on your spec, what you're asking for is just all possible rotations of the list. For example:
def rotations(lst):
for i in range(len(lst)):
print(lst)
lst.append(lst.pop(0))
I have a question that I haven't quite found a good solution to. I'm looking for a better way to append function output to two or more lists, without using temp variables. Example below:
def f():
return 5,6
a,b = [], []
for i in range(10):
tmp_a, tmp_b = f()
a.append(tmp_a)
b.append(temp_b)
I've tried playing around with something like zip(*f()), but haven't quite found a solution that way.
Any way to remove those temp vars would be super helpful though, thanks!
Edit for additional info:
In this situation, the number of outputs from the function will always equal the number of lists that are being appended to. The main reason I'm looking to get rid of temps is for the case where there are maybe 8-10 function outputs, and having that many temp variables would get messy (though I don't really even like having two).
def f():
return 5,6
a,b = zip(*[f() for i in range(10)])
# this will create two tuples of elements 5 and 6 you can change
# them to list by type casting it like list(a), list(b)
First solution: we make a list of all results, then transpose it
def f(i):
return i, 2*i
# First make a list of all your results
l = [f(i) for i in range(5)]
# [(0, 0), (1, 2), (2, 4), (3, 6), (4, 8)]
# then transpose it using zip
a, b = zip(*l)
print(list(a))
print(list(b))
# [0, 1, 2, 3, 4]
# [0, 2, 4, 6, 8]
Or, all in one line:
a, b = zip(*[f(i) for i in range(5)])
A different solution, building the lists at each iteration, so that you can use them while they're being built:
def f(i):
return 2*i, i**2, i**3
doubles = []
squares = []
cubes = []
results = [doubles, squares, cubes]
for i in range(1, 4):
list(map(lambda res, val: res.append(val), results, f(i)))
print(results)
# [[2], [1], [1]]
# [[2, 4], [1, 4], [1, 8]]
# [[2, 4, 6], [1, 4, 9], [1, 8, 27]]
print(cubes)
# [1, 8, 27]
Note about list(map(...)): in Python3, map returns a generator, so we must use it if we want the lambda to be executed.list does it.
For your specific case, the zip answers are great.
Using itertools.cycle and itertools.chain is a different approach from the existing answers that might come in handy if you have a lot of pre-existing lists that you want to append to in a round-robin fashion. It also works when your function returns more values than you have lists.
>>> from itertools import cycle, chain
>>> a, b = [], [] # new, empty lists only for demo purposes
>>> for l, v in zip(cycle([a, b]), (chain(*(f() for i in range(10))))):
... l.append(v)
...
>>> a
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]
>>> b
[6, 6, 6, 6, 6, 6, 6, 6, 6, 6]
I'd do
tmp = f()
a.append(tmp[0])
b.append(tmp[1])
Not sure how pythonic it is for you though.
For some context, I'm trying to enumerate the number of unique situations that can occur when calculating the Banzhaf power indices for four players, when there is no dictator and there are either four or five winning coalitions.
I am using the following code to generate a set of lists that I want to iterate over.
from itertools import chain, combinations
def powerset(iterable):
s = list(iterable)
return chain.from_iterable(map(list, combinations(s, r)) for r in range(2, len(s)+1))
def superpowerset(iterable):
s = powerset(iterable)
return chain.from_iterable(map(list, combinations(s, r)) for r in range(4, 6))
set_of_lists = superpowerset([1,2,3,4])
However, two lists in this set shouldn't be considered unique if they are equivalent under remapping.
Using the following list as an example:
[[1, 2], [1, 3], [2, 3], [1, 2, 4]]
If each element 2 is renamed to 3 and vice-versa, we would get:
[[1, 3], [1, 2], [3, 2], [1, 3, 4]]
The order in each sub-list is unimportant, and the order of the sub-lists is also un-important. Thus, the swapped list can be rewritten as:
[[1, 2], [1, 3], [2, 3], [1, 3, 4]]
There are 4 values, so there are P(4,4)=24 possible remappings that could occur (including the trivial mapping).
Is there any way to check this easily? Or, even better, is there are way to avoid generating these lists to begin with?
I'm not even sure how I would go about transforming the first list into the second list (but could brute force it from there). Also, I'm not restricted to data type (to a certain extent) and using frozenset would be fine.
Edit: The solution offered by tobias_k answers the "checking" question but, as noted in the comments, I think I have the wrong approach to this problem.
This is probably no complete solution yet, but it might show you a direction to investigate further.
You could map each element to some characteristics concerning the "topology", how it is "connected" with other elements. You have to be careful not to take the ordering in the sets into account, or -- obviously -- the element itself. You could, for example, consider how often the element appears, in what sized groups it appears, and something like this. Combine those metrics to a key function, sort the elements by that key, and assign them new names in that order.
def normalize(lists):
items = set(x for y in lists for x in y)
counter = itertools.count()
sorter = lambda x: sorted(len(y) for y in lists if x in y)
mapping = {k: next(counter) for k in sorted(items, key=sorter)}
return tuple(sorted(tuple(sorted(mapping[x] for x in y)) for y in lists))
This maps your two example lists to the same "normalized" list:
>>> normalize([[1, 2], [1, 3], [2, 3], [1, 2, 4]])
((0, 1), (0, 2), (1, 2), (1, 2, 3))
>>> normalize([[1, 3], [1, 2], [3, 2], [1, 3, 4]])
((0, 1), (0, 2), (1, 2), (1, 2, 3))
When applied to all the lists, it gets the count down from 330 to 36. I don't know if this is minimal, but it looks like a good start.
>>> normalized = set(map(normalize, set_of_lists))
>>> len(normalized)
36
I don't quite understand how iterators have memory in Python.
>>> l1 = [1, 2, 3, 4, 5, 6]
>>> l2 = [2, 3, 4, 5, 6, 7]
>>> iz = izip(l1, l2)
We still require O(min(l1, l2)) memory as we need to load the lists l1 and l2 in memory.
I thought one of the main uses of iterators was to save memory - yet it does not seem to be useful here.
Similarly, the code below is unclear to me:
>>> l1 = ( n for n in [1, 2, 3, 4, 5, 6] )
>>> l2 = ( n for n in [2, 3, 4, 5, 6, 7] )
>>> iz = izip(l1, l2)
We need to load the lists before converting them into generators, right? This means we'll waste memory. So - what is the point of generators here as well.
This is the only case that makes sense to me:
def build_l1():
for n in xrange(1, 6):
yield n
def build_l2:
for n in xrange(2, 7):
yield n
l1 = build_l1()
l2 = build_l2()
iz = izip(l1, l2)
None of the arrays is being loaded into memory. Hence we're in O(1) memory.
How does the memory usage of the iterator functions in Python work? The first two cases seem to use O(min(l1, l2)) memory. I thought the main point of iterators was to save memory, which makes the first two cases seem useless.
Your examples are too simplistic. Consider this:
nums = [1, 2, 3, 4, 5, 6]
nums_it = (n for n in nums)
nums_it is a generator that returns all items unmodified from nums. Clearly you do not have any advantage. But consider this:
squares_it = (n ** 2 for n in nums)
and compare it with:
squares_lst = [n ** 2 for n in nums]
With squares_it, we are generating the squares of nums on the fly only when requested. With squares_lst, we are generating all of them at once and storing them in a new list.
So, when you do:
for n in squares_it:
print(n)
it's like if you were doing:
for n in nums:
print(n ** 2)
But when you do:
for n in squares_lst:
print(n)
it's like if you were doing:
squares_lst = []
for n in nums:
squares_lst.append(n ** 2)
for n in squares_lst:
print(n)
If you don't need (or don't have) the list nums, then you can save even more space by using:
squares_it = (n ** 2 for n in xrange(1, 7))
Generators and iterators also provide another significant advantage (which may actually be a disadvantage, depending on the situation): they are evaluated lazily.
Also, generators and iterators may yield an infinite number of elements. An example is itertools.count() that yields 0, 1, 2, 3, ... without never ending.
>>> l1 = [1, 2, 3, 4, 5, 6]
>>> l2 = [2, 3, 4, 5, 6, 7]
>>> iz = izip(l1, l2)
We still require O(min(l1, l2)) memory as we need to load the lists l1 and l2 in memory.
With zip you need storage for the two original lists plus the zipped list. With izip you don't store the zipped list.
Big O notation isn't particularly helpful here if you have to work with a real physical machine instead of some abstract concept of a machine. There's a hidden constant multiplier on your O(n) calculations that could influence the practicality of the code well before n tends to infinity.
>>> l1 = ( n for n in [1, 2, 3, 4, 5, 6] )
>>> l2 = ( n for n in [2, 3, 4, 5, 6, 7] )
>>> iz = izip(l1, l2)
We need to load the lists before converting them into generators, right? This means we'll waste memory. So - what is the point of generators here as well.
No point to generators here. Any time you see n for n in <expr> without either a more complex expression before the for or an if <expr> filter after it, that's a code smell as you could just have used the original sequence directly. The generators only become useful when you transform the input values into something else or filter them.
I have a numerical list:
myList = [1, 2, 3, 100, 5]
Now if I sort this list to obtain [1, 2, 3, 5, 100].
What I want is the indices of the elements from the
original list in the sorted order i.e. [0, 1, 2, 4, 3]
--- ala MATLAB's sort function that returns both
values and indices.
If you are using numpy, you have the argsort() function available:
>>> import numpy
>>> numpy.argsort(myList)
array([0, 1, 2, 4, 3])
http://docs.scipy.org/doc/numpy/reference/generated/numpy.argsort.html
This returns the arguments that would sort the array or list.
Something like next:
>>> myList = [1, 2, 3, 100, 5]
>>> [i[0] for i in sorted(enumerate(myList), key=lambda x:x[1])]
[0, 1, 2, 4, 3]
enumerate(myList) gives you a list containing tuples of (index, value):
[(0, 1), (1, 2), (2, 3), (3, 100), (4, 5)]
You sort the list by passing it to sorted and specifying a function to extract the sort key (the second element of each tuple; that's what the lambda is for. Finally, the original index of each sorted element is extracted using the [i[0] for i in ...] list comprehension.
myList = [1, 2, 3, 100, 5]
sorted(range(len(myList)),key=myList.__getitem__)
[0, 1, 2, 4, 3]
I did a quick performance check on these with perfplot (a project of mine) and found that it's hard to recommend anything else but
np.argsort(x)
(note the log scale):
Code to reproduce the plot:
import perfplot
import numpy as np
def sorted_enumerate(seq):
return [i for (v, i) in sorted((v, i) for (i, v) in enumerate(seq))]
def sorted_enumerate_key(seq):
return [x for x, y in sorted(enumerate(seq), key=lambda x: x[1])]
def sorted_range(seq):
return sorted(range(len(seq)), key=seq.__getitem__)
b = perfplot.bench(
setup=np.random.rand,
kernels=[sorted_enumerate, sorted_enumerate_key, sorted_range, np.argsort],
n_range=[2 ** k for k in range(15)],
xlabel="len(x)",
)
b.save("out.png")
The answers with enumerate are nice, but I personally don't like the lambda used to sort by the value. The following just reverses the index and the value, and sorts that. So it'll first sort by value, then by index.
sorted((e,i) for i,e in enumerate(myList))
Updated answer with enumerate and itemgetter:
sorted(enumerate(a), key=lambda x: x[1])
# [(0, 1), (1, 2), (2, 3), (4, 5), (3, 100)]
Zip the lists together: The first element in the tuple will the index, the second is the value (then sort it using the second value of the tuple x[1], x is the tuple)
Or using itemgetter from the operatormodule`:
from operator import itemgetter
sorted(enumerate(a), key=itemgetter(1))
Essentially you need to do an argsort, what implementation you need depends if you want to use external libraries (e.g. NumPy) or if you want to stay pure-Python without dependencies.
The question you need to ask yourself is: Do you want the
indices that would sort the array/list
indices that the elements would have in the sorted array/list
Unfortunately the example in the question doesn't make it clear what is desired because both will give the same result:
>>> arr = np.array([1, 2, 3, 100, 5])
>>> np.argsort(np.argsort(arr))
array([0, 1, 2, 4, 3], dtype=int64)
>>> np.argsort(arr)
array([0, 1, 2, 4, 3], dtype=int64)
Choosing the argsort implementation
If you have NumPy at your disposal you can simply use the function numpy.argsort or method numpy.ndarray.argsort.
An implementation without NumPy was mentioned in some other answers already, so I'll just recap the fastest solution according to the benchmark answer here
def argsort(l):
return sorted(range(len(l)), key=l.__getitem__)
Getting the indices that would sort the array/list
To get the indices that would sort the array/list you can simply call argsort on the array or list. I'm using the NumPy versions here but the Python implementation should give the same results
>>> arr = np.array([3, 1, 2, 4])
>>> np.argsort(arr)
array([1, 2, 0, 3], dtype=int64)
The result contains the indices that are needed to get the sorted array.
Since the sorted array would be [1, 2, 3, 4] the argsorted array contains the indices of these elements in the original.
The smallest value is 1 and it is at index 1 in the original so the first element of the result is 1.
The 2 is at index 2 in the original so the second element of the result is 2.
The 3 is at index 0 in the original so the third element of the result is 0.
The largest value 4 and it is at index 3 in the original so the last element of the result is 3.
Getting the indices that the elements would have in the sorted array/list
In this case you would need to apply argsort twice:
>>> arr = np.array([3, 1, 2, 4])
>>> np.argsort(np.argsort(arr))
array([2, 0, 1, 3], dtype=int64)
In this case :
the first element of the original is 3, which is the third largest value so it would have index 2 in the sorted array/list so the first element is 2.
the second element of the original is 1, which is the smallest value so it would have index 0 in the sorted array/list so the second element is 0.
the third element of the original is 2, which is the second-smallest value so it would have index 1 in the sorted array/list so the third element is 1.
the fourth element of the original is 4 which is the largest value so it would have index 3 in the sorted array/list so the last element is 3.
If you do not want to use numpy,
sorted(range(len(seq)), key=seq.__getitem__)
is fastest, as demonstrated here.
The other answers are WRONG.
Running argsort once is not the solution.
For example, the following code:
import numpy as np
x = [3,1,2]
np.argsort(x)
yields array([1, 2, 0], dtype=int64) which is not what we want.
The answer should be to run argsort twice:
import numpy as np
x = [3,1,2]
np.argsort(np.argsort(x))
gives array([2, 0, 1], dtype=int64) as expected.
Most easiest way you can use Numpy Packages for that purpose:
import numpy
s = numpy.array([2, 3, 1, 4, 5])
sort_index = numpy.argsort(s)
print(sort_index)
But If you want that you code should use baisc python code:
s = [2, 3, 1, 4, 5]
li=[]
for i in range(len(s)):
li.append([s[i],i])
li.sort()
sort_index = []
for x in li:
sort_index.append(x[1])
print(sort_index)
We will create another array of indexes from 0 to n-1
Then zip this to the original array and then sort it on the basis of the original values
ar = [1,2,3,4,5]
new_ar = list(zip(ar,[i for i in range(len(ar))]))
new_ar.sort()
`
s = [2, 3, 1, 4, 5]
print([sorted(s, reverse=False).index(val) for val in s])
For a list with duplicate elements, it will return the rank without ties, e.g.
s = [2, 2, 1, 4, 5]
print([sorted(s, reverse=False).index(val) for val in s])
returns
[1, 1, 0, 3, 4]
Import numpy as np
FOR INDEX
S=[11,2,44,55,66,0,10,3,33]
r=np.argsort(S)
[output]=array([5, 1, 7, 6, 0, 8, 2, 3, 4])
argsort Returns the indices of S in sorted order
FOR VALUE
np.sort(S)
[output]=array([ 0, 2, 3, 10, 11, 33, 44, 55, 66])
Code:
s = [2, 3, 1, 4, 5]
li = []
for i in range(len(s)):
li.append([s[i], i])
li.sort()
sort_index = []
for x in li:
sort_index.append(x[1])
print(sort_index)
Try this, It worked for me cheers!
firstly convert your list to this:
myList = [1, 2, 3, 100, 5]
add a index to your list's item
myList = [[0, 1], [1, 2], [2, 3], [3, 100], [4, 5]]
next :
sorted(myList, key=lambda k:k[1])
result:
[[0, 1], [1, 2], [2, 3], [4, 5], [3, 100]]
A variant on RustyRob's answer (which is already the most performant pure Python solution) that may be superior when the collection you're sorting either:
Isn't a sequence (e.g. it's a set, and there's a legitimate reason to want the indices corresponding to how far an iterator must be advanced to reach the item), or
Is a sequence without O(1) indexing (among Python's included batteries, collections.deque is a notable example of this)
Case #1 is unlikely to be useful, but case #2 is more likely to be meaningful. In either case, you have two choices:
Convert to a list/tuple and use the converted version, or
Use a trick to assign keys based on iteration order
This answer provides the solution to #2. Note that it's not guaranteed to work by the language standard; the language says each key will be computed once, but not the order they will be computed in. On every version of CPython, the reference interpreter, to date, it's precomputed in order from beginning to end, so this works, but be aware it's not guaranteed. In any event, the code is:
sizediterable = ...
sorted_indices = sorted(range(len(sizediterable)), key=lambda _, it=iter(sizediterable): next(it))
All that does is provide a key function that ignores the value it's given (an index) and instead provides the next item from an iterator preconstructed from the original container (cached as a defaulted argument to allow it to function as a one-liner). As a result, for something like a large collections.deque, where using its .__getitem__ involves O(n) work (and therefore computing all the keys would involve O(n²) work), sequential iteration remains O(1), so generating the keys remains just O(n).
If you need something guaranteed to work by the language standard, using built-in types, Roman's solution will have the same algorithmic efficiency as this solution (as neither of them rely on the algorithmic efficiency of indexing the original container).
To be clear, for the suggested use case with collections.deque, the deque would have to be quite large for this to matter; deques have a fairly large constant divisor for indexing, so only truly huge ones would have an issue. Of course, by the same token, the cost of sorting is pretty minimal if the inputs are small/cheap to compare, so if your inputs are large enough that efficient sorting matters, they're large enough for efficient indexing to matter too.