itertools.permutations generates where its elements are treated as unique based on their position, not on their value. So basically I want to avoid duplicates like this:
>>> list(itertools.permutations([1, 1, 1]))
[(1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1)]
Filtering afterwards is not possible because the amount of permutations is too large in my case.
Does anybody know of a suitable algorithm for this?
Thank you very much!
EDIT:
What I basically want is the following:
x = itertools.product((0, 1, 'x'), repeat=X)
x = sorted(x, key=functools.partial(count_elements, elem='x'))
which is not possible because sorted creates a list and the output of itertools.product is too large.
Sorry, I should have described the actual problem.
class unique_element:
def __init__(self,value,occurrences):
self.value = value
self.occurrences = occurrences
def perm_unique(elements):
eset=set(elements)
listunique = [unique_element(i,elements.count(i)) for i in eset]
u=len(elements)
return perm_unique_helper(listunique,[0]*u,u-1)
def perm_unique_helper(listunique,result_list,d):
if d < 0:
yield tuple(result_list)
else:
for i in listunique:
if i.occurrences > 0:
result_list[d]=i.value
i.occurrences-=1
for g in perm_unique_helper(listunique,result_list,d-1):
yield g
i.occurrences+=1
a = list(perm_unique([1,1,2]))
print(a)
result:
[(2, 1, 1), (1, 2, 1), (1, 1, 2)]
EDIT (how this works):
I rewrote the above program to be longer but more readable.
I usually have a hard time explaining how something works, but let me try.
In order to understand how this works, you have to understand a similar but simpler program that would yield all permutations with repetitions.
def permutations_with_replacement(elements,n):
return permutations_helper(elements,[0]*n,n-1)#this is generator
def permutations_helper(elements,result_list,d):
if d<0:
yield tuple(result_list)
else:
for i in elements:
result_list[d]=i
all_permutations = permutations_helper(elements,result_list,d-1)#this is generator
for g in all_permutations:
yield g
This program is obviously much simpler:
d stands for depth in permutations_helper and has two functions. One function is the stopping condition of our recursive algorithm, and the other is for the result list that is passed around.
Instead of returning each result, we yield it. If there were no function/operator yield we would have to push the result in some queue at the point of the stopping condition. But this way, once the stopping condition is met, the result is propagated through all stacks up to the caller. That is the purpose of
for g in perm_unique_helper(listunique,result_list,d-1): yield g
so each result is propagated up to caller.
Back to the original program:
we have a list of unique elements. Before we can use each element, we have to check how many of them are still available to push onto result_list. Working with this program is very similar to permutations_with_replacement. The difference is that each element cannot be repeated more times than it is in perm_unique_helper.
Because sometimes new questions are marked as duplicates and their authors are referred to this question it may be important to mention that sympy has an iterator for this purpose.
>>> from sympy.utilities.iterables import multiset_permutations
>>> list(multiset_permutations([1,1,1]))
[[1, 1, 1]]
>>> list(multiset_permutations([1,1,2]))
[[1, 1, 2], [1, 2, 1], [2, 1, 1]]
This relies on the implementation detail that any permutation of a sorted iterable are in sorted order unless they are duplicates of prior permutations.
from itertools import permutations
def unique_permutations(iterable, r=None):
previous = tuple()
for p in permutations(sorted(iterable), r):
if p > previous:
previous = p
yield p
for p in unique_permutations('cabcab', 2):
print p
gives
('a', 'a')
('a', 'b')
('a', 'c')
('b', 'a')
('b', 'b')
('b', 'c')
('c', 'a')
('c', 'b')
('c', 'c')
Roughly as fast as Luka Rahne's answer, but shorter & simpler, IMHO.
def unique_permutations(elements):
if len(elements) == 1:
yield (elements[0],)
else:
unique_elements = set(elements)
for first_element in unique_elements:
remaining_elements = list(elements)
remaining_elements.remove(first_element)
for sub_permutation in unique_permutations(remaining_elements):
yield (first_element,) + sub_permutation
>>> list(unique_permutations((1,2,3,1)))
[(1, 1, 2, 3), (1, 1, 3, 2), (1, 2, 1, 3), ... , (3, 1, 2, 1), (3, 2, 1, 1)]
It works recursively by setting the first element (iterating through all unique elements), and iterating through the permutations for all remaining elements.
Let's go through the unique_permutations of (1,2,3,1) to see how it works:
unique_elements are 1,2,3
Let's iterate through them: first_element starts with 1.
remaining_elements are [2,3,1] (ie. 1,2,3,1 minus the first 1)
We iterate (recursively) through the permutations of the remaining elements: (1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)
For each sub_permutation, we insert the first_element: (1,1,2,3), (1,1,3,2), ... and yield the result.
Now we iterate to first_element = 2, and do the same as above.
remaining_elements are [1,3,1] (ie. 1,2,3,1 minus the first 2)
We iterate through the permutations of the remaining elements: (1, 1, 3), (1, 3, 1), (3, 1, 1)
For each sub_permutation, we insert the first_element: (2, 1, 1, 3), (2, 1, 3, 1), (2, 3, 1, 1)... and yield the result.
Finally, we do the same with first_element = 3.
You could try using set:
>>> list(itertools.permutations(set([1,1,2,2])))
[(1, 2), (2, 1)]
The call to set removed duplicates
A naive approach might be to take the set of the permutations:
list(set(it.permutations([1, 1, 1])))
# [(1, 1, 1)]
However, this technique wastefully computes replicate permutations and discards them. A more efficient approach would be more_itertools.distinct_permutations, a third-party tool.
Code
import itertools as it
import more_itertools as mit
list(mit.distinct_permutations([1, 1, 1]))
# [(1, 1, 1)]
Performance
Using a larger iterable, we will compare the performances between the naive and third-party techniques.
iterable = [1, 1, 1, 1, 1, 1]
len(list(it.permutations(iterable)))
# 720
%timeit -n 10000 list(set(it.permutations(iterable)))
# 10000 loops, best of 3: 111 µs per loop
%timeit -n 10000 list(mit.distinct_permutations(iterable))
# 10000 loops, best of 3: 16.7 µs per loop
We see more_itertools.distinct_permutations is an order of magnitude faster.
Details
From the source, a recursion algorithm (as seen in the accepted answer) is used to compute distinct permutations, thereby obviating wasteful computations. See the source code for more details.
This is my solution with 10 lines:
class Solution(object):
def permute_unique(self, nums):
perms = [[]]
for n in nums:
new_perm = []
for perm in perms:
for i in range(len(perm) + 1):
new_perm.append(perm[:i] + [n] + perm[i:])
# handle duplication
if i < len(perm) and perm[i] == n: break
perms = new_perm
return perms
if __name__ == '__main__':
s = Solution()
print s.permute_unique([1, 1, 1])
print s.permute_unique([1, 2, 1])
print s.permute_unique([1, 2, 3])
--- Result ----
[[1, 1, 1]]
[[1, 2, 1], [2, 1, 1], [1, 1, 2]]
[[3, 2, 1], [2, 3, 1], [2, 1, 3], [3, 1, 2], [1, 3, 2], [1, 2, 3]]
Here is a recursive solution to the problem.
def permutation(num_array):
res=[]
if len(num_array) <= 1:
return [num_array]
for num in set(num_array):
temp_array = num_array.copy()
temp_array.remove(num)
res += [[num] + perm for perm in permutation(temp_array)]
return res
arr=[1,2,2]
print(permutation(arr))
The best solution to this problem I have seen uses Knuth's "Algorithm L" (as noted previously by Gerrat in the comments to the original post):
http://stackoverflow.com/questions/12836385/how-can-i-interleave-or-create-unique-permutations-of-two-stings-without-recurs/12837695
Some timings:
Sorting [1]*12+[0]*12 (2,704,156 unique permutations):
Algorithm L → 2.43 s
Luke Rahne's solution → 8.56 s
scipy.multiset_permutations() → 16.8 s
To generate unique permutations of ["A","B","C","D"] I use the following:
from itertools import combinations,chain
l = ["A","B","C","D"]
combs = (combinations(l, r) for r in range(1, len(l) + 1))
list_combinations = list(chain.from_iterable(combs))
Which generates:
[('A',),
('B',),
('C',),
('D',),
('A', 'B'),
('A', 'C'),
('A', 'D'),
('B', 'C'),
('B', 'D'),
('C', 'D'),
('A', 'B', 'C'),
('A', 'B', 'D'),
('A', 'C', 'D'),
('B', 'C', 'D'),
('A', 'B', 'C', 'D')]
Notice, duplicates are not created (e.g. items in combination with D are not generated, as they already exist).
Example: This can then be used in generating terms of higher or lower order for OLS models via data in a Pandas dataframe.
import statsmodels.formula.api as smf
import pandas as pd
# create some data
pd_dataframe = pd.Dataframe(somedata)
response_column = "Y"
# generate combinations of column/variable names
l = [col for col in pd_dataframe.columns if col!=response_column]
combs = (combinations(l, r) for r in range(1, len(l) + 1))
list_combinations = list(chain.from_iterable(combs))
# generate OLS input string
formula_base = '{} ~ '.format(response_column)
list_for_ols = [":".join(list(item)) for item in list_combinations]
string_for_ols = formula_base + ' + '.join(list_for_ols)
Creates...
Y ~ A + B + C + D + A:B + A:C + A:D + B:C + B:D + C:D + A:B:C + A:B:D + A:C:D + B:C:D + A:B:C:D'
Which can then be piped to your OLS regression
model = smf.ols(string_for_ols, pd_dataframe).fit()
model.summary()
It sound like you are looking for itertools.combinations() docs.python.org
list(itertools.combinations([1, 1, 1],3))
[(1, 1, 1)]
Bumped into this question while looking for something myself !
Here's what I did:
def dont_repeat(x=[0,1,1,2]): # Pass a list
from itertools import permutations as per
uniq_set = set()
for byt_grp in per(x, 4):
if byt_grp not in uniq_set:
yield byt_grp
uniq_set.update([byt_grp])
print uniq_set
for i in dont_repeat(): print i
(0, 1, 1, 2)
(0, 1, 2, 1)
(0, 2, 1, 1)
(1, 0, 1, 2)
(1, 0, 2, 1)
(1, 1, 0, 2)
(1, 1, 2, 0)
(1, 2, 0, 1)
(1, 2, 1, 0)
(2, 0, 1, 1)
(2, 1, 0, 1)
(2, 1, 1, 0)
set([(0, 1, 1, 2), (1, 0, 1, 2), (2, 1, 0, 1), (1, 2, 0, 1), (0, 1, 2, 1), (0, 2, 1, 1), (1, 1, 2, 0), (1, 2, 1, 0), (2, 1, 1, 0), (1, 0, 2, 1), (2, 0, 1, 1), (1, 1, 0, 2)])
Basically, make a set and keep adding to it. Better than making lists etc. that take too much memory..
Hope it helps the next person looking out :-) Comment out the set 'update' in the function to see the difference.
You can make a function that uses collections.Counter to get unique items and their counts from the given sequence, and uses itertools.combinations to pick combinations of indices for each unique item in each recursive call, and map the indices back to a list when all indices are picked:
from collections import Counter
from itertools import combinations
def unique_permutations(seq):
def index_permutations(counts, index_pool):
if not counts:
yield {}
return
(item, count), *rest = counts.items()
rest = dict(rest)
for indices in combinations(index_pool, count):
mapping = dict.fromkeys(indices, item)
for others in index_permutations(rest, index_pool.difference(indices)):
yield {**mapping, **others}
indices = set(range(len(seq)))
for mapping in index_permutations(Counter(seq), indices):
yield [mapping[i] for i in indices]
so that [''.join(i) for i in unique_permutations('moon')] returns:
['moon', 'mono', 'mnoo', 'omon', 'omno', 'nmoo', 'oomn', 'onmo', 'nomo', 'oonm', 'onom', 'noom']
This is my attempt without resorting to set / dict, as a generator using recursion, but using string as input. Output is also ordered in natural order:
def perm_helper(head: str, tail: str):
if len(tail) == 0:
yield head
else:
last_c = None
for index, c in enumerate(tail):
if last_c != c:
last_c = c
yield from perm_helper(
head + c, tail[:index] + tail[index + 1:]
)
def perm_generator(word):
yield from perm_helper("", sorted(word))
example:
from itertools import takewhile
word = "POOL"
list(takewhile(lambda w: w != word, (x for x in perm_generator(word))))
# output
# ['LOOP', 'LOPO', 'LPOO', 'OLOP', 'OLPO', 'OOLP', 'OOPL', 'OPLO', 'OPOL', 'PLOO', 'POLO']
ans=[]
def fn(a, size):
if (size == 1):
if a.copy() not in ans:
ans.append(a.copy())
return
for i in range(size):
fn(a,size-1);
if size&1:
a[0], a[size-1] = a[size-1],a[0]
else:
a[i], a[size-1] = a[size-1],a[i]
https://www.geeksforgeeks.org/heaps-algorithm-for-generating-permutations/
Came across this the other day while working on a problem of my own. I like Luka Rahne's approach, but I thought that using the Counter class in the collections library seemed like a modest improvement. Here's my code:
def unique_permutations(elements):
"Returns a list of lists; each sublist is a unique permutations of elements."
ctr = collections.Counter(elements)
# Base case with one element: just return the element
if len(ctr.keys())==1 and ctr[ctr.keys()[0]] == 1:
return [[ctr.keys()[0]]]
perms = []
# For each counter key, find the unique permutations of the set with
# one member of that key removed, and append the key to the front of
# each of those permutations.
for k in ctr.keys():
ctr_k = ctr.copy()
ctr_k[k] -= 1
if ctr_k[k]==0:
ctr_k.pop(k)
perms_k = [[k] + p for p in unique_permutations(ctr_k)]
perms.extend(perms_k)
return perms
This code returns each permutation as a list. If you feed it a string, it'll give you a list of permutations where each one is a list of characters. If you want the output as a list of strings instead (for example, if you're a terrible person and you want to abuse my code to help you cheat in Scrabble), just do the following:
[''.join(perm) for perm in unique_permutations('abunchofletters')]
I came up with a very suitable implementation using itertools.product in this case (this is an implementation where you want all combinations
unique_perm_list = [''.join(p) for p in itertools.product(['0', '1'], repeat = X) if ''.join(p).count() == somenumber]
this is essentially a combination (n over k) with n = X and somenumber = k
itertools.product() iterates from k = 0 to k = X subsequent filtering with count ensures that just the permutations with the right number of ones are cast into a list. you can easily see that it works when you calculate n over k and compare it to the len(unique_perm_list)
Adapted to remove recursion, use a dictionary and numba for high performance but not using yield/generator style so memory usage is not limited:
import numba
#numba.njit
def perm_unique_fast(elements): #memory usage too high for large permutations
eset = set(elements)
dictunique = dict()
for i in eset: dictunique[i] = elements.count(i)
result_list = numba.typed.List()
u = len(elements)
for _ in range(u): result_list.append(0)
s = numba.typed.List()
results = numba.typed.List()
d = u
while True:
if d > 0:
for i in dictunique:
if dictunique[i] > 0: s.append((i, d - 1))
i, d = s.pop()
if d == -1:
dictunique[i] += 1
if len(s) == 0: break
continue
result_list[d] = i
if d == 0: results.append(result_list[:])
dictunique[i] -= 1
s.append((i, -1))
return results
import timeit
l = [2, 2, 3, 3, 4, 4, 5, 5, 6, 6]
%timeit list(perm_unique(l))
#377 ms ± 26 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
ltyp = numba.typed.List()
for x in l: ltyp.append(x)
%timeit perm_unique_fast(ltyp)
#293 ms ± 3.37 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
assert list(sorted(perm_unique(l))) == list(sorted([tuple(x) for x in perm_unique_fast(ltyp)]))
About 30% faster but still suffers a bit due to list copying and management.
Alternatively without numba but still without recursion and using a generator to avoid memory issues:
def perm_unique_fast_gen(elements):
eset = set(elements)
dictunique = dict()
for i in eset: dictunique[i] = elements.count(i)
result_list = list() #numba.typed.List()
u = len(elements)
for _ in range(u): result_list.append(0)
s = list()
d = u
while True:
if d > 0:
for i in dictunique:
if dictunique[i] > 0: s.append((i, d - 1))
i, d = s.pop()
if d == -1:
dictunique[i] += 1
if len(s) == 0: break
continue
result_list[d] = i
if d == 0: yield result_list
dictunique[i] -= 1
s.append((i, -1))
May be we can use set here to obtain unique permutations
import itertools
print('unique perms >> ', set(itertools.permutations(A)))
What about
np.unique(itertools.permutations([1, 1, 1]))
The problem is the permutations are now rows of a Numpy array, thus using more memory, but you can cycle through them as before
perms = np.unique(itertools.permutations([1, 1, 1]))
for p in perms:
print p
Related
I'm trying to find the most efficient way to find all combinations of numbers that equal a given sum.
For example: Looking to find all 3-number combos whose sum = 3
Desired Output:
[0,0,3], [0,3,0], [3,0,0], [1,1,1], [1,2,0], [1,2,0], [2,1,0], [2,0,1]
In terms of application I'm trying to generate lists of all of 3-numbered combinations which equal 100. I tried achieving this by creating a list containing numbers 0 - 100 to be used as an input argument and the following code below:
def weightList():
l=[]
for x in range(0,101):
l.append(x)
return l
def subset_sum(numbers, target, partial=[]):
s = sum(partial)
# check if the partial sum is equals to target
if (s == target)&(len(partial) == LoanCount):
print(partial)
if s >= target:
return # if we reach the number why bother to continue
for i in range(len(numbers)):
n = numbers[i]
remaining = numbers[i+1:]
subset_sum(remaining, target, partial + [n])
o=weightList()
subset_sum(o,100)
The problem is that it takes too long because of the amount of iterations it must perform.
I also need to include combinations repeating elements and different order/sequence of elements. This method does not. Can anyone provide a much faster/efficient method for me to obtain my desired out come?
you can use a generator to find all possible combinations of 3 numbers that equal a given sum:
def gen_combo_target(s):
for i in range(s + 1):
s2 = s - i
for j in range(s2 + 1):
yield (i, j , s - i - j)
list(gen_combo_target(3))
output:
[(0, 0, 3),
(0, 1, 2),
(0, 2, 1),
(0, 3, 0),
(1, 0, 2),
(1, 1, 1),
(1, 2, 0),
(2, 0, 1),
(2, 1, 0),
(3, 0, 0)]
Take a look at the stars and bars problem. The thing is, even the fastest possible solution will still be very slow because of the sheer number of answers (which can be on the order of N!), and any correct solution needs to get all of them.
Here's my shot at a solution:
def get_combinations(n, k): # gets all lists of size k summing to n
ans = []
def solve(rem, depth, k, cur):
if depth == k:
ans.append(cur)
elif depth == k-1:
solve(0, depth+1, k, cur + [rem])
else:
for i in range(rem+1):
solve(rem-i, depth+1, k, cur+[i])
solve(n, 0, k, [])
return ans
print (get_combinations(3,3))
It's about as fast as it can get as far as finding all the possible lists. However, if you wanted to just find the number of such lists, you could do it much faster using the stars and bars method referenced above.
You can also use generator and python itertools like this:
from itertools import product
def comb(nx):
list_gen = (n for n in range(nx+1))
list_ = (list(list_gen))
prod = [seq for seq in product(list_, list_, list_) if sum(seq) == nx]
print(list(prod))
result:
[(0, 0, 3), (0, 1, 2), (0, 2, 1), (0, 3, 0), (1, 0, 2), (1, 1, 1), (1, 2, 0), (2, 0, 1), (2, 1, 0), (3, 0, 0)]
and it can run about 10 us in my machine:
9.53 µs ± 177 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Given a list of numbers, how many different ways can you add them together to get a sum S?
Example:
list = [1, 2]
S = 5
1) 1+1+1+1+1 = 5
2) 1+1+1+2 = 5
3) 1+2+2 = 5
4) 2+1+1+1 = 5
5) 2+2+1 = 5
6) 1+2+1+1 = 5
7) 1+1+2+1 = 5
8) 2+1+2 = 5
Answer = 8
This is what I've tried, but it only outputs 3 as the answer
lst = [1, 2]
i = 1
result = 0
while i <= 5:
s_list = [sum(comb) for comb in combinations_with_replacement(lst, i)]
for val in s_list:
if val == 5:
result += 1
i+= 1
print(result)
However, this outputs three. I believe it outputs three because it doesn't account for the different order you can add the numbers in. Any ideas on how to solve this.
The problem should work for much larger data: however, I give this simple example to give the general idea.
Using both itertools.combinations_with_replacement and permutations:
import itertools
l = [1,2]
s = 5
res = []
for i in range(1, s+1):
for tup in itertools.combinations_with_replacement(l, i):
if sum(tup) == s:
res.extend(list(itertools.permutations(tup, i)))
res = list(set(res))
print(res)
[(1, 2, 2),
(2, 2, 1),
(1, 1, 2, 1),
(1, 2, 1, 1),
(2, 1, 1, 1),
(1, 1, 1, 2),
(2, 1, 2),
(1, 1, 1, 1, 1)]
print(len(res))
# 8
How about using dynamic programming? I believe it's more easy to understand and can be implemented easily.
def cal(target, choices, record):
min_choice = min(choices)
if min_choice > target:
return False
for i in range(0, target+1):
if i == 0:
record.append(1)
elif i < min_choice:
record.append(0)
elif i == min_choice:
record.append(1)
else:
num_solution = 0
j = 0
while j < len(choices) and i-choices[j] >= 0:
num_solution += record[i-choices[j]]
j += 1
record.append(num_solution)
choices = [1, 2]
record = []
cal(5, choices, record)
print(record)
print(f"Answer:{record[-1]}")
The core idea here is using an extra record array to record how many ways can be found to get current num, e.g. record[2] = 2 means we can use to ways to get a sum of 2 (1+1 or 2).
And we have record[target] = sum(record[target-choices[i]]) where i iterates over choices. Try to think, the way of getting sum=5 must be related with the way of getting sum=4 and so on.
Use Dynamic Programming.
We suppose that your list consists of [1,2,5] so we have this recursive function :
f(n,[1,2,5]) = f(n-1,[1,2,5]) + f(n-2,[1,2,5]) + f(n-5,[1,2,5])
Because if the first number in sum is 1 then you have f(n-1,[1,2,5]) options for the rest and if it is 2 you have f(n-2,[1,2,5]) option for the rest and so on ...
so start from f(1) and work your way up with Dynamic programming. this solution in the worst case is O(n^2) and this happens when your list has O(n) items.
Solution would be something like this:
answers = []
lst = [1,2]
number = 5
def f(target):
val = 0
for i in lst: #O(lst.count())
current = target - i
if current > 0:
val += answers[current-1]
if lst.__contains__(target): #O(lst.count())
val += 1
answers.insert(target,val)
j = 1;
while j<=number: #O(n) for while loop
f(j)
j+=1
print(answers[number-1])
here is a working version.
You'd want to use recursion to traverse through each possibility for each stage of addition, and pass back the numbers used once you've reached a number that is equal to the expected.
def find_addend_combinations(sum_value, addend_choices, base=0, history=None):
if history is None: history = []
if base == sum_value:
return tuple(history)
elif base > sum_value:
return None
else:
results = []
for v in addend_choices:
r = find_addend_combinations(sum_value, addend_choices, base + v,
history + [v])
if isinstance(r, tuple):
results.append(r)
elif isinstance(r, list):
results.extend(r)
return results
You could write the last part a list comprehension but I think this way is clearer.
Combinations with the elements in a different order are considered to equivalent. For example, #3 and #5 from your list of summations are considered equivalent if you are only talking about combinations.
In contrast, permutations consider the two collections unique if they are comprised of the same elements in a different order.
To get the answer you are looking for you need to combine both concepts.
First, use your technique to find combinations that meet your criteria
Next, permute the collection of number from the combination
Finally, collect the generated permutations in a set to remove duplicates.
[ins] In [01]: def combination_generator(numbers, k, target):
...: assert k > 0, "Must be a positive number; 'k = {}".format(k)
...: assert len(numbers) > 0, "List of numbers must have at least one element"
...:
...: for candidate in (
...: {'numbers': combination, 'sum': sum(combination)}
...: for num_elements in range(1, k + 1)
...: for combination in itertools.combinations_with_replacement(numbers, num_elements)
...: ):
...: if candidate['sum'] != target:
...: continue
...: for permuted_candidate in itertools.permutations(candidate['numbers']):
...: yield permuted_candidate
...:
[ins] In [02]: {candidate for candidate in combination_generator([1, 2], 5, 5)}
Out[02]:
{(1, 1, 1, 1, 1),
(1, 1, 1, 2),
(1, 1, 2, 1),
(1, 2, 1, 1),
(1, 2, 2),
(2, 1, 1, 1),
(2, 1, 2),
(2, 2, 1)}
I've created a cartesian product and printed from the following code:
A = [0, 3, 5]
B = [5, 10, 15]
C = product(A, B)
for n in C:
print(n)
And we have a result of
(0, 5)
(0, 10)
(0, 15)
(3, 5)
(3, 10)
(3, 15)
(5, 5)
(5, 10)
(5, 15)
Is it possible to check the sum of each set within the cartesian product to a target value of 10? Can we then return or set aside the sets that match?
Result should be:
(0, 10)
(5, 5)
Finally, how can I count the frequency of numbers in my resulting subset to show that 0 appeared once, 10 appeared once, and 5 appeared twice? I would appreciate any feedback or guidance.
You can use a list comprehension like this:
C = [p for p in product(A, B) if sum(p) == 10]
And to count the frequency of the numbers in the tuples, as #Amadan suggests, you can use collections.Counter after using itertools.chain.from_iterable to chain the sub-lists into one sequence:
from collections import Counter
from itertools import chain
Counter(chain.from_iterable(C))
which returns, given your sample input:
Counter({5: 2, 0: 1, 10: 1})
which is an instance of a dict subclass, so you can iterate over its items for output:
for n, freq in Counter({5: 2, 0: 1, 10: 1}).items():
print('{}: {}'.format(n, freq))
I want find all possible ways to map a series of (contiguous) integers M = {0,1,2,...,m} to another series of integers N = {0,1,2,...,n} where m > n, subject to the constraint that only contiguous integers in M map to the same integer in N.
The following piece of python code comes close (start corresponds to the first element in M, stop-1 corresponds to the last element in M, and nbins corresponds to |N|):
import itertools
def find_bins(start, stop, nbins):
if (nbins > 1):
return list(list(itertools.product([range(start, ii)], find_bins(ii, stop, nbins-1))) for ii in range(start+1, stop-nbins+2))
else:
return [range(start, stop)]
E.g
In [20]: find_bins(start=0, stop=5, nbins=3)
Out[20]:
[[([0], [([1], [2, 3, 4])]),
([0], [([1, 2], [3, 4])]),
([0], [([1, 2, 3], [4])])],
[([0, 1], [([2], [3, 4])]),
([0, 1], [([2, 3], [4])])],
[([0, 1, 2], [([3], [4])])]]
However, as you can see the output is nested, and for the life of me, I cant find a way to properly amend the code without breaking it.
The desired output would look like this:
In [20]: find_bins(start=0, stop=5, nbins=3)
Out[20]:
[[(0), (1), (2, 3, 4)],
[(0), (1, 2), (3, 4)],
[(0), (1, 2, 3), (4)],
[(0, 1), (2), (3, 4)],
[(0, 1), (2, 3), (4)],
[(0, 1, 2), (3), (4)]]
I suggest a different approach: a partitioning into n non-empty bins is uniquely determined by the n-1 distinct indices marking the boundaries between the bins, where the first marker is after the first element, and the final marker before the last element. itertools.combinations() can be used directly to generate all such index tuples, and then it's just a matter of using them as slice indices. Like so:
def find_nbins(start, stop, nbins):
from itertools import combinations
base = range(start, stop)
nbase = len(base)
for ixs in combinations(range(1, stop - start), nbins - 1):
yield [tuple(base[lo: hi])
for lo, hi in zip((0,) + ixs, ixs + (nbase,))]
Then, e.g.,
for x in find_nbins(0, 5, 3):
print(x)
displays:
[(0,), (1,), (2, 3, 4)]
[(0,), (1, 2), (3, 4)]
[(0,), (1, 2, 3), (4,)]
[(0, 1), (2,), (3, 4)]
[(0, 1), (2, 3), (4,)]
[(0, 1, 2), (3,), (4,)]
EDIT: Making it into 2 problems
Just noting that there's a more general underlying problem here: generating the ways to break an arbitrary sequence into n non-empty bins. Then the specific question here is applying that to the sequence range(start, stop). I believe viewing it that way makes the code easier to understand, so here it is:
def gbins(seq, nbins):
from itertools import combinations
base = tuple(seq)
nbase = len(base)
for ixs in combinations(range(1, nbase), nbins - 1):
yield [base[lo: hi]
for lo, hi in zip((0,) + ixs, ixs + (nbase,))]
def find_nbins(start, stop, nbins):
return gbins(range(start, stop), nbins)
This does what I want; I will gladly accept simpler, more elegant solutions:
def _split(start, stop, nbins):
if (nbins > 1):
out = []
for ii in range(start+1, stop-nbins+2):
iterator = itertools.product([range(start, ii)], _split(ii, stop, nbins-1))
for item in iterator:
out.append(item)
return out
else:
return [range(start, stop)]
def _unpack(nested):
unpacked = []
if isinstance(nested, (list, tuple)):
for item in nested:
if isinstance(item, tuple):
for subitem in item:
unpacked.extend(_unpack(subitem))
elif isinstance(item, list):
unpacked.append([_unpack(subitem) for subitem in item])
elif isinstance(item, int):
unpacked.append([item])
return unpacked
else: # integer
return nested
def find_nbins(start, stop, nbins):
nested = _split(start, stop, nbins)
unpacked = [_unpack(item) for item in nested]
return unpacked
I want to generate all permutations of an n by n list with max value n-1, for example, for n=3, all the possible lists are as follows
[0,0,0]
[0,0,1]
[0,0,2]
[0,1,0]
[0,1,1]
[0,1,2]
[0,2,0]
...
[2,2,2]
I realize this grows very large very quickly (there are n^n permutations). I currently have the following working code which uses recursion
def generatePermutations(allPerms, curPerm, curIndex, n):
#allPerms is a reference to the full set of permutations
#curIndex is the index which is currently being changed
if (curIndex == n - 1):
for i in range(n):
curPerm[curIndex] = i
allPerms.append(list(curPerm))
else:
for i in range(n):
curPerm[curIndex] = i
#recursively generate all permutations past our curIndex
generatePermutations(allPerms, curPerm, curIndex + 1, n)
allPermutations = []
currentPermutation = []
n = 4
for i in range(n):
currentPermutation.append(0)
generatePermutations(allPermutations, currentPermutation, 0, n)
In trying to find a non-recursive solution I've hit a wall, I'm thinking there would have to be n number of nested for loops, which I can't figure out how to do for arbitrary n. The only ideas I've had are doing some kind of fancy adding of functions containing the loops to a list to be run somehow, or even more absurdly, generating the code programmatically and passing it to an eval call. My gut tells me these are more complicated than necessary. Can anyone think of a solution? thanks!
Simple way, with a library call:
import itertools
def lists(n):
return itertools.product(xrange(n), repeat=n)
This returns an iterator, rather than a list. You can get a list if you want by calling list on the result.
If you want to do this without foisting the job onto itertools, you can count in base n, incrementing the last digit and carrying whenever you hit n:
def lists(n):
l = [0]*n
while True:
yield l[:]
for i in reversed(xrange(n)):
if l[i] != n-1:
break
l[i] = 0
else:
# All digits were n-1; we're done
return
l[i] += 1
You can use itertools.permutations() to handle the whole problem for you, in one go:
from itertools import permutations
allPermutations = list(permutations(range(4))
The documentation includes Python code that details alternative implementations in Python for that function; a version using itertools.product(), for example:
from itertools import product
def permutations(iterable, r=None):
pool = tuple(iterable)
n = len(pool)
r = n if r is None else r
for indices in product(range(n), repeat=r):
if len(set(indices)) == r:
yield tuple(pool[i] for i in indices)
However, your expected output is just a product of range(3) over 3:
>>> from itertools import product
>>> for p in product(range(3), repeat=3):
... print p
...
(0, 0, 0)
(0, 0, 1)
(0, 0, 2)
(0, 1, 0)
(0, 1, 1)
(0, 1, 2)
(0, 2, 0)
(0, 2, 1)
(0, 2, 2)
(1, 0, 0)
(1, 0, 1)
(1, 0, 2)
(1, 1, 0)
(1, 1, 1)
(1, 1, 2)
(1, 2, 0)
(1, 2, 1)
(1, 2, 2)
(2, 0, 0)
(2, 0, 1)
(2, 0, 2)
(2, 1, 0)
(2, 1, 1)
(2, 1, 2)
(2, 2, 0)
(2, 2, 1)
(2, 2, 2)
Permutations is a much shorter sequence:
>>> from itertools import permutations
>>> for p in permutations(range(3)):
... print p
...
(0, 1, 2)
(0, 2, 1)
(1, 0, 2)
(1, 2, 0)
(2, 0, 1)
(2, 1, 0)