Aim: I'd like to get (or be able to work with) a set of all possible permutations obtained from a list of strings.
Example in Python:
import pandas as pd
import itertools
list1 = ['A', 'A', 'B', 'B']
# Get all permutations
list1_perm = list(itertools.permutations(list1))
len(list1_perm)
24
list1_perm
[('A', 'A', 'B', 'B'),
('A', 'A', 'B', 'B'),
('A', 'B', 'A', 'B'),
('A', 'B', 'B', 'A'),
('A', 'B', 'A', 'B'),
('A', 'B', 'B', 'A'),
('A', 'A', 'B', 'B'),
('A', 'A', 'B', 'B'),
('A', 'B', 'A', 'B'),
('A', 'B', 'B', 'A'),
('A', 'B', 'A', 'B'),
('A', 'B', 'B', 'A'),
('B', 'A', 'A', 'B'),
('B', 'A', 'B', 'A'),
('B', 'A', 'A', 'B'),
('B', 'A', 'B', 'A'),
('B', 'B', 'A', 'A'),
('B', 'B', 'A', 'A'),
('B', 'A', 'A', 'B'),
('B', 'A', 'B', 'A'),
('B', 'A', 'A', 'B'),
('B', 'A', 'B', 'A'),
('B', 'B', 'A', 'A'),
('B', 'B', 'A', 'A')]
Since for my analysis ('A', 'A', 'B', 'B') is the same as ('A', 'A', 'B', 'B'), (although the 'A' may have changed the position), I do:
# Get set of permutations
set1_perm = set(itertools.permutations(list1))
len(set1_perm)
6
set1_perm
{('A', 'A', 'B', 'B'),
('A', 'B', 'A', 'B'),
('A', 'B', 'B', 'A'),
('B', 'A', 'A', 'B'),
('B', 'A', 'B', 'A'),
('B', 'B', 'A', 'A')}
Now this is great, but the list I want to work with has 481 strings, with 5 unique strings with different frequencies:
len(real_list)
481
len(set(real_list))
5
# Count number of times each unique value appears
pd.Series(real_list).value_counts()
A 141
B 116
C 80
D 78
E 66
This is not a problem for itertools.permutations(real_list), but when I want to get the set, it takes ages. This is because the number of permutations is 9.044272819E+1082.
What I want to do is:
First I want to know the number of unique elements in that permutation space, i.e. the length of the set. To get the number of unique elements it might be possible to do it analytically, however since the frequency of each unique element is different I don't how to do that.
Second I would like to be able to get a sample of those unique elements in the set of permutations.
I'd appreciate any help provided.
Best,
Alejandro
Calculating the number of unique permutations is simply a matter of applying a formula - we know that were we to have n distinct elements, we would have n! permutations. Then to account for repeated permutations we must divide by each count of permutations of repeated letters. This is a multinomial coefficient.
So a simple implementation to generate the unique count may look something like
from math import factorial
from functools import reduce
from collections import Counter
def perm_cnt(l):
denom = reduce(lambda x,y: x*factorial(y), Counter(l).values())
return factorial(len(l)) // denom
Then sampling from the unique permutations is probably most simply achieved by just ensuring your sample values remain unique, as opposed to trying to generate all of the unique values and then sampling. There is a recipe in the itertools module, random_permutation, which could be useful for this.
def random_permutation(iterable, r=None):
"Random selection from itertools.permutations(iterable, r)"
pool = tuple(iterable)
r = len(pool) if r is None else r
return tuple(random.sample(pool, r))
So creating a unique sample might look something like
def uniq_sample(l, size):
s = set()
perm_size = perm_cnt(l)
cnt = 0
while cnt < min(perm_size, size):
samp = random_permutation(l)
if samp not in s:
s.add(samp)
cnt += 1
return s
Demo
>>> perm_cnt(list1)
6
>>> perm_cnt(['a']*3 + ['b']*5 + ['d']*2)
2520
>>> perm_cnt(np.random.randint(10, size=20))
105594705216000
>>> uniq_sample(list1, 4)
{('A', 'A', 'B', 'B'),
('B', 'A', 'A', 'B'),
('B', 'A', 'B', 'A'),
('B', 'B', 'A', 'A')}
Related
I am creating a program, in python, that allows me to generate NFT Art based on the given assets.
Obviously the number of arts that can be generated varies according to the assets (layers and layer images) and this is precisely the problem, how can I calculate the possible combinations also counting the optional layers?
To be clearer:
for example I have 4 layers:
l1 = ["A","B"]
l2 = ["C"]
l3 = ["D","E"] #optional
l4 = ["F","G"] #optional
where l3 and l4 are optional. So the combinations I expect are:
1. ["A","C"]
2. ["B","C"]
3. ["A","C","D"]
4. ["A","C","E"]
5. ["B","C","D"]
6. ["B","C","E"]
7. ["A","C","F"]
8. ["A","C","G"]
9. ["B","C","F"]
10. ["B","C","G"]
11. ["A","C","D","F"]
12. ["A","C","D","G"]
13. ["A","C","E","F"]
14. ["A","C","E","G"]
15. ["B","C","D","F"]
16. ["B","C","D","G"]
17. ["B","C","E","F"]
18. ["B","C","E","G"]
How can I get to that? I tried with itertools.product but obviusly it take into account all lists
One way to do this is with the powerset recipe from the itertools docs. Chaining together the products of 'required lists' with every subset of the 'optional-list-set' gives a generator that produces each possibility once:
def powerset(iterable):
"""powerset([1,2,3]) --> () (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)"""
s = list(iterable)
return chain.from_iterable(combinations(s, r) for r in range(len(s) + 1))
def product_with_optional(required_sequences, optional_sequences):
return chain.from_iterable(product(*required_sequences, *optionals)
for optionals in powerset(optional_sequences))
optional_combinations = product_with_optional(required_sequences=[l1, l2],
optional_sequences=[l3, l4])
which gives:
1 ('A', 'C')
2 ('B', 'C')
3 ('A', 'C', 'D')
4 ('A', 'C', 'E')
5 ('B', 'C', 'D')
6 ('B', 'C', 'E')
7 ('A', 'C', 'F')
8 ('A', 'C', 'G')
9 ('B', 'C', 'F')
10 ('B', 'C', 'G')
11 ('A', 'C', 'D', 'F')
12 ('A', 'C', 'D', 'G')
13 ('A', 'C', 'E', 'F')
14 ('A', 'C', 'E', 'G')
15 ('B', 'C', 'D', 'F')
16 ('B', 'C', 'D', 'G')
17 ('B', 'C', 'E', 'F')
18 ('B', 'C', 'E', 'G')
I'm assuming the ordering of the optional layers matter, so you can just iteratively create all combinations of the optional layers, then use itertools.product on the layers + optional_layers to generate the lists.
import itertools
from pprint import pprint
l1 = ["A","B"]
l2 = ["C"]
l3 = ["D","E"] #optional
l4 = ["F","G"] #optional
layers = [l1, l2]
optional_layers = [l3, l4]
results = []
results += itertools.product(*layers)
for i in range(len(optional_layers) + 1):
comb = itertools.combinations(optional_layers, r=i)
for c in comb:
results += itertools.product(*layers, *c)
pprint(results)
Output
[('A', 'C'),
('B', 'C'),
('A', 'C'),
('B', 'C'),
('A', 'C', 'D'),
('A', 'C', 'E'),
('B', 'C', 'D'),
('B', 'C', 'E'),
('A', 'C', 'F'),
('A', 'C', 'G'),
('B', 'C', 'F'),
('B', 'C', 'G'),
('A', 'C', 'D', 'F'),
('A', 'C', 'D', 'G'),
('A', 'C', 'E', 'F'),
('A', 'C', 'E', 'G'),
('B', 'C', 'D', 'F'),
('B', 'C', 'D', 'G'),
('B', 'C', 'E', 'F'),
('B', 'C', 'E', 'G')]
so I will do my best to explain what I'm looking for,
at the moment I have a 100 item list that I want to repetitively shuffle using a set pattern to first check if the pattern will eventually bring me back to where I began
and 2 to print the result of each loop to a text file.
so using a 3 item list as my example
[a,b,c]
and the shuffle pattern [3 1 2]
where the 3rd item becomes the first.
the first item becomes the second
and the second item becomes the 3rd
on a loop would generate the following patterns
[a,b,c]
[3,1,2]
[c,a,b]
[b,c,a]
[a,b,c]
but I have a list at the moment of 100 items that I need to find every single arrangement for a few different patterns I would like to test out.
does anyone know of a way to do this in python please.
You can define function and call this function multi times like below:
>>> def func(lst, ptr):
... return [lst[idx-1] for idx in ptr]
>>> lst = ['a','b','c']
>>> ptr = [3,1,2]
>>> for _ in range(5):
... lst = func(lst, ptr)
... print(lst)
['c', 'a', 'b']
['b', 'c', 'a']
['a', 'b', 'c']
['c', 'a', 'b']
['b', 'c', 'a']
You could use numpy advanced integer indexing if your list contains a numeric type:
import numpy as np
original_list=[1,2,3]
numpy_array = np.array(original_list)
pattern = [2,1,0]
print(numpy_array[pattern])
>>> array([3, 2, 1])
def rearrange(pattern : list,L:list):
new_list = []
for i in pattern :
new_list.append(L[i-1])
return new_list
print(rearrange([3,1,2],['a','b','c']))
output :
['c', 'a', 'b']
Itertools could be what you need.
import itertools
p = itertools.permutations(['a','b','c', 'd'])
list(p)
Output:
[('a', 'b', 'c', 'd'),
('a', 'b', 'd', 'c'),
('a', 'c', 'b', 'd'),
('a', 'c', 'd', 'b'),
('a', 'd', 'b', 'c'),
('a', 'd', 'c', 'b'),
('b', 'a', 'c', 'd'),
('b', 'a', 'd', 'c'),
('b', 'c', 'a', 'd'),
('b', 'c', 'd', 'a'),
('b', 'd', 'a', 'c'),
('b', 'd', 'c', 'a'),
('c', 'a', 'b', 'd'),
('c', 'a', 'd', 'b'),
('c', 'b', 'a', 'd'),
('c', 'b', 'd', 'a'),
('c', 'd', 'a', 'b'),
('c', 'd', 'b', 'a'),
('d', 'a', 'b', 'c'),
('d', 'a', 'c', 'b'),
('d', 'b', 'a', 'c'),
('d', 'b', 'c', 'a'),
('d', 'c', 'a', 'b'),
('d', 'c', 'b', 'a')]
I have a list of tuples as below.
x = [('b', 'c'),
('c',),
('a', 'c', 'b'),
('b', 'c', 'a', 'd'),
('b', 'c', 'a'),
('a', 'b'),
('a', 'b', 'c', 'd'),
('a', 'c', 'b', 'd'),
('b',),
('c', 'a'),
('a', 'b', 'c'),
('a',)]
I want to give input like ('a') then it should give output like,
[('a', 'c', 'b'), ('a', 'b'),('a', 'b', 'c', 'd'),('a', 'c', 'b', 'd'),('a', 'b', 'c')]
#everything starts with a. But not "a".
or for input of ('a','b') it should give an output of
[('a', 'b', 'c', 'd'),('a', 'b', 'c')]
#everything start with ('a','b') but not ('a','b') itself.
I tried to use but no success.
print(filter(lambda x: ("a","b") in x, x))
>>> <filter object at 0x00000214B3A545F8>
def f(lst, target):
return [t for t in lst if len(t) > len(target) and all(a == b for a, b in zip(t, target))]
so that:
f(x, ('a', 'b'))
returns:
[('a', 'b', 'c', 'd'), ('a', 'b', 'c')]
Tuples are matched lexicographically in python, meaning that there elements are compared pair by pair, regardless of their type.
You can extract the portion of each tuple of the same length as your prefix and compare with ==:
def find_prefixes(prefix, sequence):
n = len(prefix)
return[x for x in sequence if x[:n] == prefix and len(x) > n]
List comprehensions of this type are indeed equivalent to filter calls, so you can do
def find_prefixes(prefix, sequence):
n = len(prefix)
return list(filter(lambda x: x[:n] == prefix and len(x) > n, sequence))
Doing a linear search is not a very efficient way to solve this problem. The data structure known as a Trie is made specifically for finding prefixes. It arranges all your data into a single tree. Here is a popular Python implementation you can use with the appropriate attribution: https://stackoverflow.com/a/11016430/2988730
Firstly, use list(filter(...)) to convert a filter object to a list, but your filter doesn't do what you want - it checks membership, not subsequence. You can check subsequence by using a slice.
Then you just need to add a check that the match is longer than the subsequence.
Also, a filter of a lambda is better written as a comprehension.
for sub in ('a',), ('a', 'b'):
n = len(sub)
out = [t for t in x if t[:n] == sub and len(t) > n]
print(out)
Output:
[('a', 'c', 'b'), ('a', 'b'), ('a', 'b', 'c', 'd'), ('a', 'c', 'b', 'd'), ('a', 'b', 'c')]
[('a', 'b', 'c', 'd'), ('a', 'b', 'c')]
list(filter(lambda y: all([y[i] == z for i,z in enumerate(inp)]) if len(y)>=len(inp) else False, x))
for
inp = ('a', 'b')
output will be
[('a', 'b'), ('a', 'b', 'c', 'd'), ('a', 'b', 'c')]
Could someone show me how to get the values from a dictionary in row order (STARTING FROM SECOND ROW) e.g. get the first value from all rows, when rows finish, move onto getting the second value from all rows until all the values have been collected (when no more columns).
E.g. here is a table:
('E', 'K', 'Y') # <- don't get the values from the first row
('B', 'C', 'B') # start getting values from this (second row)
('C', 'B', 'F')
('F', 'C', 'A')
('C', 'C', 'C')
('B', 'C', 'B')
('E', 'B', 'F')
('B', 'B', 'F')
('D', 'A', 'A')
('A', 'D', 'F')
The table above should print the values:
BCFCBEBDACBCCCBBADBFACBFFAF
Creating an encode and decoder program. stuck with printing values in correct order.
Thanks.
If you have all the tuples in a list you can use zip and join :
>>> l=[('E', 'K', 'Y') ,('B', 'C', 'B') ,('C', 'B', 'F'),('F', 'C', 'A'),('C', 'C', 'C'),('B', 'C', 'B'),('E', 'B', 'F'),('B', 'B', 'F'),('D', 'A', 'A')]
>>> ''.join([''.join(i) for i in zip(*l[1:])])
'BCFCBEBDCBCCCBBABFACBFFA'
Alterantive way, that gets a list of characters instead of str:
print([v for r in (row for row in zip(*l[1:])) for v in r])
#['B', 'C', 'F', 'C', 'B', 'E', 'B', 'D', 'A', 'C', 'B', 'C', 'C', 'C', 'B', 'B', 'A', 'D', 'B', 'F', 'A', 'C', 'B', 'F', 'F', 'A', 'F']
If the data structure is a list of tuples, meaning:
l =[('C', 'B', 'F'),
('F', 'C', 'A'),
('C', 'C', 'C'),
('B', 'C', 'B'),
('E', 'B', 'F'),
('B', 'B', 'F'),
('D', 'A', 'A'),
('A', 'D', 'F')]
Something like this could solve the problem:
string = ""
for i in range(3):
for element in range(1, len(l)):
string+=l[element][i]
I'm making a specialized utility similar to John the Ripper, and I'd like to use a loop that returns all strings up to x characters that can be formed from the string. For example, if the "seed" string is abcd, it should return:
a
b
c
d
aa
ab
ac
and so on. If the character limit is 10, it would generate aaaaaaaaaa, abcddcbaaa, and so on. Is there a simple for loop to do this, or is it more complicated than that?
I'll self-plagiarize from this answer and add a maximum length:
from itertools import product
def multiletters(seq, max_length):
for n in range(1, max_length+1):
for s in product(seq, repeat=n):
yield ''.join(s)
which gives
>>> list(multiletters("abc", 2))
['a', 'b', 'c', 'aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']
>>> list(multiletters("abcd", 4))[:8]
['a', 'b', 'c', 'd', 'aa', 'ab', 'ac', 'ad']
and so on.
def all_strings(alphabet, length_limit=None):
n_letters = len(alphabet)
length = 0
n_strings = 1
buf = []
while True:
for i in xrange(0, n_strings):
k = i
for j in xrange(length - 1, -1, -1):
buf[j] = alphabet[k % n_letters]
k /= n_letters
yield ''.join(buf)
length += 1
if length == length_limit:
break
n_strings *= n_letters
buf.append(alphabet[0])
for s in all_strings('abcd', length_limit=4):
print s
As pointed out in the comment's use itertools.premutations or even better take a look #DSM's answer, as this one misses the doubles:
In [194]: from itertools import chain, permutations
In [195]: s = 'abcd'
In [196]: map(''.join,chain.from_iterable(permutations(s,x)
for x in range(1,len(s)+1)))
Out[196]:
['a',
'b',
'c',
'd',
'ab',
'ac',
'ad',
'ba',
'bc',
'bd',
...
'dbca',
'dcab',
'dcba']
Anyway, here's a version of #DSM's answer that returns a list:
from itertools import product
def ms(seq, max_length):
return [''.join(s) for n in range(1, max_length+1)
for s in product(seq,repeat=n)]
Use itertools.permuataions.
for i in range(2,4):
tuples = itertools.permutations('abca' , i)
print( list(tuples))
The example code sequence generates:
[('a', 'b'), ('a', 'c'), ('a', 'a'), ('b', 'a'), ('b', 'c'), ('b', 'a'), ('c', 'a'), ('c', 'b'), ('c', 'a'), ('a', 'a'), ('a', 'b'), ('a', 'c')]
[('a', 'b', 'c'), ('a', 'b', 'a'), ('a', 'c', 'b'), ('a', 'c', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c'), ('b', 'a', 'c'), ('b', 'a', 'a'), ('b', 'c', 'a'), ('b', 'c', 'a'), ('b', 'a', 'a'), ('b', 'a', 'c'), ('c', 'a', 'b'), ('c', 'a', 'a'), ('c', 'b', 'a'), ('c', 'b', 'a'), ('c', 'a', 'a'), ('c', 'a', 'b'), ('a', 'a', 'b'), ('a', 'a', 'c'), ('a', 'b', 'a'), ('a', 'b', 'c'), ('a', 'c', 'a'), ('a', 'c', 'b')]