I have a project and I am trying to do a complex cryptographic method. Is it normal to use nested loops a lot? Or did I miss something?
I intend to create a method which tries all strings to find password. For example, when I input ['A','B'] it should create each of these one by one:
['A', 'B', 'AA', 'AB', 'BA', 'BB', 'AAA', 'AAB', 'ABA', 'ABB', 'BAA', 'BAB', 'BBA', 'BBB', (triple permutations), (quadruple permutations), and it goes on ...]
My code:
def rulefinder():
global externalrul1
global externalrul2
rul2 = uniquestring1[0]
rul1 = uniquestring1[0]
for n in range (0,3):
for m in range (0, len(uniquestring1)):
for z in range(0, n+1):
for k in range (0,3):
for i in range(0, len(uniquestring1)):
for o in range(0, k+1):
for y in range (0, len(uniquestring1)):
rul2 = rul2[:-1] + uniquestring1[y]
for x in range (0, len(uniquestring1)):
rul1= rul1[:-1] + uniquestring1[x]
code=""
for cha in Genx1:
if cha==uniquestring1[0]:
code +=codereplacer(rul1)
elif cha==uniquestring1[1]:
code +=codereplacer(rul2)
print(code)
print(uniquestring1[0],rul1)
print(uniquestring1[1],rul2)
print(LastString)
if code == LastString:
axiom1=uniquestring1[0]
axiom2=uniquestring1[1]
externalrul1=rul1
externalrul2=rul2
print('rules are found')
print("First RULE:", uniquestring1[0], rul1)
print("Second RULE:", uniquestring1[1], rul2)
findsubgeneration(code, axiom1, rul1, axiom2, rul2)
return
rul1 = rul1[:o] + uniquestring1[i] + rul1[(o + 1):]
rul1 += codereplacer(uniquestring1[i])
rul2 = rul2[:z] + uniquestring1[m] + rul2[(z + 1):]
rul1 =""
rul2 += codereplacer(uniquestring1[m])
You are doing things in a very MATLAB-ic way (lots of loops, and iterating only on the index, instead of elements of iterables). Pythonic way is much more efficient (uses generators under the hood), and is much cleaner:
import itertools
l = ['A','B']
n = 5 # The size of your "expanding permutations"
res = []
for i in range(1,n):
res.extend(map(''.join, list(itertools.product(l, repeat=i))))
print res
Result:
['A', 'B', 'AA', 'AB', 'BA', 'BB', 'AAA', 'AAB', 'ABA', 'ABB', 'BAA', 'BAB', 'BBA', 'BBB',
'AAAA', 'AAAB', 'AABA', 'AABB', 'ABAA', 'ABAB', 'ABBA', 'ABBB', 'BAAA', 'BAAB', 'BABA', 'BABB', 'BBAA', 'BBAB', 'BBBA', 'BBBB']
Caution: As long as you are working with a small list, you should be fine but as l grows, the result is going to grow exponentially, and may eat out your memory. Hence, instead of doing an extend to the res list, you can write the results inside the loop into disk, on the fly.
Related
I would like to generate a list of combinations. I will try to simplify my problem to make it understandable.
We have 3 variables :
x : number of letters
k : number of groups
n : number of letters per group
I would like to generate using python a list of every possible combinations, without any duplicate knowing that : i don't care about the order of the groups and the order of the letters within a group.
As an example, with x = 4, k = 2, n = 2 :
# we start with 4 letters, we want to make 2 groups of 2 letters
letters = ['A','B','C','D']
# here would be a code that generate the list
# Here is the result that is very simple, only 3 combinations exist.
combos = [ ['AB', 'CD'], ['AC', 'BD'], ['AD', 'BC'] ]
Since I don't care about the order of or within the groups, and letters within a group, ['AB', 'CD'] and ['DC', 'BA'] is a duplicate.
This is a simplification of my real problem, which has those values : x = 12, k = 4, n = 3. I tried to use some functions from itertools, but with that many letters my computer freezes because it's too many combinations.
Another way of seeing the problem : you have 12 players, you want to make 4 teams of 3 players. What are all the possibilities ?
Could anyone help me to find an optimized solution to generate this list?
There will certainly be more sophisticated/efficient ways of doing this, but here's an approach that works in a reasonable amount of time for your example and should be easy enough to adapt for other cases.
It generates unique teams and unique combinations thereof, as per your specifications.
from itertools import combinations
# this assumes that team_size * team_num == len(players) is a given
team_size = 3
team_num = 4
players = list('ABCDEFGHIJKL')
unique_teams = [set(c) for c in combinations(players, team_size)]
def duplicate_player(combo):
"""Returns True if a player occurs in more than one team"""
return len(set.union(*combo)) < len(players)
result = (combo for combo in combinations(unique_teams, team_num) if not duplicate_player(combo))
result is a generator that can be iterated or turned into a list with list(result). On kaggle.com, it takes a minute or so to generate the whole list of all possible combinations (a total of 15400, in line with the computations by #beaker and #John Coleman in the comments). The teams are tuples of sets that look like this:
[({'A', 'B', 'C'}, {'D', 'E', 'F'}, {'G', 'H', 'I'}, {'J', 'K', 'L'}),
({'A', 'B', 'C'}, {'D', 'E', 'F'}, {'G', 'H', 'J'}, {'I', 'K', 'L'}),
({'A', 'B', 'C'}, {'D', 'E', 'F'}, {'G', 'H', 'K'}, {'I', 'J', 'L'}),
...
]
If you want, you can cast them into strings by calling ''.join() on each of them.
Another solution (players are numbered 0, 1, ...):
import itertools
def equipartitions(base_count: int, group_size: int):
if base_count % group_size != 0:
raise ValueError("group_count must divide base_count")
return set(_equipartitions(frozenset(range(base_count)), group_size))
def _equipartitions(base_set: frozenset, group_size: int):
if not base_set:
yield frozenset()
for combo in itertools.combinations(base_set, group_size):
for rest in _equipartitions(base_set.difference(frozenset(combo)), group_size):
yield frozenset({frozenset(combo), *rest})
all_combinations = [
[tuple(team) for team in combo]
for combo in equipartitions(12, 3)
]
print(all_combinations)
print(len(all_combinations))
And another:
import itertools
from typing import Iterable
def equipartitions(players: Iterable, team_size: int):
if len(players) % team_size != 0:
raise ValueError("group_count must divide base_count")
return _equipartitions(set(players), team_size)
def _equipartitions(players: set, team_size: int):
if not players:
yield []
return
first_player, *other_players = players
for other_team_members in itertools.combinations(other_players, team_size-1):
first_team = {first_player, *other_team_members}
for other_teams in _equipartitions(set(other_players) - set(first_team), team_size):
yield [first_team, *other_teams]
all_combinations = [
{''.join(sorted(team)) for team in combo} for combo in equipartitions(players='ABCDEFGHIJKL', team_size=3)
]
print(all_combinations)
print(len(all_combinations))
Firstly, you can use a list comprehension to give you all of the possible combinations (regardless of the duplicates):
comb = [(a,b) for a in letters for b in letters if a != b]
And, afterwards, you can use the sorted function to sort the tuples. After that, to remove the duplicates, you can convert all of the items to a set and then back to a list.
var = [tuple(sorted(sub)) for sub in comb]
var = list(set(var))
You could use the list comprehension approach, which has a time complexity of O(n*n-1), or you could use a more verbose way, but with a slightly better time complexity of O(n^2-n)/2:
comb = []
for first_letter_idx, _ in enumerate(letters):
for sec_letter_idx in range(first_letter_idx + 1, len(letters)):
comb.append(letters[first_letter_idx] + letters[sec_letter_idx])
print(comb)
comb2 = []
for first_letter_idx, _ in enumerate(comb):
for sec_letter_idx in range(first_letter_idx + 1, len(comb)):
if (comb[first_letter_idx][0] not in comb[sec_letter_idx]
and comb[first_letter_idx][1] not in comb[sec_letter_idx]):
comb2.append([comb[first_letter_idx], comb[sec_letter_idx]])
print(comb2)
This algorithm needs more work to handle dynamic inputs. Maybe with recursion.
Use combination from itertools
from itertools import combinations
x = list(combinations(['A','B','C','D'],2))
t = []
for i in (x):
t.append(i[0]+i[1]) # concatenating the strings and adding in a list
g = []
for i in range(0,len(t),2):
for j in range(i+1,len(t)):
g.append([t[i],t[j]])
break
print(g)
I got a list, in one (the last) column is a string of comma separated items:
temp = ['AAA', 'BBB', 'CCC-DDD', 'EE,FFF,FFF,EE']
Now I want to remove the duplicates in that column.
I tried to make a list out of every column:
e = [s.split(',') for s in temp]
print e
Which gave me:
[['AAA'], ['BBB'], ['CCC-DDD'], ['EE', 'FFF', 'FFF', 'EE']]
Now I tried to remove the duplicates with:
y = list(set(e))
print y
What ended up in an error
TypeError: unhashable type: 'list'
I'd appreciate any help.
Edit:
I didn't exactly said what the end result should be. The list should look like that
temp = ['AAA', 'BBB', 'CCC-DDD', 'EE', 'FFF']
Just the duplicates should get removed in the last column.
Apply set on the elements of the list not on the list of lists. You want your set to contain the strings of each list, not the lists.
e = [list(set(x)) for x in e]
You can do it directly as well:
e = [list(set(s.split(','))) for s in temp]
>>> e
[['AAA'], ['BBB'], ['CCC-DDD'], ['EE', 'FFF']]
you may want sorted(set(s.split(','))) instead to ensure lexicographic order (sets aren't ordered, even in python 3.7)
for a flat, ordered list, create a flat set comprehension and sort it:
e = sorted({x for s in temp for x in s.split(',')})
result:
['AAA', 'BBB', 'CCC-DDD', 'EE', 'FFF']
Here is solution, that uses itertools.chain method
import itertools
temp = ['AAA', 'BBB', 'CCC-DDD', 'EE,FFF,FFF,EE']
y = list(set(itertools.chain(*[s.split(',') for s in temp])))
# ['EE', 'FFF', 'AAA', 'BBB', 'CCC-DDD']
a = ['AAA', 'BBB', 'CCC-DDD', 'EE,FFF,FFF,EE']
b = [s.split(',') for s in a]
c = []
for i in b:
c = c + i
c = list(set(c))
['EE', 'FFF', 'AAA', 'BBB', 'CCC-DDD']
Here is a pure functional way to do it in Python:
from functools import partial
split = partial(str.split, sep=',')
list(map(list, map(set, (map(split, temp)))))
[['AAA'], ['BBB'], ['CCC-DDD'], ['EE', 'FFF']]
Or as I see the answer doesn't need lists inside of a list:
from itertools import chain
list(chain(*map(set, (map(split, temp)))))
['AAA', 'BBB', 'CCC-DDD', 'EE', 'FFF']
This is the code that i have a problem with.
def permute(word):
letters = list(word)
print(type(letters))
for letter in letters:
letter_copy = letters.remove(letter)
rtrn_list = letter + permute(letter_copy)
return rtrn_list
w = 'ABC'
print(permute(w))
i am new to programming. someone please say where the problem is. Thanks in advance
Find your problem my comparing to this implementation.
def permute(string):
'''
Recursively finds all possible combinations of the
elements -- or permuations -- of an input string and
returns them as a list.
>>>permute('abc')
['abc', 'acb', 'bac', 'bca', 'cab', 'cba']
'''
output = []
if len(string) == 1:
output = [string]
else:
for i, let in enumerate(string):
for perm in permute(string[:i] + string[i + 1:]):
#print('Let is', let)
#print('Perm is', perm)
output += [let + perm]
return output
permute('abc')
Out[ ]:
['abc', 'acb', 'bac', 'bca', 'cab', 'cba']
For permutations you can use python builtin from itertools:
from itertools import permutations
p = []
for t in permutations('abc'):
p.append(''.join(t))
print(p)
Output is:
['abc', 'acb', 'bac', 'bca', 'cab', 'cba']
I want to create a simple string generator and here is how it will work
I declare a pattern_string = "abcdefghijklmnopqrstuvwxyz"
My starting string lets say starting_string = "qywtx"
Now I want to generate strings as follows:
Check the last character in my starting_stringagainst the pattern string.
Last character is x. We find this character in the find it in the pattern_string:
abcdefghijklmnopqrstuvw x yz
and see that next character is y so I want output qywty.
...
However, when I reach the z, I want my string to increment second last character and set the last character to the first character of the starting_pattern so it will be qywra and so on...
Now questions:
Can I use REGEX to achieve that?
Are there any libraries out there that already handle such generation?
The following will generate the next string according to your description.
def next(s, pat):
l = len(s)
for i in range(len(s) - 1, -1, -1): # find the first non-'z' from the back
if s[i] != pat[-1]: # if you find it
# leave everything before i as is, increment at i, reset rest to all 'a's
return s[:i] + pat[pat.index(s[i]) + 1] + (l - i - 1) * pat[0]
else: # this is only reached for s == 'zzzzz'
return (l + 1) * pat[0] # and generates 'aaaaaa' (just my assumption)
>>> import string
>>> pattern = string.ascii_lowercase # 'abcde...xyz'
>>> s = 'qywtx'
>>> s = next(s, pattern) # 'qywty'
>>> s = next(s, pattern) # 'qywtz'
>>> s = next(s, pattern) # 'qywua'
>>> s = next(s, pattern) # 'qywub'
For multiple 'z' in the end:
>>> s = 'foozz'
>>> s = next(s, lower) # 'fopaa'
For all 'z', start over with 'a' of incremented length:
>>> s = 'zzz'
>>> s = next(s, lower) # 'aaaa'
To my knowledge there is no library function to do that. One that comes close is itertools.product:
>>> from itertools import product
>>> list(map(''.join, product('abc', repeat=3)))
['aaa', 'aab', 'aac', 'aba', 'abb', 'abc', 'aca', 'acb', 'acc', 'baa',
'bab', 'bac', 'bba', 'bbb', 'bbc', 'bca', 'bcb', 'bcc', 'caa', 'cab',
'cac', 'cba', 'cbb', 'cbc', 'cca', 'ccb', 'ccc']
But that doesn't not work with an arbitrary start string. This behaviour could be mimicked by combining it with itertools.dropwhile but that has the serious overhead of skipping all the combinations before the start string (which in the case of an alphabet of 26 and a start string towards the end pretty much renders that approach useless):
>>> list(dropwhile(lambda s: s != 'bba', map(''.join, product('abc', repeat=3))))
['bba', 'bbb', 'bbc', 'bca', 'bcb', 'bcc', 'caa', 'cab', 'cac', 'cba', 'cbb', 'cbc', 'cca', 'ccb', 'ccc']
How can I format this function so that it works recursively? I want to go multiple levels deeper if possible, not just until 5.
permutations is a list with different permutations, and those individual permutation can also have permutations, etc. I want to rank them based on some calculations I do in get_permutations and return the new order of permutations. A good way to look at it is probably a large list of lists of lists of lists of lists. First I want to change the order of the first level, than one step deeper etc. But eventually I return the string based on those permutations and not the permutation itself (if that matters), with res1...res5 being the strings. I'm not smart enough to get it to work recursively, even though I know it should be possible...Any ideas?
permutations, res1 = get_permutations(str, 1)
for p2 in permutations:
permutations_2, res2 = get_permutations(p2,2)
for p3 in permutations_2:
permutations_3, res3 = get_permutations(p3,3)
for p4 in permutations_3:
permutations_4, res4 = get_permutations(p4, 4)
for p5 in permutations_4:
permutations_5, res5 = get_permutations(p5,5)
res4 += res5
res3 += res4
res2 += res3
res1 += res2
return res1
EDIT: this returns a single (best) permutation. That is what the result is for. So not a list of possible permutations, as mentioned in the answers. E.g. if we have a lists of lists of lists, if first sorts the list based on all subinformation, then sorts the multiple lists of lists based on the previous sorts and all subinformation and then sorts the lists of lists of lists based on the previous 2 sorts.
A recursive generator function that yields permutations in the expected order with regard to the original string:
def get_permutations(a):
if len(a) <= 1:
yield a
else:
for i in xrange(len(a)):
for p in get_permutations(a[:i]+a[i+1:]):
yield ''.join([a[i]])+p
>>> a = '123'
>>> list(get_permutations(a))
['123', '132', '213', '231', '312', '321']
The recursive principle here:
Base case: strings of lengthes (0, 1) have only one permutation: themselves.
Recursion: for each letter in the string, remove it and prepend it to each permutation of the string's remainder.
An example below, This method works by nesting for loops in a recursive manner repeat-times. We then accumulate the result of the sub-solutions, appending to a result list:
result = []
def permutations(alphabet, repeat, total = ''):
if repeat >= 1:
for i in alphabet:
# Add the subsolutions.
permutations(alphabet, repeat - 1, total + i)
else:
result.append(total)
return result
Sample Outputs:
permutations('ab', 3) ->
$ ['aaa', 'aab', 'aba', 'abb', 'baa', 'bab', 'bba', 'bbb']
permutations('ab', 3) ->
$ ['aaa', 'aab', 'aac', 'aba', 'abb', 'abc', 'aca', 'acb', 'acc', 'baa',
'bab', 'bac', 'bba', 'bbb', 'bbc', 'bca', 'bcb', 'bcc', 'caa', 'cab',
'cac', 'cba', 'cbb', 'cbc', 'cca', 'ccb', 'ccc']
permutations('ab', 1) ->
$ ['a', 'b']
Source: a previous answer of mine.