I would like to make a list of 81-tuple by using three elements, namely 1,2,3 in python.
I tried to find a solution, and then I found these useful links:
How to use itertools to compute all combinations with repeating elements?
and
Which itertools generator doesn't skip any combinations?
According to the above links, I should do the following
import itertools
list = []
for p in itertools.product(range(1, 3 + 1), repeat=81):
list.append(p)
print(list)
But, my computer hangs. I think there is too much data in the list.
I want to know whether there is a command that prints only first 100-elements in list or the 101th to 200th in the list.
You can use itertools.islice:
p = itertools.product(range(1, 3 + 1), repeat=81)
s = itertools.islice(p, 101, 200)
print(list(s))
This will, however, iterate through all the elements until it reaches the starting index of the slice. So for ranges towards the end of an iterator with a gazillion elements (yours has 3**81 = 443426488243037769948249630619149892803 or in other words: too many to process let alone store), this will run into similar problems.
For those later ranges, you would have to calculate the n-th element by hand and generate successors from there... See How to select specific item from cartesian product without calculating every other item for some inspiration.
Related
I'm exploring probability with the use of python and I want to resolve this kind of problem. I have 5 couples and from these couples, I want to extract each group of three, that doesn't contain persons from a married couple.
import itertools as it
married_couples = [["Mary","John"],["Homer","Marge"],["Beauty","Beast"],["Abigail","George"],["Marco","Luisa"]]
all_persons = []
for couples in married_couples:
for person in couples:
all_persons.append(person)
sample_space = list(it.combinations(all_persons,3))
# Better solution with generator:
sample_space = [s for t in it.combinations(married_couples, 3) for s in it.product(*t)]
Now I would like to proceed excluding from the sample space all results that contains people from the same married couple and I try:
correct_results = sample_space.copy()
for couple in married_couples:
for res in sample_space:
if couple[0] in res and couple[1] in res:
if res in correct_results:
correct_results.remove(res)
Edit: I edited and put also the generator solution inside, so you can use this code for the purpose
The problem is in
if couple[0] and couple[1] in res:
because it tests not that the couple is in res, but that the first element of the couple is not null and the second is in res.
You should use:
if couple[0] in res and couple[1] in res:
Here's a much more efficient way to do this:
r = [s for t in itertools.combinations(married_couples, 3) for s in itertools.product(*t)]
If you just need to iterate over this, and don't need it as an actual expanded list, then you can instead create a generator:
iter = (s for t in itertools.combinations(married_couples, 3) for s in itertools.product(*t))
Then you can do:
for triple in iter:
...
Both solutions work as follows: There are two levels to it. At the top level, it calls itertools.combinations(married_couples, 3). This generates all triplets from the original set of married couples. Then, for each triplet, it uses iterrools.product(*t) to generate all 8 combinations that take one from each of the three pairs in the triplet.
I am trying to find faster solution than the current brute force one, which assumes looping over all elements and is super inefficient on the larger lists.
To be more specific, I am having a list of N elements. Each of these elements has another list of variable number of elements. The goal is to find top 3 products of all lists, given that only one number from any list might be used at any time.
So for instance, if we had lists(dictionaries):
MAIN LIST: {'a':0.97,'b':0.88,'c':0.77}
A SUB-LIST: {'a1':0.68,'a2':0.13,'a3':0.04}
B SUB-LIST: {'b1':0.77,'b2':0.66,'b3':0.02}
C SUB-LIST: {'c1':0.99,'c2':0.92,'c3':0.13}
RESULT: 1.c*c1 2.c*c2 3.b*b1
I wanna get the top number from all possible products. Sorting the lists, iterating over all elements and adding them to the temporary list with top3 items is the current solution, but becomes problematic while dealing with lists having more than 1,000 elements.
Does anyone know any sort of algorithm that might be used here, given that length of each list can exceed 1000 elements?
Alternatively, if finding 3 is not possible with good time complexity - do you know any algorithm that might be used to finding e.g. top1 product between all the lists?
You can cut down on the multiplications by getting the top 3 of each perfix letter (using heapq nlargest)
Then apply the factors on these lists of top 3s and pick the top 3 out of those.
Something like
from heapq import nlargest
# group the sub list by their key letter
sub_lists = dict()
for sl in [A,B,C]:
k = next(iter(sl))[0]
sub_lists.setdefault(k,[]).extend(sl.items())
top3s = [ (pk,pv*main_list[k]) for k,prods in sub_lists ]
for pk,pv in nlargest(3,prods,key=lambda kv:kv[1]) ]
top3 = nlargest(3,top3s,key=lambda kv:kv[1])
Note: You didn't provide any code or data so I could not actually test this
a = [[1,2,3,4],[2,3,4,5],[5,6,7,8],[4,3,2,3]]
output: (1,2,5,4),(1,2,5,3),(1,2,5,2)...
I have multiple lists that I'm trying to find all possible combinations of that's similar to the above code. However due to the amount of lists and the amount of items in the list, my computer can't process it. So I was wondering if there was a way I could filter some combinations out by adding all the items(numbers) together. ex: (1,2,5,4) = 12; And then if that list total is under 15 then delete that list and move on to the next combination. I would have to do something like this anyway after I got all the combinations so I figured why not do it earlier.
My code which generates all possible combinations:
list(itertools.product(*a))
Any ideas on how I can implement this filtering concept?
You can use a list comprehension:
from itertools import product
a = [[1,2,3,4],[2,3,4,5],[5,6,7,8],[4,3,2,3]]
l = [e for e in product(*a) if sum(e) > 15]
However be aware that the generation of the initial products might be the bottleneck. If you have too many elements (only 256 in your example) there is a possibility that your computer can never generate them all, and the filtering won't change anything.
I have a list of strings which looks like this:
['(num1, num2):1', '(num3, num4):1', '(num5, num6):1', '(num7, num8):1']
What I try to achieve is to reduce this list and combine every two elements and I want to do this until there is only one big string element left.
So the intermediate list would look like this:
['((num1, num2):1,(num3, num4):1)', '((num5, num6):1,(num7, num8):1)']
The complicated thing is (as you can see in the intermediate list), that two strings need to be wrapped in paranthesis. So for the above mentioned starting point the final result should look like this:
(((num_1,num_2):1,(num_3,num_4):1),((num_5,num_6):1,(num_7,num_8):1))
Of course this should work in a generic way also for 8, 16 or more string elements in the starting list. Or to be more precise it should work for an=2(n+1).
Just to be very specific how the result should look with 8 elements:
'((((num_1,num_2):1,(num_3,num_4):1),((num_5,num_6):1,(num_7,num_8):1)),(((num_9,num_10):1,(num_11,num_12):1),((num_13,num_14):1,(num_15,num_16):1)))'
I already solved the problem using nested for loops but I thought there should be a more functional or short-cut solution.
I also found this solution on stackoverflow:
import itertools as it
l = [map( ",".join ,list(it.combinations(my_list, l))) for l in range(1,len(my_list)+1)]
Although, the join isn't bad, I still need the paranthesis. I tried to use:
"{},{}".format
instead of .join but this seems to be to easy to work :).
I also thought to use reduce but obviously this is not the right function. Maybe one can implement an own reduce function or so?
I hope some advanced pythonics can help me.
Sounds like a job for the zip clustering idiom: zip(*[iter(x)]*n) where you want to break iterable x into size n chunks. This will discard "leftover" elements that don't make up a full chunk. For x=[1, 2, 3], n=2 this would yield (1, 2)
def reducer(l):
while len(l) > 1:
l = ['({},{})'.format(x, y) for x, y in zip(*[iter(l)]*2)]
return l
reducer(['(num1, num2):1', '(num3, num4):1', '(num5, num6):1', '(num7, num8):1'])
# ['(((num1, num2):1,(num3, num4):1),((num5, num6):1,(num7, num8):1))']
This is an explanation of what is happening in zip(*[iter(l)]*2)
[iter(l)*2] This creates an list of length 2 with two times the same iterable element or to be more precise with two references to the same iter-object.
zip(*...) does the extracting. It pulls:
Loop
the first element from the first reference of the iter-object
the second element from the second reference of the iter-object
Loop
the third element from the first reference of the iter-object
the fourth element from the second reference of the iter object
Loop
the fifth element from the first reference of the iter-object
the sixth element from the second reference of the iter-object
and so on...
Therefore we have the extracted elements available in the for-loop and can use them as x and y for further processing.
This is really handy.
I also want to point to this thread since it helped me to understand the concept.
I come from C++ as my 1st programming lenguage, Im just getting into python and im looking for a way to switch numbers from a list, in C++ this would be done using pointers moving them around with loops, however this time I need to generate all the permutations of a list A to a list B in Python
List A (the starter list) and list B (the result list)
A= 1234
B= 4231
The program has to show all the possible combinations in order, by moving only 2 numbers at he same time until the A list becomes the B list (the following example is simplified to 4 numbers and might not show all the combinations)
[1,2,3,4]
[1,2,4,3]
[1,4,2,3]
[4,1,2,3]
[4,2,1,3]
[4,2,3,1]
In order to acomplish this, I have found the itertools module, that contains a lot of functions, but havent been able to implement to many functions so far, the following code kind of does what its needed however it does not move the numbers in pairs nor in order
import itertools
from itertools import product, permutations
A = ([1,2,3,4])
B = ([4,2,3,1])
print "\n"
print (list(permutations(sorted(B),4)))
Im thinking about adding a while ( A != B ) then stop the permutations, i already tried this but im not familiar with pythons syntax, any help about how can i accomplish this would be appreciated
I am assuming sorting of the input list is not really required ,
from itertools import permutations
A = ([4, 3, 2, 1])
B = ([1,2,4, 3])
def print_combinations(start, target):
# use list(permutations(sorted(start), len(start))) if sorting of start is really required
all_perms = list(permutations(start, len(start)))
if tuple(target) not in all_perms:
# return empty list if target is not found in all permutations
return []
# return all combinations till target(inclusive)
# using list slicing
temp = all_perms[: all_perms.index(tuple(target)) + 1]
return temp
print print_combinations(A, B)
On the assumption that you are asking the best way to solve this permutation question - here is a different answer:
Think of all the permutations as a set. itertools.permutations generates all those permutations in some order. That is just what you want if you want to find all or some of those permutations. But that's not what you are looking for. You are trying to find paths through those permutations. itertools.permutations generates all the permutations in an order but not necessarily the order you want. And certainly not all the orders: it only generates them once.
So, you could generate all the permutations and consider them as the nodes of a network. Then you could link nodes whenever they are connected by a single swap, to get a graph. This is called the permutohedron. Then you could do a search on that graph to find all the loop-free paths from a to b that you are interested in. This is certainly possible but it's not really optimal. Building the whole graph in advance is an unnecessary step since it can easily be generated on demand.
Here is some Python code that does just that: it generates a depth first search over the permutohedron by generating the neighbours for a node when needed. It doesn't use itertools though.
a = (1,2,3,4)
b = (4,2,3,1)
def children(current):
for i in range(len(a)-1):
yield (current[:i] + (current[i+1],current[i]) +
current[i+2:])
def dfs(current,path,path_as_set):
path.append(current)
path_as_set.add(current)
if current == b:
yield path
else:
for next_perm in children(current):
if next_perm in path_as_set:
continue
for path in dfs(next_perm,path,path_as_set):
yield path
path.pop()
path_as_set.remove(current)
for path in dfs(a,[],set()):
print(path)
If you are really interested in using itertools.permutations, then the object you are trying to study is actually:
itertools.permutations(itertools.permutations(a))
This generates all possible paths through the set of permutations. You could work through this rejecting any that don't start at a and that contain steps that are not a single swap. But that is a very bad approach: this list is very long.
It's not completely clear what you are asking. I think you are asking for a pythonic way to swap two elements in a list. In Python it is usual to separate data structures into immutable and mutable. In this case you could be talking about tuples or lists.
Let's assume you want to swap elements i and j, with j larger.
For immutable tuples the pythonic approach will be to generate a new tuple via slicing like this:
next = (current[:i] + current[j:j+1] + current[i+1:j]
+ current[i:i+1] + current[j+1:])
For mutable lists it would be pythonic to do much the same as C++, though it's prettier in Python:
list[i],list[j] = list[j],list[i]
Alternatively you could be asking about how to solve your permutation question, in which case the answer is that itertools does not really provide much help. I would advise depth first search.
I guess following is a simpler way, I had nearly same issue (wanted swapped number) in a list (append a copy of list to itself list = list + list and then run :
from itertools import combinations_with_replacement
mylist = ['a', 'b']
list(set(combinations_with_replacement(mylist + mylist, r=2)))
results:
[('a', 'b'), ('b', 'a'), ('b', 'b'), ('a', 'a')]