We Keep Coding

Filter Cartesian product so it fits in memory

I have a list of lists
y = [[a, b, c], ...]
I want to generate the Cartesian product between all of the lists. itertools.product in python does this but all at once so I can't hold them in memory.
I only need a subset of the output though. For arguments sake say only those that sum to 1. I need to sequentially obtain results so I can drop those that that are unneeded.
Essentially
[x for x in itertools.product(*y) if sum(x)==1]
where itertools.product does not eagerly evaluate.

Just replace your square brackets with ( and ): that will make your resulting expression a generator (that evaluates lazily) and not a list (that will have all of its content in memory).
(x for x in itertools.product(*y) if sum(x)==1)

itertools.product is already lazy. All functions in the itertools module are - they return an iterator, never lists. So just loop over the product and filter:
for prod in itertools.product(*y):
if sum(prod) == 1:
print(filtered_result)
Alternatively, the built-in function filter(function, iterable) also:
Construct[s] an iterator from those elements of iterable for which function returns true.
So you can also iterate over the derived filtered iterator from the product iterator:
for filtered_result in filter(lambda x: sum(x) == 1, product(*y)):
print(filtered_result)

Find n permutations of a list of length p python

So I have an array of about 40 items and need to find n permutations of the list. I know I can use itertools.permutations but that would give me all 40! different orderings which would take longer.
Is there a way I can create exactly n permutations of the list?

itertools.permutations gives you an iterator over the permutations of your list. You can pull out values one by one, the function does not precompute anything.
If you just want the first n permutations that are produced without any further constraints, islice the result of permutations.
>>> from itertools import permutations, islice
>>> perms = islice(permutations(yourlist), n)
Computing of permutations will only start once you iterate over perms - either directly with a for loop or by making a container (list, tuple, set, ...) out of it.

Converting combinations of integers into list of sums of these combinations

I need to do this: I have list of integers, i make all possible combinations containing 3 of these numbers, and output is list containing sums of these combinations. (not sum of all combinations together, but for each combination one sum)
My algorithm do it in this way: make list of all possible combinations, then count sum of each combination and save it into list, what is too uneffective if number of integers is large. Here is little sample in python:
import itertools
array=[1,2,3,4]
combs=[]
els = [list(x) for x in itertools.combinations(array, 3)]
combs.extend(els)
result=list(map(sum, combs))
Could you please come up with some more effective solution? Maybe if there is some way of making it a loop where there isn't made a list of combinations at first and counted sum of each combination at second but rather it straight counts sum of just created combination, saves it into list and then continues to another one until all combinations and sums are done.

sum() works with any iterable, not just lists. You could make the code more efficient by applying it directly to the combinations' tuples:
result = [sum(c) for c in itertools.combinations(array, 3)]
If you want to process the result lazily, you can use a generator expression as well:
result = (sum(c) for c in itertools.combinations(array, 3))

Cartesian Product of Sets where No Elements are Identical under Permutations in Python

I have some sets I would like to take the Cartesian product of, which is working well. However, I want to remove all elements of this new set which are identical under a permutation of the elements.
For example, take the following code:
import itertools as ittools
x = 2
y = 3
z = 5
flist = list(ittools.product([x,y,z],repeat=3))
for f in flist:
print reduce(lambda a,b: a*b, f)
This code find the Cartesian product of the set {2,3,5} and returns the product of all three components of each element in the resulting set. However, some numbers appear multiple times, i.e. 12 can be written as 2*2*3, 2*3*2, or 3*2*2. I would like to remove all but one instance of these duplicates.
I know that this fundamentally a combinatorics problem, but this seems like there is probably a nice solution in Python that doesn't involve doing an extra pass over the list like I did here to compute some identifier for each element of the Cartesian product.

You want combinations_with_replacement, not product:
itertools.combinations_with_replacement([x, y, z], 3)

Use a dict to map each unique product to the most recently seen tuple.
d = {reduce(operator.mul, f): f for f in flist}
If you would need to treat tuples that aren't permutations of each other as distinct elements, you'll need a more complicated key that incorporates a canonical representation of the tuple.
from operator import mul
d = {(tuple(sorted(f)), reduce(mul, f)): f for f in flist}
Actually, once you do that, you don't need to map the tuple/product pair to a tuple; you can just maintain a set of pairs:
d = {(tuple(sorted(f)), reduce(mul, f)) for f in flist}
In any case, retrieving just the tuples is as simple as
tuples = d.values() # In the first two cases
tuples = {x for x,y in d} # In the third case

Making a list comprehension for a dict within a list within a list

I need to create a list comprehension that extracts values from a dict within a list within a list, and my attempts so far are failing me. The object looks like this:
MyList=[[{'animal':'A','color':'blue'},{'animal':'B','color':'red'}],[{'animal':'C','color':'blue'},{'animal':'D','color':'Y'}]]
I want to extract the values for each element in the dict/list/list so that I get two new lists:
Animals=[[A,B],[C,D]]
Colors=[[blue,red],[blue,Y]]
Any suggestions? Doesn't necessarily need to use a list comprehension; that's just been my starting point so far. Thanks!

Animals = [[d['animal'] for d in sub] for sub in MyList]
Colors = [[d['color'] for d in sub] for sub in MyList]
Gives the desired result:
[['A', 'B'], ['C', 'D']]
[['blue', 'red'], ['blue', 'Y']] # No second 'red'.
What I have done here is take each sub-list, then each dictionary, and then access the correct key.

In a single assignment (with a single list comprehension, and the help of map and zip):
Colors, Animals = map(list,
zip(*[map(list,
zip(*[(d['color'], d['animal']) for d in a]))
for a in MyList]))
If you are fine with tuples, you can avoid the two calls to map => list
EDIT:
Let's see it in some details, by decomposing the nested comprehension.
Let's also assume MyList have m elements, for a total of n objects (dictionaries).
[[d for d in sub] for sub in MyList]
This would iterate through every dictionary in the sublists. For each of them, we create a couple with its color property in the first element and its animal property in the second one:
(d['color'], d['animal'])
So far, this will take time proportional to O(n) - exatly n elements will be processed.
print [[(d['color'], d['animal']) for d in sub] for sub in MyList]
Now, for each of the m sublists of the original list, we have one list of couples that we need to unzip, i.e. transform it into two lists of singletons. In Python, unzip is performed using the zip function by passing a variable number of tuples as arguments (the arity of the first tuple determines the number of tuples it outputs). For instance, passing 3 couples, we get two lists of 3 elements each
>>> zip((1,2), (3,4), (5,6)) #Prints [(1, 3, 5), (2, 4, 6)]
To apply this to our case, we need to pass array of couples to zip as a variable number of arguments: that's done using the splat operator, i.e. *
[zip(*[(d['color'], d['animal']) for d in sub]) for sub in MyList]
This operation requires going through each sublist once, and in turn through each one of the couples we created in the previous step. Total running time is therefore O(n + n + m) = O(n), with approximatively 2*n + 2*m operations.
So far we have m sublists, each one containing two tuples (the first one will gather all the colors for the sublist, the second one all the animals). To obtain two lists with m tuples each, we apply unzip again
zip(*[zip(*[(d['color'], d['animal']) for d in sub]) for sub in MyList]
This will require an additional m steps - the running time will therefore stay O(n), with approximatively 2*n + 4*m operations.
For sake of simplicity we left out mapping tuples to lists in this analysis - which is fine if you are ok with tuples instead.
Tuples are immutable, however, so it might not be the case.
If you need lists of lists, you need to apply the list function to each tuple: once for each of the m sublists (with a total of 2*n elements), and once for each of the 2 first level lists, i.e. Animals and Colors, (which have a total of m elements each). Assuming list requires time proportional to the length of the sequence it is applied to, this extra step requires 2*n + 2*m operations, which is still O(n).