I'm trying to include multiple operations in the lambda function with variables that have different lengths, i.e. something like:
$ serial_result = map(lambda x,y:(x**2,y**3), range(20), range(10))
but this doesn't work. Could someone tell me how to get around this?
I understand that:
$ serial_result = map(lambda x,y:(x**2,y**3), range(0,20,2), range(10))
works because the arrays of "x" and "y" have the same length.
If you want the product of range items you can use itertools.product :
>>> from itertools import product
>>> serial_result = map(lambda x:(x[0]**2,x[1]**3), product(range(20), range(10)))
If you want to pass the pairs to lambda like second case you can use itertools.zip_longest (in python 2 use izip_longest)and pass a fillvalue to fill the missed items,
>>> from itertools import zip_longest
>>> serial_result = map(lambda x:(x[0]**2,x[1]**3), zip_longest(range(20), range(10),fillvalue=1))
Note that if you are in python 2 you can pass multiple argument to lambda as a tuple :
>>> serial_result = map(lambda (x,y):(x**2,y**3), product(range(20), range(10)))
See the difference of izip_longest and product in following example :
>>> list(product(range(5),range(3)))
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2)]
>>> list(zip_longest(range(5),range(3)))
[(0, 0), (1, 1), (2, 2), (3, None), (4, None)]
>>> list(zip_longest(range(5),range(3),fillvalue=1))
[(0, 0), (1, 1), (2, 2), (3, 1), (4, 1)]
It sounds like you may be confused as to how exactly you want to use all the values in these two variables. There are several ways combine them...
If you want a result for every combination of an element in a and an element in b: itertools.product(a, b).
If you want to stop once you get to the end of the shorter: zip(a, b)
If you want to continue on until you've used all of the longest: itertools.zip_longer(a, b) (izip_longer in python 2). Once a runs out of elements it will be filled in with None, or a default you provide.
Related
Is there a simple way to append an integer to each item in an itertools iterator? If I use itertools.product, I do not receive the expected output. For example:
>>> for i in itertools.product(itertools.combinations(np.arange(4),2),(4,)):
... print(i)
...
((0, 1), 4)
((0, 2), 4)
((0, 3), 4)
((1, 2), 4)
((1, 3), 4)
((2, 3), 4)
But I would expect (and I want) is
>>> for i in itertools.product(itertools.combinations(np.arange(4),2),(4,)):
... print(i)
...
(0, 1, 4)
(0, 2, 4)
(0, 3, 4)
(1, 2, 4)
(1, 3, 4)
(2, 3, 4)
I know that I can "flatten" the output, but I would rather construct the iterator to produce tuples, not tuples of tuples.
I have many different iterators floating around, and I want to keep the code the same for products of itertool iterators and plain itertool iterators
These two alternatives each produce an iterator. In the first case, the iterator is created by a generator expression. In the second, the iterator is created by the use of a generator function.
In [9]: for i in (tup + (4,) for tup in itertools.combinations(np.arange(4),2)):
...: print(i)
...:
(0, 1, 4)
(0, 2, 4)
(0, 3, 4)
(1, 2, 4)
(1, 3, 4)
(2, 3, 4)
In [10]:
A generator function might be more readable at the call site, especially if the function name describes its behavior.
import itertools
import numpy as np
def adder(it, addend):
for x in it:
yield x + addend
for i in adder(itertools.combinations(np.arange(4),2), (4,)):
print(i)
I'm trying to dig into some code I found online here to better understand Python.
This is the code fragment I'm trying to get a feel for:
from itertools import chain, product
def generate_groupings(word_length, glyph_sizes=(1,2)):
cartesian_products = (
product(glyph_sizes, repeat=r)
for r in range(1, word_length + 1)
)
Here, word_length is 3.
I'm trying to evaluate the contents of the cartesian_products generator. From what I can gather after reading the answer at this SO question, generators do not iterate (and thus, do not yield a value) until they are called as part of a collection, so I've placed the generator in a list:
list(cartesian_products)
Out[6]:
[<itertools.product at 0x1025d1dc0>,
<itertools.product at 0x1025d1e10>,
<itertools.product at 0x1025d1f50>]
Obviously, I now see inside the generator, but I was hoping to get more specific information than the raw details of the itertools.product objects. Is there a way to accomplish this?
if you don't care about exhausting the generator, you can use:
list(map(list,cartesian_products))
You will get the following for word_length = 3
Out[1]:
[[(1,), (2,)],
[(1, 1), (1, 2), (2, 1), (2, 2)],
[(1, 1, 1),
(1, 1, 2),
(1, 2, 1),
(1, 2, 2),
(2, 1, 1),
(2, 1, 2),
(2, 2, 1),
(2, 2, 2)]]
I have a list of tuples that looks like this:
lst = [(0, 0), (2, 3), (4, 3), (5, 1)]
What is the best way to accumulate the sum of the first and secound tuple elements? Using the example above, I'm looking for the best way to produce this list:
new_lst = [(0, 0), (2, 3), (6, 6), (11, 7)]
I am looking for a solution in Python 2.6
I would argue the best solution is itertools.accumulate() to accumulate the values, and using zip() to split up your columns and merge them back. This means the generator just handles a single column, and makes the method entirely scalable.
>>> from itertools import accumulate
>>> lst = [(0, 0), (2, 3), (4, 3), (5, 1)]
>>> list(zip(*map(accumulate, zip(*lst))))
[(0, 0), (2, 3), (6, 6), (11, 7)]
We use zip() to take the columns, then apply itertools.accumulate() to each column, then use zip() to merge them back into the original format.
This method will work for any iterable, not just sequences, and should be relatively efficient.
Prior to 3.2, accumulate can be defined as:
def accumulate(iterator):
total = 0
for item in iterator:
total += item
yield total
(The docs page gives a more generic implementation, but for this use case, we can use this simple implementation).
How about this generator:
def accumulate_tuples(iterable):
accum_a = accum_b = 0
for a, b in iterable:
accum_a += a
accum_b += b
yield accum_a, accum_b
If you need a list, just call list(accumulate_tuples(your_list)).
Here's a version that works for arbitrary length tuples:
def accumulate_tuples(iterable):
it = iter(iterable):
accum = next(it) # initialize with the first value
yield accum
for val in it: # iterate over the rest of the values
accum = tuple(a+b for a, b in zip(accum, val))
yield accum
>> reduce(lambda x,y: (x[0] + y[0], x[1] + y[1]), lst)
(11, 7)
EDIT. I can see your updated question. To get the running list you can do:
>> [reduce(lambda x,y: (x[0]+y[0], x[1]+y[1]), lst[:i]) for i in range(1,len(lst)+1)]
[(0, 0), (2, 3), (6, 6), (11, 7)]
Not super efficient, but at least it works and does what you want :)
This works for any length of tuples or other iterables.
from collections import defaultdict
def accumulate(lst):
sums = defaultdict(int)
for item in lst:
for index, subitem in enumerate(item):
sums[index] += subitem
yield [sums[index] for index in xrange(len(sums))]
print [tuple(x) for x in accumulate([(0, 0), (2, 3), (4, 3), (5, 1)])]
In Python 2.7+ you would use a Counter instead of defaultdict(int).
This is a really poor way (in terms of performance) to do this because list.append is expensive, but it works.
last = lst[0]
new_list = [last]
for t in lst[1:]:
last += t
new_list.append(last)
Simple method:
>> x = [(0, 0), (2, 3), (4, 3), (5, 1)]
>>> [(sum(a for a,b in x[:t] ),sum(b for a,b in x[:t])) for t in range(1,len(x)+1)]
[(0, 0), (2, 3), (6, 6), (11, 7)]
lst = [(0, 0), (2, 3), (4, 3), (5, 1)]
lst2 = [lst[0]]
for idx in range(1, len(lst)):
newItem = [0,0]
for idx2 in range(0, idx + 1):
newItem[0] = newItem[0] + lst[idx2][0]
newItem[1] = newItem[1] + lst[idx2][1]
lst2.append(newItem)
print(lst2)
You can use the following function
>>> def my_accumulate(lst):
new_lst = [lst[0]]
for x, y in lst[1:]:
new_lst.append((new_lst[-1][0]+x, new_lst[-1][1]+y))
return new_lst
>>> lst = [(0, 0), (2, 3), (4, 3), (5, 1)]
>>> my_accumulate(lst)
[(0, 0), (2, 3), (6, 6), (11, 7)]
Changed my code to a terser version:
lst = [(0, 0), (2, 3), (4, 3), (5, 1)]
def accumulate(the_list):
the_item = iter(the_list)
accumulator = next(the_item)
while True:
yield accumulator
accumulator = tuple(x+y for (x,y) in zip (accumulator, next(the_item)))
new_lst = list(accumulate(lst))
It is easy to implement a regular double sort:
pairs = [(1, 2), (2, 1), (1, 3), (2, 4), (3, 1)]
sorted(pairs,key=lambda x: (x[0],x[1]))
# out: [(1, 2), (1, 3), (2, 1), (2, 4), (3, 1)]
I am interested how to do it with the second elements in the reverse order. This can be easily implemented by grouping the pairs by the first item at first and then adding the the sorted second items together. I have implemented this both using itertools.groupby and defaultdict. Still, it remains far more complex, than the regular double sort, so i wonder, if there is a neat trick to do it in a more concise way.
double_sort(pairs)
# out: [(1, 3), (1, 2), (2, 4), (2, 1), (3, 1)]
PS! I know how to do it with numpy.argsort and would mostly like to see a standard lib approach.
This will work for numbers and similar data types
sorted(pairs, key=lambda x: (x[0], -x[1]))
This will work for all comparable types only in Python 2 :(
sorted(pairs, lambda x, y: cmp(x[0], y[0]) or cmp(y[1], x[1]))
I am currently working with a script that has lists that looks like this:
example = [ ((2,1),(0,1)), ((0,1),(2,1)), ((2,1),(0,1)) ]
Now turning this list to a set returns:
set( [ ((2,1),(0,1)), ((0,1),(2,1)) ] )
For my purposes I need to recognize these tuples as being equal as well. I dont care about retaining the order. All solutions I can think of is really messy so if anyone has any idea I would be gratefull.
It sounds like you may be off using frozensets instead of tuples.
>>> x = [((2, 1), (0, 1)), ((0, 1), (2, 1)), ((2, 1), (0, 1))]
>>> x
[((2, 1), (0, 1)), ((0, 1), (2, 1)), ((2, 1), (0, 1))]
>>> set(frozenset(ts) for ts in x)
set([frozenset([(0, 1), (2, 1)])])
In [10]: set(tuple(sorted(elt)) for elt in example)
Out[10]: set([ ((0, 1), (2, 1)) ])
First transform all elements to sets too. Then make a set of the whole list.