I was trying to implement the reverse function of itertools.izip on Python 2.7.1. The thing is that I find a problem, and I don't have an explantion.
Solution 1, iunzip_v1 works perfectly. But solution 2. iunzip_v2, doesn't works as expected. Til now, I haven't found any relevant information about this problem, and reading the PEP about generators, it sound it should work, but it doesn't.
import itertools
from operator import itemgetter
def iunzip_v1(iterable):
_tmp, iterable = itertools.tee(iterable, 2)
iters = itertools.tee(iterable, len(_tmp.next()))
return tuple(itertools.imap(itemgetter(i), it) for i, it in enumerate(iters))
def iunzip_v2(iterable):
_tmp, iterable = itertools.tee(iterable, 2)
iters = itertools.tee(iterable, len(_tmp.next()))
return tuple((elem[i] for elem in it) for i, it in enumerate(iters))
result:
In [17]: l
Out[17]: [(0, 0, 0), (1, 2, 3), (2, 4, 6), (3, 6, 9), (4, 8, 12)]
In [18]: map(list, iunzip.iunzip_v1(l))
Out[18]: [[0, 1, 2, 3, 4], [0, 2, 4, 6, 8], [0, 3, 6, 9, 12]]
In [19]: map(list, iunzip.iunzip_v2(l))
Out[19]: [[0, 3, 6, 9, 12], [0, 3, 6, 9, 12], [0, 3, 6, 9, 12]]
Seems that iunzip_v2 is using the last value, so the generators aren't keeping the value while they are created inside the first generator.
I'm missing something and I don't know what is.
Thanks in advance if something can clarify me this situation.
UPDATE:
I've found the explanation here PEP-289, my first read was at PEP-255.
The solution I'm trying to implement is a lazy one, so:
zip(*iter) or izip(*...)
doesn't work for me, because *arg expand the argument list.
You're reinventing the wheel in a crazy way. izip is its own inverse:
>>> list(izip(*izip(range(10), range(10))))
[(0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)]
But that doesn't quite answer your question, does it?
The problem with your nested generators is a scoping problem that happens because the innermost generators don't get used until the outermost generator has already run:
def iunzip_v2(iterable):
_tmp, iterable = itertools.tee(iterable, 2)
iters = itertools.tee(iterable, len(_tmp.next()))
return tuple((elem[i] for elem in it) for i, it in enumerate(iters))
Here, you generate three generators, each of which uses the same variable, i. Copies of this variable are not made. Then, tuple exhausts the outermost generator, creating a tuple of generators:
>>> iunzip_v2((range(3), range(3)))
(<generator object <genexpr> at 0x1004d4a50>, <generator object <genexpr> at 0x1004d4aa0>, <generator object <genexpr> at 0x1004d4af0>)
At this point, each of these generators will execute elem[i] for each element of it. And since i is now equal to 3 for all three generators, you get the last element each time.
The reason the first version works is that itemgetter(i) is a closure, with its own scope -- so every time it returns a function, it generates a new scope, within which the value of i does not change.
Ok this is a bit tricky. When you use a name like i the value it stands for is looked up just during runtime. In this code:
return tuple((elem[i] for elem in it) for i, it in enumerate(iters))
you return a number of generators, (elem[i] for elem in it) and each of them uses the same name i. When the function returns, the loop in tuple( .. for i in .. ) has ended and i has been set to it's final value (3 in your example). Once you evaluate these generators to lists, they all create the same values because they are using the same i.
Btw:
unzip = lambda zipped: zip(*zipped)
Related
I am new to Python and I need help with following.
I do have a list a = [range(1, 50, 10)]
and another list b = [2, 4, 5, 8, 12, 34]
Now, I have a function that calculates something, let's call it "SomeFunction", that takes two arguments.
I want to pass to this SomeFunction for element from list a each element of list b. So, I want values for: SomeFunction(1, 2), SomeFunction(1, 4), SomeFunction(1, 5), ..... SomeFunction(50, 2), SomeFunction(50, 4), SomeFunction(50, 5), etc.
I think it should be done somehow with for loops, but I do not know how...
You'd need a nested for loop:
a = range(1, 50, 10)
b = [2, 4, 5, 8, 12, 34]
for aval in a:
for bval in b:
print(aval, bval) # or any other function call
This just goes through all values in b for each value in a. (Note that you don't want range inside square brackets as you have in your question, I removed those).
A more advanced version: "all pairs of values" is also known as the Cartesian product, which is provided by the itertools module. Thus you could write
from itertools import product
for aval, bval in product(a, b):
print(aval, bval)
Try this:
a = range(1, 50, 10)
b = [2, 4, 5, 8, 12, 34]
for i in a:
for n in b:
print(SomeFunction(i, n))
I'm trying to practice Python exercises, but using list comprehension to solve problems rather than the beginner style loops shown in the book. There is one example where it asks for a list of numbers to be put into a list of even numbers only, BUT they must be in sublists so that if the numbers follow after one another without being interrupted by an odd number, they should be put into a sublist together:
my_list = [2,3,5,7,8,9,10,12,14,15,17,25,31,32]
desired_output = [[2],[8],[10,12,14],[32]]
So you can see in the desired output above, 10,12,14 are evens that follow on from one another without being interrupted by an odd, so they get put into a sublist together. 8 has an odd on either side of it, so it gets put into a sublist alone after the odds are removed.
I can put together an evens list easily using list comprehension like this below, but I have no idea how to get it into sublists like the desired output shows. Could someone please suggest an idea for this using list comprehension (or generators, I don't mind which as I'm trying to learn both at the moment). Thanks!
evens = [x for x in my_list if x%2==0]
print(evens)
[2, 8, 10, 12, 14, 32]
As explained in the comments, list comprehensions should not be deemed "for beginners" - first focus on writing your logic using simple for loops.
When you're ready, you can look at comprehension-based methods. Here's one:
from itertools import groupby
my_list = [2,3,5,7,8,9,10,12,14,15,17,25,31,32]
condition = lambda x: all(i%2==0 for i in x)
grouper = (list(j) for _, j in groupby(my_list, key=lambda x: x%2))
res = filter(condition, grouper)
print(list(res))
# [[2], [8], [10, 12, 14], [32]]
The main point to note in this solution is nothing is computed until you call list(res). This is because filter and generator comprehensions are lazy.
You mentioned also wanting to learn generators, so here is a version that's also a bit more readable, imho.
from itertools import groupby
def is_even(n):
return n%2 == 0
def runs(lst):
for even, run in groupby(lst, key=is_even):
if even:
yield list(run)
if __name__ == '__main__':
lst = [2, 3, 5, 7, 8, 9, 10, 12, 14, 15, 17, 25, 31, 32]
res = list(runs(lst))
print(res)
Incidentally, if you absolutely, positively want to implement it as a list comprehension, this solutions falls out of the above quite naturally:
[list(run) for even, run in groupby(lst, key=is_even) if even]
If you don't want to use itertools, there's another way to do it with list comprehensions.
First, take the indices of the odd elements:
[i for i,x in enumerate(my_list) if x%2==1]
And add two sentinels: [-1] before and [len(my_list)] after:
odd_indices = [-1]+[i for i,x in enumerate(my_list) if x%2==1]+[len(my_list)]
# [-1, 1, 2, 3, 5, 9, 10, 11, 12, 14]
You have now something like that:
[2,3,5,7,8,9,10,12,14,15,17,25,31,32]
^---^-^-^---^-----------^--^--^--^----^
You can see your sequences. Now, take the elements between those indices. To do that, zip odd_indices with itself to get the intervals as tuples:
zip(odd_indices, odd_indices[1:])
# [(-1, 1), (1, 2), (2, 3), (3, 5), (5, 9), (9, 10), (10, 11), (11, 12), (12, 14)]
even_groups = [my_list[a+1:b] for a,b in zip(odd_indices, odd_indices[1:])]
# [[2], [], [], [8], [10, 12, 14], [], [], [], [32]]
You just have to filter the non empty lists:
even_groups = [my_list[a+1:b] for a,b in zip(odd_indices, odd_indices[1:]) if a+1<b]
# [[2], [8], [10, 12, 14], [32]]
You can merge the two steps into one comprehension list, but that is a bit unreadable:
>>> my_list = [2,3,5,7,8,9,10,12,14,15,17,25,31,32]
>>> [my_list[a+1:b] for l1 in [[-1]+[i for i,x in enumerate(my_list) if x%2==1]+[len(my_list)]] for a,b in zip(l1, l1[1:]) if b>a+1]
[[2], [8], [10, 12, 14], [32]]
As pointed by #jpp, prefer basic loops until you feel comfortable. And maybe avoid those nested list comprehensions forever...
The code below gives all the permutations for the array, I want to write a function that can use one permutation at a time (I want to process one permutation at a time for example [2, 3, 5, 4, 1, 6] without generating them all in advance, to be able to use it in an objective function)
s = np.array([1, 2, 3, 4, 5, 6])
from itertools import permutations
for s_temp in permutations(s):
print (s_temp)
You have several posibilities, starting that permutations returns a iterator so your code will just work by changing the print for whatever you need to call.
from itertools import permutations
def process(s):
for s_temp in permutations(s):
call_your_stuff(s_temp)
Also once you have the permutations object you can get the next item on it calling next:
from itertools import permutations
s = permutations(range(3))
s
<itertools.permutations object at 0x000000000377CFC0>
next(s)
(0, 1, 2)
If you want to process the same function for each of the permutations you can just use map, just replace the lambda function in the example for whatever you need to call:
s = permutations(range(3))
map(lambda (x, y, z): x+y-z, s)
[-1, 1, -1, 3, 1, 3]
You can collect the data into a list for further processing:
s = list(permutations(range(3)))
s
[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]
Notice that if you dont collect the data (into a list, tuple, etc) once you consume the permutations object iterator that data will be "lost" (you will have to recalculate again)
I am practicing exercises for functional programming concepts using python. I came across this problem. I have tried a lot and couldn't find a solution using functional programming constructs like map/reduce, closures.
Problem: Given a list of numbers
list = [10, 9, 8, 7, 6, 5, 4, 3]
Find sum of difference in each pair using Map/Reduce or any functional programming concepts e.g
[[10 -9] + [8 - 7] + [6 -5] + [4 - 3]] = 4
For me tricky part is isolating pairs using map/reduce/recursion/closure
The recursive relationship you are looking for is
f([4, 3, 2, 1]) = 4 - 3 + 2 - 1 = 4 - (3 - 2 + 1) = 4 - f([3, 2, 1])
One of the mantras followed by many functional programmers is the following:
Data should be organized into data structures that mirror the processing you want to perform on it
Applying this to your question, you're running into a simple problem: the list data structure doesn't encode in any way the relationship between the pairs that you want to operate on. So map/reduce operations, since they work on the structure of lists, don't have any natural visibility into the pairs! This means you're "swimming against the current" of these operations, so to speak.
So the first step should be to organize the data as a list or stream of pairs:
pairs = [(10, 9), (8, 7), (6, 5), (4, 3)]
Now after you've done this, it's trivial to use map to take the difference of the elements of each pair. So basically I'm telling you to split the problem into two easier subproblems:
Write a function that groups lists into pairs like that.
Use map to compute the difference of each pair.
And the hint I'll give is that neither map nor reduce is particularly useful for step #1.
using itertools.starmap:
l = [10, 9, 8, 7, 6, 5, 4, 3]
from operator import sub
from itertools import starmap
print(sum(starmap(sub, zip(*[iter(l)] * 2))))
4
Or just a lambda:
print(sum(map(lambda x: sub(*x), zip(*[iter(l)] * 2))))
Or range and itemgetter:
from operator import itemgetter as itgt
print(sum(itgt(*range(0, len(l), 2))(l)) - sum(itgt(*range(1, len(l), 2))(l)))
Another try may be isolating pairs-You may need to sort the list beforehand
>>>import operator
>>>l=[10, 9, 8, 7, 6, 5, 4, 3]
>>>d= zip(l,l[1:])
>>>w=[d[i] for i in range(0,len(d),2)]#isolate pairs i.e. [(10, 9), (8, 7), (6, 5), (4, 3)]
>>>reduce(operator.add,[reduce(operator.sub,i) for i in w])
>>>4
You can do it in simple way.
l = [10, 9, 8, 7, 6, 5, 4, 3]
reduce(lambda x, y : y - x, l) * -1
Im trying to write a function that creates set of dynamic sublists each containing 5 elements from a list passed to it.Here's my attempt at the code
def sublists(seq):
i=0
x=[]
while i<len(seq)-1:
j=0
while j<5:
X.append(seq[i]) # How do I change X after it reaches size 5?
#return set of sublists
EDIT:
Sample input: [1,2,3,4,5,6,7,8,9,10]
Expected output: [[1,2,3,4,5],[6,7,8,9,10]]
Well, for starters, you'll need to (or at least should) have two lists, a temporary one and a permanent one that you return (Also you will need to increase j and i or, more practically, use a for loop, but I assume you just forgot to post that).
EDIT removed first code as the style given doesn't match easily with the expected results, see other two possibilities.
Or, more sensibly:
def sublists(seq):
x=[]
for i in range(0,len(seq),5):
x.append(seq[i:i+5])
return x
Or, more sensibly again, a simple list comprehension:
def sublists(seq):
return [seq[i:i+5] for i in range(0,len(seq),5)]
When given the list:
l = [1,2,3,4,5,6,7,8,9,10]
They will return
[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]
Have you considered using itertools.combinations(...)?
For example:
>>> from itertools import combinations
>>> l = [1,2,3,4,5,6]
>>> list(combinations(l, 5))
[(1, 2, 3, 4, 5), (1, 2, 3, 4, 6), (1, 2, 3, 5, 6), (1, 2, 4, 5, 6), (1, 3, 4, 5, 6), (2, 3, 4, 5, 6)]
By "dynamic sublists", do you mean break up the list into groups of five elements? This is similar to your approach:
def sublists(lst, n):
ret = []
i = 0
while i < len(lst):
ret.append(seq[i:i+n])
i += n
return ret
Or, using iterators:
def sublists(seq, n):
it = iter(seq)
while True:
r = list(itertools.islice(it, 5))
if not r:
break
yield r
which will return an iterator of lists over list of length up to five. (If you took out the list call, weird things would happen if you didn't access the iterators in the same order.)