I have this function:
import concurrent.futures
def function_abc(a,b):
#do something
I call this function with this:
with concurrent.futures.ThreadPoolExecutor() as executor:
executor.map(function_abc, a, b)
But this does not work.
If I remove the argument "b":
with concurrent.futures.ThreadPoolExecutor() as executor:
executor.map(function_abc, a)
It works anyway. But I don't get the additional argument inside of this function.
What am I doing wrong?
Pass your arguments as separate iterables, one for a and another for b:
In [2]: with concurrent.futures.ThreadPoolExecutor() as ex:
...: m = ex.map(pow, (2, 2, 2, 2), (2, 3, 4, 5))
...:
...: for item in m:
...: print(item)
...:
4
8
16
32
To be more explicit:
ex.map(func_of_a_and_b, (a1, a2, a3, a4, ...), (b1, b2, b3, b4, ...))
You can pass in a list of tuples instead. Tuples are defined as (a, b, c...) and cannot be modified, but you can access them the same as lists (with an index, reverse index, slicing).
So basically nested lists but python somelist = [sometuple, sometuple, sometuple]
For example:
## Lets say this function multiplies a and b
# This is a compact and fast way
def function_abc(tup):
return tup[0]*tup[1]
## This makes it possible to modify the variables (not the tuple) and a bit nicer for more complicated code
## This also has lists
def function_abc(tup):
a = tup[0]
b = tup[1]
a = a/2
#tup[0] = tup[0]/2 <- Will show error as tuples are immutable
return a*b
def combine(a, b):
if len(a) != len(b):
print("Wrong lengths!")
l = []
for i in range(0, len(a)):
l.append((a[i], b[i]))
return l
## In map:
a = [1, 2, 3]
b = [10, 9, 8]
results = list(map(function_abc, combine(a, b))) # Should work
print(results)
Note that you will need a combine function to combine the 2 lists into a list of tuples
EDIT: Fixed code with combine function. What was I thinking?
Related
I have two lists
a = [1,2,3]
b = [9,10]
I want to combine (zip) these two lists into one list c such that
c = [(1,9), (2,10), (3, )]
Is there any function in standard library in Python to do this?
Normally, you use itertools.zip_longest for this:
>>> import itertools
>>> a = [1, 2, 3]
>>> b = [9, 10]
>>> for i in itertools.zip_longest(a, b): print(i)
...
(1, 9)
(2, 10)
(3, None)
But zip_longest pads the shorter iterable with Nones (or whatever value you pass as the fillvalue= parameter). If that's not what you want then you can use a comprehension to filter out the Nones:
>>> for i in (tuple(p for p in pair if p is not None)
... for pair in itertools.zip_longest(a, b)):
... print(i)
...
(1, 9)
(2, 10)
(3,)
but note that if either of the iterables has None values, this will filter them out too. If you don't want that, define your own object for fillvalue= and filter that instead of None:
sentinel = object()
def zip_longest_no_fill(a, b):
for i in itertools.zip_longest(a, b, fillvalue=sentinel):
yield tuple(x for x in i if x is not sentinel)
list(zip_longest_no_fill(a, b)) # [(1, 9), (2, 10), (3,)]
Another way is map:
a = [1, 2, 3]
b = [9, 10]
c = map(None, a, b)
Although that will too contain (3, None) instead of (3,). To do that, here's a fun line:
c = (tuple(y for y in x if y is not None) for x in map(None, a, b))
It's not too hard to just write the explicit Python to do the desired operation:
def izip_short(a, b):
ia = iter(a)
ib = iter(b)
for x in ia:
try:
y = next(ib)
yield (x, y)
except StopIteration:
yield (x,)
break
for x in ia:
yield (x,)
for y in ib:
yield (None, y)
a = [1, 2, 3]
b = [9, 10]
list(izip_short(a, b))
list(izip_short(b, a))
I wasn't sure how you would want to handle the b sequence being longer than the a sequence, so I just stuff in a None for the first value in the tuple in that case.
Get an explicit iterator for each sequence. Run the a iterator as a for loop, while manually using next(ib) to get the next value from the b sequence. If we get a StopIteration on the b sequence, we break the loop and then for x in ia: gets the rest of the a sequence; after that for y in ib: will do nothing because that iterator is already exhausted. Alternatively, if the first for x in ia: loop exhausts the a iterator, the second for x in ia: does nothing but there could be values left in the b sequence and the for y in ib: loop collects them.
Single line:
c = zip(a, b) + [(x,) for x in a[len(b):]] + [(x,) for x in b[len(a):]]
If you want to reuse this:
def mergeUsNicely(a, b):
def tupleMe(val):
return (val,)
return zip(a, b) + map(tupleMe, a[len(b):]) + map(tupleMe, b[len(a):])
This answer is an extension of the top answer that allows for arbitrary inputs instead of only two.
import itertools
sentinel = object()
def zip_longest_no_fill(*args):
for i in itertools.zip_longest(*args, fillvalue=sentinel):
yield tuple(x for x in i if x is not sentinel)
I got this problem:
Given an Iterable i, two disjoint sets a and b (where usually a and b each contain far more elements than i) remove all the elements from a that are found in i and add all the elements of i not shared by a to b.
For example:
i = [0, 3]
a = {1, 2, 3, 4}
b = {5, 6, 7, 8}
should result in
a = {1, 2, 4}
b = {0, 5, 6, 7, 8}
In terms of this diagram, the grey subsets are supposed to be empty. After the operation, I want the light blue subsets to be joined to b and the yellow overlap to be removed from a.
To solve it I wrote this Python function:
def foo(i, a, b):
set_i = set(i)
b |= set_i - a
a -= set_i
This works but it is not very pretty and still does things (the difference of a and set_i) twice.
What would be a more efficient way?
Can't you just do it straightforward as you described it:
def foo(i, a, b):
for el in i:
if el in a:
a.remove(el)
else:
b.add(el)
I wouldn't mutate a and b in place, especially since they are provided as arguments to a function, a safe version would be
def foo(i, a, b):
a_new = a.copy()
b_new = b.copy()
for el in i:
if el in a:
a_new.remove(el)
else:
b_new.add(el)
return a_new, b_new
It will be more efficient in the general case since i being an iterable (which could be an actual iterator), you only iterate over it once. If you convert it to a set, internally it will be iterated over once, and then again at least two times for each set operation.
You could also use groupby from itertools like this:
from itertools import groupby
def foo(i, a, b):
for in_a, i_group in groupby(i, key=lambda x: x in a):
if in_a:
a -= set(i_group)
else:
b |= set(i_group)
Try:
def foo(i, a, b):
a -= set(i)
b |= a
Example, I have the following lists in Python
A = [1,2,3]
B = [4,5,6]
I would need to create a new list C such that the elements should be paired as a separate lists based on their index numbers. i.e., C = [[[1,4],[2,5],[3,6]], [[1,4],[2,5],[3,6]]]
I have written a code to that but it is purely structured. Can you please provide if there is any better way other than the below code?
A = [1,2,3]
B = [4,5,6]
D = [a,b,c]
E = [d,e,f]
C = []
list = []
for i in range(len(A)):
list.append([A[i],B[i]])
C.append(list)
list = []
for i in range(len(D)):
list.append([D[i],E[i]])
C.append(list)
I have to repeat this code if I have multiple cases similar to the above one in my code. Which is poor in structuring.
Can someone suggest any better method for the problem?
You can use zip, convert that to list
list(zip(a,b))
which will give list of tuples. If you want list of lists, you can go:
[[i,j] for i,j in zip(a,b)]
You can try something like this in one line.
C = [[A[i],B[i]] for i in range(len(A))]
Be careful if A and B don't have the same length though!
Your problem has a simple solution found in Python's built-in functions,
The 'zip' function takes 2 arguments (iterables) and return a single iterator.
Here is an example code:
a = [1, 2, 3]
b = [4, 5, 6]
c = list(zip(a, b))
# c -> [(1, 4), (2, 5), (3, 6)]
# if you want the elements to be list instead of tuple you can do
c = [[i, j] for i,j in zip(a, b)]
# c -> [[1, 4], [2, 5], [3, 6]]
you can use zip for this
c= zip(a,b)
c=list(c)
if you convert in set then
c=set(c)
Using numpy, the following would be efficient.
import numpy as np
A = [1,2,3]
B = [4,5,6]
a = np.array(A)
b = np.array(B)
c = np.vstack((a, b)).T
print c
C = c.tolist()
print C
lower case letters are all numpy arrays, upper case are python arrays
I have 3 lists:
a = [1,2,3]
b = [4,5,6]
c = ['SUM','AVG','MAX']
I need to achieve:
d = [5,3.5,6]
Note that a and b are just an example. In reality I have a few hundred arrays with about one hundred elements. So I would prefer an interesting "Pythonian way" to achieve the d array, rather than looping around.
Something like similar to map, for example:
d= [calculate(c, x) for x in [a,b]]
Is this possible?
Here's one way to do this:
op_dict = {
'SUM' : lambda x, y: x + y,
'AVG' : lambda x, y: (x + y) / 2.0,
'MAX' : max
}
a = [1,2,3]
b = [4,5,6]
c = ['SUM','AVG','MAX']
d = [op_dict[z](x, y) for x, y, z in zip(a, b, c)]
Here's a more powerful version that can handle any number of lists, but they must still all be the same length:
op_dict = {
'SUM' : lambda *x: sum(x),
'AVG' : lambda *x: sum(x) / float(len(x)),
'MAX' : max
}
ops = ['SUM', 'AVG', 'MAX']
a = [1, 2, 3]
b = [4, 5, 6]
c = [7, 8, 9]
d = [10, 11, 12]
args = [ops, a, b, c, d]
r = [op_dict[x[0]](*x[1:]) for x in zip(*args)]
If you have less functions than elements in the lists you can cycle the functions:
a = [1, 2, 3]
b = [4, 5, 6]
c = ['SUM', 'AVG', 'MAX']
mapped = {'SUM': sum, 'AVG': lambda x: sum(x, 0.0) / len(x), 'MAX': max}
from itertools import cycle
def calc(funcs, *args):
_fncs = list(map(mapped.get, funcs))
return [f(it) for f, it in zip(cycle(_fncs), zip(*args))]
If c is shorter than a and b then the functions will be cycled, if the lists are different lengths you can use itertools.izip_longest but you will have to decide what an appropriate fillvalue is.
If you are using python 3 there is also a builtin for the average:
mapped = {'SUM': sum, 'AVG': mean, 'MAX': max}
from itertools import cycle
from statistics import mean
def calc(funcs, *args):
_fncs = list(map(mapped.get, funcs))
return [f(it) for f, it in zip(cycle(_fncs), zip(*args))]
If they are always going to be even lengths you don't need the cycle:
def calc(funcs, *args):
_fncs = map(mapped.get, funcs)
return [f(it) for f, it in zip(_fncs, zip(*args))]
How about:
def sum(a, b):
return a + b
def avg(a, b):
return a + b / 2.0
def calculate(a, b, c):
mapping = {'SUM': sum,
'AVG': avg,
'MAX': max}
return mapping[c](a,b)
d = [calculate(a1,b1,c1) for a1,b1,c1 in zip(a, b, c)]
So, working from the bottom up:
Zip is great for iterating through multiple lists at once. a1,b1,c1 are the items in a, b, and c at the same index.
Calculate will do a different calculation depending on c.
the mapping dict simply stores functions ( notice the lack of parens, we don't call the function. We are storing the function itself. )
The we grab the correct function, determined by the key we pass in, and call it with two arguments.
edit: I love how we've all given essentially the same answer, within a small period of time. Kinda reassures you that this is the way to do it.
Obviously syntatic sugar like lambdas help keep the code looking simpler.
I have two lists
a = [1,2,3]
b = [9,10]
I want to combine (zip) these two lists into one list c such that
c = [(1,9), (2,10), (3, )]
Is there any function in standard library in Python to do this?
Normally, you use itertools.zip_longest for this:
>>> import itertools
>>> a = [1, 2, 3]
>>> b = [9, 10]
>>> for i in itertools.zip_longest(a, b): print(i)
...
(1, 9)
(2, 10)
(3, None)
But zip_longest pads the shorter iterable with Nones (or whatever value you pass as the fillvalue= parameter). If that's not what you want then you can use a comprehension to filter out the Nones:
>>> for i in (tuple(p for p in pair if p is not None)
... for pair in itertools.zip_longest(a, b)):
... print(i)
...
(1, 9)
(2, 10)
(3,)
but note that if either of the iterables has None values, this will filter them out too. If you don't want that, define your own object for fillvalue= and filter that instead of None:
sentinel = object()
def zip_longest_no_fill(a, b):
for i in itertools.zip_longest(a, b, fillvalue=sentinel):
yield tuple(x for x in i if x is not sentinel)
list(zip_longest_no_fill(a, b)) # [(1, 9), (2, 10), (3,)]
Another way is map:
a = [1, 2, 3]
b = [9, 10]
c = map(None, a, b)
Although that will too contain (3, None) instead of (3,). To do that, here's a fun line:
c = (tuple(y for y in x if y is not None) for x in map(None, a, b))
It's not too hard to just write the explicit Python to do the desired operation:
def izip_short(a, b):
ia = iter(a)
ib = iter(b)
for x in ia:
try:
y = next(ib)
yield (x, y)
except StopIteration:
yield (x,)
break
for x in ia:
yield (x,)
for y in ib:
yield (None, y)
a = [1, 2, 3]
b = [9, 10]
list(izip_short(a, b))
list(izip_short(b, a))
I wasn't sure how you would want to handle the b sequence being longer than the a sequence, so I just stuff in a None for the first value in the tuple in that case.
Get an explicit iterator for each sequence. Run the a iterator as a for loop, while manually using next(ib) to get the next value from the b sequence. If we get a StopIteration on the b sequence, we break the loop and then for x in ia: gets the rest of the a sequence; after that for y in ib: will do nothing because that iterator is already exhausted. Alternatively, if the first for x in ia: loop exhausts the a iterator, the second for x in ia: does nothing but there could be values left in the b sequence and the for y in ib: loop collects them.
Single line:
c = zip(a, b) + [(x,) for x in a[len(b):]] + [(x,) for x in b[len(a):]]
If you want to reuse this:
def mergeUsNicely(a, b):
def tupleMe(val):
return (val,)
return zip(a, b) + map(tupleMe, a[len(b):]) + map(tupleMe, b[len(a):])
This answer is an extension of the top answer that allows for arbitrary inputs instead of only two.
import itertools
sentinel = object()
def zip_longest_no_fill(*args):
for i in itertools.zip_longest(*args, fillvalue=sentinel):
yield tuple(x for x in i if x is not sentinel)