I have 3 lists:
a1 = range(10)
a2 = range(10,20)
a3 = range(20,30)
I need to do the following:
For each list, get max of every 5 element blocks, so hypothetically:
a1_maxes = [max1_a1, max2_a1]
a2_maxes = [max1_a2, max2_a2]
a3_maxes = [max1_a3, max2_a3]
Sum each "maxes" list, so:
for each i:
sum_i = sum(ai_maxes)
Take the max of these 3 sums, so:
max(sum_1, sum_2, sum_3)
I could not get myself to use map() here. What would be the most Pythonic (concise) way to do this? Thanks.
a1 = range(10)
a2 = range(10,20)
a3 = range(20,30)
print(max(sum(x[i:i+5]) for x in (a1,a2,a3) for i in xrange(0,len(a1),5)))
135
Just get the sumof each chunk x[i:i+5]
To make it more obvious, the lists become split into the following chucks:
print(list(x[i:i+5]) for x in [a1,a2,a3] for i in xrange(0,len(a1),5))
[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24], [25, 26, 27, 28, 29]]
Then max just gets the largest sum:
If you want the highest two elements from each check:
mx_pair = max(sorted(x[i:i+5])[-2:] for x in (a1,a2,a3) for i in xrange(0,len(a1),5))
print(sum(mx_pair))
57
If the answer should be 53:
from itertools import izip,imap
def chunks(l):
for i in xrange(0,len(l), 5):
yield l[i:i+5]
sums = (max(izip(*ele)) for ele in imap(chunks,(a1,a2,a3)))
print(sum(max(sums)))
Let's break this into pieces.
The first point is that you probably don't want separate a1, a2, and a3` variables; if you're going to have to do the exact same thing repeatedly to multiple values, and then iterate over those values, they probably belong in a list. So:
a = [a1, a2, a3]
Now, how do you split an iterable into 5-element pieces? There are a number of ways to do it, from the grouper function in the itertools recipes to zipping slices to iterating over slices. I'll use grouper:
grouped = [grouper(sublist, 5) for sublist in a]
Now we just want the max value of each group, so:
maxes = [[max(group) for group in sublist] for sublist in a]
And now, we want to sum up each sublist:
sums = [sum(sublist) for sublist in maxes]
And finally, we want to take the max of these sums:
maxsum = max(sums)
Now, given that each of these list comprehensions is only being used as a one-shot iterable, we might as well turn them into generator expressions. And if you want to, you can merge some of the steps together:
maxsum = max(sum(max(group) for group in grouper(sublist, 5)) for sublist in a)
And, having done that, you don't actually need a to be created explicitly, because it only appears once:
maxsum = max(sum(max(group) for group in grouper(sublist, 5))
for sublist in a1, a2, a3)
Related
I would like to know how to use the python random.sample() function within a for-loop to generate multiple sample lists that are not identical.
For example, right now I have:
for i in range(3):
sample = random.sample(range(10), k=2)
This will generate 3 sample lists containing two numbers each, but I would like to make sure none of those sample lists are identical. (It is okay if there are repeating values, i.e., (2,1), (3,2), (3,7) would be okay, but (2,1), (1,2), (5,4) would not.)
If you specifically need to "use random.sample() within a for-loop", then you could keep track of samples that you've seen, and check that new ones haven't been seen yet.
import random
seen = set()
for i in range(3):
while True:
sample = random.sample(range(10), k=2)
print(f'TESTING: {sample = }') # For demo
fr = frozenset(sample)
if fr not in seen:
seen.add(fr)
break
print(sample)
Example output:
TESTING: sample = [0, 7]
[0, 7]
TESTING: sample = [0, 7]
TESTING: sample = [1, 5]
[1, 5]
TESTING: sample = [7, 0]
TESTING: sample = [3, 5]
[3, 5]
Here I made seen a set to allow fast lookups, and I converted sample to a frozenset so that order doesn't matter in comparisons. It has to be frozen because a set can't contain another set.
However, this could be very slow with different inputs, especially a larger range of i or smaller range to draw samples from. In theory, its runtime is infinite, but in practice, random's number generator is finite.
Alternatives
There are other ways to do the same thing that could be much more performant. For example, you could take a big random sample, then chunk it into the desired size:
n = 3
k = 2
upper = 10
sample = random.sample(range(upper), k=k*n)
for chunk in chunks(sample, k):
print(chunk)
Example output:
[6, 5]
[3, 0]
[1, 8]
With this approach, you'll never get any duplicate numbers like [[2,1], [3,2], [3,7]] because the sample contains all unique numbers.
This approach was inspired by Sven Marnach's answer on "Non-repetitive random number in numpy", which I coincidentally just read today.
it looks like you are trying to make a nested list of certain list items without repetition from original list, you can try below code.
import random
mylist = list(range(50))
def randomlist(mylist,k):
length = lambda : len(mylist)
newlist = []
while length() >= k:
newlist.append([mylist.pop(random.randint(0, length() - 1)) for I in range(k)])
newlist.append(mylist)
return newlist
randomlist(mylist,6)
[[2, 20, 36, 46, 14, 30],
[4, 12, 13, 3, 28, 5],
[45, 37, 18, 9, 34, 24],
[31, 48, 11, 6, 19, 17],
[40, 38, 0, 7, 22, 42],
[23, 25, 47, 41, 16, 39],
[8, 33, 10, 43, 15, 26],
[1, 49, 35, 44, 27, 21],
[29, 32]]
This should do the trick.
import random
import math
# create set to store samples
a = set()
# number of distinct elements in the population
m = 10
# sample size
k = 2
# number of samples
n = 3
# this protects against an infinite loop (see Safety Note)
if n > math.comb(m, k):
print(
f"Error: {math.comb(m, k)} is the number of {k}-combinations "
f"from a set of {m} distinct elements."
)
exit()
# the meat
while len(a) < n:
a.add(tuple(sorted(random.sample(range(m), k = k))))
print(a)
With a set you are guaranteed to get a collection with no duplicate elements. In a set, you would be allowed to have (1, 2) and (2, 1) inside, which is why sorted is applied. So if [1, 2] is drawn, sorted([1, 2]) returns [1, 2]. And if [2, 1] is subsequently drawn, sorted([2, 1]) returns [1, 2], which won't be added to the set because (1, 2) is already in the set. We use tuple because objects in a set have to be hashable and list objects are not.
I hope this helps. Any questions, please let me know.
Safety Note
To avoid an infinite loop when you change 3 to some large number, you need to know the maximum number of possible samples of the type that you desire.
The relevant mathematical concept for this is a combination.
Suppose your first argument of random.sample() is range(m) where
m is some arbitrary positive integer. Note that this means that the
sample will be drawn from a population of m distinct members
without replacement.
Suppose that you wish to have n samples of length k in total.
The number of possible k-combinations from the set of m distinct elements is
m! / (k! * (m - k)!)
You can get this value via
from math import comb
num_comb = comb(m, k)
comb(m, k) gives the number of different ways to choose k elements from m elements without repetition and without order, which is exactly what we want.
So in the example above, m = 10, k = 2, n = 3.
With these m and k, the number of possible k-combinations from the set of m distinct elements is 45.
You need to ensure that n is less than 45 if you want to use those specific m and k and avoid an infinite loop.
I find myself in a unique situation in which I need to multiply single elements within a listed pair of numbers where each pair is nested within a parent list of elements. For example, I have my pre-defined variables as:
output = []
initial_list = [[1,2],[3,4],[5,6]]
I am trying to calculate an output such that each element is the product of a unique combination (always of length len(initial_list)) of a single element from each pair. Using my example of initial_list, I am looking to generate an output of length pow(2 * len(initial_list)) that is scable for any "n" number of pairs in initial_list (with a minimum of 2 pairs). So in this case each element of the output would be as follows:
output[0] = 1 * 3 * 5
output[1] = 1 * 3 * 6
output[2] = 1 * 4 * 5
output[3] = 1 * 4 * 6
output[4] = 2 * 3 * 5
output[5] = 2 * 3 * 6
output[6] = 2 * 4 * 5
output[7] = 2 * 4 * 6
In my specific case, the order of output assignments does not matter other than output[0], which I need to be equivalent to the product of the first element in each pair in initial_list. What is the best way to proceed to generate an output list such that each element is a unique combination of every element in each list?
...
My initial approach consisted of using;
from itertools import combinations
from itertools import permutations
from itertools import product
to somehow generate a list of every possible combination then multiply the products together and append each product to the output list, but I couldn't figure out a wait to implement the tools successfully. I have since tried to create a recursive function that combines for x in range(2): with nested recursion recalls, but once again I cannot figured out a solution.
Someone more experienced and smarter than me please help me out; Any and all help is appreciated! Thank you!
Without using any external library
def multi_comb(my_list):
"""
This returns the multiplication of
every possible combinationation of
the `my_list` of type [[a1, a2], [b1, b2], ...]
Arg: List
Return: List
"""
if not my_list: return [1]
a, b = my_list.pop(0)
result = multi_comb(my_list)
left = [a * i for i in result]
right = [b * i for i in result]
return (left + right)
print(multi_comb([[1, 2], [3, 4], [5, 6]]))
# Output
# [15, 18, 20, 24, 30, 36, 40, 48]
I am using reccursion to get the result. Here's the visual illustration of how this works.
Instead of taking a top-down approach, we can take bottom-up approach to better understand how this program works.
At the last step, a and b becomes 5 and 6 respectively. Calling multi_comb() with empty list returns [1] as a result. So left and right becomes [5] and [6]. Thus we return [5, 6] to our previous step.
At the second last step, a and b was 3 and 4 respectively. From the last step we got [5, 6] as a result. After multiplying each of the values inside the result with a and b (notice left and right), we return the result [15, 18, 20, 24] to our previous step.
At our first step, that is our starting step, we had a and b as 1 and 2 respectively. The value returned from our last step becomes our result, ie, [15, 18, 20, 24]. Now we multiply both a and b with this result and return our final output.
Note:
This program works only if list is in the form [ [a1, a2], [b1, b2], [c1, c2], ... ] as told by the OP in the comments. The problem of solving the list containing the sub-list of n items will be little different in code, but the concept is same as in this answer.
This problem can also be solved using dynamic programming
output = [1, ]
for arr in initial_list:
output = [a * b for a in arr for b in product]
This problem is easy to solve if you have just one subarray -- the output is the given subarray.
Suppose you solved the problem for the first n - 1 subarrays, and you got the output. The new subarray is appended. How the output should change? The new output is all pair-wise products of the previous output and the "new" subarray.
Look closely, there's an easy pattern. Let there be n sublists, and 2 elements in each: at index 0 and 1. Now, the indexes selected can be represented as a binary string of length n.
It'll start with 0000..000, then 0000...001, 0000...010 and so on. So all you need to do is:
n = len(lst)
for i in range(2**n):
binary = bin(i)[2:] #get binary representation
for j in range(n):
if binary[j]=="1":
#include jth list's 1st index in product
else:
#include jth list's 0th index in product
The problem would a scalable solution would be, since you're generating all possible pairs, the time complexity will be O(2^N)
Your idea to use itertools.product is great!
import itertools
initial_list = [[1,2],[3,4],[5,6]]
combinations = list(itertools.product(*initial_list))
# [(1, 3, 5), (1, 3, 6), (1, 4, 5), (1, 4, 6), (2, 3, 5), (2, 3, 6), (2, 4, 5), (2, 4, 6)]
Now, you can get the product of each tuple in combination using for-loops, or using functools.reduce, or you can use math.prod which was introduced in python 3.8:
import itertools
import math
initial_list = [[1,2],[3,4],[5,6]]
output = [math.prod(c) for c in itertools.product(*initial_list)]
# [15, 18, 20, 24, 30, 36, 40, 48]
import itertools
import functools
import operator
initial_list = [[1,2],[3,4],[5,6]]
output = [functools.reduce(operator.mul, c) for c in itertools.product(*initial_list)]
# [15, 18, 20, 24, 30, 36, 40, 48]
import itertools
output = []
for c in itertools.product(*initial_list):
p = 1
for x in c:
p *= x
output.append(p)
# output == [15, 18, 20, 24, 30, 36, 40, 48]
Note: if you are more familiar with lambdas, operator.mul is pretty much equivalent to lambda x,y: x*y.
itertools.product and math.prod are a nice fit -
from itertools import product
from math import prod
input = [[1,2],[3,4],[5,6]]
output = [prod(x) for x in product(*input)]
print(output)
[15, 18, 20, 24, 30, 36, 40, 48]
I'll do my best to explain my concern.
This problem is a practice activity for me as I learn python.
How could I check one by one if the element of the first lists is bigger than the element on the second lists then subtract?
Example:
# inputted values
list1 = [50,40,90,30]
list2 = [40,50,40,20]
# desired output:
output = [10,40,50,10]
You needs some real lists, not just four variables. Then do this.
list1=[50,40, 90, 30]
list2=[40, 50, 40, 20]
output=[p1-p2 if p1>p2 else p1 for (p1, p2) in zip(list1, list2)]
print(output) #==> [10, 40, 50, 10]
You essentially need to iterate over both lists together and compare each element pairwise, then the output will be element from list1 if it is smaller than the element in list2, or else the output will be the difference of list2's element and list1's element
Two ways to achieve this
List-comprehension by simultaneously iterating on both lists
list1 = [50,40,90,30]
list2 = [40,50,40,20]
print([list1[idx] if list1[idx] < list2[idx] else list1[idx] - list2[idx] for idx in range(len(list1))])
Output is [10, 40, 50, 10]
List-comprehension by iterating on the iterator created by zipping both lists together
list1 = [50,40,90,30]
list2 = [40,50,40,20]
print([item1 if item1 < item2 else item1 - item2 for item1, item2 in zip(list1, list2)])
Output is [10, 40, 50, 10]
I don't understand why we are all answering a new python dev with list comps, I believe that a newly minted dev should first start with an easier syntax, and let them learn about the python features (like list comps) that can improve their code.</rant>
To answer your question,
1. you can iterate multiple lists of the same length using python's built-in zip which takes multiple iterables as arguments and returns a tuple with the current element of each iterator at the same index, the function works like this;
list1 = [50,40,90,30]
list2 = [40,50,40,20]
for i1, i2 in zip(list1, list2):
print(i1, i2)
# result
50, 40
40, 50
90, 40
30, 20
and then 2. you can start adding your logic at each iteration;
# if you want to store the results use the results list.
results = []
list1 = [50,40,90,30]
list2 = [40,50,40,20]
for i1, i2 in zip(list1, list2):
if i1 > i2:
result.append(i1 - i2)
# or if you don't need to store the results, just use...
print(i1 - i2)
else:
result.append(i1)
# same here...
print(i1)
# result
10
40
50
10
# results == [10, 40, 50, 10]
It might be an overkill to use numpy but the syntax is clear and concise:
>>> import numpy as np
>>> a1 = np.array([50, 40, 90, 30])
>>> a2 = np.array([40, 50, 40, 20])
>>> np.where(a1 >= a2, a1 - a2, a1)
array([10, 40, 50, 10])
I want to multiply an element of a list with all other elements.
For example:
def product(a,b,c):
return (a*b, a*c, a*b*c)
I have done this
def product(*args):
list = []
for index,element in enumerate(args):
for i in args:
if (args[index]*i) not in list:
list.append(args[index]*i)
return list
but this gives me [a*a, a*b,a*c, b*b] etc. I don't want the a*a, b*b, c*c bit in there.
you could check for equality
if (args[index]*i) not in list and args[index] != i:
itertools is your friend here:
from itertools import combinations
from functools import reduce, partial
from operator import mul
# Make a sum-like function for multiplication; I'd call it product,
# but that overlaps a name in itertools and our own function
multiplyall = partial(reduce, mul)
def product(*args):
# Loop so you get all two elements combinations, then all three element, etc.
for n in range(2, len(args) + 1):
# Get the combinations for the current combo count
for comb in combinations(args, n):
# Compute product and yield it
# yielding comb as well just for illustration
yield comb, multiplyall(comb)
I made it a generator function, because frankly, almost any function that's just slowly building a list element by element and returning it should really be a generator function (if the caller wants a list, they just do mylist = list(generatorfunc(...))), making it easier to use iteratively without blowing main memory when many arguments are passed.
Example usage:
>>> for pieces, prod in product(2, 3, 4):
print ' * '.join(map(str, pieces)), '=', prod
Which outputs:
2 * 3 = 6
2 * 4 = 8
3 * 4 = 12
2 * 3 * 4 = 24
So if the values are 2, 3, 4, 5 you want all and only these products:
2*3=6, 2*4=8, 2*5=10, 2*3*4=24, 2*3*5=30, 2*4*5=40, 2*3*4*5=120
This means taking all combinations of 3, 4, 5 and then multiplying them togther with 2. The itertools module has a combinations function, and reduce can be used in conjunction with operator.mul to do the calculation:
def product(first, *other):
for n in range(1, len(other) + 1):
for m in combinations(other, n):
yield reduce(mul, m, first)
list(product(2, 3, 4, 5))
Output:
[6, 8, 10, 24, 30, 40, 120]
Does your list have duplicate elements, like [2, 3, 4, 2]?
If it does not, here is a one liner:
First, with tags to illustrate the pattern:
a = ['a1','a2','a3']
lsta = [[x+y for y in [z for z in a if z != x]] for x in a]
lsta
[['a1a2', 'a1a3'], ['a2a1', 'a2a3'], ['a3a1', 'a3a2']]
And here, with numbers:
a =[2,3,4,5]
print [[x*y for y in [z for z in a if z != x]] for x in a]
[[6, 8, 10], [6, 12, 15], [8, 12, 20], [10, 15, 20]]
or the sum of the products, if you wish:
a =[2,3,4,5]
print [sum([x*y for y in [z for z in a if z != x]]) for x in a]
[24, 33, 40, 45]
If the list has duplicates, it gets more complicated. Do you want the first occurrence and the second occurrence of 2 in [2,3,4,2] to be separately calculated (you might need that for some purposes even though you will get the same value for both)?
consider x = [10,10,20,20,20,30]
How do i form another list_x1 which contains only same values example: list_x1 = [10,10]
and list_x2 =[20,20] and list_x3 =[30] ?
You can use counter.
from collections import Counter
x = [10, 10, 20, 20, 20, 30]
my_counter = Counter(x)
d = {'list_x{0}'.format(key): [key] * my_counter[key] for key in my_counter}
>>> d
{'list_x10': [10, 10], 'list_x20': [20, 20, 20], 'list_x30': [30]}
One of the issues with your request is that you would need to pre-assign variables, which aren't initially know. I've used a dictionary as a container to hold them.
For a list, [10] * 3 results in [10, 10, 10]. So, [k] * my_counter multiplies the unique key value by the number of occurrences.
With itertools.groupby
>>> from itertools import groupby
>>> x = [10,10,20,20,20,30]
>>> [list(g) for k, g in groupby(x)]
[[10, 10], [20, 20, 20], [30]]
Perhaps the best way is #Alexander's idea with collections, but I always find it helpful to look at more 'native' python code to see what's going on. So here's a way to do it:
x = [10,10,20,20,20,30]
def foo(iterable):
for val in iterable:
cnt = iterable.count(val)
iterable = list(filter(lambda x: x != val, iterable))
if cnt:
yield [val]*cnt
for _ in foo(x):
print(_)
Note that the complexity factor is going to be fairly high. Certainly not O(n) because you have to:
Iterate through each of the values in our main for val in iterable
Iterate through each of the values every time we call iterable.count
Iterate through each of the values when we filter() them to prevent duplicates.
Using collections.Counter:
>>> def list_gen(in_list, elem):
... count = collections.Counter(in_list)
... return [elem] * count[elem]
...
>>> a
[1, 2, 3, 2, 3]
>>> list_gen(a, 2)
[2, 2]
This isn't exactly what you're looking for, but this code will generate a list of lists separating the values.
x = [10, 10, 20, 20, 20, 30]
uniques = set(x)
output = []
for unique in uniques:
unique_count = x.count(unique)
temp = []
for i in range(0, unique_count):
temp.append(unique)
output.append(temp)
You can then use list comprehensions on output