Suppose I have an array as follows:
A = np.array([a,b,c,d,e,f])
Each element in the array has a value that is equal to a scalar (e.g. maybe a is equal to 2, b is equal to 7, etc.). But a and b take a long time to evaluate, so it might take a couple of minutes for Python to determine the value a, a couple more minutes to determine the value of b, and similarly for the rest of the elements.
If there a way to define a function (lets call it sumfirstn) that evaluates the first n terms of A and computes their sum? For example, sumfirstn(A,2) would return a+b, sumfirstn(A,3) would return a+b+c, etc.
But (and this is important) sumfirstn(A,2) must not evaluate c, d, e, and f. It must only evaluate the relevant terms that are being added (to reduce computation time). Of course, A[0:n].sum() does the trick, but this requires Python computing the entire array A, which I don't want to do!
Any ideas? A doesn't have to be an array, could a list, data-frame, or be in some other form... whatever works.
It looks like you need a generic solution to sum the first n items of any iterable, which not necessarily has to support indexing, slicing or a sum method.
If I understand correctly, the following should suffice as a generic solution.
>>> from itertools import islice
>>>
>>> def first_n_sum(iterable, n):
... return sum(islice(iterable, n))
Demo:
>>> first_n_sum([1,2,3,4], 3)
6
>>> first_n_sum((x for x in (1, 2, 3, 4)), 2)
3
>>> first_n_sum([], 123)
0
>>> from collections import OrderedDict
>>> first_n_sum(OrderedDict([(1, 'a'), (2, 'b'), (-3, 'c'), (100, 'd')]), 3)
0
Note that there will be no error if the iterable cannot produce n values, just like slicing beyond the length of an array is okay. In this case the sum of all the elements will be returned.
Supposing that a,b,c,d,e,f are functions we can do something like:
def func():
return 1+2+3
def sumfirstn(arr_of_functions, n):
return sum([arr_of_functions[i]() for i in range(n)])
A = np.array([func, func, func, func, func, func])
sumfirstn(A,2)
I have to use functional programming to implement the following function takes in a list of numbers from 0 to 9. The goal is to find the five consecutive elements of the list that have the greatest product. The function should return tuple of the index of the greatest product and the value of the greatest product without using the max function.
I can easily implement this without functional programming but I am having trouble implementing it without any loops.
This is my approach so far but the part that I am stuck on is how to loop through the array to find those consecutive five numbers without loops. I am trying to use map to do that but I don't think it is correct. Is it possible to incorporate enumerate in any way? Any help is appreciated.
def find_products(L):
val = map(lambda a: reduce(lambda x,y: x*y, L),L)
print (val)
This doesn't have any explicit loops or call the max function. The function assumes that there're at least five elements in the input list and outputs a tuple (start_index, max_product).
from functools import reduce, partial
import operator
def f(l):
win = zip(l, l[1:], l[2:], l[3:], l[4:])
products = map(partial(reduce, operator.mul), win)
return reduce(lambda x, y: x if x[1] > y[1] else y, enumerate(products))
In [2]: f([1, 2, 3, 4, 7, 8, 9])
Out[2]: (2, 6048)
In [3]: f([2, 6, 7, 9, 1, 4, 3, 5, 6, 1, 2, 4])
Out[3]: (1, 1512)
win = zip(l, l[1:], l[2:], l[3:], l[4:]) creates a sliding window iterator of size 5 over the input list. products = map(partial(reduce, operator.mul), win) is an iterator calling partial(reduce, operator.mul) (translates to reduce(operator.mul, ...)) on every element of win. reduce(lambda x, y: x if x[1] > y[1] else y, enumerate(products)) adds a counter to products and returns the index-value pair with the highest value.
If you need a more general version and/or the input list is large you'd use itertools.islice:
from itertools import islice
def f(l, n=5):
win = zip(*(islice(l, i, None) for i in range(n)))
...
The code above uses a generator expression which is a loop, technically. A pure functional version of that might look like
from itertools import islice
def f(l, n=5):
win = zip(*map(lambda i: islice(l, i, None), range(n)))
...
from functools import reduce #only for python3, python2 doesn't need import
def find_products(L):
if len(L)==0:
return 0
if len(L) <= 5:
return reduce( lambda x,y:x*y, L)
pdts = ( reduce(lambda a,b:a*b,L[pos:pos+5]) for pos in range(len(L)-4)) # or pdts = map(lambda pos: reduce(lambda a,b:a*b,L[pos:pos+5],0),range(len(L)-4))
mx = reduce(lambda x,y: x if x>y else y, pdts)
return mx
pdts contains all the possible 5 tuple products, and then using reduce to mimic the max function, we find the maximum among the products.
You could do the following:
For each start index in range(0, len(L) - 5)
Map the index to the tuple of start and the product of items L[start:start + 5]
Reduce the tuples to the one that has the highest product
Get the first value of the resulting tuple = the start index of the 5 elements that have the highest product
Return the slice L[result:result + 5]
This algorithm could be further improved to avoid re-calculating sub-products, but use a "rolling product", that is updated as you reduce from left to right, dividing by the element that was dropped, and multiplying by the new element that was added.
Here is a Haskell solution, which is purely functional:
import Data.List
multiply :: [Int] -> Int
multiply = foldr (*) 1
consecutiveProducts :: [Int] -> [(Int,Int)]
consecutiveProducts xs =
[(i,multiply $ take 5 h) | (i,h) <- zipped, length h >= 5]
where
indices = reverse [0..(length xs)]
zipped = zip indices (tails xs)
myComp (i1,h1) (i2,h2) = compare h2 h1
main = print $ head $ sortBy myComp $ consecutiveProducts [4,5,3,1,5,3,2,3,5]
Here is what it does:
Starting in the last line, it computes the consecutive products from that list.
tails xs gives all the subsets starting with different starting values:
> tails [4,5,3,1,5,3,2,3,5]
[[4,5,3,1,5,3,2,3,5],[5,3,1,5,3,2,3,5],[3,1,5,3,2,3,5],[1,5,3,2,3,5],[5,3,2,3,5],[3,2,3,5],[2,3,5],[3,5],[5],[]]
From these tails we only take those that are at least 5 elements long.
Then we zip them with natural numbers such that we have the starting index associated with it.
From each of the subsets we take the first five elements.
These five elements are passed to the multiply function. There those are reduced to a single number, the product.
After that we go back to the last line, we sort the list by the product value descending.
From the resulting list we only take the first element.
And then we print the result, which is (5,450) for my input data.
This solution uses reduce to calculate a 5-value product, list comprehension for generating all of those products, tuple creation for having the index to go with each, reduce again to get the best tuple.
An if else operator is used to catch the case when there are no 5 values in the input.
from functools import reduce
def find_products(values):
return None if len(values) < 5 else reduce(
lambda best, this: this if this[1] > best[1] else best,
[(i, reduce(lambda a,b: a*b, values[i:i+5], 1)) for i in range(0, len(values)-4)]
)
result = find_products([1, 0, 8, 3, 5, 1, 0, 2, 2, 3, 2, 2, 1])
print (result)
Output for the example call is:
(7, 48)
A Pure Python Solution using recursion
First, we need to create a recursive function to find the product of a list:
def product(l, i=0, s=1):
s *= l[i]
if i+1 < len(l):
return product(l, i+1, s)
return s
which we can do some tests for:
>>> product([1, 2, 3])
6
>>> product([1, 1, 1])
3
>>> product([2, 2, 2])
8
Then, we can use this function in another recursive function to solve your problem:
def find_products(l, i=0, t=(0, -1)):
p = product(l[i:i+5])
if p > t[1]:
t = (i, p)
if i+5 < len(l):
return find_products(l, i+1, t)
return t
which works!
Here are some tests to show it working:
>>> find_products([1, 1, 5, 5, 5, 5, 5, 1, 1])
(2, 3125)
>>> find_products([1, 1, 1, 1, 1, 0, 0, 0, 0])
(0, 1)
>>> find_products([1, 4, 5, 2, 7, 9, 3, 1, 1])
(1, 2520)
want one liner using max and without max try this
from numpy import prod
l=[2,6,7,9,1,4,3]
max([prod(l[i:i+5]) for i in range(len(l))])
sorted([prod(l[i:i+5]) for i in range(len(l))])[-1] // without max
Imperative paradigm is often:
state = state0
while condition:
# change state
This is the "natural" way of programming for lot of people and you know how do that in this way.
The pure functional paradigm forbid variables, which have some advantages . It works with functions which communicates through parameters(IN) and return values(OUT). It frequently uses recursive functions.
A generic functional recursive scheme is :
f = lambda *args : result(*args) if condition(*args) else f(*newparams(*args))
Here we can find a solution with (l,i,imax,prodmax) as parameters, and:
condition = lambda l,i,_,__ : i>=len(l)-5
result = lambda _,__,*args : args
newparams = lambda l,i,imax,prodmax: (l, i+1, imax, prodmax) \
if l[i]*l[i+1]*l[i+2]*l[i+3]*l[i+4] <= prodmax \
else (l, i+1, i, l[i]*l[i+1]*l[i+2]*l[i+3]*l[i+4])
None other than functions have been defined.
You can even define no functions to do that, see here for example, but readability suffers even more.
Run :
In [1]: f([random.randint(0,9) for i in range (997)],0,0,0)
Out[1]: (386, 59049)
Python limits this approach by setting recursive depth to 2000, and from Python 3, by hiding functional tools in the module functools.
I dont know how to properly ask this question but here is what I am trying to do.
lists = []
for x in range(3):
for y in range(3):
if x!=y:
lists.append([x,y])
Is there a simple solution so it doesnt give me lists that are the same but reversed:
for example [2,0] and [0,2]?
I know I could go through the lists and remove them afterwards but is there a solution to not even make the list? (sorry my english isnt perfect)
You can use itertools.combinations
>>> from itertools import combinations
>>> list(combinations(range(3), 2))
[(0, 1), (0, 2), (1, 2)]
With the above example we take any combination of two elements from range(3) without repeating any elements.
Sure: if you add all pairs with y > x instead of all possible pairs, only one of each pair (x, y) and (y, x) will appear.
lists = []
for x in range(3):
for y in range(x + 1, 3):
lists.append([x,y])
If you don't want those "duplicates", you want a combination
a combination is a way of selecting items from a collection, such that (unlike permutations) the order of selection does not matter
>>> import itertools
>>> list(itertools.combinations(iterable=range(3), r=2))
[(0, 1), (0, 2), (1, 2)]
Above I have used combinations() from the Python module itertools.
Explanation
I've set r=2 because you want a subsequence length of 2 (the form you described as [x, y])
The iterable=range(3) parameter is just a list of elements that are going to be used to make combinations of, so range(3) would result in [0, 1, 2]
The list() applied to the end result is simply to force the output to be printed out to the console because otherwise itertools.combinations returns an iterable that you iterate through to pull the elements one by one.
Easy:
for x in range(3):
for y in range(x, 3):
lists.append([x,y])
Can anyone help me one this one?
I am trying to find a way to count the range between 2 list on integers; and to get each step necessary to get from one list to then next
using these 2 arrays:
a = [1,1,1]
b = [3,4,3]
I'd like to arrive to a sequence of in-between values:
[[2,2,2], [None,3,None]]
Thanks
a
This is quite simple to do with itertools.zip_longest() and a list comprehension:
>>> import itertools
>>> list(itertools.zip_longest(*[range(i+1, j) for i, j in zip(a, b)]))
[(2, 2, 2), (None, 3, None)]
Note that in 2.x itertools.zip_longest() doesn't exist - it's called itertools.izip_longest() instead.
This works by zip()ing the values together so we get the bounds, then we generate the range we need (adding one to the lower bound as you seem to not want to include it), then we separate them out into parts, using itertools.zip_longest() (which also introduces the None values).
A variation of Lattywares Answer that works in python 2.5 and below where izip_longest is not available:
map(None, *[range(x + 1, y) for x, y in zip(a, b)])
Up to this point in time, in Python, I've only ever seen list comprehensions that specify the inclusion of one element at time. For example, they're all in the following form
[f(x) for x in <iterable>]
Is there any way to specify more than one element at a time? For example,
[f(x), g(x) for x in <iterable>]
I'm asking because I want to create a list comprehension that calculates all the divisors of some number x, but I want to do this as efficiently as possible. Right now I'm using the following,
[y for y in range(1,x+1) if x%y == 0]
but I'd like to do something like this,
[y, x/y for y in range(1, sqrt(x)+1) if x%y == 0]
as this would be more efficient. Btw, I have no practical reason for doing this. It's simply a challenge problem that somebody told me and the goal is to do it with the smallest, most efficient list comprehension possible.
Thanks in advance.
Edit: Ok, it looks like I have to use tuples. I was trying to avoid that though as I'd then have to make another function to flatten the list which would make my solution longer.
Edit 2: Just in case anyone stumbles upon this question, the answer to what I wanted to do in my original question is this:
[y for x in <iterable> for y in f(x), g(x)]
It uses nested for loops in the list comprehension to get the job done.
You can flatten it in place
[y if i else x/y for y in range(1, sqrt(x)+1) for i in 0,1 if x%y == 0]
You can assign to tuples
[(y, x/y) for y in range(1, int(sqrt(x))+1) if x%y == 0]
Not really related to your basic question, but your example: I had to convert the 2nd parameter of range() to an int since sqrt() resulted in a float in my test code.
Update re Edit in post:
To flatten this list of tuples:
In [24]: s
Out[24]: [(1, 20), (2, 10), (4, 5)]
use this:
In [25]: import operator
create a tuple:
In [26]: reduce(operator.add, s, ())
Out[26]: (1, 20, 2, 10, 4, 5)
create a list:
In [27]: list(reduce(operator.add, s, ()))
Out[27]: [1, 20, 2, 10, 4, 5]
Note: In a helpful comment #jamylak points out that reduce and operator.add run order O(N^2), and that using itertools.chain would be much more efficient. This becomes more important as the size of the list grows and should be considered in that case.
Yes, you can do it. You just need to put parentheses around the element. What you get is a list of tuples.
>>> [(x, x+1) for x in range(5)]
[(0, 1), (1, 2), (2, 3), (3, 4), (4, 5)]
Oh so close:
[(y, x/y) for y in range(1, sqrt(x)+1) if x%y == 0]
It is possible to generate a list of tuples, and these of course can hold multiple values.
Yes absolutely, just use parenthesis around the value:
[(y, x/y) for y in range(1, sqrt(x)+1) if x%y == 0]