Is it possible to do the below with list comprehension? Trying to store the maximum value that has been seen at any given point through the loop.
def test(input):
a = input[0]
b = []
for i in input:
a = max(i,a)
b.append(a)
return b
print test([-5,6,19,4,5,20,1,30])
# returns [-5, 6, 19, 19, 19, 20, 20, 30]
You can use itertools.accumulate with the max builtin in Python 3:
from itertools import accumulate
lst = [-5,6,19,4,5,20,1,30]
r = list(accumulate(lst, max)) #[i for i in accumulate(lst, max)]
print(r)
# [-5, 6, 19, 19, 19, 20, 20, 30]
What you present here is a typical form of what is known in functional programming as scan.
A way to do this with list comprehension that is inefficient is:
[max(input[:i]) for i in range(1,n+1)]
But this will run in O(n2).
You can do this with list comprehension given you use a function with side effects: like the following:
def update_and_store(f,initial=None):
cache = [initial]
def g(x):
cache[0] = f(cache[0],x)
return cache[0]
return g
You can then use:
h = update_and_store(max,a[0])
[h(x) for x in a]
Or you can use a dictonaries setdefault() like:
def update_and_store(f):
c = {}
def g(x):
return c.setdefault(0,f(c.pop(0,x),x))
return g
and call it with:
h = update_and_store(max)
[h(x) for x in a]
like #AChampion says.
But functions with side-effects are rather unpythonic and not declarative.
But you better use a scanl or accumulate approach like the one offered by itertools:
from itertools import accumulate
accumulate(input,max)
If using NumPy is permitted, then you can use NumPy:
import numpy as np
np.maximum.accumulate([-5,6,19,4,5,20,1,30])
# array([-5, 6, 19, 19, 19, 20, 20, 30])
Related
#!/usr/bin/python2
s=5 # input any int
l1=[5,8,2,1,17] # input some int numbers
l2=[] # will be out
for i in l1:
s+=i
l2.append(s)
print l1,l2
[5, 8, 2, 1, 17]
[10, 18, 20, 21, 38]
I would like to replace the above for cycle with listcomprehension.
I generally use list comprehensions but I don't know how could I do with the above for cycle.
For a list comprehension without using libraries or the overhead of sum(), you could use an internal list to store the accumulator.
l1 = [5,8,2,1,17]
s = 5
R = [a.append(a.pop(0)+n) or a[0] for a in [[s]] for n in l1]
print(R)
[10, 18, 20, 21, 38]
If you're using Python 3.8+ you can use the walrus operator to manage the accumulation:
R = [a := n + (a if i else s) for i,n in enumerate(l1)]
Here you can see use of sum(). However I would not prefer it if the list is too long for the repetitive calculations by sum()
>>> s=5
>>>
>>> l1=[5,8,2,1,17]
>>> [s+sum(l1[:i+1]) for i,x in enumerate(l1)]
[10, 18, 20, 21, 38]
>>>
List comprehensions are much better suited to situations where you generate an output item independently for each input item. Here, having to sum the items makes it rather inconvenient.
You could use itertools.accumulate, though, which is designed to do that by default:
from itertools import accumulate
l1 = [5, 8, 2, 1, 17]
s = 5
l2 = list(accumulate(l1, initial=s))[1:]
print(l2)
# [10, 18, 20, 21, 38]
(Note that we have to exclude the first value, which would be the initial s, as you don't want it in your output.)
You have the list-comprehension option
res = [s + sum(l1[:i+1]) for i,x in enumerate(l1)]
Or using some libs along with itertools.accumulate
from operator import add
from itertools import accumulate
import numpy as np
res = np.array(list(accumulate(l1, add))) + s
Is it possible to unpack a list of numbers in to list indices? For example I have a lists with in a list containing numbers like this:
a = [[25,26,1,2,23], [15,16,11,12,10]]
I need to place them in a pattern so i did something like this
newA = []
for lst in a:
new_nums = [lst[4],lst[2],lst[3],lst[0],lst[1]]
newA.append(new_nums)
print (newA) # prints -->[[23, 1, 2, 25, 26], [10, 11, 12, 15, 16]]
so instead of writing new_nums = [lst[4],lst[2],lst[3],lst[0],lst[1]] , i thought of defining a pattern as list called pattern = [4,2,3,0,1] and then unpack these in to those indices of lst to create new order of lst.
Is there a fine way to do this.
Given a list of indices called pattern, you can use a list comprehension like so:
new_lst = [[lst[i] for i in pattern] for lst in a]
operator.itemgetter provides a useful mapping function:
from operator import itemgetter
a = [[25,26,1,2,23], [15,16,11,12,10]]
f = itemgetter(4,2,3,0,1)
print [f(x) for x in a]
[(23, 1, 2, 25, 26), (10, 11, 12, 15, 16)]
Use list(f(x)) if you want list-of-lists instead of list-of-tuples.
If you're not opposed to using numpy, then try something like:
import numpy as np
pattern = [4, 2, 3, 0, 1]
newA = [list(np.array(lst)[pattern]) for lst in a]
Hope it helps.
In pure Python, you can use a list comprehension:
pattern = [4,2,3,0,1]
newA = []
for lst in a:
new_nums = [lst[i] for i in pattern]
newA.append(new_nums)
In numpy, you may use the fancy indexing feature:
>>> [np.array(lst)[pattern].tolist() for lst in a]
[[23, 1, 2, 25, 26], [10, 11, 12, 15, 16]]
it is slower than other, but it is another option. you can sort the list based on your pattern
a = [[25,26,1,2,23], [15,16,11,12,10]]
pattern = [4,2,3,0,1]
[sorted(line,key=lambda x:pattern.index(line.index(x))) for line in a]
[[23, 1, 2, 25, 26], [10, 11, 12, 15, 16]]
consider x = [10,10,20,20,20,30]
How do i form another list_x1 which contains only same values example: list_x1 = [10,10]
and list_x2 =[20,20] and list_x3 =[30] ?
You can use counter.
from collections import Counter
x = [10, 10, 20, 20, 20, 30]
my_counter = Counter(x)
d = {'list_x{0}'.format(key): [key] * my_counter[key] for key in my_counter}
>>> d
{'list_x10': [10, 10], 'list_x20': [20, 20, 20], 'list_x30': [30]}
One of the issues with your request is that you would need to pre-assign variables, which aren't initially know. I've used a dictionary as a container to hold them.
For a list, [10] * 3 results in [10, 10, 10]. So, [k] * my_counter multiplies the unique key value by the number of occurrences.
With itertools.groupby
>>> from itertools import groupby
>>> x = [10,10,20,20,20,30]
>>> [list(g) for k, g in groupby(x)]
[[10, 10], [20, 20, 20], [30]]
Perhaps the best way is #Alexander's idea with collections, but I always find it helpful to look at more 'native' python code to see what's going on. So here's a way to do it:
x = [10,10,20,20,20,30]
def foo(iterable):
for val in iterable:
cnt = iterable.count(val)
iterable = list(filter(lambda x: x != val, iterable))
if cnt:
yield [val]*cnt
for _ in foo(x):
print(_)
Note that the complexity factor is going to be fairly high. Certainly not O(n) because you have to:
Iterate through each of the values in our main for val in iterable
Iterate through each of the values every time we call iterable.count
Iterate through each of the values when we filter() them to prevent duplicates.
Using collections.Counter:
>>> def list_gen(in_list, elem):
... count = collections.Counter(in_list)
... return [elem] * count[elem]
...
>>> a
[1, 2, 3, 2, 3]
>>> list_gen(a, 2)
[2, 2]
This isn't exactly what you're looking for, but this code will generate a list of lists separating the values.
x = [10, 10, 20, 20, 20, 30]
uniques = set(x)
output = []
for unique in uniques:
unique_count = x.count(unique)
temp = []
for i in range(0, unique_count):
temp.append(unique)
output.append(temp)
You can then use list comprehensions on output
I would like to know if there is a built in function in python for the equivalent Haskell scanl, as reduce is the equivalent of foldl.
Something that does this:
Prelude> scanl (+) 0 [1 ..10]
[0,1,3,6,10,15,21,28,36,45,55]
The question is not about how to implement it, I already have 2 implementations, shown below (however, if you have a more elegant one please feel free to show it here).
First implementation:
# Inefficient, uses reduce multiple times
def scanl(f, base, l):
ls = [l[0:i] for i in range(1, len(l) + 1)]
return [base] + [reduce(f, x, base) for x in ls]
print scanl(operator.add, 0, range(1, 11))
Gives:
[0, 1, 3, 6, 10, 15, 21, 28, 36, 45, 55]
Second implementation:
# Efficient, using an accumulator
def scanl2(f, base, l):
res = [base]
acc = base
for x in l:
acc = f(acc, x)
res += [acc]
return res
print scanl2(operator.add, 0, range(1, 11))
Gives:
[0, 1, 3, 6, 10, 15, 21, 28, 36, 45, 55]
Thank you :)
You can use this, if its more elegant:
def scanl(f, base, l):
for x in l:
base = f(base, x)
yield base
Use it like:
import operator
list(scanl(operator.add, 0, range(1,11)))
Python 3.x has itertools.accumulate(iterable, func= operator.add). It is implemented as below. The implementation might give you ideas:
def accumulate(iterable, func=operator.add):
'Return running totals'
# accumulate([1,2,3,4,5]) --> 1 3 6 10 15
# accumulate([1,2,3,4,5], operator.mul) --> 1 2 6 24 120
it = iter(iterable)
total = next(it)
yield total
for element in it:
total = func(total, element)
yield total
Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), which gives the possibility to name the result of an expression, we can use a list comprehension to replicate a scan left operation:
acc = 0
scanned = [acc := acc + x for x in [1, 2, 3, 4, 5]]
# scanned = [1, 3, 6, 10, 15]
Or in a generic way, given a list, a reducing function and an initialized accumulator:
items = [1, 2, 3, 4, 5]
f = lambda acc, x: acc + x
accumulator = 0
we can scan items from the left and reduce them with f:
scanned = [accumulator := f(accumulator, x) for x in items]
# scanned = [1, 3, 6, 10, 15]
I had a similar need. This version uses the python list comprehension
def scanl(data):
'''
returns list of successive reduced values from the list (see haskell foldl)
'''
return [0] + [sum(data[:(k+1)]) for (k,v) in enumerate(data)]
>>> scanl(range(1,11))
gives:
[0, 1, 3, 6, 10, 15, 21, 28, 36, 45, 55]
As usual, the Python ecosystem is also overflowing with solutions:
Toolz has an accumulate capable of taking a user-supplied function as an argument. I tested it with lambda expressions.
https://github.com/pytoolz/toolz/blob/master/toolz/itertoolz.py
https://pypi.python.org/pypi/toolz
as does more_itertools
http://more-itertools.readthedocs.io/en/stable/api.html
I did not test the version from more-itertools, but it also can take a user-supplied function.
This question already has answers here:
How to find the cumulative sum of numbers in a list?
(25 answers)
Closed 8 years ago.
can you help me with code which returns partial sum of numbers in text file?
I must import text file, then make a code for partial sums without tools ..etc.
My input:
4
13
23
21
11
The output should be (without brackets or commas):
4
17
40
61
72
I was trying to make code in python, but could only do total sum and not partial one.
If i use the += operator for generator, it gives me an error!
Well, since everyone seems to be giving their favourite idiom for solving the problem, how about itertools.accumulate in Python 3:
>>> import itertools
>>> nums = [4, 13, 23, 21, 11]
>>> list(itertools.accumulate(nums))
[4, 17, 40, 61, 72]
There are a number of ways to create your sequence of partial sums. I think the most elegant is to use a generator.
def partial_sums(iterable):
total = 0
for i in iterable:
total += i
yield total
You can run it like this:
nums = [4, 13, 23, 21, 11]
sums = list(partial_sums(nums)) # [ 4, 17, 40, 61, 72]
Edit To read the data values from your file, you can use another generator, and chain them together. Here's how I'd do it:
with open("filename.in") as f_in:
# Sums generator that "feeds" from a generator expression that reads the file
sums = partial_sums(int(line) for line in f_in)
# Do output:
for value in sums:
print(value)
# If you need to write to a file, comment the loop above and uncomment this:
# with open("filename.out", "w") as f_out:
# f_out.writelines("%d\n" % value for value in sums)
numpy.cumsum will do what you want.
If you're not using numpy, you can write your own.
def cumsum(i):
s = 0
for elt in i:
s += elt
yield s
try this:
import numpy as np
input = [ 4, 13, 23, 21, 11 ]
output = []
output.append(input[0])
for i in np.arange(1,len(input)):
output.append(input[i] + input[i-1])
print output
Use cumulative sum in numpy:
import numpy as np
input = np.array([4, 13, 23, 21 ,11])
output = input.cumsum()
Result:
print output
>>>array([ 4, 17, 40, 61, 72])
Or if you need a list, you may convert output to list:
output = list(output)
print output
>>>[4, 17, 40, 61, 72]
This is an alternative solution using reduce:
nums = [4, 13, 23, 21, 11]
partial_sum = lambda a, b: a + [a[-1] + b]
sums = reduce(partial_sum, nums[1:], nums[0:1])
Pluses in lambda are not the same operator, the first one is list concatenation and the second one is sum of two integers. Altough Blckknght's may be more clear, this one is shorter and works in Python 2.7.
something like this:
>>> lst = [4, 13, 23, 21 ,11]
>>> [sum(lst[:i+1]) for i, x in enumerate(lst)]
[4, 17, 40, 61, 72]