Related
How to convert this for loop:
smoke_ray = [18, 14]
total = 0
for i in smoke_ray:
total += i
Into a list comprehension? I tried:
smoke_ray = [18, 14]
total = 0
[total += i for i in smoke_ray]
Is the problem the += operator?
that’s where I get an error
Updated with full code:
days = [
{ "day_name": "wed",
"smoked_at": {
'15:30': 1,
'16:30': 1,
'16:50': 2,
'17:30': 1,
'18:30': 1,
'20:20': 1,
'21:30': 1,
'22:30': 1,
'25:00': 5
}
},
{ "day_name": "thurs",
"smoked_at": {
'08:15': 1,
'08:40': 1,
'09:20': 1,
'10:00': 1,
'11:20': 1,
'11:38': 1,
'12:10': 1,
'13:00': 1,
'14:26': 1,
'15:40': 1,
'17:08': 1,
'18:10': 1,
'19:30': 1,
'20:20': 1,
'22:00': 1,
'23:00': 1,
'25:00': 2
}
}
]
smoke_ray = []
for i in days:
print(i["day_name"])
smokes = i["smoked_at"].values()
smokes_day = sum(smokes)
print(smokes_day)
smoke_ray.append(i)
total = 0
for i in smoke_ray:
total += i
print(total)
When trying to convert the last for loop to a list comprehension (are you telling me it’s not a shorthand way of writing a loop? I heard it was faster)
I get this error:
File "compiler.py", line 47
[total += i for i in smoke_ray]
^
SyntaxError: invalid syntax
When trying to use sum, it just won’t work:
sum(smoke_ray)
wed
14
thurs
18
Traceback (most recent call last):
File "compiler.py", line 47, in
sum(smoke_ray)
TypeError: unsupported operand type(s) for +: 'int' and 'dict'
You don't even need a list comprehension. Just use sum:
total = sum(smoke_ray)
Looking at the code you've now added for context, I think the issue you're having isn't actually in summing the list, it's in constructing the list itself. When I run your original code, with the for loop, that also fails for me.
I think the problem is in the line smoke_ray.append(i): Here, you're appending an entire element of the dictionary (e.g: { "day_name": "...", "smoked_at": { ... } }) to the smoke_ray list. Then, it doesn't make sense to sum over values in that list, since they're dictionaries. If you wanted to add each smokes_day to the list, and then sum over those, you'd do smoke_ray.append(smoke_day) within that loop. Then, you should be able to just use sum as mentioned in other answers to sum over the list.
Edit: This isn't to say that there aren't more improvements that could be done to the code btw, a simple change that would preserve the original structure would be to change your for loop to be something like this:
total = 0
for i in days:
print(i['day_name'])
smokes = i['smoked_at'].values()
smokes_day = sum(smokes)
print(smokes_day)
total += smokes_day
print(total)
That way you can sum the values like you want within one loop, without the need to construct another list/use a list comprehension.
If you want to see the sum value list wise then you can use
import itertools
smoke_ray = [18, 14]
print(list(itertools.accumulate(smoke_ray)))
This will show you the sum of the series by element
Output
[18, 32]
List comprehension is specifically an operation to transform an input list into an output list via two actions:
transformation of each element (also known as “mapping”)
filtering based on some filtering criterion
That’s it.
The operation you want is fundamentally different: you want to accumulate the elements in a list using a given operation. This action is also known as a “reduction” and is available in the Python library via functools.reduce.
So you could write
import functools
import operator
functools.reduce(operator.add, smoke_ray)
… but this is such a common operation that there’s a shorthand for it: as mentioned, you can also just use sum(smoke_ray).
You can do it in multiple ways, since you asking here for list comprehension (although list comprehension is a bad option here, a battery option is sum(your_list) ) try this:
sum([i for i in [18, 14]])
24
I've been trying to find a relevant question, though I can't seem to search for the right words and all I'm finding is how to check if a list contains an intersection.
Basically, I need to split a list once a certain sequence of numbers is found, similar to doing str.split(sequence)[0], but with lists instead. I have working code, though it doesn't seem very efficient (also no idea if raising an error was the right way to go about it), and I'm sure there must be a better way to do it.
For the record, long_list could potentially have a length of a few million values, which is why I think iterating through them all might not be the best idea.
long_list = [2,6,4,2,7,98,32,5,15,4,2,6,43,23,95,10,31,5,1,73]
end_marker = [6,43,23,95]
end_marker_len = len(end_marker)
class SuccessfulTruncate(Exception):
pass
try:
counter = 0
for i in range(len(long_list)):
if long_list[i] == end_marker[counter]:
counter += 1
else:
counter = 0
if counter == end_marker_len:
raise SuccessfulTruncate()
except SuccessfulTruncate:
long_list = long_list[:2 + i - end_marker_len]
else:
raise IndexError('sequence not found')
>>> long_list
[2,6,4,2,7,98,32,5,15,4,2]
Ok, timing a few answers with a big list of 1 million values (the marker is very near the end):
Tim: 3.55 seconds
Mine: 2.7 seconds
Dan: 0.55 seconds
Andrey: 0.28 seconds
Kasramvd: still executing :P
I have working code, though it doesn't seem very efficient (also no idea if raising an error was the right way to go about it), and I'm sure there must be a better way to do it.
I commented on the exception raising in my comment
Instead of raising an exception and catching it in the same try/except you can just omit the try/except and do if counter == end_marker_len: long_list = long_list[:2 + i - end_marker_len]. Successful is not a word thats fitting for an exception name. Exceptions are used to indicate that something failed
Anyway, here is a shorter way:
>>> long_list = [2,6,4,2,7,98,32,5,15,4,2,6,43,23,95,10,31,5,1,73]
>>> end_marker = [6,43,23,95]
>>> index = [i for i in range(len(long_list)) if long_list[i:i+len(end_marker)] == end_marker][0]
>>> long_list[:index]
[2, 6, 4, 2, 7, 98, 32, 5, 15, 4, 2]
List comprehension inspired by this post
As a more pythonic way instead of multiple slicing you can use itertools.islice within a list comprehension :
>>> from itertools import islice
>>> M,N=len(long_list),len(end_maker)
>>> long_list[:next((i for i in range(0,M) if list(islice(long_list,i,i+N))==end_marker),0)]
[2, 6, 4, 2, 7, 98, 32, 5, 15, 4, 2]
Note that since the default value of next function is 0 if it doesn't find any match it will returns the whole of long_list.
In my solution used approach with index method:
input = [2,6,4,2,7,98,32,5,15,4,2,6,43,23,95,10,31,5,1,73]
brk = [6,43,23,95]
brk_len = len(brk)
brk_idx = 0
brk_offset = brk_idx + brk_len
try:
while input[brk_idx:brk_offset] != brk:
brk_idx = input.index(brk[0], brk_idx + 1)
brk_offset = brk_idx + brk_len
except ValueError:
print("Not found")
else:
print(input[:brk_idx])
If the values are of limited range, say fit in bytes (this can also be adapted to larger types), why not then encode the lists so that the string method find could be used:
long_list = [2,6,4,2,7,98,32,5,15,4,2,6,43,23,95,10,31,5,1,73]
end_marker = [6,43,23,95]
import struct
long_list_p = struct.pack('B'*len(long_list), *long_list)
end_marker_p = struct.pack('B'*len(end_marker), *end_marker)
print long_list[:long_list_p.find(end_marker_p)]
Prints:
[2, 6, 4, 2, 7, 98, 32, 5, 15, 4, 2]
I tried using bytes as in but the find method they had didn't work:
print long_list[:bytes(long_list).find(bytes(end_marker))]
I have three lists
vr=[5, 10, 15, 20, 25]
vr_bin = [1,2,3,4,5,6,7,8,9,10]
crf= [0.0357, 0.7124, 0.1707, 0.0142, 0.0328, 0.0291, 0.0244, 0.0206, 0.0334, 1.7124]
where vr_bin and crf are of same length. I have to take first value of vr (for example 5) and compare it with vr_bin (for which value of i, vr_bin[i] <= vr[0]) then sum crf up to crf[i].
In our example, up to vr_bin[4] will satisfy our condition (since vr_bin[4] is 5 and vr[0] is 5). So I want to calculate crf[0]+crf[1]+crf[2]+crf[3]+crf[4]. I tried with the following code
total_crf = crf[vr_bin <= vr].sum()
But I am getting the following error AttributeError: 'float' object has no attribute 'sum' Please suggest a method for solving this. Thankyou
sum is a function, not a method, but the bigger problem is that the crf[vr_bin <= vr] notation you're trying to use doesn't do what you think it does in Python. Instead, you have to iterate over the lists the long way, summing up as you go. (Fortunately, you can iterate over vr_bin and crf at the same time with zip.)
result = 0
for (v, c) in zip(vr_bin, crf):
if v <= vr[0]:
result += c
else:
break
You are getting that error because sum() is a function, not a method:
>>> x = [1, 2]
>>> sum(x)
3
>>> x.sum() #Error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'sum'
>>>
Here is code that works:
ind = [i for i in range(len(vr_bin)) if vr_bin[i] <= vr[0]]
round(sum(crf[i] for i in range(max(ind)+1)), 3)
Which runs as:
>>> vr=[5, 10, 15, 20, 25]
>>> vr_bin = [1,2,3,4,5,6,7,8,9,10]
>>> crf= [0.0357, 0.7124, 0.1707, 0.0142, 0.0328, 0.0291, 0.0244, 0.0206, 0.0334, 1.7124]
>>> ind = [i for i in range(len(vr_bin)) if vr_bin[i] <= vr[0]]
>>> round(sum(crf[i] for i in range(max(ind)+1)), 3)
0.933
>>>
I wanted to know if is safe ( documented behaviour? ) to delete the domain space of an iterator in execution in Python.
Consider the code:
import os
import sys
sampleSpace = [ x*x for x in range( 7 ) ]
print sampleSpace
for dx in sampleSpace:
print str( dx )
if dx == 1:
del sampleSpace[ 1 ]
del sampleSpace[ 3 ]
elif dx == 25:
del sampleSpace[ -1 ]
print sampleSpace
'sampleSpace' is what I call 'the domain space of an iterator' ( if there is a more appropriate word/phrase, lemme know ).
What I am doing is deleting values from it while the iterator 'dx' is running through it.
Here is what I expect from the code :
Iteration versus element being pointed to (*):
0: [*0, 1, 4, 9, 16, 25, 36]
1: [0, *1, 4, 9, 16, 25, 36] ( delete 2nd and 5th element after this iteration )
2: [0, 4, *9, 25, 36]
3: [0, 4, 9, *25, 36] ( delete -1th element after this iteration )
4: [0, 4, 9, 25*] ( as the iterator points to nothing/end of list, the loop terminates )
.. and here is what I get:
[0, 1, 4, 9, 16, 25, 36]
0
1
9
25
[0, 4, 9, 25]
As you can see - what I expect is what I get - which is contrary to the behaviour I have had from other languages in such a scenario.
Hence - I wanted to ask you if there is some rule like "the iterator becomes invalid if you mutate its space during iteration" in Python?
Is it safe ( documented behaviour? ) in Python to do stuff like this?
From the Python tutorial:
It is not safe to modify the sequence
being iterated over in the loop (this
can only happen for mutable sequence
types, such as lists). If you need to
modify the list you are iterating over
(for example, to duplicate selected
items) you must iterate over a copy.
The slice notation makes this
particularly convenient:
>>> for x in a[:]: # make a slice copy of the entire list
... if len(x) > 6: a.insert(0, x)
...
>>> a
['defenestrate', 'cat', 'window', 'defenestrate']
Generally speaking no, it's not safe and you may get unpredictable behaviour. Iterators aren't required to behave in an specific way under these circumstances.
What's happening in your example is
# list is [0, 1, 4, 9, 16, 25, 36]
if dx == 1:
# we're at index 1 when this is true
del sampleSpace[ 1 ]
# we've removed the item at index 1, and the iterator will move to the next valid position - still index 1, but in a mutated list. We got lucky in this case
# the list now contains [0, 4, 9, 16, 25, 36]
del sampleSpace[ 3 ]
# we remove the item at index 3 which is (now) value 16
# the list now contains [0, 4, 9, 25, 36]
elif dx == 25:
del sampleSpace[ -1 ]
# we remove the final item, list now looks like
# the list now contains [0, 4, 9, 25]
What do you mean by safe? Your code happens not to raise any errors, but it is a distinct possibility of course, consider this:
>>> a = range(3)
>>> for i in a:
del a
Traceback (most recent call last):
File "<pyshell#13>", line 2, in <module>
del a
NameError: name 'a' is not defined
>>> a
[0, 1, 2]
>>> for i in a:
del a[i+1]
Traceback (most recent call last):
File "<pyshell#27>", line 2, in <module>
del a[i+1]
IndexError: list assignment index out of range
It is not clear why would you want to do this, but there is no additional rules applicable to iterators. They're acting exactly as any other type would.
In a Pylons webapp, I need to take a string such as "<3, 45, 46, 48-51, 77" and create a list of ints (which are actually IDs of objects) to search on.
Any suggestions on ways to do this? I'm new to Python, and I haven't found anything out there that helps with this kind of thing.
The list would be: [1, 2, 3, 45, 46, 48, 49, 50, 51, 77]
Use parseIntSet from here
I also like the pyparsing implementation in the comments at the end.
The parseIntSet has been modified here to handle "<3"-type entries and to only spit out the invalid strings if there are any.
#! /usr/local/bin/python
import sys
import os
# return a set of selected values when a string in the form:
# 1-4,6
# would return:
# 1,2,3,4,6
# as expected...
def parseIntSet(nputstr=""):
selection = set()
invalid = set()
# tokens are comma seperated values
tokens = [x.strip() for x in nputstr.split(',')]
for i in tokens:
if len(i) > 0:
if i[:1] == "<":
i = "1-%s"%(i[1:])
try:
# typically tokens are plain old integers
selection.add(int(i))
except:
# if not, then it might be a range
try:
token = [int(k.strip()) for k in i.split('-')]
if len(token) > 1:
token.sort()
# we have items seperated by a dash
# try to build a valid range
first = token[0]
last = token[len(token)-1]
for x in range(first, last+1):
selection.add(x)
except:
# not an int and not a range...
invalid.add(i)
# Report invalid tokens before returning valid selection
if len(invalid) > 0:
print "Invalid set: " + str(invalid)
return selection
# end parseIntSet
print 'Generate a list of selected items!'
nputstr = raw_input('Enter a list of items: ')
selection = parseIntSet(nputstr)
print 'Your selection is: '
print str(selection)
And here's the output from the sample run:
$ python qq.py
Generate a list of selected items!
Enter a list of items: <3, 45, 46, 48-51, 77
Your selection is:
set([1, 2, 3, 45, 46, 77, 48, 49, 50, 51])
I've created a version of #vartec's solution which I feel is more readable:
def _parse_range(numbers: str):
for x in numbers.split(','):
x = x.strip()
if x.isdigit():
yield int(x)
elif x[0] == '<':
yield from range(0, int(x[1:]))
elif '-' in x:
xr = x.split('-')
yield from range(int(xr[0].strip()), int(xr[1].strip())+1)
else:
raise ValueError(f"Unknown range specified: {x}")
In the process, the function became a generator :)
rng = "<3, 45, 46, 48-51, 77"
ids = []
for x in map(str.strip,rng.split(',')):
if x.isdigit():
ids.append(int(x))
continue
if x[0] == '<':
ids.extend(range(1,int(x[1:])+1))
continue
if '-' in x:
xr = map(str.strip,x.split('-'))
ids.extend(range(int(xr[0]),int(xr[1])+1))
continue
else:
raise Exception, 'unknown range type: "%s"'%x
First, you'll need to figure out what kind of syntax you'll accept. You current have three in your example:
Single number: 45, 46
Less than operator
Dash ranging: 48-51
After that, it's just a matter of splitting the string into tokens, and checking the format of the token.
I also had to do something similar for an app lately.
If you don't need concrete numbers but just a way to see whether a given number is in the range, you might consider parsing it to a Python expression you can eval into a lambda. For example <3, 5-10, 12 could be func=(lambda x:x<3 or (5 <= x <= 10) or x==12)). Then you can just call the lambda, func(11) to see if 11 belongs in there.
>>> print range.__doc__
range([start,] stop[, step]) -> list of integers
Return a list containing an arithmetic progression of integers.
range(i, j) returns [i, i+1, i+2, ..., j-1]; start (!) defaults to 0.
When step is given, it specifies the increment (or decrement).
For example, range(4) returns [0, 1, 2, 3]. The end point is omitted!
These are exactly the valid indices for a list of 4 elements.
>>> range(33,44)
[33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43]
>>> range(1,3)
[1, 2]
I imagine you could iterate your list, and call range appropriately.
>>> def lessThan(n) :
... return range(n+1)
...
>>> lessThan(4)
[0, 1, 2, 3, 4]
>>> def toFrom(n,m):
... return range(n,m)
...
>>> toFrom(33,44)
[33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43]
Then split the string on commas, and for each bit, parse it enough to figure out what function to call, catenating the lists returned.
Anything more and I'd have written it for you.