I came across an interesting bit of code in a QC review and was surprised by its behavior. I am curious if it is documented anywhere.
for i in range(0, my_array.max(), 3)[:]:
# other code here
I was curious about the need for the [:] after range, so I tested it:
>>> range(0, 10, 3)
range(0, 10, 3)
>>> range(0, 10, 3)[:]
range(0, 12, 3)
The actual sequence defined by these ranges is identical, but I do not see this slicing behavior documented anywhere in the Python range documentation, so I was curious what is actually going on here.
For a moment let's pretend that range still returned a list. Slicing the range object returns a range object which would act as if you were slicing the underlying list. Instead of doing this with a list though, the range object is able to take care of it in constant time using arithmetic.
>>> range(0, 90, 2)[10:23]
range(20, 46, 2)
>>> list(range(0, 90, 2)[10:23])
[20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44]
When you do something like:
range(0, 10, 3)[:]
Python slices it with arithmetic.
My assumption is that when determining the final element it rounds up. It tries to compute the zeroth element in the range to start with. This will be start + step * 0 = 0.
Then Python tries to get the ending element. There are (10 - 0) // 3 + 1 = 4 elements in the range, so the ending element is start + step * n_elements = 0 + 3 * 4 = 12.
I think a few things are mixed here.
range produces slicing behavior, because slicing with non-default indexes makes sense:
>>> list(range(10, 20)[3:7])
[13, 14, 15, 16]
There is an idiom of copying a list (which is mutable) by producing a slice with all default indexes: some_list[:] is equivalent to something like [x for x in some_list].
There is the strange code that does [:] for the range object (or the actual list, if it's Python 2) which makes no sense. The generated range object / list is not referenced anywhere else anyway.
Python documentation lists slicing among "Common Sequence Operations" in a chapter named "Sequence Types — list, tuple, range" (emph. mine). So it's documented, but few people ever read it.
Related
I'm trying to write a function that will take as input a string of intervals e.g "0-0,4-8,20-21,43-45" to produce all numbers within each of the ranges meaning: [0, 4, 5, 6, 7, 8, 20, 21, 43, 44, 45]
The exercise requires to do so using generators. I managed to parse the input through a generator but I can't do the same for populating the numbers. I'm trying to int() each number so I could leverage the range() to produce all numbers within the edges.
Here's my conceptual code - how can I produce the numbers within each interval?
def parse_ranges(arg):
arg = arg.split(",")
parsed= (line.split("-") for line in arg)
#Trying to parse each character to int to use range()
intervals= (int(i) for i in number for number in parsed)
# Even if I had the characters parsed to int, I still don't know how to produce the range
ranges = (range(interval[0],interval[1]) interval for interval in intervals)
return ranges
print(list(parse_ranges("0-0,4-8,20-21,43-45")))
def parse_ranges(arg):
arg = arg.split(",")
parsed = ((line.split("-")) for line in arg)
for pair in parsed:
yield from range(int(pair[0]), int(pair[1])+1)
print(list(parse_ranges("0-0,4-8,20-21,43-45")))
Out: [0, 4, 5, 6, 7, 8, 20, 21, 43, 44, 45]
If you want to pass values from a generator INSIDE another generator directly out to the consumer of the outer generator, you need to use the "yield from" expression. (Also, note that you need to extend the "to" end of each range by +1, since the range endpoint is not inclusive.)
Two pieces that you seem to missing are: the second argument of range() needs to be one beyond what you want; you can pass control from one generator to another via yield from:
def parse_ranges(arg):
for start, stop in (interval.split('-') for interval in arg.split(',')):
yield from range(int(start), int(stop) + 1)
print(*parse_ranges("0-0,4-8,20-21,43-45"))
OUTPUT
% python3 test.py
0 4 5 6 7 8 20 21 43 44 45
%
This question already has answers here:
Strange result when removing item from a list while iterating over it
(8 answers)
Closed 7 years ago.
I'm trying to move the odd numbers in this list to the end without using an external list. When I run the code, it moves half of the odd numbers over, and leaves the other half where they were. The overall output is correct if I run the code twice on the list, but I should only have to run it once, right? What am I doing wrong?
a = [3, 4, 55, 13, 6, 19, 33, 10, 11, 45]
for ind, val in enumerate(a):
if val % 2 != 0:
a.pop(ind)
a.append(val)
print a
Thanks.
This is because, as a general rule, you shouldn't iterate through and modify the same list at the same time. The indices get thrown off!
As you go through and modify the list, not all of the elements actually get cycled through. The a that you are popping is a different list with different indicies after your first change to the list, but you are still using the enumeration of the original version for your loop.
You could use pythons sorting methods (sorted() or someList.sort()) and pass a special key function to it:
>>> sorted(a, key = lambda element: element%2 != 0)
[4, 6, 10, 3, 55, 13, 19, 33, 11, 45]
This is possible because Python sorts are guaranteed to be stable und therefore when multiple records have the same key, their original order is preserved.
Your approach has two problems,
you modify the list on which you iterate
you don't test if the number that slides into the position occupied by an odd number is odd itself
To obviate with 1., you simply can iterate on the indices of your list but to obviate 2., you must start from the end of the list and going towards the beginning, using negative indices.
Executing the following code
a = [3, 4, 55, 13, 6, 19, 33, 10, 11, 45]
for i in range(len(a)):
j = -1-i
if a[j]%2:
a.append(a[j]) ; a.pop(j-1) # j-1 because our list is temporarily longer...
print a
gives you the following output
[4, 6, 10, 45, 11, 33, 19, 13, 55, 3]
I have a question about how the python function for works. I have a program I am writing which involved generating the prime factorization of several numbers.
for a in range(2,25):
print primefactorize(a)
When I tested it in the range above, the last list to pop up in the terminal was the prime factorization output for 24, not 25. Does the function
for a in range(1,x):
only run through each value up to x-1?
Thanks
I think your confusion is with range
range(2,25)
[2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]
From the documentation:
This is a versatile function to create lists containing arithmetic
progressions. It is most often used in for loops. The arguments must
be plain integers. If the step argument is omitted, it defaults to 1.
If the start argument is omitted, it defaults to 0. The full form
returns a list of plain integers [start, start + step, start + 2 *
step, ...]. If step is positive, the last element is the largest start
+ i * step less than stop; if step is negative, the last element is the smallest start + i * step greater than stop. step must not be zero
(or else ValueError is raised). Example:
In math this is called an start inclusive, stop exclusive interval and looks like this:
[start, stop)
I often work with multidimensional arrays whose array indices are generated from a complicated user-specified set.
I'm looking for a library with classes for representing complicated sets with an arbitrary number of indices, and arbitrarily complicated predicates. Given a set description, the desired output would be a generator. This generator would in turn produce either dicts or tuples which correspond to the multidimensional array indices.
Does such a library exist?
Example
Suppose we had the following user-specified set (in set-builder notation), which represents the indices of some array variable x[i][j]:
{i in 1..100, j in 1..50: i >= 20, j >= 21, 2*(i + j) <= 100}
I'd like to put this into some sort of a lazy class (a generator expression perhaps) that will allow me to lazily evaluate the elements of the set to generate the indices for my array. Suppose this class were called lazyset; this would be the desired behavior:
>>> S = lazyset("{i in 1..100, j in 1..50: i >= 20, j >= 21, 2*(i+j) <= 100}")
>>> S
<generator object <genexpr> at 0x1f3e7d0>
>>> next(S)
{'i': 20, 'j': 21}
>>> next(S)
{'i': 20, 'j': 22}
I'm thinking I could roll my own using generator expressions, but this almost seems like a solved problem. So I thought I'd asked if anyone's come across an established library that handles this (to some extent, at least). Does such a library exist?
This looks more like a constraint-solver problem to me:
import constraint as c
p = c.Problem()
p.addVariable(0, range(1,101))
p.addVariable(1, range(1,51))
p.addConstraint(lambda i: i >= 20, [0])
p.addConstraint(lambda j: j >= 21, [1])
p.addConstraint(c.MaxSumConstraint(50))
indices = ((s[0], s[1]) for s in p.getSolutionIter()) # convert to tuple generator
then if you do
for ij in indices:
print ij
you get
(29, 21)
(28, 22)
(28, 21)
(27, 23)
(27, 22)
(27, 21)
...
(20, 25)
(20, 24)
(20, 23)
(20, 22)
(20, 21)
Although I am not certain if this specifically (the set-builder notation) is supported by scipy. I think scipy is your best bet regardless.
There is support for sparse arrays/sets in scipy so you can easily let it handle the allocation of those without actually allocating the space :)
I have a generator that takes a number as an argument and yields other numbers.
I want to use the numbers yielded by this generator and pass them as arguments to the same generator, creating a chain of some length.
For example, mygenerator(2) yields 5, 4 and 6. Apply mygenerator to each of these numbers, over and over again to the numbers yielded. The generator always yields bigger numbers than the one passed as argument, and for 2 different numbers will never yield the same number.
mygenerator(2): 4 5
mygenerator(4) : 10 11 12
mygenerator(5): 9 300 500
So the set (9,10,11,12,300,500) has "distance" 2 from the original number, 2. If I apply it to the number 9, I will get a set of numbers with distance "3" from the original 2.
Essentially what I want is to create a set that has a specified distance from a given number and I have problems figuring out how to do that in Python. Help much appreciated :)
Suppose our generator yields square and cube of given number that way it will output unique
so if we want to get numbers at dist D in simplest case we can recursively get numbers at dist D-1 and then apply generator to them
def mygen(N):
yield N**2
yield N**3
def getSet(N, dist):
if dist == 0:
return [N]
numbers = []
for n in getSet(N, dist-1):
numbers += list(mygen(n))
return numbers
print getSet(2,0)
print getSet(2,1)
print getSet(2,2)
print getSet(2,3)
output is
[2]
[4, 8]
[16, 64, 64, 512]
[256, 4096, 4096, 262144, 4096, 262144, 262144, 134217728]
This solution does not require to keep all results in memory: (in case it doesn't fit in memory etc)
def grandKids(generation, kidsFunc, val):
layer = [val]
for i in xrange(generation):
layer = itertools.chain.from_iterable(itertools.imap(kidsFunc, layer))
return layer
Example:
def kids(x): # children indices in a 1-based binary heap
yield x*2
yield x*2+1
>>> list(grandKids(3, kids, 2))
[16, 17, 18, 19, 20, 21, 22, 23]
Btw, solution in Haskell:
grandKids generation kidsFunc val =
iterate (concatMap kidsFunc) [val] !! generation
I have just started learning Python so bear with me if my answer seems a tad amateurish. What you could do is use a list of lists to populate the values returned from the myGenerator function.
So for eg. with 2 as the starting argument your data-structure would resemble something like
resDataSet = [[2],
[4, 5],
[9, 10, 11, 12, 300 , 500]
...
]
The row index should give you the distance and you can use methods like extend to add on more data to your list.