Bear with me while I explain my question. Skip down to the bold heading if you already understand extended slice list indexing.
In python, you can index lists using slice notation. Here's an example:
>>> A = list(range(10))
>>> A[0:5]
[0, 1, 2, 3, 4]
You can also include a stride, which acts like a "step":
>>> A[0:5:2]
[0, 2, 4]
The stride is also allowed to be negative, meaning the elements are retrieved in reverse order:
>>> A[5:0:-1]
[5, 4, 3, 2, 1]
But wait! I wanted to see [4, 3, 2, 1, 0]. Oh, I see, I need to decrement the start and end indices:
>>> A[4:-1:-1]
[]
What happened? It's interpreting -1 as being at the end of the array, not the beginning. I know you can achieve this as follows:
>>> A[4::-1]
[4, 3, 2, 1, 0]
But you can't use this in all cases. For example, in a method that's been passed indices.
My question is:
Is there any good pythonic way of using extended slices with negative strides and explicit start and end indices that include the first element of a sequence?
This is what I've come up with so far, but it seems unsatisfying.
>>> A[0:5][::-1]
[4, 3, 2, 1, 0]
It is error-prone to change the semantics of start and stop. Use None or -(len(a) + 1) instead of 0 or -1. The semantics is not arbitrary. See Edsger W. Dijkstra's article "Why numbering should start at zero".
>>> a = range(10)
>>> start, stop, step = 4, None, -1
Or
>>> start, stop, step = 4, -(len(a) + 1), -1
>>> a[start:stop:step]
[4, 3, 2, 1, 0]
Or
>>> s = slice(start, stop, step)
>>> a[s]
[4, 3, 2, 1, 0]
When s is a sequence the negative indexes in s[i:j:k] are treated specially:
If i or j is negative, the index is relative to the end of the string:
len(s) + i or len(s) + j is substituted. But note that -0 is still 0.
that is why len(range(10)[4:-1:-1]) == 0 because it is equivalent to range(10)[4:9:-1].
Ok, I think this is probably as good as I will get it. Thanks to Abgan for sparking the idea. This relies on the fact that None in a slice is treated as if it were a missing parameter. Anyone got anything better?
def getReversedList(aList, end, start, step):
return aList[end:start if start!=-1 else None:step]
edit: check for start==-1, not 0
This is still not ideal, because you're clobbering the usual behavior of -1. It seems the problem here is two overlapping definitions of what's supposed to happen. Whoever wins takes away otherwise valid invocations looking for the other intention.
[ A[b] for b in range(end,start,stride) ]
Slower, however you can use negative indices, so this should work:
[ A[b] for b in range(9, -1, -1) ]
I realize this isn't using slices, but thought I'd offer the solution anyway if using slices specifically for getting the result isn't a priority.
I believe that the following doesn't satisfy you:
def getReversedList(aList, end, start, step):
if step < 0 and start == 0:
return aList[end::step]
return aList[end:start:step]
or does it? :-)
But you can't use that if you are
storing your indices in variables for
example.
Is this satisfactory?
>>> a = range(10)
>>> start = 0
>>> end = 4
>>> a[4:start-1 if start > 0 else None:-1]
[4, 3, 2, 1, 0]
As you say very few people fully understand everything that you can do with extended slicing, so unless you really need the extra performance I'd do it the "obvious" way:
rev_subset = reversed(data[start:stop])
a[4::-1]
Example:
Python 2.6 (r26:66714, Dec 4 2008, 11:34:15)
[GCC 4.0.1 (Apple Inc. build 5488)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a = list(range(10))
>>> a
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> a[4:0:-1]
[4, 3, 2, 1]
>>> a[4::-1]
[4, 3, 2, 1, 0]
>>>
The reason is that the second term is interpreted as "while not index ==". Leaving it out is "while index in range".
I know this is an old question, but in case someone like me is looking for answers:
>>> A[5-1::-1]
[4, 3, 2, 1, 0]
>>> A[4:1:-1]
[4, 3, 2]
You can use a slice(start, stop, step) object, which is such that
s=slice(start, stop, step)
print a[s]
is the same as
print a[start : stop : step]
and, moreover, you can set any of the arguments to None to indicate nothing in between the colons. So in the case you give, you can use slice(4, None, -1).
Related
This question already has answers here:
Reversing a list slice in python
(3 answers)
Closed 3 years ago.
Suppose we have this list:
>>> a = [x for x in range(10)]
>>> print(a)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Separately, both ways to slice work as expected:
>>> a[3:8]
[3, 4, 5, 6, 7]
>>> a[::-1]
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
But, when combined:
>>> a[3:8:-1]
[]
I would expect it to be [7, 6, 5 ,4, 3] or perhaps [6, 5, 4, 3, 2] (if reversing happened first).
It is also interesting to consider what happens when either start or stop parameters are not passed:
>>> a[:5:-1]
[9, 8, 7, 6]
This is almost what I would expect, only its one item short. Tested this with numpy and it seems to behave in the same way.
Whats going on here?
With
a[3:8:-1]
The start and stop positions of the slice aren't adjusted based on the step. With a negative step, you're having it go backwards from 3, but there are no elements with indices in the range 3 to 8 counting back from 3, so you get an empty list.
You need to set the start and stop accordingly:
a[8:3:-1]
Which will count back from 8 to 4.
a[3:8:-1] instructs python to start from 3 and go to 8 by steps of -1
This creates an empty list: it's not possible to reach 8 from 3 by adding -1 (just like list(range(3,8,-1)) which gives an empty list too)
When you do a[:5:-1] then start is the default start, which python sets to "end of list" so it "works"
Same as when you do a[::-1] the start & stop are the default ones, and python understands that they're from end to start (else this notation wouldn't be useable)
This behavior is explained in the documentation.
The slice of s from i to j is defined as the sequence of items with index k such that i <= k < j. If i or j is greater than len(s), use len(s). If i is omitted or None, use 0. If j is omitted or None, use len(s). If i is greater than or equal to j, the slice is empty.
The slice of s from i to j with step k.... stopping when j is reached (but never including j). When k is positive, i and j are reduced to len(s) if they are greater. When k is negative, i and j are reduced to len(s) - 1 if they are greater. If i or j are omitted or None, they become “end” values (which end depends on the sign of k).
I understand that given an iterable such as
>>> it = [1, 2, 3, 4, 5, 6, 7, 8, 9]
I can turn it into a list and slice off the ends at arbitrary points with, for example
>>> it[1:-2]
[2, 3, 4, 5, 6, 7]
or reverse it with
>>> it[::-1]
[9, 8, 7, 6, 5, 4, 3, 2, 1]
or combine the two with
>>> it[1:-2][::-1]
[7, 6, 5, 4, 3, 2]
However, trying to accomplish this in a single operation produces in some results that puzzle me:
>>> it[1:-2:-1]
[]
>>>> it[-1:2:-1]
[9, 8, 7, 6, 5, 4]
>>>> it[-2:1:-1]
[8, 7, 6, 5, 4, 3]
Only after much trial and error, do I get what I'm looking for:
>>> it[-3:0:-1]
[7, 6, 5, 4, 3, 2]
This makes my head hurt (and can't help readers of my code):
>>> it[-3:0:-1] == it[1:-2][::-1]
True
How can I make sense of this? Should I even be pondering such things?
FWYW, my code does a lot of truncating, reversing, and listifying of iterables, and I was looking for something that was faster and clearer (yes, don't laugh) than list(reversed(it[1:-2])).
This is because in a slice like -
list[start:stop:step]
start is inclusive, resultant list starts at index start.
stop is exclusive, that is the resultant list only contains elements till stop - 1 (and not the element at stop).
So for your caseit[1:-2] - the 1 is inclusive , that means the slice result starts at index 1 , whereas the -2 is exclusive , hence the last element of the slice index would be from index -3.
Hence, if you want the reversed of that, you would have to do it[-3:0:-1] - only then -3 would be included in the sliced result, and the sliced result would go upto 1 index.
The important things to understand in your slices are
Start will be included in the slice
Stop will NOT be included in the slice
If you want to slice backwards, the step value should be a negative value.
Basically the range which you specify is a half-open (half-closed) range.
When you say it[-3:0:-1] you are actually starting from the third element from the back, till we reach 0 (not including zero), step one element at a time backwards.
>>> it[-3:0:-1]
[7, 6, 5, 4, 3, 2]
Instead, you can realize the start value like this
>>> it[len(it)-3 : 0 : -1]
[7, 6, 5, 4, 3, 2]
I think the other two answers disambiguate the usage of slicing and give a clearer image of how its parameters work.
But, since your question also involves readability -- which, let's not forget, is a big factor especially in Python -- I'd like to point out how you can improve it slightly by assigning slice() objects to variables thus removing all those hardcoded : separated numbers.
Your truncate and reverse slice object could, alternatively, be coded with a usage implying name :
rev_slice = slice(-3, 0, -1)
In some other config-like file. You could then use it in its named glory within slicing operations to make this a bit more easy on the eyes :
it[rev_slice] # [7, 6, 5, 4, 3, 2]
This might be a trivial thing to mention, but I think it's probably worth it.
Why not create a function for readability:
def listify(it, start=0, stop=None, rev=False):
if stop is None:
the_list = it[start:]
else:
the_list = it[start:stop]
if rev:
return the_list[::-1]
else:
return the_list
listify(it, start=1, stop=-2) # [2, 3, 4, 5, 6, 7]
listify(it, start=1, stop=-2, rev=True) # [7, 6, 5, 4, 3, 2]
A good way to intuitively understand the Python slicing syntax is to see how it maps to the corresponding C for loop.
A slice like
x[a:b:c]
gives you the same elements as
for (int i = a; i < b; i += c) {
...
}
The special cases are just default values:
a defaults to 0
b defaults to len(x)
c defaults to 1
Plus one more special case:
if c is negative, then a and b are swapped and the < is inverted to a >
In [122]: a = range(10)
In [123]: a[: : -1]
Out[123]: [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
Could you explain the expression a[: : -1]?
a[:] is clearly understandable -> "start form the beginning(space before the colon) and retrieve the list upto the end (space after the colon)"
But I am not getting what the two colons are actually doing in the expression a[: : -1].
A slice takes three arguments, just like range: start, stop and step:
[0, 1, 2, 3, 4, 5][0:4:2] == list(range(0, 4, 2)) # every second element from 0 to 3
The negative step causes the slice to work backwards through the iterable. Without a start and stop (i.e. just the step [::-1]) it starts from the end, as it is working backwards.
The third argument (after two :'s) is the step size. -1 can be interpreted as stepping backwards. In other words, reversing the list.
Try with -2 step size i.e., a[::-2], You'll get:
[9, 7, 5, 3, 1]
Hope this helps.
More elaborate answers and explanations here Explain Python's slice notation
Imagine some numpy array, e.g. x = np.linspace(1,10).
x[i:j] gives me a view into x for the range [i,j).
I love that I can also do x[i:-k] which excludes the last k elements.
However, in order to include the last element I need to do x[i:].
My question is this: How do I combine these two notations if I for instance need to loop over k.
Say that I want to do this:
l = list()
for k in [5,4,3,2,1]:
l.append(x[:-k])
l.append(x[:])
What annoys me is that last line. In this simple example of course it doesn't do much of a difference, but sometimes this becomes much more annoying. What I miss is something more DRY-like.
The following snippet course does NOT yield the desired result, but represents the style of code I seek:
l = list()
for k in [5,4,3,2,1,0]:
l.append(x[:-k])
It's a bit of a pain, but since -0 is the same as 0, there is no easy solution.
One way to do it would be:
l = list()
for k in [5,4,3,2,1,0]:
l.append(x[:-k or None])
This is because when k is 0, -k or None is None, and x[:None] will do what you want. For other values of k, -k or None will be -k.
I am not sure if I like it myself though.
You can't, because -0 doesn't slice that way in python (it becomes 0)
You could just do the old school:
l = list()
for k in [5,4,3,2,1,0]:
l.append(x[:len(x)-k])
The value None, in a slice, is the same as putting nothing there. In other words, x[:None] is the same as x[:]. So:
l = list()
for k in [-5,-4,-3,-2,-1,None]:
l.append(x[:k])
However… this code is a lot easier to write as a list comprehension:
l = [x[:k] for k in (-5,-4,-3,-2,-1,None)]
Or… you might want to look at whatever it is you're trying to do and see if there's a higher-level abstraction that makes sense, or maybe just another way to organize things that's more readable (even if it's a bit more verbose). For example, depending on what x actually represents, this might be more understandable (or it might be less, of course):
l = []
for k in range(6):
l.insert(0, x)
x = x[:-1]
Perhaps this, then:
l = list()
for k in [5,4,3,2,1,None]:
l.append(x[:-k if k else None])
If x is simply range(10), the above code will produce:
[[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5, 6],
[0, 1, 2, 3, 4, 5, 6, 7],
[0, 1, 2, 3, 4, 5, 6, 7, 8],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]
Numpy tentative tutorial suggests that a[ : :-1] is a reversed a. Can someone explain me how we got there?
I understand that a[:] means for each element of a (with axis=0). Next : should denote the number of elements to skip (or period) from my understanding.
It isn't numpy, it's Python.
In Python, there are slices for sequence/iterable, which come in the following syntax
seq[start:stop:step] => a slice from start to stop, stepping step each time.
All the arguments are optional, but a : has to be there for Python to recognize this as a slice.
Negative values, for step, also work to make a copy of the same sequence/iterable in reverse order:
>>> L = range(10)
>>> L[::-1]
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
And numpy follows that "rule" like any good 3rd party library..
>>> a = numpy.array(range(10))
>>> a[::-1]
array([9, 8, 7, 6, 5, 4, 3, 2, 1, 0])
See this link
As others have noted, this is a python slicing technique, and numpy just follows suit. Hopefully this helps explain how it works:
The last bit is the stepsize. The 1 indicates to step by one element at a time, the - does that in reverse.
Blanks indicate the first and last, unless you have a negative stepsize, in which case they indicate last and first:
In [1]: import numpy as np
In [2]: a = np.arange(5)
In [3]: a
Out[3]: array([0, 1, 2, 3, 4])
In [4]: a[0:5:1]
Out[4]: array([0, 1, 2, 3, 4])
In [5]: a[0:5:-1]
Out[5]: array([], dtype=int64)
In [6]: a[5:0:-1]
Out[6]: array([4, 3, 2, 1])
In [7]: a[::-2]
Out[7]: array([4, 2, 0])
Line 5 gives an empty array since it tries to step backwards from the 0th element to the 5th.
The slice doesn't include the 'endpoint' (named last element) so line 6 misses 0 when going backwards.
This isn't specific to numpy, the slice a[::-1] is equivalent to slice(None, None, -1), where the first argument is the start index, the second argument is the end index, and the third argument is the step. None for start or stop will have the same behavior as using the beginning or end of the sequence, and -1 for step will iterate over the sequence in reverse order.
You can use the reversed Python built-in:
import numpy as np
bins = np.arange(0.0, 1.1, .1)
for i in reversed(bins):
print(i)