I would like to get slice objects from subscript notation. So far, I have used a hacky trick to exploit Python built-in subscript to slice conversion features:
class Slice:
def __getitem__(self, item):
return item
For example, Slice()[1:2] will return slice(1,2,None).
Does anyone know of a more straightforward way?
If you must use subscript notation, then your current solution is the most compact besides maybe a dynamic class created with type:
>>> Slice = type('', (), {'__getitem__': lambda _, x: x})()
>>> Slice[1:2]
slice(1, 2, None)
>>> Slice[1:2:3]
slice(1, 2, 3)
>>>
But code like this is usually hard to understand/maintain/extend/etc.
Instead, I would recommend that you use slice, which allows you to create slice objects directly:
>>> slice(1, 2)
slice(1, 2, None)
>>> slice(1, 2, 3)
slice(1, 2, 3)
>>>
The built-in was made specifically for this purpose (well, that and a few others such as type-checking with isinstance) and is therefore very portable as well as pythonic.
Related
I have a function with one of the parameters as numpy.ndarray. It's mutable so it can not be cached by lru_cache.
Is there any existing solution for this?
Possibly the simplest way of doing so is to memoize a version taking only immutable objects.
Say your function takes an np.array, and let's assume it's a 1d array. Fortunately, it is easily translated to a tuple:
import numpy as np
a = np.array([1, 2, 3, 4])
>> tuple(a)
(1, 2, 3, 4)
and vice versa:
>> np.array(tuple(a))
array([1, 2, 3, 4])
So you get something like
# Function called by the rest of your program
array_foo(a) # `a` is an `np.array`
...
return tuple_foo(tuple(a))
then memoize instead this function:
# Internal implementation
#functools.lru_cache
tuple_foo(t) # `t` is a tuple
...
a = np.array(t)
So I can create a reverse iterator on a list:
list(reversed([0,1,2,3]))
[3, 2, 1, 0]
I assume this simply calls getitem from index len(...)-1 to 0. But then I cannot also do this:
list(reversed(xrange(4)))
[3, 2, 1, 0]
Now I am a bit confused. Does this create the list from xrange(4) and then reverse it? If not, how does it know what the last element is and how to go backwards? I read the documentation but it didn't help.
reversed() looks for a __reversed__ special method on the object. List objects provide this, and so does xrange():
>>> xrange(4).__reversed__()
<rangeiterator object at 0x106e2fa50>
The iterator object simply produces the values in reverse, no list object is produced.
For objects that do not implement __reversed__, the reversed() function uses the length and the __getitem__ method; e.g. reversed() is essentially equivalent to:
def reversed(seq):
try:
return seq.__reversed__()
except AttributeError:
return (seq[i] for i in xrange(len(seq) - 1, -1 , -1))
where the second part is a generator expression, evaluating lazily. The generator then accesses each item in turn starting at index (length - 1) all the way down to index 0.
reversed() can only take a sequence -- if it took generic iterators, it couldn't know what the final value was without exhausting the iterator and storing all the values.
Luckily, xrange returns an xrange object that works as a sequence:
>>> x = xrange(10)
>>> len(x)
10
>>> x[9]
9
It also happens to have an actual __reversed__ method, but that's a special case of having all the sequence methods.
Just compare the two:
In [2]: reversed(xrange(4))
Out[2]: <rangeiterator at 0x7fa83291bde0>
In [3]: list(reversed(xrange(4)))
Out[3]: [3, 2, 1, 0]
In [4]: reversed([0,1,2,3])
Out[4]: <listreverseiterator at 0x7fa8328be2d0>
In [5]: list(reversed([0,1,2,3]))
Out[5]: [3, 2, 1, 0]
Is there are a way to use Numpy's multidimensional array slicing without using the [slice, slice] syntax?
I need to be able to use it from normal function calls, but I haven't found a way to use the slice object to do it.
I cannot use the syntax [(slice,slice)] for my program because [] is special syntax outside of regular function calls.
The language I am using is Hy, a Lisp for Python, and it does not support this syntax. More importantly, it shouldn't support this syntax. Numpy, however, doesn't seem to support multidimensional slicing without using the [] syntax.
What's tripping me up is that the mix of C and Python in the Numpy source makes it difficult to discern how the [slice,slice] is implemented.
It may not even be possible to circumvent this syntax.
EDIT:
The answer provided below by #Joe Kington allows one to slice Numpy matrices like so:
x = np.array([list(range(5)) for x in list(range(5))])
x.getitem(slice(1,4))
array([[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]])
x.getitem(tuple([slice(1,4),slice(1,4)]))
array([[1, 2, 3],
[1, 2, 3],
[1, 2, 3]])
From your description, it seems like you're asking what function calls are used to implement slicing and slice assignment.
Python uses the "special" methods __getitem__ and __setitem__ to implement and/or allow customization of how slicing works. Any class that implements these can be sliced. There's actually nothing numpy-specific about this.
In other words
x = arr[4:10, 9:15, ::-1]
x[0] = 100
is translated into
x = arr.__getitem__((slice(4, 6), slice(9, 10), slice(None, None, -1)))
x.__setitem__(0, 100)
For example:
class Foo(object):
def __getitem__(self, index):
print 'Getting', index
def __setitem__(self, index, val):
print 'Setting', index, 'to', val
f = Foo()
print 'Getting...'
f[:]
f[4:10, ::-1, ...]
print 'Equivalently:'
f.__getitem__(slice(None))
f.__getitem__((slice(4, 10), slice(None, None, -1), Ellipsis))
print 'Setting...'
f[0] = 1
f[5:10, 100] = 2
f[...] = 100
print 'Equivalently:'
f.__setitem__(0, 1)
f.__setitem__((slice(5,10), 100), 2)
f.__setitem__(Ellipsis, 100)
Also, it can be handy to know about numpy.index_exp (or equivalently, np.s_). It's nothing fancy -- it just translates slicing into the equivalent tuple, etc. It's quite similar to our Foo class above. For example:
In [1]: np.index_exp[10:4, ::-1, ...]
Out[1]: (slice(10, 4, None), slice(None, None, -1), Ellipsis)
I suspect you are trying to pass the slice through as a parameter?
def do_slice(sl, mystring):
return mystring[sl]
sl = slice(0,2)
mystr = "Hello"
print do_slice(sl, mystr)
Is there an equivalent of cons in Python? (any version above 2.5)
If so, is it built in? Or do I need easy_install do get a module?
WARNING AHEAD: The material below may not be practical!
Actually, cons needs not to be primitive in Lisp, you can build it with λ.
See Use of lambda for cons/car/cdr definition in SICP for details. In Python, it is translated to:
def cons(x, y):
return lambda pair: pair(x, y)
def car(pair):
return pair(lambda p, q: p)
def cdr(pair):
return pair(lambda p, q: q)
Now, car(cons("a", "b")) should give you 'a'.
How is that? Prefix Scheme :)
Obviously, you can start building list using cdr recursion. You can define nil to be the empty pair in Python.
def nil(): return ()
Note that you must bind variable using = in Python. Am I right? Since it may mutate the variable, I'd rather define constant function.
Of course, this is not Pythonic but Lispy, not so practical yet elegant.
Exercise: Implement the List Library http://srfi.schemers.org/srfi-1/srfi-1.html of Scheme in Python. Just kidding :)
In Python, it's more typical to use the array-based list class than Lisp-style linked lists. But it's not too hard to convert between them:
def cons(seq):
result = None
for item in reversed(seq):
result = (item, result)
return result
def iter_cons(seq):
while seq is not None:
car, cdr = seq
yield car
seq = cdr
>>> cons([1, 2, 3, 4, 5, 6])
(1, (2, (3, (4, (5, (6, None))))))
>>> iter_cons(_)
<generator object uncons at 0x00000000024D7090>
>>> list(_)
[1, 2, 3, 4, 5, 6]
Note that Python's lists are implemented as vectors, not as linked lists. You could do lst.insert(0, val), but that operation is O(n).
If you want a data structure that behaves more like a linked list, try using a Deque.
In Python 3, you can use the splat operator * to do this concisely by writing [x, *xs]. For example:
>>> x = 1
>>> xs = [1, 2, 3]
>>> [x, *xs]
[1, 1, 2, 3]
If you prefer to define it as a function, that is easy too:
def cons(x, xs):
return [x, *xs]
You can quite trivially define a class that behaves much like cons:
class Cons(object):
def __init__(self, car, cdr):
self.car = car
self.cdr = cdr
However this will be a very 'heavyweight' way to build basic data structures, which Python is not optimised for, so I would expect the results to be much more CPU/memory intensive than doing something similar in Lisp.
No. cons is an implementation detail of Lisp-like languages; it doesn't exist in any meaningful sense in Python.
In Numpy (and Python in general, I suppose), how does one store a slice-index, such as (...,0,:), in order to pass it around and apply it to various arrays? It would be nice to, say, be able to pass a slice-index to and from functions.
Python creates special objects out of the slice syntax, but only inside the square brackets for indexing. You can either create those objects by hand (in this case, (...,0,:) is (Ellipsis, 0, slice(None, None, None)), or you can create a little helper object:
class ExtendedSliceMaker(object):
def __getitem__(self, idx):
return idx
>>> ExtendedSliceMaker()[...,0,:]
(Ellipsis, 0, slice(None, None, None))
Use s_ in NumPy:
In [1]: np.s_[...,0,:]
Out[1]: (Ellipsis, 0, slice(None, None, None))
The equivalent to (...,0,:) should be...
>>> myslice = (..., 0, slice(None, None))
>>> myslice
(Ellipsis, 0, slice(None, None, None))
The neat thing about Python is that you can actually make a class to inspect how these things are represented. Python uses the magic method __getitem__ to handle indexing operations, so we'll make a class that overloads this to show us what was passed in, instantiate the class, and "index in" to the instance:
class foo:
def __getitem__(self, index): print index
foo()[...,0,:]
And our result is:
(Ellipsis, 0, slice(None, None, None))
Ellipsis and slice are builtins, and we can read their documentation:
help(Ellipsis)
help(slice)
I think you want to just do myslice = slice(1,2) to for example define a slice that will return the 2nd element (i.e. myarray[myslice] == myarray[1:2])