I have a function with one of the parameters as numpy.ndarray. It's mutable so it can not be cached by lru_cache.
Is there any existing solution for this?
Possibly the simplest way of doing so is to memoize a version taking only immutable objects.
Say your function takes an np.array, and let's assume it's a 1d array. Fortunately, it is easily translated to a tuple:
import numpy as np
a = np.array([1, 2, 3, 4])
>> tuple(a)
(1, 2, 3, 4)
and vice versa:
>> np.array(tuple(a))
array([1, 2, 3, 4])
So you get something like
# Function called by the rest of your program
array_foo(a) # `a` is an `np.array`
...
return tuple_foo(tuple(a))
then memoize instead this function:
# Internal implementation
#functools.lru_cache
tuple_foo(t) # `t` is a tuple
...
a = np.array(t)
Related
When calling numpy.array in the following two ways:
>>> np.array((1,2,3,4))
array([1, 2, 3, 4])
>>> np.array([1,2,3,4])
array([1, 2, 3, 4])
I notice it returns two seemingly identical ndarrays.
Are both of these ndarrays identical? Why?
We can behave in a general way here. Suppose we don't know what X = np.array([1,2,3,4]) and Y = np.array((1,2,3,4)) are. If we print it, we can see an output which is the result of a secret built-in methods X.__repr__ and Y.__repr__. You can see here for sure that both X and Y has the same representations. It doesn't mean, however, that they are the same because they can be instances of different classes with the same representations. To make it sure, I usually use X.__class__ and Y.__class__. So both X and Y are instances of the same class np.ndarray.
From the doc of random.shuffle(x[, random]), it says:
The optional argument random is a 0-argument function returning a random float in [0.0, 1.0); by default, this is the function random()
Could someone please explain what 0-argument function means and give an example of random.shuffle() with the optional argument random? I searched but couldn't find any example for that case. Also, what did it mean by "this is the function random()"? Does this refer to the optional argument?
It means you can pass in the name of a function which does not require an argument.
def foo():
return 0.5
is such a function.
def bar(limit):
return limit
is not, because it requires an argument limit.
Usage example:
random.shuffle(itemlist, random=foo)
If the input itemlist was [1, 2, 3] it will now be [1, 3, 2]. I have established this experimentally, and I suppose how exactly the shuffle operation uses the output from the random function could change between Python versions.
The default if you don't specify anything is the function random().
One possible use case for this is if you want predictable output e.g. for a test case. Another is if you want nonuniform distribution - for example, a random function which prefers small values over large ones, or implements e.g. Poisson or normal distribution.
A 0-argument function has an empty argument list.
def random_seed(): # Zero arguments here.
return MY_CONFIG.predictble_random_seed # Some imaginary config.
random.shuffle(some_list, random_seed) # Always the same shuffling.
The point of this arrangement is to allow to control the predictability of the shuffling. You can return a really random number (from timer, /dev/urandom, etc, as random.random() does) in production, and a controlled number in a test environment:
def get_random_generator(environment):
if environment == 'test':
return lambda: 0.5 # A 0-argument callable returning a constant.
else:
return random.random # A function returning a random number.
# ...
# The below is predictable when testing in isolation,
# unpredictable when running in production.
# We suppose that `environment` has values like 'test' and 'prod'.
random.shuffle(entries, get_random_generator(environment))
The random parameter is the seed. Then if you always use the same seed, it always reorder your array with the same logic. See the exemple. 5 is at index 4 and go to 0. 6 go to 4 (Old index of 5) then if we reuse the same seed, 6 go the index 0 because 6 is at index 4 like 5 at the first shuffle
Exemple:
>>> import random
>>> r = random.random()
>>> r
0.4309619702601998
>>> x = [1, 2, 3, 4, 5, 6]
>>> random.shuffle(x, lambda: r)
>>> x
[5, 1, 4, 2, 6, 3]
>>> random.shuffle(x, lambda: r)
>>> x
[6, 5, 2, 1, 3, 4]
>>> x = [1, 2, 3, 4, 5, 6]
>>> random.shuffle(x, lambda: r)
>>> x
[5, 1, 4, 2, 6, 3]
Source
I'am trying to clip the values in arrays. I found the function np.clip()
and it did what I need. However, the way it modifies the array values in the list of arrays makes me confused. Here is the code:
import numpy as np
a = np.arange(5)
b = np.arange(5)
for x in [a,b]:
np.clip(x, 1, 3, out=x)
result
>>> a
array([1, 1, 2, 3, 3])
>>> b
array([1, 1, 2, 3, 3])
The values of a and b have been changed without being assigned while the function np.clip() only works with x.
There are some questions related, but they use index of the list, e.g Modifying list elements in a for loop, Changing iteration variable inside for loop in Python.
Can somebody explain for me how the function np.clip() can directly modify the value of list values.
It's not because of the np.clip function. It's because you use loop on a list of mutable, so the value of the element can be modified. You can see here Immutable vs Mutable types for more information.
I have a numpy array that is changed by a function.
After calling the function I want to proceed with the initial value of the array (value before calling the modifying function)
# Init of the array
array = np.array([1, 2, 3])
# Function that modifies array
func(array)
# Print the init value [1,2,3]
print(array)
Is there a way to pass the array by value or am I obligated to make a deep copy?
As I mentioned, np.ndarray objects are mutable data structures. This means that any variables that refer to a particular object will all reflect changes when a change is made to the object.
However, keep in mind that most numpy functions that transform arrays return new array objects, leaving the original unchanged.
What you need to do in this scenario depends on exactly what you're doing. If your function modifies the same array in place, then you'll need to pass a copy to the function. You can do this with np.ndarray.copy.
You could use the following library (https://pypi.python.org/pypi/pynverse) that inverts your function and call it, like so :
from pynverse import inversefunc
cube = (lambda x: x**3)
invcube = inversefunc(cube)
arr = func(arr)
In [0]: arr
Out[0]: array([1, 8, 27], dtype=int32)
In [1]: invcube(arr)
Out[1]: array([1, 2, 3])
Is there are a way to use Numpy's multidimensional array slicing without using the [slice, slice] syntax?
I need to be able to use it from normal function calls, but I haven't found a way to use the slice object to do it.
I cannot use the syntax [(slice,slice)] for my program because [] is special syntax outside of regular function calls.
The language I am using is Hy, a Lisp for Python, and it does not support this syntax. More importantly, it shouldn't support this syntax. Numpy, however, doesn't seem to support multidimensional slicing without using the [] syntax.
What's tripping me up is that the mix of C and Python in the Numpy source makes it difficult to discern how the [slice,slice] is implemented.
It may not even be possible to circumvent this syntax.
EDIT:
The answer provided below by #Joe Kington allows one to slice Numpy matrices like so:
x = np.array([list(range(5)) for x in list(range(5))])
x.getitem(slice(1,4))
array([[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]])
x.getitem(tuple([slice(1,4),slice(1,4)]))
array([[1, 2, 3],
[1, 2, 3],
[1, 2, 3]])
From your description, it seems like you're asking what function calls are used to implement slicing and slice assignment.
Python uses the "special" methods __getitem__ and __setitem__ to implement and/or allow customization of how slicing works. Any class that implements these can be sliced. There's actually nothing numpy-specific about this.
In other words
x = arr[4:10, 9:15, ::-1]
x[0] = 100
is translated into
x = arr.__getitem__((slice(4, 6), slice(9, 10), slice(None, None, -1)))
x.__setitem__(0, 100)
For example:
class Foo(object):
def __getitem__(self, index):
print 'Getting', index
def __setitem__(self, index, val):
print 'Setting', index, 'to', val
f = Foo()
print 'Getting...'
f[:]
f[4:10, ::-1, ...]
print 'Equivalently:'
f.__getitem__(slice(None))
f.__getitem__((slice(4, 10), slice(None, None, -1), Ellipsis))
print 'Setting...'
f[0] = 1
f[5:10, 100] = 2
f[...] = 100
print 'Equivalently:'
f.__setitem__(0, 1)
f.__setitem__((slice(5,10), 100), 2)
f.__setitem__(Ellipsis, 100)
Also, it can be handy to know about numpy.index_exp (or equivalently, np.s_). It's nothing fancy -- it just translates slicing into the equivalent tuple, etc. It's quite similar to our Foo class above. For example:
In [1]: np.index_exp[10:4, ::-1, ...]
Out[1]: (slice(10, 4, None), slice(None, None, -1), Ellipsis)
I suspect you are trying to pass the slice through as a parameter?
def do_slice(sl, mystring):
return mystring[sl]
sl = slice(0,2)
mystr = "Hello"
print do_slice(sl, mystr)