I have been trying (with some success) to write vectorized integration calls with the numpy vectorize function but every once and a while I get stuck with issues of how Python treats tuples.
I want to write variants of integrate.quad that can integrate vector-valued functions over a grid of points. Similarly I want to create a version of integrate.nquad that integrates over n-dimensional domains, and can compute these integrals over a grid of points (i.e an integral with an n-dimensional domain, vector output, computed along a lattice of points in k-dimensional space).
For example:
import numpy as np
from scipy import integrate
def vecint(F, I, *args):
componentintegrals = [integrate.nquad(f, I, args) for f in F]
retint = [CI[0] for CI in componentintegrals]
if (len(retint)==1):
retint = retint[0]
reterr = np.sqrt(sum(CI[1]**2 for CI in componentintegrals))
return retint, reterr
vecint takes as input an list of many-variable functions and treats it as an integration problem where one integrates a vector-valued function. This code works just fine, eg:
print(vecint([lambda x,y: np.sin(x), lambda x,y: np.cos(y)], [[0,np.pi],[0,np.pi]] ) )
print(vecint([lambda x: np.sin(x)], [[0,np.pi]]))
print(vecint([lambda x: np.cos(x), lambda x: np.sin(x)], [[0,np.pi]] ))
## and we can pass additional arguments.
print(vecint( [lambda x,k: np.sin(x)+k], [[0,np.pi]], 1) )
All the above calls work as expected. The trouble for me starts when I try to vectorize these functions. Vectorizing integrate.quad goes fine...
def quad_1vz1(f, I, *args):
return np.vectorize(lambda n: integrate.quad(f, I[0], I[1], (n,)+args)[0])
as expected. The above code allows for calls such as:
quad_1vz1(lambda x,k: np.sin(kx), [0,np.pi])(K)
where K=np.mgrid[0:1:6j], etc. These are the integrals
$$\int_0^{\pi} \sin(kx) dx$$
for various values of $k$.
The problem occurs when I try to replace integrate.quad with the vecint function above. eg:
## let's vectorize a 1-dimensional integral of a vector-valued function with one parameter.
def vecint_2vz1(F, I, *args):
#print(I, args)
return np.vectorize(lambda n: vecint( F, I, (n,)+args )[0])
def f1(t,k):
return np.cos(t)+k
def f2(t,k):
return np.sin(t)+k
K = np.mgrid[0:1:6j]
print( vecint_2vz1( [f1,f2], [[0,np.pi]] )(K) )
The above results in a "ValueError: setting an array element with a sequence."
When vecint is vectorized here, the elements of K are sent as 1-element tuples of 1-element tuples, i.e. the extra argument might be something like a ((0,),).
I suspect to avoid this I have to do some crafty unpacking/repacking of arguments.... but I'm a little confused as to what Python is thinking.
It appears as if Python sometimes auto-casts 1-tuples to the value contained inside... and sometimes it does not. This has me confused. I feel like I'm missing something elementary.
Python's output on execution:
ValueError Traceback (most recent call last)
<ipython-input-131-8cb6f52e7d30> in <module>()
17 ## integral of (cos(t)+k, sin(t)+k)dt for various k's.
18
---> 19 print( vecint_2vz1( [f1,f2], [[0,np.pi]] )(K) )
/usr/local/lib/python3.4/dist-packages/numpy/lib/function_base.py in __call__(self, *args, **kwargs)
1809 vargs.extend([kwargs[_n] for _n in names])
1810
-> 1811 return self._vectorize_call(func=func, args=vargs)
1812
1813 def _get_ufunc_and_otypes(self, func, args):
/usr/local/lib/python3.4/dist-packages/numpy/lib/function_base.py in _vectorize_call(self, func, args)
1882 if ufunc.nout == 1:
1883 _res = array(outputs,
-> 1884 copy=False, subok=True, dtype=otypes[0])
1885 else:
1886 _res = tuple([array(_x, copy=False, subok=True, dtype=_t)
ValueError: setting an array element with a sequence.
Further if I put a litle print(args) line in vecint, it prints out items such as ((a,),) where a are the elements of the mgrid K.
It's hard to trace what is going on in your code. We're trying to keep track of what quad takes, what your vecint does, lambdas and vectorize.
Why does quad_1vz1 run, but quad_2vz1 doesn't? Is quad unpacking the tuples? I don't know.
I tried to simplify things with:
def foo2(*args):
def foo(*args):
print(args)
return 1
return np.vectorize(lambda n: foo((n,)+args))
which produces:
In [148]: foo2()(np.arange(3))
((0,),)
((0,),)
((1,),)
((2,),)
Out[148]: array([1, 1, 1])
In [153]: foo2(3)(np.arange(2))
((0, 3),)
((0, 3),)
((1, 3),)
Out[153]: array([1, 1])
Note that since I did not specify an output type in vectorize it runs one step just to figure out what the output is like. Hence I see ((0,),) twice. That has given other SO problems if the initial type is different from later ones (e.g. integer v float).
The 2 tuple levels are the produce of my own (n,)+args, and the passage of *args and vectorize. I'd have to experiment more to sort out the responsibility of the 2nd two.
vectorize is a poor tool if you just want to iterate over one variable. It doesn't add speed, and it is hard to apply correctly. It is more useful if you have several variables that you want to broadcast together.
vectorize with 2 arguments:
def foo2(*args):
def foo(*args):
print(args)
return sum(*args)
return np.vectorize(lambda x,y: foo((x,y)+args))
In [164]: foo2(10)(np.arange(2),np.arange(3,5)[:,None])
((0, 3, 10),)
((0, 3, 10),)
((1, 3, 10),)
((0, 4, 10),)
((1, 4, 10),)
Out[164]:
array([[13, 14],
[14, 15]])
In [166]: 10+np.arange(2)+np.arange(3,5)[:,None]
Out[166]:
array([[13, 14],
[14, 15]])
Note that I change the inner print to print(*args), I see (0, 3, 10). You need to be careful when using *args and args - one's a tuple, the other isn't (or is it the other way around?).
If my function returns an array, I get your error:
def foo2(*args):
def foo(*args):
print(*args)
return np.array(*args)
return np.vectorize(lambda x,y: foo((x,y)+args))
.....:
In [197]: foo2(10)(np.arange(2),np.arange(3,5)[:,None])
...
ValueError: setting an array element with a sequence.
but ok with one step
In [203]: foo2(3)(1,2)
(1, 2, 3)
(1, 2, 3)
Out[203]: array([1, 2, 3])
I can specify and object otypes:
def foo2(*args):
def foo(*args):
print(*args)
return np.array(*args)
return np.vectorize(lambda x,y: foo((x,y)+args),otypes=[object])
.....:
In [206]: foo2(3)(1,2)
(1, 2, 3)
Out[206]: array([1, 2, 3], dtype=object)
In [207]: foo2(3)([1,1],[2,3])
(1, 2, 3)
(1, 3, 3)
Out[207]: array([array([1, 2, 3]), array([1, 3, 3])], dtype=object)
In [208]: foo2(10)(np.arange(2),np.arange(3,5)[:,None])
(0, 3, 10)
(1, 3, 10)
(0, 4, 10)
(1, 4, 10)
Out[208]:
array([[array([ 0, 3, 10]), array([ 1, 3, 10])],
[array([ 0, 4, 10]), array([ 1, 4, 10])]], dtype=object)
These can be stacked into an array, though in this 2d result I have to flatten as well (provided the inner arrays are all the same size).
In [225]: np.vstack(np.ravel(A))
Out[225]:
array([[ 0, 3, 10],
[ 1, 3, 10],
[ 0, 4, 10],
[ 1, 4, 10]])
I vaguely recall SO questions about vectorize object otypes, but I don't recall whether there were problems with this (in some versions) or it was just a solution.
Related
I have 2 2D-arrays. I am trying to convolve along the axis 1. np.convolve doesn't provide the axis argument. The answer here, convolves 1 2D-array with a 1D array using np.apply_along_axis. But it cannot be directly applied to my use case. The question here doesn't have an answer.
MWE is as follows.
import numpy as np
a = np.random.randint(0, 5, (2, 5))
"""
a=
array([[4, 2, 0, 4, 3],
[2, 2, 2, 3, 1]])
"""
b = np.random.randint(0, 5, (2, 2))
"""
b=
array([[4, 3],
[4, 0]])
"""
# What I want
c = np.convolve(a, b, axis=1) # axis is not supported as an argument
"""
c=
array([[16, 20, 6, 16, 24, 9],
[ 8, 8, 8, 12, 4, 0]])
"""
I know I can do it using np.fft.fft, but it seems like an unnecessary step to get a simple thing done. Is there a simple way to do this? Thanks.
Why not just do a list comprehension with zip?
>>> np.array([np.convolve(x, y) for x, y in zip(a, b)])
array([[16, 20, 6, 16, 24, 9],
[ 8, 8, 8, 12, 4, 0]])
>>>
Or with scipy.signal.convolve2d:
>>> from scipy.signal import convolve2d
>>> convolve2d(a, b)[[0, 2]]
array([[16, 20, 6, 16, 24, 9],
[ 8, 8, 8, 12, 4, 0]])
>>>
One possibility could be to manually go the way to the Fourier spectrum, and back:
n = np.max([a.shape, b.shape]) + 1
np.abs(np.fft.ifft(np.fft.fft(a, n=n) * np.fft.fft(b, n=n))).astype(int)
# array([[16, 20, 6, 16, 24, 9],
# [ 8, 8, 8, 12, 4, 0]])
Would it be considered too ugly to loop over the orthogonal dimension? That would not add much overhead unless the main dimension is very short. Creating the output array ahead of time ensures that no memory needs to be copied about.
def convolvesecond(a, b):
N1, L1 = a.shape
N2, L2 = b.shape
if N1 != N2:
raise ValueError("Not compatible")
c = np.zeros((N1, L1 + L2 - 1), dtype=a.dtype)
for n in range(N1):
c[n,:] = np.convolve(a[n,:], b[n,:], 'full')
return c
For the generic case (convolving along the k-th axis of a pair of multidimensional arrays), I would resort to a pair of helper functions I always keep on hand to convert multidimensional problems to the basic 2d case:
def semiflatten(x, d=0):
'''SEMIFLATTEN - Permute and reshape an array to convenient matrix form
y, s = SEMIFLATTEN(x, d) permutes and reshapes the arbitrary array X so
that input dimension D (default: 0) becomes the second dimension of the
output, and all other dimensions (if any) are combined into the first
dimension of the output. The output is always 2-D, even if the input is
only 1-D.
If D<0, dimensions are counted from the end.
Return value S can be used to invert the operation using SEMIUNFLATTEN.
This is useful to facilitate looping over arrays with unknown shape.'''
x = np.array(x)
shp = x.shape
ndims = x.ndim
if d<0:
d = ndims + d
perm = list(range(ndims))
perm.pop(d)
perm.append(d)
y = np.transpose(x, perm)
# Y has the original D-th axis last, preceded by the other axes, in order
rest = np.array(shp, int)[perm[:-1]]
y = np.reshape(y, [np.prod(rest), y.shape[-1]])
return y, (d, rest)
def semiunflatten(y, s):
'''SEMIUNFLATTEN - Reverse the operation of SEMIFLATTEN
x = SEMIUNFLATTEN(y, s), where Y, S are as returned from SEMIFLATTEN,
reverses the reshaping and permutation.'''
d, rest = s
x = np.reshape(y, np.append(rest, y.shape[-1]))
perm = list(range(x.ndim))
perm.pop()
perm.insert(d, x.ndim-1)
x = np.transpose(x, perm)
return x
(Note that reshape and transpose do not create copies, so these functions are extremely fast.)
With those, the generic form can be written as:
def convolvealong(a, b, axis=-1):
a, S1 = semiflatten(a, axis)
b, S2 = semiflatten(b, axis)
c = convolvesecond(a, b)
return semiunflatten(c, S1)
I am writing a jury-rigged PyTorch version of scipy.linalg.toeplitz, which currently has the following form:
def toeplitz_torch(c, r=None):
c = torch.tensor(c).ravel()
if r is None:
r = torch.conj(c)
else:
r = torch.tensor(r).ravel()
# Flip c left to right.
idx = [i for i in range(c.size(0)-1, -1, -1)]
idx = torch.LongTensor(idx)
c = c.index_select(0, idx)
vals = torch.cat((c, r[1:]))
out_shp = len(c), len(r)
n = vals.stride(0)
return torch.as_strided(vals[len(c)-1:], size=out_shp, stride=(-n, n)).copy()
But torch.as_strided currently does not support negative strides. My function, therefore, throws the error:
RuntimeError: as_strided: Negative strides are not supported at the moment, got strides: [-1, 1].
My (perhaps incorrect) understanding of as_strided is that it inserts the values of the first argument into a new array whose size is specified by the second argument and it does so by linearly indexing those values in the original array and placing them at subscript-indexed strides given by the final argument.
Both the NumPy and PyTorch documentation concerning as_strided have scary warnings about using the function with "extreme care" and I don't understand this function fully, so I'd like to ask:
Is my understanding of as_strided correct?
Is there a simple way to rewrite this so negative strides work?
Will I be able to pass a gradient w.r.t c (or r) through toeplitz_torch?
> 1. Is my understanding of as_strided correct?
The stride is an interface for your tensor to access the underlying contiguous data buffer. It does not insert values, no copies of the values are done by torch.as_strided, the strides define the artificial layout of what we refer to as multi-dimensional array (in NumPy) or tensor (in PyTorch).
As Andreas K. puts it in another answer:
Strides are the number of bytes to jump over in the memory in order to get from one item to the next item along each direction/dimension of the array. In other words, it's the byte-separation between consecutive items for each dimension.
Please feel free to read the answers over there if you have some trouble with strides. Here we will take your example and look at how it is implemented with as_strided.
The example given by Scipy for linalg.toeplitz is the following:
>>> toeplitz([1,2,3], [1,4,5,6])
array([[1, 4, 5, 6],
[2, 1, 4, 5],
[3, 2, 1, 4]])
To do so they first construct the list of values (what we can refer to as the underlying values, not actually underlying data): vals which is constructed as [3 2 1 4 5 6], i.e. the Toeplitz column and row flattened.
Now notice the arguments passed to np.lib.stride_tricks.as_strided:
values: vals[len(c)-1:] notice the slice: the tensors show up smaller, yet the underlying values remain, and they correspond to those of vals. Go ahead and compare the two with storage_offset: it's just an offset of 2, the values are still there! How this works is that it essentially shifts the indices such that index=0 will refer to value 1, index=1 to 4, etc...
shape: given by the column/row inputs, here (3, 4). This is the shape of the resulting object.
strides: this is the most important piece: (-n, n), in this case (-1, 1)
The most intuitive thing to do with strides is to describe a mapping between the multi-dimensional space: (i, j) ∈ [0,3[ x [0,4[ and the flattened 1D space: k ∈ [0, 3*4[. Since the strides are equal to (-n, n) = (-1, 1), the mapping is -n*i + n*j = -1*i + 1*j = j-i. Mathematically you can describe your matrix as M[i, j] = F[j-i] where F is the flattened values vector [3 2 1 4 5 6].
For instance, let's try with i=1 and j=2. If you look at the Topleitz matrix above M[1, 2] = 4. Indeed F[k] = F[j-i] = F[1] = 4
If you look closely you will see the trick behind negative strides: they allow you to 'reference' to negative indices: for instance, if you take j=0 and i=2, then you see k=-2. Remember how vals was given with an offset of 2 by slicing vals[len(c)-1:]. If you look at its own underlying data storage it's still [3 2 1 4 5 6], but has an offset. The mapping for vals (in this case i: 1D -> k: 1D) would be M'[i] = F'[k] = F'[i+2] because of the offset. This means M'[-2] = F'[0] = 3.
In the above I defined M' as vals[len(c)-1:] which basically equivalent to the following tensor:
>>> torch.as_strided(vals, size=(len(vals)-2,), stride=(1,), storage_offset=2)
tensor([1, 4, 5, 6])
Similarly, I defined F' as the flattened vector of underlying values: [3 2 1 4 5 6].
The usage of strides is indeed a very clever way to define a Toeplitz matrix!
> 2. Is there a simple way to rewrite this so negative strides work?
The issue is, negative strides are not implemented in PyTorch... I don't believe there is a way around it with torch.as_strided, otherwise it would be rather easy to extend the current implementation and provide support for that feature.
There are however alternative ways to solve the problem. It is entirely possible to construct a Toeplitz matrix in PyTorch, but that won't be with torch.as_strided.
We will do the mapping ourselves: for each element of M indexed by (i, j), we will find out the corresponding index k which is simply j-i. This can be done with ease, first by gathering all (i, j) pairs from M:
>>> i, j = torch.ones(3, 4).nonzero().T
(tensor([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2]),
tensor([0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]))
Now we essentially have k:
>>> j-i
tensor([ 0, 1, 2, 3, -1, 0, 1, 2, -2, -1, 0, 1])
We just need to construct a flattened tensor of all possible values from the row r and column c inputs. Negative indexed values (the content of c) are put last and flipped:
>>> values = torch.cat((r, c[1:].flip(0)))
tensor([1, 4, 5, 6, 3, 2])
Finally index values with k and reshape:
>>> values[j-i].reshape(3, 4)
tensor([[1, 4, 5, 6],
[2, 1, 4, 5],
[3, 2, 1, 4]])
To sum it up, my proposed implementation would be:
def toeplitz(c, r):
vals = torch.cat((r, c[1:].flip(0)))
shape = len(c), len(r)
i, j = torch.ones(*shape).nonzero().T
return vals[j-i].reshape(*shape)
> 3. Will I be able to pass a gradient w.r.t c (or r) through toeplitz_torch?
That's an interesting question because torch.as_strided doesn't have a backward function implemented. This means you wouldn't have been able to backpropagate to c and r! With the above method, however, which uses 'backward-compatible' builtins, the backward pass comes free of charge.
Notice the grad_fn on the output:
>>> toeplitz(torch.tensor([1.,2.,3.], requires_grad=True),
torch.tensor([1.,4.,5.,6.], requires_grad=True))
tensor([[1., 4., 5., 6.],
[2., 1., 4., 5.],
[3., 2., 1., 4.]], grad_fn=<ViewBackward>)
This was a quick draft (that did take a little while to write down), I will make some edits. If you have some questions or remarks, don't hesitate to comment! I would be interested in seeing other answers as I am not an expert with strides, this is just my take on the problem.
Let's say I have an array 3x3 a and would like to upsample it to a 30x30 array b with nearest neighbor interpolation.
Is it possible to use a technique which does not actually store repeated values? Something similar on how broadcasting works in numpy.
e.g. I would like to have an object such that when I call b[x, x] with 0 < x < 10 I get a[0, 0].
I don't believe there is any way to do this using numpy. The way broadcasting works in numpy is that each axis has a "stride" parameter which controls how to calculate the next element along the axis. So for example:
In [1]: a = np.arange(10)
In [2]: a
Out[2]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
In [3]: b = a[::2]
In [4]: b
Out[4]: array([0, 2, 4, 6, 8])
In [5]: b.strides
Out[5]: (16,)
In [6]: a.strides
Out[6]: (8,)
In this case you can see that b is just a view of a obtained by doubling the stride along the first dimension. So when you access b[1], the offset is calculated as b.__array_interface__['data'][0] + b.strides[0].
In your case you essentially want a nonlinear stride, which isn't supported.
You could of course achieve this affect yourself by calculating the indexes yourself, i.e.:
a[x//10,x//10]
You could achieve an object like the one you describe by creating a class that wraps a numpy array and implements a custom __getitem__ method. That could look something like the below, where factor is the factor with which you want to upsample.
class UpSampled:
__slots__ = ('arr', 'factor')
def __init__(self, arr, factor):
self.arr = arr
self.factor = factor
def __getitem__(self, key):
return self.arr[key // self.factor]
You would then use it like below:
o = UpSampled(np.array([
UpSampled(np.array([0, 1, 2]), 10),
UpSampled(np.array([3, 4, 5]), 10),
UpSampled(np.array([6, 7, 8]), 10),
]), 10)
print(o[23][13]) # prints 7
If you need the object to be iterable you would then also implement __next__ and __iter__:
class UpSampled:
__slots__ = ('arr', 'factor', '__index')
def __init__(self, arr, factor):
self.arr = arr
self.factor = factor
self.__index = 0
def __getitem__(self, key):
return self.arr[key // self.factor]
def __iter__(self):
self.__index = 0
return self
def __next__(self):
try:
result = self[self.__index]
except IndexError:
raise StopIteration
self.__index += 1
return result
Though I'm not sure that would work well with libraries that specifically expect a numpy array.
Is it generally safe to provide the input array as the optional out argument to a ufunc in numpy, provided the type is correct? For example, I have verified that the following works:
>>> import numpy as np
>>> arr = np.array([1.2, 3.4, 4.5])
>>> np.floor(arr, arr)
array([ 1., 3., 4.])
The array type must be either compatible or identical with the output (which is a float for numpy.floor()), or this happens:
>>> arr2 = np.array([1, 3, 4], dtype = np.uint8)
>>> np.floor(arr2, arr2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: ufunc 'floor' output (typecode 'e') could not be coerced to provided output parameter (typecode 'B') according to the casting rule ''same_kind''
So given that an array of proper type, is it generally safe to apply ufuncs in-place? Or is floor() an exceptional case? The documentation does not make it clear, and neither do the following two threads that have tangential bearing on the question:
Numpy modify array in place?
Numpy Ceil and Floor "out" Argument
EDIT:
As a first order guess, I would assume it is often, but not always safe, based on the tutorial at http://docs.scipy.org/doc/numpy/user/c-info.ufunc-tutorial.html. There does not appear to be any restriction on using the output array as a temporary holder for intermediate results during the computation. While something like floor() and ciel() may not require temporary storage, more complex functions might. That being said, the entire existing library may be written with that in mind.
The out parameter of a numpy function is the array where the result is written. The main advantage of using out is avoiding the allocation of new memory where it is not necessary.
Is it safe to use write the output of a function on the same array passed as input? There is no general answer, it depends on what the function is doing.
Two examples
Here are two examples of ufunc-like functions:
In [1]: def plus_one(x, out=None):
...: if out is None:
...: out = np.zeros_like(x)
...:
...: for i in range(x.size):
...: out[i] = x[i] + 1
...: return out
...:
In [2]: x = np.arange(5)
In [3]: x
Out[3]: array([0, 1, 2, 3, 4])
In [4]: y = plus_one(x)
In [5]: y
Out[5]: array([1, 2, 3, 4, 5])
In [6]: z = plus_one(x, x)
In [7]: z
Out[7]: array([1, 2, 3, 4, 5])
Function shift_one:
In [11]: def shift_one(x, out=None):
...: if out is None:
...: out = np.zeros_like(x)
...:
...: n = x.size
...: for i in range(n):
...: out[(i+1) % n] = x[i]
...: return out
...:
In [12]: x = np.arange(5)
In [13]: x
Out[13]: array([0, 1, 2, 3, 4])
In [14]: y = shift_one(x)
In [15]: y
Out[15]: array([4, 0, 1, 2, 3])
In [16]: z = shift_one(x, x)
In [17]: z
Out[17]: array([0, 0, 0, 0, 0])
For the function plus_one there is no problem: the expected result is obtained when the parameters x and out are the same array. But the function shift_one gives a surprising result when the parameters x and out are the same array because the array
Discussion
For function of the form out[i] := some_operation(x[i]), such as plus_one above but also the functions floor, ceil, sin, cos, tan, log, conj, etc, as far as I know it is safe to write the result in the input using parameter out.
It is also safe for functions taking two input parameters of the form ``out[i] := some_operation(x[i], y[i]) such as the numpy function add, multiply, subtract.
For the other functions, it is case-by-case. As illustrated bellow, the matrix multiplication is not safe:
In [18]: a = np.arange(4).reshape((2,2))
In [19]: a
Out[19]:
array([[0, 1],
[2, 3]])
In [20]: b = (np.arange(4) % 2).reshape((2,2))
In [21]: b
Out[21]:
array([[0, 1],
[0, 1]], dtype=int32)
In [22]: c = np.dot(a, b)
In [23]: c
Out[23]:
array([[0, 1],
[0, 5]])
In [24]: d = np.dot(a, b, out=a)
In [25]: d
Out[25]:
array([[0, 1],
[0, 3]])
Last remark: if the implementation is multithreaded, the result of an unsafe function may even be non-deterministic because it depends on the order on which the array elements are processed.
This is an old question, but there is an updated answer:
Yes, it is safe. In the Numpy documentation, we see that as of v1.13:
Operations where ufunc input and output operands have memory overlap are defined to be the same as for equivalent operations where there is no memory overlap. Operations affected make temporary copies as needed to eliminate data dependency. As detecting these cases is computationally expensive, a heuristic is used, which may in rare cases result in needless temporary copies. For operations where the data dependency is simple enough for the heuristic to analyze, temporary copies will not be made even if the arrays overlap, if it can be deduced copies are not necessary. As an example, np.add(a, b, out=a) will not involve copies.
Is there a function in numpy or scipy (or some other library) that generalizes the idea of cumsum and cumprod to arbitrary function. For example, consider the (theoretical) function
cumf( func, array)
func is a function that accepts two floats, and returns a float. Particular cases
lambda x,y: x+y
and
lambda x,y: x*y
are cumsum and cumprod respectively. For example, if
func = lambda x,prev_x: x^2*prev_x
and I apply it to:
cumf(func, np.array( 1, 2, 3) )
I would like
np.array( 1, 4, 9*4 )
The ValueError above is still a bug using Numpy 1.20.1 (with Python 3.9.1).
Luckily a workaround was discovered that uses casting:
https://groups.google.com/forum/#!topic/numpy/JgUltPe2hqw
import numpy as np
uadd = np.frompyfunc(lambda x, y: x + y, 2, 1)
uadd.accumulate([1,2,3], dtype=object).astype(int)
# array([1, 3, 6])
Note that since the custom operation works on an object type, it won't benefit from the efficient memory management of numpy. So the operation may be slower than one that didn't need casting to object for extremely large arrays.
NumPy's ufuncs have accumulate():
In [22]: np.multiply.accumulate([[1, 2, 3], [4, 5, 6]], axis=1)
Out[22]:
array([[ 1, 2, 6],
[ 4, 20, 120]])
Unfortunately, calling accumulate() on a frompyfunc()'ed Python function fails with a strange error:
In [32]: uadd = np.frompyfunc(lambda x, y: x + y, 2, 1)
In [33]: uadd.accumulate([1, 2, 3])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
ValueError: could not find a matching type for <lambda> (vectorized).accumulate,
requested type has type code 'l'
This is using NumPy 1.6.1 with Python 2.7.3.