I have a complicated linear system $y = Ax$ where I cannot specify the matrix A, I can however write a function that computes Ax, and I have made this into a linear operator.
I need to find $A^T$.
I have tried finding $A^T$ by hand but it is becoming tricky.
I found that scipy has a built in function .transpose(), I have tried using this with a simple example,
def mv(v):
return np.array([2*v[0]- v[1], 3*v[1]])
A = LinearOperator((2,2), matvec=mv)
C = A.transpose()
but then when I try to use this it doesn't seem to work. I tried comparing the results
A.matvec(np.ones(2))
array([1., 3.])
C.rmatvec(np.ones(2))
array([1., 3.])
but the results are the same? I'm not sure why this is, surely the second result should be [2, 2].
For a 2d array, M:
In [28]: M = np.array([[1,3,2],[2,1,2],[5,2,1]]); x=np.array([1,2,3])
The transpose lets us switch the order:
In [29]: M#x
Out[29]: array([13, 10, 12])
In [30]: x#M.T
Out[30]: array([13, 10, 12])
Is that what your C does? Implementing the rmatvec inplace of the A.matvec.
Trying to use matvec on C produces an error. C itself is a LinearOperator, one that (somehow) references the operations defined for A.
In [40]: C.matvec([1,1])
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
Cell In[40], line 1
----> 1 C.matvec([1,1])
File ~\miniconda3\lib\site-packages\scipy\sparse\linalg\_interface.py:232, in LinearOperator.matvec(self, x)
229 if x.shape != (N,) and x.shape != (N,1):
230 raise ValueError('dimension mismatch')
--> 232 y = self._matvec(x)
234 if isinstance(x, np.matrix):
235 y = asmatrix(y)
File ~\miniconda3\lib\site-packages\scipy\sparse\linalg\_interface.py:583, in _TransposedLinearOperator._matvec(self, x)
581 def _matvec(self, x):
582 # NB. np.conj works also on sparse matrices
--> 583 return np.conj(self.A._rmatvec(np.conj(x)))
File ~\miniconda3\lib\site-packages\scipy\sparse\linalg\_interface.py:535, in _CustomLinearOperator._rmatvec(self, x)
533 func = self.__rmatvec_impl
534 if func is None:
--> 535 raise NotImplementedError("rmatvec is not defined")
536 return self.__rmatvec_impl(x)
NotImplementedError: rmatvec is not defined
The key is that for C, matvec is implemented with a A.rmatvec:
np.conj(self.A._rmatvec(np.conj(x)))
I haven't worked a lot with this LinearOperator class, but I view it was an abstract class that can be used in iterative solvers in much the same as a 'conventional' 2d array, except that all it has to define is one or more operations like matvec. In your case it's the mv function.
From the LinearOpertor docs:
shape : tuple
Matrix dimensions (M, N).
matvec : callable f(v)
Returns returns A * v.
rmatvec : callable f(v)
Returns A^H * v, where A^H is the conjugate transpose of A.
contrived example
Define a rmatvec for A:
In [51]: def mv(v):
...: return np.array([2*v[0]- v[1], 3*v[1]])
...: def rmv(v):
...: return np.array([2*v[1]- v[0], 3*v[0]])
...: A = linalg.LinearOperator((2,2), matvec=mv, rmatvec=rmv)
...: C = A.transpose()
In [52]: A.matvec((1,6))
Out[52]: array([-4, 18])
In [53]: A.rmatvec((1,6))
Out[53]: array([11, 3])
In [54]: A.rmatvec((6,1))
Out[54]: array([-4, 18])
Notice how C just switches the roles of matvec and rmatvec:
In [55]: C.matvec((1,6))
Out[55]: array([11, 3])
In [56]: C.rmatvec((1,6))
Out[56]: array([-4, 18])
transpose will also change the shape of the operator. If A takes (2,) and returns a (3,), it has shape (3,2); C has shape (2,3), consistent with being a right-hand operator:
In [58]: def mv(v):
...: return np.array([2*v[0]- v[1], 3*v[1],0])
...: A = linalg.LinearOperator((3,2), matvec=mv)
...: C = A.transpose()
In [59]: A
Out[59]: <3x2 _CustomLinearOperator with dtype=float64>
In [60]: C
Out[60]: <2x3 _TransposedLinearOperator with dtype=float64>
To find the transpose of a function you would need to use automatic differentiation which python has in built tools for. I managed, in the end, to find the transpose of my linear operator by hand.
Related
I have to write a python function where i need to compute the vector
For A is n by n and xn is n by 1
r_n = Axn - (xn^TAxn)xn
Im using numpy but .T doesn't work on vectors and when I just do
r_n = A#xn - (xn#A#xn)#xn but xn#A#xn gives me a scaler.
I've tried changing the A with the xn but nothing seems to work.
Making a 3x1 numpy array like this...
import numpy as np
a = np.array([1, 2, 3])
...and then attempting to take its transpose like this...
a_transpose = a.T
...will, confusingly, return this:
# [1 2 3]
If you want to define a (column) vector whose transpose you can meaningfully take, and get a row vector in return, you need to define it like this:
a = np.reshape(np.array([1, 2, 3]), (3, 1))
print(a)
# [[1]
# [2]
# [3]]
a_transpose = a.T
print(a_transpose)
# [[1 2 3]]
If you want to define a 1 x n array whose transpose you can take to get an n x 1 array, you can do it like this:
a = np.array([[1, 2, 3]])
and then get its transpose by calling a.T.
If A is (n,n) and xn is (n,1):
A#xn - (xn#A#xn)#xn
(n,n)#(n,1) - ((n,1)#(n,n)#(n,1)) # (n,1)
(n,1) error (1 does not match n)
If xn#A#xn gives scalar that's because xn is (n,) shape; as per np.matmul docs that's a 2d with two 1d arrays
(n,)#(n,n)#(n,) => (n,)#(n,) -> scalar
I think you want
(1,n) # (n,n) # (n,1) => (1,1)
Come to think of it that (1,1) array should be same single values as the scalar.
Sample calculation; 1st with the (n,) shape:
In [6]: A = np.arange(1,10).reshape(3,3); x = np.arange(1,4)
In [7]: A#x
Out[7]: array([14, 32, 50]) # (3,3)#(3,)=>(3,)
In [8]: x#A#x # scalar
Out[8]: 228
In [9]: (x#A#x)#x
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[9], line 1
----> 1 (x#A#x)#x
ValueError: matmul: Input operand 0 does not have enough dimensions (has 0, gufunc core with signature (n?,k),(k,m?)->(n?,m?) requires 1)
matmul does not like to work with scalars. But we can use np.dot instead, or simply multiply:
In [10]: (x#A#x)*x
Out[10]: array([228, 456, 684]) # (3,)
In [11]: A#x - (x#A#x)*x
Out[11]: array([-214, -424, -634])
Change the array to (3,1):
In [12]: xn = x[:,None]; xn.shape
Out[12]: (3, 1)
In [13]: A#xn - (xn.T#A#xn)*xn
Out[13]:
array([[-214],
[-424],
[-634]]) # same numbers but in (3,1) shape
I have a 2 dimensional numpy matrix whose type is numpy.ndarray of uint8.
Now when I perform np.square on the array, it does not return the expected result.
Here's an example of the code on the console and its output after I create the numpy array:
arr[0, 0] outputs 203
Now,
np.square(203) outputs 41209
But,
np.square(arr[0, 0]) outputs 249
This odd behavior was also observed for the np.sqrt method.
According to numpy.org np.uint8is Unsigned integer (0 to 255)
import numpy as np
arr = np.array([[203, 32, 45, 34], [34,322,780,54]])
arr1 = arr.astype('uint8')
arr2 = arr.astype('uint16')
sq = np.square(203)
sq8 = np.square(arr1[0,0])
sq16 = np.square(arr2[0,0])
sq, sq8, sq16, type(sq), type(sq8), type(sq16)
output:
(41209, 249, 41209, numpy.intc, numpy.uint8, numpy.uint16)
41209 is 249 in uint8:
num = np.array([41209])
num1 = arr.astype('uint8')
num1
>>> array([249], dtype=uint8)
np.square like other ufunc can return results with various dtypes. Looks like it 'prefers' to return a matching dtype:
In [109]: np.square(np.array([203], 'uint8'),dtype=int)
Out[109]: array([41209])
In [110]: np.square(np.array([203], 'uint8'),dtype='uint8')
Out[110]: array([249], dtype=uint8)
In [111]: np.square(np.array([203], 'uint8'))
Out[111]: array([249], dtype=uint8)
In [112]: np.square(np.array([203], 'uint8'),dtype='uint8')
Out[112]: array([249], dtype=uint8)
In [113]: np.square(np.array([203], 'uint8'),dtype='uint16')
Out[113]: array([41209], dtype=uint16)
In [114]: np.square(np.array([203], 'uint8'),dtype='int')
Out[114]: array([41209])
In [115]: np.square(np.array([203], 'uint8'),dtype='float')
Out[115]: array([41209.])
https://numpy.org/doc/stable/reference/ufuncs.html#casting-rules
and under signature parameter.
In [118]: np.square.types
Out[118]:
['b->b',
'B->B', # uint8
'h->h',
'H->H',
'i->i',
...]
There are more details in handling of casting and dtypes, but the basic point is that if using 'exotic' dtypes like unit8 beware of overflow etc. Even with 'int32' overflow can be problem, but at much larger numbers. A more common problem arise when doing some calculation on integers that results in floats.
A recent SO about dtypes with /= operator.
numpy.array's have bizarre behavior with /= operator?
Many times I've had to ask SO questions - what's the shape, what's the dtype. Those are fundamental properties of a numpy array. Getting those right is 80% of the debugging battle.
So I would like to apply a function over two 3d matrixes with numpy and I can't figure out how. I read about numpy.apply_over_axes() but can't make it work.
This is my code now:
c = np.random.beta(2,3,size=(2,80))
def my_func(a,b):
xi = np.matmul(b, c)
spe = np.power(a - xi, 2)
return spe.sum()
a = np.zeros(shape=(5,1000,80))
b = np.random.beta(2,3,size=(5,1000,2))
np.apply_over_axes(func=my_func,a=[a,b],axes=[0,0,0])
Which doesnt work and returns
could not broadcast input array from shape (5,1000,80) into shape (5,1000)
I will like to iterate though a and b and apply my_func to every vector of the 3rd dimension.
This would do the job but with normal for loops:
results = []
for i in range(len(a)): #5 Iterations
for j in range(len(a[i])): #1000 Iterations
results.append(my_func(a[i][j], b[i][j]))
I would like to obtain this results this but using numpy functions.
The contraction operation hidden in np.matmul(b, c) can be achieved through np.tensordot(b, c, axes=[2, 0]), where the [2, 0] indicates that the third axis in b is contracted with the first axis in c. That is, np.tensordot(b, c, axes=[2, 0]).shape is (5, 1000, 80). From there on, ordinary broadcasting applies, and your code boils down to
a = np.zeros(shape=(5, 1000, 80))
b = np.random.beta(2, 3, size=(5, 1000, 2))
c = np.random.beta(2, 3, size=(2, 80))
xi = np.tensordot(b, c, axes=[2, 0])
spe = np.power(a - xi, 2)
results2 = spe.sum(axis=2)
Let's check that this indeed matches what you get by simply using loops:
In [55]: results = np.array(results).reshape(5, 1000)
In [56]: np.allclose(results, results2)
Out[56]: True
Running your code in an ipython session:
In [88]: c = np.random.beta(2,3,size=(2,80))
...:
...: def my_func(a,b):
...: xi = np.matmul(b, c)
...:
...: spe = np.power(a - xi, 2)
...: return spe.sum()
...:
...: a = np.zeros(shape=(5,1000,80))
...: b = np.random.beta(2,3,size=(5,1000,2))
...:
...: np.apply_over_axes(func=my_func,a=[a,b],axes=[0,0,0])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-88-c5e5a66c9d0a> in <module>
10 b = np.random.beta(2,3,size=(5,1000,2))
11
---> 12 np.apply_over_axes(func=my_func,a=[a,b],axes=[0,0,0])
<__array_function__ internals> in apply_over_axes(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/numpy/lib/shape_base.py in apply_over_axes(func, a, axes)
485
486 """
--> 487 val = asarray(a)
488 N = a.ndim
489 if array(axes).ndim == 0:
/usr/local/lib/python3.6/dist-packages/numpy/core/_asarray.py in asarray(a, dtype, order)
83
84 """
---> 85 return array(a, dtype, copy=False, order=order)
86
87
ValueError: could not broadcast input array from shape (5,1000,80) into shape (5,1000)
You should have shown us the full error with traceback.
That traceback shows us that it is trying make one array from your list of two. Since the shapes don't match, it raises an error. With a different mismatch it would have created a (2,) object array, which would just move raise problems later:
In [89]: np.array([a,b])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-89-964832cdcfcd> in <module>
----> 1 np.array([a,b])
ValueError: could not broadcast input array from shape (5,1000,80) into shape (5,1000)
But the problem is you did not read the docs, or take them seriously:
func : function
This function must take two arguments, `func(a, axis)`.
a : array_like
Input array.
axes : array_like
Axes over which `func` is applied; the elements must be integers.
a is supposed to be an array, not a list of two arrays. func is supposed to take an axis parameter, not another array. And I don't know what you are trying to do with [0,0,0]. For 3d arrays [0,1] might apply, but not a repeated 0.
your loop
With a bit better numpy style:
In [91]: results = []
...: for i in range(a.shape[0]): #5 Iterations
...: for j in range(a.shape[1]): #1000 Iterations
...: results.append(my_func(a[i,j], b[i,j]))
...:
In [92]: np.array(results).shape
Out[92]: (5000,)
rework my_func
To do this without the loops, we need to use whole-array functions within my_func. There isn't a numpy apply that compiles python code - you have to look at numba or cython for that.
xi=np.matmul(b,c). b is (5,1000,2), c is (2,80). matmul is happy doing a dot combining the last axis of b with the 2nd to the last of c.
In [93]: xi = np.matmul(b,c)
In [94]: xi.shape
Out[94]: (5, 1000, 80)
That matches a, so
In [97]: spe = np.power(a-xi,2)
In [98]: spe.shape
Out[98]: (5, 1000, 80)
then sum on that last axis:
In [99]: res = spe.sum(axis=2)
In [100]: res.shape
Out[100]: (5, 1000)
which matches your loop:
In [101]: np.allclose(res.ravel(), np.array(results))
Out[101]: True
Except for the last sum, your myfunc runs with the whole arrays.
In [103]: my_func(a,b)
Out[103]: 46883.49325596101
I am new to python/numpy.
I need to do the following calculation:
for an array of discrete times t, calculate $e^{At}$ for a $2\times 2$ matrix $A$
What I did:
def calculate(t_,x_0,v_0,omega_0,c):
# define A
a_11,a_12, a_21, a_22=0,1,-omega_0^2,-c
A =np.matrix([[a_11,a_12], [a_21, a_22]])
print A
# use vectorization
temps = np.array(t_)
A_ = np.array([A for k in range (1,n+1,1)])
temps*A_
x_=scipy.linalg.expm(temps*A)
v_=A*scipy.linalg.expm(temps*A)
return x_,v_
n=10
omega_0=1
c=1
x_0=1
v_0=1
t_ = [float(5*k*np.pi/n) for k in range (1,n+1,1)]
x_, v_ = calculate(t_,x_0,v_0,omega_0,c)
However, I get this error when multiplying A_ (array containing n times A ) and temps (containg the times for which I want to calculate exp(At) :
ValueError: operands could not be broadcast together with shapes (10,) (10,2,2)
As I understand vectorization, each element in A_ would be multiplied by element at the same index from temps; but I think i don't get it right.
Any help/ comments much appreciated
A pure numpy calculation of t_ is (creates an array instead of a list):
In [254]: t = 5*np.arange(1,n+1)*np.pi/n
In [255]: t
Out[255]:
array([ 1.57079633, 3.14159265, 4.71238898, 6.28318531, 7.85398163,
9.42477796, 10.99557429, 12.56637061, 14.13716694, 15.70796327])
In [256]: a_11,a_12, a_21, a_22=0,1,-omega_0^2,-c
In [257]: a_11
Out[257]: 0
In [258]: A = np.array([[a_11,a_12], [a_21, a_22]])
In [259]: A
Out[259]:
array([[ 0, 1],
[-3, -1]])
In [260]: t.shape
Out[260]: (10,)
In [261]: A.shape
Out[261]: (2, 2)
In [262]: A_ = np.array([A for k in range (1,n+1,1)])
In [263]: A_.shape
Out[263]: (10, 2, 2)
A_ is np.ndarray. I made A a np.ndarray as well; yours is np.matrix, but your A_ will still be np.ndarray. np.matrix can only be 2d, where as A_ is 3d.
So t * A will be array elementwise multiplication, hence the broadcasting error, (10,) (10,2,2).
To do that elementwise multiplication right you need something like
In [264]: result = t[:,None,None]*A[None,:,:]
In [265]: result.shape
Out[265]: (10, 2, 2)
But if you want matrix multiplication of the (10,) with (10,2,2), then einsum does it easily:
In [266]: result1 = np.einsum('i,ijk', t, A_)
In [267]: result1
Out[267]:
array([[ 0. , 86.39379797],
[-259.18139392, -86.39379797]])
np.dot can't do it because its rule is 'last with 2nd to last'. tensordot can, but I'm more comfortable with einsum.
But that einsum expression makes it obvious (to me) that I can get the same thing from the elementwise *, by summing on the 1st axis:
In [268]: (t[:,None,None]*A[None,:,:]).sum(axis=0)
Out[268]:
array([[ 0. , 86.39379797],
[-259.18139392, -86.39379797]])
Or (t[:,None,None]*A[None,:,:]).cumsum(axis=0) to get a 2x2 for each time.
This is what I would do.
import numpy as np
from scipy.linalg import expm
A = np.array([[1, 2], [3, 4]])
for t in np.linspace(0, 5*np.pi, 20):
print(expm(t*A))
No attempt at vectorization here. The expm function applies to one matrix at a time, and it surely takes the bulk of computation time. No need to worry about the cost of multiplying A by a scalar.
Is it generally safe to provide the input array as the optional out argument to a ufunc in numpy, provided the type is correct? For example, I have verified that the following works:
>>> import numpy as np
>>> arr = np.array([1.2, 3.4, 4.5])
>>> np.floor(arr, arr)
array([ 1., 3., 4.])
The array type must be either compatible or identical with the output (which is a float for numpy.floor()), or this happens:
>>> arr2 = np.array([1, 3, 4], dtype = np.uint8)
>>> np.floor(arr2, arr2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: ufunc 'floor' output (typecode 'e') could not be coerced to provided output parameter (typecode 'B') according to the casting rule ''same_kind''
So given that an array of proper type, is it generally safe to apply ufuncs in-place? Or is floor() an exceptional case? The documentation does not make it clear, and neither do the following two threads that have tangential bearing on the question:
Numpy modify array in place?
Numpy Ceil and Floor "out" Argument
EDIT:
As a first order guess, I would assume it is often, but not always safe, based on the tutorial at http://docs.scipy.org/doc/numpy/user/c-info.ufunc-tutorial.html. There does not appear to be any restriction on using the output array as a temporary holder for intermediate results during the computation. While something like floor() and ciel() may not require temporary storage, more complex functions might. That being said, the entire existing library may be written with that in mind.
The out parameter of a numpy function is the array where the result is written. The main advantage of using out is avoiding the allocation of new memory where it is not necessary.
Is it safe to use write the output of a function on the same array passed as input? There is no general answer, it depends on what the function is doing.
Two examples
Here are two examples of ufunc-like functions:
In [1]: def plus_one(x, out=None):
...: if out is None:
...: out = np.zeros_like(x)
...:
...: for i in range(x.size):
...: out[i] = x[i] + 1
...: return out
...:
In [2]: x = np.arange(5)
In [3]: x
Out[3]: array([0, 1, 2, 3, 4])
In [4]: y = plus_one(x)
In [5]: y
Out[5]: array([1, 2, 3, 4, 5])
In [6]: z = plus_one(x, x)
In [7]: z
Out[7]: array([1, 2, 3, 4, 5])
Function shift_one:
In [11]: def shift_one(x, out=None):
...: if out is None:
...: out = np.zeros_like(x)
...:
...: n = x.size
...: for i in range(n):
...: out[(i+1) % n] = x[i]
...: return out
...:
In [12]: x = np.arange(5)
In [13]: x
Out[13]: array([0, 1, 2, 3, 4])
In [14]: y = shift_one(x)
In [15]: y
Out[15]: array([4, 0, 1, 2, 3])
In [16]: z = shift_one(x, x)
In [17]: z
Out[17]: array([0, 0, 0, 0, 0])
For the function plus_one there is no problem: the expected result is obtained when the parameters x and out are the same array. But the function shift_one gives a surprising result when the parameters x and out are the same array because the array
Discussion
For function of the form out[i] := some_operation(x[i]), such as plus_one above but also the functions floor, ceil, sin, cos, tan, log, conj, etc, as far as I know it is safe to write the result in the input using parameter out.
It is also safe for functions taking two input parameters of the form ``out[i] := some_operation(x[i], y[i]) such as the numpy function add, multiply, subtract.
For the other functions, it is case-by-case. As illustrated bellow, the matrix multiplication is not safe:
In [18]: a = np.arange(4).reshape((2,2))
In [19]: a
Out[19]:
array([[0, 1],
[2, 3]])
In [20]: b = (np.arange(4) % 2).reshape((2,2))
In [21]: b
Out[21]:
array([[0, 1],
[0, 1]], dtype=int32)
In [22]: c = np.dot(a, b)
In [23]: c
Out[23]:
array([[0, 1],
[0, 5]])
In [24]: d = np.dot(a, b, out=a)
In [25]: d
Out[25]:
array([[0, 1],
[0, 3]])
Last remark: if the implementation is multithreaded, the result of an unsafe function may even be non-deterministic because it depends on the order on which the array elements are processed.
This is an old question, but there is an updated answer:
Yes, it is safe. In the Numpy documentation, we see that as of v1.13:
Operations where ufunc input and output operands have memory overlap are defined to be the same as for equivalent operations where there is no memory overlap. Operations affected make temporary copies as needed to eliminate data dependency. As detecting these cases is computationally expensive, a heuristic is used, which may in rare cases result in needless temporary copies. For operations where the data dependency is simple enough for the heuristic to analyze, temporary copies will not be made even if the arrays overlap, if it can be deduced copies are not necessary. As an example, np.add(a, b, out=a) will not involve copies.