We Keep Coding

Writing a Transpose a vector in python

I have to write a python function where i need to compute the vector
For A is n by n and xn is n by 1
r_n = Axn - (xn^TAxn)xn
Im using numpy but .T doesn't work on vectors and when I just do
r_n = A#xn - (xn#A#xn)#xn but xn#A#xn gives me a scaler.
I've tried changing the A with the xn but nothing seems to work.

Making a 3x1 numpy array like this...
import numpy as np
a = np.array([1, 2, 3])
...and then attempting to take its transpose like this...
a_transpose = a.T
...will, confusingly, return this:
# [1 2 3]
If you want to define a (column) vector whose transpose you can meaningfully take, and get a row vector in return, you need to define it like this:
a = np.reshape(np.array([1, 2, 3]), (3, 1))
print(a)
# [[1]
# [2]
# [3]]
a_transpose = a.T
print(a_transpose)
# [[1 2 3]]
If you want to define a 1 x n array whose transpose you can take to get an n x 1 array, you can do it like this:
a = np.array([[1, 2, 3]])
and then get its transpose by calling a.T.

If A is (n,n) and xn is (n,1):
A#xn - (xn#A#xn)#xn
(n,n)#(n,1) - ((n,1)#(n,n)#(n,1)) # (n,1)
(n,1) error (1 does not match n)
If xn#A#xn gives scalar that's because xn is (n,) shape; as per np.matmul docs that's a 2d with two 1d arrays
(n,)#(n,n)#(n,) => (n,)#(n,) -> scalar
I think you want
(1,n) # (n,n) # (n,1) => (1,1)
Come to think of it that (1,1) array should be same single values as the scalar.
Sample calculation; 1st with the (n,) shape:
In [6]: A = np.arange(1,10).reshape(3,3); x = np.arange(1,4)
In [7]: A#x
Out[7]: array([14, 32, 50]) # (3,3)#(3,)=>(3,)
In [8]: x#A#x # scalar
Out[8]: 228
In [9]: (x#A#x)#x
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[9], line 1
----> 1 (x#A#x)#x
ValueError: matmul: Input operand 0 does not have enough dimensions (has 0, gufunc core with signature (n?,k),(k,m?)->(n?,m?) requires 1)
matmul does not like to work with scalars. But we can use np.dot instead, or simply multiply:
In [10]: (x#A#x)*x
Out[10]: array([228, 456, 684]) # (3,)
In [11]: A#x - (x#A#x)*x
Out[11]: array([-214, -424, -634])
Change the array to (3,1):
In [12]: xn = x[:,None]; xn.shape
Out[12]: (3, 1)
In [13]: A#xn - (xn.T#A#xn)*xn
Out[13]:
array([[-214],
[-424],
[-634]]) # same numbers but in (3,1) shape

Numpy broadcasting and structured dtype: how to handle a vector as an entity?

I'm fairly new to NumPy, so it's quite possible that I'm missing something fundamental. Don't hesitate to ask "stupid" questions about "basic" things!
I'm trying to write some functions that manipulate vectors. I'd like them to work on single vectors, as well as on arrays of vectors, like most of NumPy's ufuncs:
import math
import numpy
def func(scalar, x, vector):
# arbitrary function
# I'm NOT looking to replace this with numpy.magic_sum_multiply()
# I'm trying to understand broadcasting & dtypes
return scalar * x + vector
print(func(
scalar=numpy.array(2),
x=numpy.array([1, 0, 0]),
vector=numpy.array([1, 0, 0]),
))
# => [3 0 0], as expected
print(func(
scalar=numpy.array(2),
x=numpy.array([1, 0, 0]),
vector=numpy.array([[1, 0, 0], [0, 1, 0]]),
))
# => [[3 0 0], [2 1 0]], as expected. x & scalar are broadcasted out to match the multiple vectors
However, when trying to use multiple scalars, things go wrong:
print(func(
scalar=numpy.array([1, 2]),
x=numpy.array([1, 0, 0]),
vector=numpy.array([1, 0, 0]),
))
# => ValueError: operands could not be broadcast together with shapes (2,) (3,)
# expected: [[2 0 0], [3 0 0]]
I'm not entirely surprised be this. After all, NumPy has no idea that I'm working with vectors that are an single entity, and not an arbitrary dimension.
I can solve this ad-hoc with some expand_dims() and/or squeeze() to add/remove axes, but that feels hacky...
So I figured that, since I'm working with vectors that are a single "entity", dtypes may be what I'm looking for:
vector_dtype = numpy.dtype([
('x', numpy.float64),
('y', numpy.float64),
('z', numpy.float64),
])
_ = numpy.array([(1, 0, 0), (0, 1, 0)], dtype=vector_dtype)
print(_.shape) # => (2,), good, we indeed have 2 vectors!
_ = numpy.array((1, 0, 0, 7), dtype=vector_dtype)
# Good, basic checking that I'm staying in 3D
# => ValueError: could not assign tuple of length 4 to structure with 3 fields.
However, I seem to loose basic math capabilities:
print(2 * _)
# => TypeError: The DTypes <class 'numpy.dtype[void]'> and <class 'numpy.dtype[uint8]'> do not have a common DType. For example they cannot be stored in a single array unless the dtype is `object`.
So my main question is: How do I solve this?
Is there some numpy.magic_broadcast_that_understands_what_I_mean() function?
Can I define math-operators (such as addition, ...) on the vector-dtype?

How do I solve this?
You are after the first-argument vectorized version of func, let's call it vfunc(vfunc is not "vectorization" stricto sensu, since the vectorization job in done internally.)
# v
def vfunc(scalars, x, vector):
# ^
return numpy.vstack([ # Assuming that's the shape you want.
scalar * x + vector for scalar in scalars
])
print(vfunc(
scalars = [2], # No need for array instance actually
x = numpy.array([1, 0, 0]),
vector = numpy.array([1, 0, 0]),
))
# => [3 0 0], as expected
print(vfunc(
scalars = [2],
x = numpy.array([1, 0, 0]),
vector = numpy.array([[1, 0, 0], [0, 1, 0]]),
))
# => [[3 0 0], [2 1 0]], as expected
print(vfunc(
scalars = [1, 2],
x = numpy.array([1, 0, 0]),
vector = numpy.array([1, 0, 0]),
))
# => # expected: [[2 0 0], [3 0 0]]
[...] dtypes may be what I'm looking for
No it is not.
Is there some numpy.magic_broadcast_that_understands_what_I_mean()
Yes. It is called numpy.vectorize but it is not worth it.
As it reads in the documentation:
The vectorize function is provided primarily for convenience, not for performance. The implementation is essentially a for loop.

ufuncs obey the same broadcasting rules as the operators. And your own function, written with numpy operators and ufuncs have to work with those as well. Your function could tweak the dimensions to translate inputs to something works with the rest of numpy. (Writing your own ufuncs is an advanced topic.)
In [64]: scalar=numpy.array([1, 2])
...: x=numpy.array([1, 0, 0])
...: vector=numpy.array([1, 0, 0])
In [65]: scalar * x + vector
Traceback (most recent call last):
File "<ipython-input-65-ad4a73833616>", line 1, in <module>
scalar * x + vector
ValueError: operands could not be broadcast together with shapes (2,) (3,)
The problem is the multiplication; regardless of what you call it, scalar is a (2,) shape array, which does not work with a (3,) array.
In [68]: scalar*x
Traceback (most recent call last):
File "<ipython-input-68-0d21729ffa15>", line 1, in <module>
scalar*x
ValueError: operands could not be broadcast together with shapes (2,) (3,)
But what do you expect to happen? What shape should the result have?
If scalar is a (2,1) shaped array, then by broadcasting this result is (2,3) - taking the 2 from scalar and 3 from the other arrays:
In [76]: scalar[:,None] * x + vector
Out[76]:
array([[2, 0, 0],
[3, 0, 0]])
This is standard numpy broadcasting, and there's nothing "hacky" about it.
I don't know what you mean by calling scalar a 'single entity'.
Structured array is a convenient way of putting arrays with diverse dtypes into one structure. Or to access "columns" of convenient 'names'.
But you can't perform math across the fields of such an array.
In [70]: z=np.array([(1, 0, 0), (0, 1, 0)], dtype=vector_dtype)
In [71]: z
Out[71]:
array([(1., 0., 0.), (0., 1., 0.)],
dtype=[('x', '<f8'), ('y', '<f8'), ('z', '<f8')])
In [72]: z.shape
Out[72]: (2,)
In [73]: z.dtype
Out[73]: dtype([('x', '<f8'), ('y', '<f8'), ('z', '<f8')])
In [74]: z['x']
Out[74]: array([1., 0.])
In [75]: 2*z['x'] # math on a single field
Out[75]: array([2., 0.])
note
There is a np.vectorize function. It takes a function that accepts only (true) scalar arguments, and applies array arguments according to the standard broadcasting rules. So even if your func was implemented with it, you'd still have to use arguments as I did in [70]. Sometimes it's convenient, but it's better to use standard numpy functions and operators where possible - better and much faster.

Dot product of a vector with each vector in another matrix

weight = np.array([[[ 0.38932115, -0.27430567]],
[[-0.04543304, -0.05643598]],
[[ 0.46912688, -0.07695298]]])
data = np.array([[-0.2056065, 0.7889058]])
like,
data = np.array([[1, 2, 3], [4, 5, 6]])
I want to take the dot product of the row in data with each row in weight, how could I accomplish this? I tried tensordot but it seems a bit convoluted / non-obvious the way axes works. Is there an easier way?

Your use of row and vector is a bit ambiguous:
In [7]: weight = np.array([[[ 0.38932115, -0.27430567]],
...:
...: [[-0.04543304, -0.05643598]],
...:
...: [[ 0.46912688, -0.07695298]]])
...:
...: data = np.array([[-0.2056065, 0.7889058]])
In [8]: weight.shape
Out[8]: (3, 1, 2)
In [9]: data.shape
Out[9]: (1, 2)
Are your rows of shape (2,) or (1,2)?
dot is a 'sum of products' function, but sum on which axis?
With einsum we can control the sum axis.
Sum on both the 1 and 2's:
In [11]: np.einsum('ijk,jk',weight, data)
Out[11]: array([-0.29644829, -0.03518134, -0.15716419]) # shape (3,)
or just the 1's:
In [12]: np.einsum('ijk,jm',weight, data)
Out[12]:
array([[[-0.08004696, 0.30713771],
[ 0.05639903, -0.21640133]],
[[ 0.00934133, -0.03584239],
[ 0.0116036 , -0.04452267]],
[[-0.09645554, 0.37009692],
[ 0.01582203, -0.06070865]]])
In [13]: _.shape
Out[13]: (3, 2, 2)
Or just the 2's:
In [14]: np.einsum('ijk,mk',weight, data)
Out[14]:
array([[[-0.29644829]],
[[-0.03518134]],
[[-0.15716419]]])
In [16]: _.shape
Out[16]: (3, 1, 1)
matmul/# also does this sum - data.T changes the (1,2) array to a (2,1). This pairs the (3,1,2) with a (2,1) to fit the "Last A with the second to the last of B" rule for dot/#.
In [17]: weight # data.T
Out[17]:
array([[[-0.29644829]],
[[-0.03518134]],
[[-0.15716419]]])
You ask about a multidimensional data. Just what do you mean by that? It already is 2d. Do you mean a (n,2) array, or a (n,1,2)? What's the relation between this n dimension and the 3 dimension of weight? No hand waving please :)

You can also use np.apply_along_axis
np.apply_along_axis(lambda x:np.dot(x,data.T),2,weight)
which gives
array([[[-0.29644829]],
[[-0.03518134]],
[[-0.15716419]]])
If data contains more than one row, this will also work, for example
weight = np.array([[[ 0.38932115, -0.27430567]],
[[-0.04543304, -0.05643598]],
[[ 0.46912688, -0.07695298]]])
data = np.array([[-0.2056065, 0.7889058],[-0.2056065, 0.7889058]])
np.apply_along_axis(lambda x:np.dot(x,data.T),2,weight)
gives you
array([[[-0.29644829, -0.29644829]],
[[-0.03518134, -0.03518134]],
[[-0.15716419, -0.15716419]]])

First transpose data before taking the dot product.
>>> weight.dot(data.T)
array([[[-0.29644829]],
[[-0.03518134]],
[[-0.15716419]]])
# Multiple rows of data.
data = np.array([[-0.2056065, 0.7889058],
[0.7889058, -.2056065]])
>>> weight.dot(data.T)
array([[[-0.29644829, 0.36353674]],
[[-0.03518134, -0.02423878]],
[[-0.15716419, 0.38591895]]])

Numpy passing input array as `out` argument to ufunc

Is it generally safe to provide the input array as the optional out argument to a ufunc in numpy, provided the type is correct? For example, I have verified that the following works:
>>> import numpy as np
>>> arr = np.array([1.2, 3.4, 4.5])
>>> np.floor(arr, arr)
array([ 1., 3., 4.])
The array type must be either compatible or identical with the output (which is a float for numpy.floor()), or this happens:
>>> arr2 = np.array([1, 3, 4], dtype = np.uint8)
>>> np.floor(arr2, arr2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: ufunc 'floor' output (typecode 'e') could not be coerced to provided output parameter (typecode 'B') according to the casting rule ''same_kind''
So given that an array of proper type, is it generally safe to apply ufuncs in-place? Or is floor() an exceptional case? The documentation does not make it clear, and neither do the following two threads that have tangential bearing on the question:
Numpy modify array in place?
Numpy Ceil and Floor "out" Argument
EDIT:
As a first order guess, I would assume it is often, but not always safe, based on the tutorial at http://docs.scipy.org/doc/numpy/user/c-info.ufunc-tutorial.html. There does not appear to be any restriction on using the output array as a temporary holder for intermediate results during the computation. While something like floor() and ciel() may not require temporary storage, more complex functions might. That being said, the entire existing library may be written with that in mind.

The out parameter of a numpy function is the array where the result is written. The main advantage of using out is avoiding the allocation of new memory where it is not necessary.
Is it safe to use write the output of a function on the same array passed as input? There is no general answer, it depends on what the function is doing.
Two examples
Here are two examples of ufunc-like functions:
In [1]: def plus_one(x, out=None):
...: if out is None:
...: out = np.zeros_like(x)
...:
...: for i in range(x.size):
...: out[i] = x[i] + 1
...: return out
...:
In [2]: x = np.arange(5)
In [3]: x
Out[3]: array([0, 1, 2, 3, 4])
In [4]: y = plus_one(x)
In [5]: y
Out[5]: array([1, 2, 3, 4, 5])
In [6]: z = plus_one(x, x)
In [7]: z
Out[7]: array([1, 2, 3, 4, 5])
Function shift_one:
In [11]: def shift_one(x, out=None):
...: if out is None:
...: out = np.zeros_like(x)
...:
...: n = x.size
...: for i in range(n):
...: out[(i+1) % n] = x[i]
...: return out
...:
In [12]: x = np.arange(5)
In [13]: x
Out[13]: array([0, 1, 2, 3, 4])
In [14]: y = shift_one(x)
In [15]: y
Out[15]: array([4, 0, 1, 2, 3])
In [16]: z = shift_one(x, x)
In [17]: z
Out[17]: array([0, 0, 0, 0, 0])
For the function plus_one there is no problem: the expected result is obtained when the parameters x and out are the same array. But the function shift_one gives a surprising result when the parameters x and out are the same array because the array
Discussion
For function of the form out[i] := some_operation(x[i]), such as plus_one above but also the functions floor, ceil, sin, cos, tan, log, conj, etc, as far as I know it is safe to write the result in the input using parameter out.
It is also safe for functions taking two input parameters of the form ``out[i] := some_operation(x[i], y[i]) such as the numpy function add, multiply, subtract.
For the other functions, it is case-by-case. As illustrated bellow, the matrix multiplication is not safe:
In [18]: a = np.arange(4).reshape((2,2))
In [19]: a
Out[19]:
array([[0, 1],
[2, 3]])
In [20]: b = (np.arange(4) % 2).reshape((2,2))
In [21]: b
Out[21]:
array([[0, 1],
[0, 1]], dtype=int32)
In [22]: c = np.dot(a, b)
In [23]: c
Out[23]:
array([[0, 1],
[0, 5]])
In [24]: d = np.dot(a, b, out=a)
In [25]: d
Out[25]:
array([[0, 1],
[0, 3]])
Last remark: if the implementation is multithreaded, the result of an unsafe function may even be non-deterministic because it depends on the order on which the array elements are processed.

This is an old question, but there is an updated answer:
Yes, it is safe. In the Numpy documentation, we see that as of v1.13:
Operations where ufunc input and output operands have memory overlap are defined to be the same as for equivalent operations where there is no memory overlap. Operations affected make temporary copies as needed to eliminate data dependency. As detecting these cases is computationally expensive, a heuristic is used, which may in rare cases result in needless temporary copies. For operations where the data dependency is simple enough for the heuristic to analyze, temporary copies will not be made even if the arrays overlap, if it can be deduced copies are not necessary. As an example, np.add(a, b, out=a) will not involve copies.

Element-wise effecient multiplication of arrays of matrices

Suppose array_1 and array_2 are two arrays of matrices of the same sizes. Is there any vectorised way of multiplying element-wise, the elements of these two arrays(which their elements' multiplication is well defined)?
The dummy code:
def mat_multiply(array_1,array_2):
size=np.shape(array_1)[0]
result=np.array([])
for i in range(size):
result=np.append(result,np.dot(array_1[i],array_2[i]),axis=0)
return np.reshape(result,(size,2))
example input:
a=[[[1,2],[3,4]],[[1,2],[3,4]]]
b=[[1,3],[4,5]]
output:
[[ 7. 15.]
[ 14. 32.]]

Contrary to your first sentence, a and b are not the same size. But let's focus on your example.
So you want this - 2 dot products, one for each row of a and b
np.array([np.dot(x,y) for x,y in zip(a,b)])
or to avoid appending
X = np.zeros((2,2))
for i in range(2):
X[i,...] = np.dot(a[i],b[i])
the dot product can be expressed with einsum (matrix index notation) as
[np.einsum('ij,j->i',x,y) for x,y in zip(a,b)]
so the next step is to index that first dimension:
np.einsum('kij,kj->ki',a,b)
I'm quite familiar with einsum, but it still took a bit of trial and error to figure out what you want. Now that the problem is clear I can compute it in several other ways
A, B = np.array(a), np.array(b)
np.multiply(A,B[:,np.newaxis,:]).sum(axis=2)
(A*B[:,None,:]).sum(2)
np.dot(A,B.T)[0,...]
np.tensordot(b,a,(-1,-1))[:,0,:]
I find it helpful to work with arrays that have different sizes. For example if A were (2,3,4) and B (2,4), it would be more obvious the dot sum has to be on the last dimension.
Another numpy iteration tool is np.nditer. einsum uses this (in C).
http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html
it = np.nditer([A, B, None],flags=['external_loop'],
op_axes=[[0,1,2], [0,-1,1], None])
for x,y,w in it:
# x, y are shape (2,)
w[...] = np.dot(x,y)
it.operands[2][...,0]
Avoiding that [...,0] step, requires a more elaborate setup.
C = np.zeros((2,2))
it = np.nditer([A, B, C],flags=['external_loop','reduce_ok'],
op_axes=[[0,1,2], [0,-1,1], [0,1,-1]],
op_flags=[['readonly'],['readonly'],['readwrite']])
for x,y,w in it:
w[...] = np.dot(x,y)
# w[...] += x*y
print C
# array([[ 7., 15.],[ 14., 32.]])

There's one more option that #hpaulj left out in his extensive and comprehensive list of options:
>>> a = np.array(a)
>>> b = np.array(b)
>>> from numpy.core.umath_tests import matrix_multiply
>>> matrix_multiply.signature
'(m,n),(n,p)->(m,p)'
>>> matrix_multiply(a, b[..., np.newaxis])
array([[[ 7],
[15]],
[[14],
[32]]])
>>> matrix_multiply(a, b[..., np.newaxis]).shape
(2L, 2L, 1L)
>>> np.squeeze(matrix_multiply(a, b[..., np.newaxis]), axis=-1)
array([[ 7, 15],
[14, 32]])
The nice thing about matrix_multiply is that, it being a gufunc, it will work not only with 1D arrays of matrices, but also with broadcastable arrays. As an example, if instead of multiplying the first matrix with the first vector, and the second matrix with the second vector, you wanted to compute all possible multiplications, you could simply do:
>>> a = np.arange(8).reshape(2, 2, 2) # to have different matrices
>>> np.squeeze(matrix_multiply(a[...,np.newaxis, :, :],
... b[..., np.newaxis]), axis=-1)
array([[[ 3, 11],
[ 5, 23]],
[[19, 27],
[41, 59]]])