I am trying to get a good understanding on broadcasting rules in numpy, but I have noticed I firstly need to get a good understanding on what 1-dimensional numpy array is. I found multiple sources saying that 1-dimensional numpy array is neither a horizontal or vertical vector. From that I'd expect that it behaves differently depending on an operation done and other component of the operation. But I can't really find a case when 1-dimensional array would behave like a column vector. For example:
a = np.arange(3)
b = np.arange(3)[:, np.newaxis]
a + b
array([[0, 1, 2],
[1, 2, 3],
[2, 3, 4]])
which indicates that a behaves like a horizontal vector. On the other hand, if we add it to horizontal vector b:
a = np.arange(3)
b = np.arange(3)[np.newaxis, :]
a + b
array([[0, 1, 4]])
a still behaves like a horizontal vector. On the other hand a seems to be indifferent to transformation with .T. So my question is - does 1-dimensional numpy arrays always mimic the horizontal vector behaviour? If not, what are the cases when they behave like standard vertical vector?
What you just came across is known as right align property of numpy arrays. When you have a vector of shape (n, ) and some other array of shape (a, b, c, d, ..., z) then numpy will always try to broadcast the vector to shape (1, 1, ...., n) and finally check if n is broadcastable with z (in other words, z is a multiple of n).
Now, if you don't want the behaviour, you will have to tell numpy explicitly, how do you want to broadcast with the other array with which you are operating by adding axis to the vector using np.newaxis. You can also use the function np.broadcast_arrays to get the broadcasted arrays.
For example,
import numpy as np
a = np.array([1, 2, 3])
b = np.eye(3)
# broadcasts a to shape (1, 3) first
# adds the vector a to rows of b
# [[1, 0, 0] [[1, 2, 3]
# [0, 1, 0] + [1, 2, 3]
# [0, 0, 1]] [1, 2, 3]]
print(a + b)
# Tell numpy explicitly, how you want
# your vector to be broadcasted
# Now, a is first broadcasted to shape (3, 1)
# and the vector a is added to the columns of b
# [[1, 0, 0] [[1, 1, 1]
# [0, 1, 0] + [2, 2, 2]
# [0, 0, 1]] [3, 3, 3]]
print(b + a[np.newaxis, :])
Related
Given two 2-D pytorch tensors:
A = torch.FloatTensor([[1,2],[3,4]])
B = torch.FloatTensor([[0,0],[1,1],[2,2]])
Is there an efficient way to calculate a tensor of shape (6, 2, 2) where each entry is a column of A times each row of B?
For example, with A and B above, the 3D tensor should have the following matrices:
[[[0, 0],
[0, 0]],
[[1, 1],
[3, 3]],
[[2, 2],
[6, 6]],
[[0, 0],
[0, 0]],
[[2, 2],
[4, 4]],
[[4, 4],
[8, 8]]]
I know how to do it via for-loop but I am wondering if could have an efficient way to save it.
Pytorch tensors implement numpy style broadcast semantics which will work for this problem.
It's not clear from the question if you want to perform matrix multiplication or element-wise multiplication. In the length 2 case that you showed the two are equivalent, but this is certainly not true for higher dimensionality! Thankfully the code is almost the same so I'll just give both options.
A = torch.FloatTensor([[1, 2], [3, 4]])
B = torch.FloatTensor([[0, 0], [1, 1], [2, 2]])
# matrix multiplication
C_mm = (A.T[:, None, :, None] # B[None, :, None, :]).flatten(0, 1)
# element-wise multiplication
C_ew = (A.T[:, None, :, None] * B[None, :, None, :]).flatten(0, 1)
Code description. A.T transposes A and the indexing with None inserts unitary dimensions so A.T[:, None, :, None] will be shape (2, 1, 2, 1) and B[None, :, None, :] is shape (1, 3, 1, 2). Since # (matrix multiplication) operates on the last two dimensions of tensors, and broadcasts the other dimensions, then the result is matrix multiplication for each column of A times each row of B. In the element-wise case the broadcasting is performed on every dimension. The result is a (2, 3, 2, 2) tensor. To turn it into a (6, 2, 2) tensor we just flatten the first two dimensions using Tensor.flatten.
How do I combine multiple column vectors into a Matrix? For example, if I have 3 10 x 1 vectors, how do I put them into a 10 x 3 matrix?
Here's what I've tried so far:
D0 =np.array([[np.cos(2*np.pi*f*time)],[np.sin(2*np.pi*f*time)],np.ones((len(time),1)).transpose()],'float').transpose()
this gives me something like this ,
[[[ 1.00000000e+00 0.00000000e+00 1.00000000e+00]]
[[ 9.99999741e-01 7.19053432e-04 1.00000000e+00]]
[[ 9.99998966e-01 1.43810649e-03 1.00000000e+00]]
...
[[ 9.99998966e-01 -1.43810649e-03 1.00000000e+00]]
[[ 9.99999741e-01 -7.19053432e-04 1.00000000e+00]]
[[ 1.00000000e+00 -2.15587355e-14 1.00000000e+00]]]
but, I don't think this is right, it looks more like an array of lists (and I couldn't do matrix multiplication with this form)...I tried numpy.concatenate as well, but that didn't work for me either...Looking into stack next....
In Matlab notation, I need to get this into a form
D0 =[cos(2*pi*f *t1), sin(2*pi*f*t1) ,1; cos(2*pi*f*t2), sin(2*pi*f*t2) ,1;....] etc
So that I can find the least squares solution s_hat:
s_hat = (D0^T D0)^-1(D0^T x)
where x is another input vector containing the samples of the sinusoid I'm trying to fit.
In Matlab, I could just type
D0 = [cos(2*np.pi*f*time),sin(2*np.pi*f*time), repmat(1,len(time),1)]
to create the D0 matrix. How do I do this in python?
Thank you!
Here you have equivalent complete examples in Matlab and Python/NumPy:
% Matlab
f = 0.1;
time = [0; 1; 2; 3];
D0 = [cos(2*pi*f*time), sin(2*pi*f*time), repmat(1,length(time),1)]
# Python
import numpy as np
f = 0.1
time = np.array([0, 1, 2, 3])
D0 = np.array([np.cos(2*np.pi*f*time), np.sin(2*np.pi*f*time), np.ones(time.size)]).T
print(D0)
Note that unlike Matlab, Python/NumPy has no special syntax to distinguish rows from columns (, vs. ; in Matlab). Similarly, a 1D NumPy array has no notion of either being a "column" or "row" vector. When merging several 1D NumPy arrays into a single 2D array, as above, each 1D array ends up as a row in the 2D array. As you want them as columns, you need to transpose the 2D array, here accomplished simply by the .T attribute.
If the arrays really are (10,1) shape, then simply concatenate:
In [60]: x,y,z = np.ones((10,1),int), np.zeros((10,1),int), np.arange(10)[:,None]
In [61]: np.concatenate([x,y,z], axis=1)
Out[61]:
array([[1, 0, 0],
[1, 0, 1],
[1, 0, 2],
[1, 0, 3],
[1, 0, 4],
[1, 0, 5],
[1, 0, 6],
[1, 0, 7],
[1, 0, 8],
[1, 0, 9]])
If they are actually 1d, you'll have to fiddle with dimensions in one way or other. For example reshape or add a dimension as I did with z above. Or use some function that does that for you:
In [62]: x,y,z = np.ones((10,),int), np.zeros((10,),int), np.arange(10)
In [63]: z.shape
Out[63]: (10,)
In [64]: np.array([x,y,z]).shape
Out[64]: (3, 10)
In [65]: np.array([x,y,z]).T # transpose
Out[65]:
array([[1, 0, 0],
[1, 0, 1],
[1, 0, 2],
[1, 0, 3],
[1, 0, 4],
[1, 0, 5],
[1, 0, 6],
[1, 0, 7],
[1, 0, 8],
[1, 0, 9]])
np.array([...]) joins the arrays on a new initial dimension. Remember in Python/numpy the first dimension is the outermost one (MATLAB is the reverse).
stack variants tweak the dimensions, and then do concatenate:
In [66]: np.stack([x,y,z],axis=1).shape
Out[66]: (10, 3)
In [67]: np.column_stack([x,y,z]).shape
Out[67]: (10, 3)
In [68]: np.vstack([x,y,z]).shape
Out[68]: (3, 10)
===
D0 =np.array([[np.cos(2*np.pi*f*time)],[np.sin(2*np.pi*f*time)],np.ones((len(time),1)).transpose()],'float').transpose()
I'm guessing f is a scalar, and time is a 1d array (shape (10,))
[np.cos(2*np.pi*f*time)]
wraps a (10,) in [], which when turned into an array becomes (1,10) shape.
np.ones((len(time),1)).transpose() is (10,1) transposed to (1,10).
np.array(....) of these creates a (3,1,10) array. Transpose of that is (10,1,3).
If you dropped the [] and shape that created (1,10) arrays:
D0 =np.array([np.cos(2*np.pi*f*time), np.sin(2*np.pi*f*time), np.ones((len(time))]).transpose()
would join 3 (10,) arrays to make (3,10), which then transposes to (10,3).
Alternatively,
D0 =np.concatenate([[np.cos(2*np.pi*f*time)], [np.sin(2*np.pi*f*time)], np.ones((1,len(time),1))], axis=0)
joins the 3 (1,10) arrays to make a (3,10), which you can transpose.
I found an interesting thing when comparing MATLAB and numpy.
MATLAB:
x = [1, 2]
n = size(X, 2)
% n = 1
Python:
x = np.array([1, 2])
n = x.shape[1]
# error
The question is: how to handle input which may be both ndarray with shape (n,) and ndarray with shape (n, m).
e.g.
def my_summation(X):
"""
X : ndarray
each column of X is an observation.
"""
# my solution for ndarray shape (n,)
# if X.ndim == 1:
# X = X.reshape((-1, 1))
num_of_sample = X.shape[1]
sum = np.zeros(X.shape[0])
for i in range(num_of_sample):
sum = sum + X[:, i]
return sum
a = np.array([[1, 2], [3, 4]])
b = np.array([1, 2])
print my_summation(a)
print my_summation(b)
My solution is forcing ndarray shape (n,) to be shape (n, 1).
The summation is used as an example. What I want is to find an elegant way to handle the possibility of matrix with only one observation(vector) and matrix with more than one observation using ndarray.
Does anyone have better solutions?
I recently learned about numpy.atleast_2d from the Python control
toolbox. You also don't need a for-loop for summation, rather use
numpy.sum.
import numpy as np
def my_summation(X):
"""
X : ndarray
each column of X is an observation.
"""
# my solution for ndarray shape (n,)
# if X.ndim == 1:
# X = X.reshape((-1, 1))
X = np.atleast_2d(X)
return np.sum(X, axis=1)
a = np.array([[1, 2], [3, 4]])
b = np.array([1, 2])
print my_summation(a)
print my_summation(b)
gives
[3 7]
[3]
In a ndarray X, len(X) would the number of elements along the first axis. So, for a 2D array, it would be the number of rows and for a 1D array, it would be the number of elements in itself. This property could be used to reshape the input array that could be a 1D or a 2D array into a 2D array output. For a 1D array as input, the output 2D array would have number of rows same as number of elements. For a 2D array input case, it would have the number of rows same as before, therefore no change with it.
To sum up, one solution would be to put a reshaping code at the top of the function definition, like so -
X = X.reshape(len(X),-1)
Sample runs -
2D Case:
In [50]: X
Out[50]:
array([[6, 7, 8, 1],
[6, 2, 3, 0],
[5, 1, 8, 6]])
In [51]: X.reshape(len(X),-1)
Out[51]:
array([[6, 7, 8, 1],
[6, 2, 3, 0],
[5, 1, 8, 6]])
1D Case:
In [53]: X
Out[53]: array([2, 5, 2])
In [54]: X.reshape(len(X),-1)
Out[54]:
array([[2],
[5],
[2]])
Let us say one has an array of 2D vectors:
v = np.array([ [1, 1], [1, 1], [1, 1], [1, 1]])
v.shape = (4, 2)
And an array of scalars:
s = np.array( [2, 2, 2, 2] )
s.shape = (4,)
I would like the result:
f(v, s) = np.array([ [2, 2], [2, 2], [2, 2], [2, 2]])
Now, executing v*s is an error. Then, what is the most efficient way to go about implementing f?
Add a new singular dimension to the vector:
v*s[:,None]
This is equivalent to reshaping the vector as (len(s), 1). Then, the shapes of the multiplied objects will be (4,2) and (4,1), which are compatible due to NumPy broadcasting rules (corresponding dimensions are either equal to each other or equal to 1).
Note that when two operands have unequal numbers of dimensions, NumPy will insert extra singular dimensions "in front" of the operand with fewer dimensions. This would make your vector (1,4) which is incompatible with (4,2). Therefore, we explicitly specify where the extra dimensions are added, in order to make the shapes compatible.
Can someone explain exactly what the axis parameter in NumPy does?
I am terribly confused.
I'm trying to use the function myArray.sum(axis=num)
At first I thought if the array is itself 3 dimensions, axis=0 will return three elements, consisting of the sum of all nested items in that same position. If each dimension contained five dimensions, I expected axis=1 to return a result of five items, and so on.
However this is not the case, and the documentation does not do a good job helping me out (they use a 3x3x3 array so it's hard to tell what's happening)
Here's what I did:
>>> e
array([[[1, 0],
[0, 0]],
[[1, 1],
[1, 0]],
[[1, 0],
[0, 1]]])
>>> e.sum(axis = 0)
array([[3, 1],
[1, 1]])
>>> e.sum(axis=1)
array([[1, 0],
[2, 1],
[1, 1]])
>>> e.sum(axis=2)
array([[1, 0],
[2, 1],
[1, 1]])
>>>
Clearly the result is not intuitive.
Clearly,
e.shape == (3, 2, 2)
Sum over an axis is a reduction operation so the specified axis disappears. Hence,
e.sum(axis=0).shape == (2, 2)
e.sum(axis=1).shape == (3, 2)
e.sum(axis=2).shape == (3, 2)
Intuitively, we are "squashing" the array along the chosen axis, and summing the numbers that get squashed together.
To understand the axis intuitively, refer the picture below (source: Physics Dept, Cornell Uni)
The shape of the (boolean) array in the above figure is shape=(8, 3). ndarray.shape will return a tuple where the entries correspond to the length of the particular dimension. In our example, 8 corresponds to length of axis 0 whereas 3 corresponds to length of axis 1.
If someone need this visual description:
There are good answers for visualization however it might help to think purely from analytical perspective.
You can create array of arbitrary dimension with numpy.
For example, here's a 5-dimension array:
>>> a = np.random.rand(2, 3, 4, 5, 6)
>>> a.shape
(2, 3, 4, 5, 6)
You can access any element of this array by specifying indices. For example, here's the first element of this array:
>>> a[0, 0, 0, 0, 0]
0.0038908603263844155
Now if you take out one of the dimensions, you get number of elements in that dimension:
>>> a[0, 0, :, 0, 0]
array([0.00389086, 0.27394775, 0.26565889, 0.62125279])
When you apply a function like sum with axis parameter, that dimension gets eliminated and array of dimension less than original gets created. For each cell in new array, the operator will get list of elements and apply the reduction function to get a scaler.
>>> np.sum(a, axis=2).shape
(2, 3, 5, 6)
Now you can check that the first element of this array is sum of above elements:
>>> np.sum(a, axis=2)[0, 0, 0, 0]
1.1647502999560164
>>> a[0, 0, :, 0, 0].sum()
1.1647502999560164
The axis=None has special meaning to flatten out the array and apply function on all numbers.
Now you can think about more complex cases where axis is not just number but a tuple:
>>> np.sum(a, axis=(2,3)).shape
(2, 3, 6)
Note that we use same technique to figure out how this reduction was done:
>>> np.sum(a, axis=(2,3))[0,0,0]
7.889432081931909
>>> a[0, 0, :, :, 0].sum()
7.88943208193191
You can also use same reasoning for adding dimension in array instead of reducing dimension:
>>> x = np.random.rand(3, 4)
>>> y = np.random.rand(3, 4)
# New dimension is created on specified axis
>>> np.stack([x, y], axis=2).shape
(3, 4, 2)
>>> np.stack([x, y], axis=0).shape
(2, 3, 4)
# To retrieve item i in stack set i in that axis
Hope this gives you generic and full understanding of this important parameter.
Some answers are too specific or do not address the main source of confusion. This answer attempts to provide a more general but simple explanation of the concept, with a simple example.
The main source of confusion is related to expressions such as "Axis along which the means are computed", which is the documentation of the argument axis of the numpy.mean function. What the heck does "along which" even mean here? "Along which" essentially means that you will sum the rows (and divide by the number of rows, given that we are computing the mean), if the axis is 0, and the columns, if the axis is 1. In the case of axis is 0 (or 1), the rows can be scalars or vectors or even other multi-dimensional arrays.
In [1]: import numpy as np
In [2]: a=np.array([[1, 2], [3, 4]])
In [3]: a
Out[3]:
array([[1, 2],
[3, 4]])
In [4]: np.mean(a, axis=0)
Out[4]: array([2., 3.])
In [5]: np.mean(a, axis=1)
Out[5]: array([1.5, 3.5])
So, in the example above, np.mean(a, axis=0) returns array([2., 3.]) because (1 + 3)/2 = 2 and (2 + 4)/2 = 3. It returns an array of two numbers because it returns the mean of the rows for each column (and there are two columns).
Both 1st and 2nd reply is great for understanding ndarray concept in numpy. I am giving a simple example.
And according to this image by #debaonline4u
https://i.stack.imgur.com/O5hBF.jpg
Suppose , you have an 2D array -
[1, 2, 3]
[4, 5, 6]
In, numpy format it will be -
c = np.array([[1, 2, 3],
[4, 5, 6]])
Now,
c.ndim = 2 (rows/axis=0)
c.shape = (2,3) (axis0, axis1)
c.sum(axis=0) = [1+4, 2+5, 3+6] = [5, 7, 9] (sum of the 1st elements of each rows, so along axis0)
c.sum(axis=1) = [1+2+3, 4+5+6] = [6, 15] (sum of the elements in a row, so along axis1)
So for your 3D array,