I have a multidimensional array and a set of scale factors that I want to apply along the first axis:
>>> data.shape, scale_factors.shape
((22, 20, 2048, 2048), (22,))
>>> data * scale_factors
ValueError: operands could not be broadcast together with shapes (22,20,2048,2048) (22,)
I can do this with apply_along_axis, but is there a vectorized way to do this? I found a similar question, but the solution is specific to a 1-D * 2-D operation. The "data" ndarray will not always be the same shape, and won't even always have the same number of dimensions. But the length of the 1-D scale_factors will always be the same as axis 0 of data.
You can try reshape the data into 2D, then broadcast scale_factor to 2D, and reshape back:
(data.reshape(data.shape[0], -1) * scale_factors[:,None]).reshape(data.shape)
Or, you can swap the 0-th axis to the last so you can broadcast:
(data.swapaxes(0,-1) * scale_factors).swapaxes(0,-1)
data * scale_factors.reshape([-1]+[1]*(len(data.shape)-1))
data * scale_factors[:,None,None,None]
Related
I am trying to normalize some Nx3 data. If X is a Nx3 array and D is a Nx1 array, in MATLAB, I can do
Y = X./D
If I do the following in Python, I get an error
X = np.random.randn(100,3)
D = np.linalg.norm(X,axis=1)
Y = X/D
ValueError: operands could not be broadcast together with shapes (100,3) (100,)
Any suggestions?
Edit: Thanks to dm2.
Y = X/D.reshape((100,1))
Another way is to use scikitlearn.
from sklearn import preprocessing
Y = preprocessing.normalize(X)
From numpy documentation on array broadcasting:
When operating on two arrays, NumPy compares their shapes
element-wise. It starts with the trailing (i.e. rightmost) dimensions
and works its way left. Two dimensions are compatible when
they are equal, or
one of them is 1
Both of your arrays have the same first dimension, but your X array is 2-dimensional, while your D array is 1-dimensional, which means the shapes of these two arrays do not meet the requirements to be broadcast together.
To make sure they do, you could reshape your D array into a 2-dimensional array of shape (100,1), which would satisfy the requirements to broadcast: rightmost dimensions are 3 and 1 (one of them is 1) and the other dimensions are equal (100 and 100).
So:
Y = X/D.reshape((-1,1))
or
Y = X/D.reshape((100,1))
or
Y = X/D[:,np.newaxis]
Should give you the result you're after.
I want to numpy.sum() my nD arrays (matrix) of shape (2,2,2...n) to 1D arrays of shape (2,).
Basically along the axis=0.
(Sum the first number of each 1d array of shape (2,) and sum the second number of the same arrays into one single resulting 1d array of shape (2,)).
However matrix.sum(axis=0) only seems to work for those of shape (2,2), while I think matrix.sum(axis=(1,2)) works for (2,2,2). But then what about (2,2,2,2) arrays and so on?
The n-dimensions have been confusing me. A generalised solution, along with a brief explanation, would be much appreciated.
EDIT: I've provided an example of the nD arrays I'm trying to sum below. I want to sum along axis=0 to get a (2,) 1D array. numpy.sum(axis=0) seems to only work for the array of shape (2,2)...
#Of shape (2,2):
[[9.99695358e-02 9.99232016e-01]
[9.00030464e-01 7.67983971e-04]].sum(axis=0) seems to work
#Of shape (2,2,2):
[[[2.02737071e-01 7.75883149e-01]
[2.02650032e-08 1.58192237e-02]]
[[7.31718878e-06 1.41793363e-03]
[4.12802168e-03 7.26350831e-06]]].sum(axis=(1,2)) seems to work
#Of shape… (2,2,2,2)
[[[[1.83819962e+00 1.02712560e-02]
[5.05122135e-02 2.80555725e-04]]
[[5.60304309e-07 5.44521143e-04]
[2.41592380e-03 1.49436734e-05]]]
[[[7.04398015e-05 7.66717944e-06]
[1.76843337e-05 1.98868986e-06]]
[[9.74010599e-02 1.12527543e-07]
[2.61427056e-04 2.70778171e-08]]]].sum(axis=?) # What axis? And how to generalise?
What do you think about reshaping x to 2D and summing along the second axis?
x.reshape(x.shape[0], -1).sum(axis=1)
I want to reshape this array: np.array(np.arange(15)) to a 3d array that is built from a 3x3 array and a 3x2 array.
I've tried to do it with the reshape method but it didn't work.
I thought that maybe reshape can get a number of tuples maybe.
a=np.array(np.arange(15)).reshape(1,((3,2),(3,3)))
but I then I saw it cant.
How can I reshape it then? is there a nice way?
a multidimensional array can't have dimensions with different size.
but if you want a tuple you will need to split the array in 2 parts, the first that match in size with the 3x3 array and the second that match the 3x2, at this point you'll have 2 one dimensional array, then reshape them
arr1 = arr1.reshape((3,3))
arr2 = arr2.reshape((3,2))
tuple = arr1, arr2
I have many numpy arrays of shape (Ni,227,227,3), where Ni of each array is different.
I want to join them and make array of shape (N1+N2+..+Nk,227,227,3) where k is the number of arrays.
I tried numpy.concatenate and numpy.append but they ask for same dimension in axis 0. I am also confused on what is axis 1 and axis 2 in my arrays.
So, the main problem here was with the one of the arrays of shape (0,) instead of (0,227,227,3).
np.concatenate(alist,axis=0) works.
I'm trying to input vectors into a numpy matrix by doing:
eigvec[:,i] = null
However I keep getting the error:
ValueError: could not broadcast input array from shape (20,1) into shape (20)
I've tried using flatten and reshape, but nothing seems to work
The shapes in the error message are a good clue.
In [161]: x = np.zeros((10,10))
In [162]: x[:,1] = np.ones((1,10)) # or x[:,1] = np.ones(10)
In [163]: x[:,1] = np.ones((10,1))
...
ValueError: could not broadcast input array from shape (10,1) into shape (10)
In [166]: x[:,1].shape
Out[166]: (10,)
In [167]: x[:,[1]].shape
Out[167]: (10, 1)
In [168]: x[:,[1]] = np.ones((10,1))
When the shape of the destination matches the shape of the new value, the copy works. It also works in some cases where the new value can be 'broadcasted' to fit. But it does not try more general reshaping. Also note that indexing with a scalar reduces the dimension.
I can guess that
eigvec[:,i] = null.flat
would work (however, null.flatten() should work too). In fact, it looks like NumPy complains because of you are assigning a pseudo-1D array (shape (20, 1)) to a 1D array which is considered to be oriented differently (shape (1, 20), if you wish).
Another solution would be:
eigvec[:,i] = null.T
where you properly transpose the "vector" null.
The fundamental point here is that NumPy has "broadcasting" rules for converting between arrays with different numbers of dimensions. In the case of conversions between 2D and 1D, a 1D array of size n is broadcast into a 2D array of shape (1, n) (and not (n, 1)). More generally, missing dimensions are added to the left of the original dimensions.
The observed error message basically said that shapes (20,) and (20, 1) are not compatible: this is because (20,) becomes (1, 20) (and not (20, 1)). In fact, one is a column matrix, while the other is a row matrix.