Want to perform an element wise comparison between an 1D and 2D array. Each element of the 1D array need to be compared (e.g. greater) against the corresponding row of 2D and a mask will be created. Here is an example:
A = np.random.choice(np.arange(0, 10), (4,100)).astype(np.float)
B = np.array([5., 4., 8., 2. ])
I want to do
A<B
so that first row of A will be compared against B[0] which is 5. and the result will be an boolean array.
If I try this I get:
operands could not be broadcast together with shapes (4,100) (4,)
Any ideas?
You need to insert an extra dimension into array B:
A < B[:, None]
This allows NumPy to properly match up the two shapes for broadcasting; B now has shape (4, 1) and the dimensions can be paired up:
(4, 100)
(4, 1)
The rule is that either the dimensions have the same length, or one of the lengths needs to be 1; here 100 can be paired with 1, and 4 can be paired with 4. Before the new dimension was inserted, NumPy tried to pair 100 with 4 which raised the error.
Related
Getting the shape of two different numpy arrays returns tuples
a.shape
Out[131]: (3,)
A.shape
Out[132]: (3, 3)
Based on the tuples, one is a one-dimensional array (number of dimensions = 1), the other is 2d. How can I detect number of dimensions similar to how type(A) will tell me one of them is a numpy.ndarray? should I just use len(a.shape)?
You should use numpy.ndarray.ndim. So
a.ndim # gives 1
and
A.ndim # gives 2
I have a numpy array of shape (29, 10) and a list of 29 elements and I want to end up with an array of shape (29,11)
I am basically converting the list to a numpy array and trying to vstack, but it complain about dimensions not being the same.
Toy example
a = np.zeros((29,10))
a.shape
(29,10)
b = np.array(['A']*29)
b.shape
(29,)
np.vstack((a, b))
ValueError: all the input array dimensions except for the concatenation axis must match exactly
Dimensions do actually match, why am I getting this error and how can I solve it?
I think you are looking for np.hstack.
np.hstack((a, b.reshape(-1,1)))
Moreover b must be 2-dimensional, that's why I used a reshape.
The problem is that you want to append a 1D array to a 2D array.
Also, for the dimension you've given for b, you are probably looking for hstack.
Try this:
a = np.zeros((29,10))
a.shape
(29,10)
b = np.array(['A']*29)[:,None] #to ensure 2D structure
b.shape
(29,1)
np.hstack((a, b))
If you do want to vertically stack, you'd need this:
a = np.zeros((29,10))
a.shape
(29,10)
b = np.array(['A']*10)[None,:] #to ensure 2D structure
b.shape
(1,10)
np.vstack((a, b))
In the documentation on broadcasting rules, is is stated that two dimensions are compatible when either:
they are equal or
one of them is 1
This becomes clear with some of the examples that are shown, such as:
A (4d array): 8 x 1 x 6 x 1
B (3d array): 7 x 1 x 5
Result (4d array): 8 x 7 x 6 x 5
This seems clear enough. However, I've been unable to find a specific example/explanation as to why 1D arrays are only broadcastable with 2d arrays, when the 1d array's shape is compatible with that of the second axis of the 2d array.
So for instance:
np.ones((2,3)) * np.arange(3)
array([[0., 1., 2.],
[0., 1., 2.]])
The 1d arange has been broadcast across the rows, as expected. However if we do:
np.ones((3, 2)) * np.arange(3)
ValueError: operands could not be broadcast together with shapes (3,2) (3,)
We get an error for incompatible shapes. This might be quite simple, but I just want to know which is the correct interpretation. Is the reason behind this, that when it comes to broadcasting rules 1d arrays are treated as column vectors, and hence shape compatibility is checked along the second axis on the 2d array? And for larger arrays, is it always checked against the last axis on the larger ndarray?
Is the reason behind this, that when it comes to broadcasting rules 1d arrays are treated as column vectors, and hence shape compatibility is checked along the second axis on the 2d array?
On the contrary, they are treated as row vectors and stacked vertically. You can see it in this figure from a related article.
You can do the same sketch in the case you presented:
A 3 x 2 # np.ones((3, 2))
B 2 x 3 # np.ones((2, 3))
C 3 # np.arange(3)
Here A*C does not work, but B*C does. This is because C gets duplicated along the first dimension (i.e., rows). You can imagine missing dimensions as being "resized" by duplicating the dimensions coming after
How can I eliminate a dummy dimension in python numpy ndarray?
For example, suppose that A.shape = (0, 1325, 3),
then how can eliminate '0' dimension so that A.shape = (1325,3).
Both 'np.sqeeze(A)' or 'A.reshape(A.shape[1:])' don't work.
You can't eliminate that 0 dimension. A dimension of length 0 is not a "dummy" dimension. It really means length 0. Since the total number of elements in the array (which you can check with a.size) is the product of the shape attribute, an array with shape (0, 1325, 3) contains 0 elements, while an array with shape (1325, 3) contains 3975 elements. If there was a way to eliminate the 0 dimension, where would that data come from?
If your array is supposed to contain data, then you probably need to look at how that array was created in the first place.
I'm trying to input vectors into a numpy matrix by doing:
eigvec[:,i] = null
However I keep getting the error:
ValueError: could not broadcast input array from shape (20,1) into shape (20)
I've tried using flatten and reshape, but nothing seems to work
The shapes in the error message are a good clue.
In [161]: x = np.zeros((10,10))
In [162]: x[:,1] = np.ones((1,10)) # or x[:,1] = np.ones(10)
In [163]: x[:,1] = np.ones((10,1))
...
ValueError: could not broadcast input array from shape (10,1) into shape (10)
In [166]: x[:,1].shape
Out[166]: (10,)
In [167]: x[:,[1]].shape
Out[167]: (10, 1)
In [168]: x[:,[1]] = np.ones((10,1))
When the shape of the destination matches the shape of the new value, the copy works. It also works in some cases where the new value can be 'broadcasted' to fit. But it does not try more general reshaping. Also note that indexing with a scalar reduces the dimension.
I can guess that
eigvec[:,i] = null.flat
would work (however, null.flatten() should work too). In fact, it looks like NumPy complains because of you are assigning a pseudo-1D array (shape (20, 1)) to a 1D array which is considered to be oriented differently (shape (1, 20), if you wish).
Another solution would be:
eigvec[:,i] = null.T
where you properly transpose the "vector" null.
The fundamental point here is that NumPy has "broadcasting" rules for converting between arrays with different numbers of dimensions. In the case of conversions between 2D and 1D, a 1D array of size n is broadcast into a 2D array of shape (1, n) (and not (n, 1)). More generally, missing dimensions are added to the left of the original dimensions.
The observed error message basically said that shapes (20,) and (20, 1) are not compatible: this is because (20,) becomes (1, 20) (and not (20, 1)). In fact, one is a column matrix, while the other is a row matrix.