I have four Numpy arrays of shapes:
(2577, 42)
(2580, 100)
(2580, 236)
(2580, 8)
(2580, 37)
When I try to concatenate all of them do except (2577, 42). I get an error:
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 2580 and the array at index 4 has size 2577
The code I am using:
dataset = np.concatenate((onehot_b, num_v, onehot_s, onehot_c, onehot_s), axis=1)
Is there a way to fix this?
The error is prety clear. You Cannot concatenate arrays of different sizes. One possible way out is convert the numpy arrays to lists and append all list lines to you dataset.
Numpy does not allow non-rectangular arrays, meaning that all sub-arrays should have the same dimension along the same axis. In your case, 2577 and 2580, are dimensions along same axis=0 that you are not stacking over (hence not adding them along that axis and they should have same length). If you can change all of them to have same first dimension shape, you can use concatenate. If you insist on stacking them, another way is just stacking arrays rather than their content:
dataset = np.asarray([onehot_b, num_v, onehot_s, onehot_c, onehot_s])
This will create an array of arrays for you.
Related
I am getting the error ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 2, the array at index 0 has size 3 and the array at index 1 has size 1 while running the below code.
for i in range(6):
print('current b', current_batch)
current_pred = model.predict(current_batch)[0]
print('current pred', current_pred)
test_predictions.append(current_pred)
print('current batch', current_batch)
print('current batch => ', current_batch[:,1:,:])
current_batch = np.append(current_batch[:,1:,:], [[current_pred]], axis=1)
getting this error
Can anyone please explain me why this is happening.
Thanks,
Basically, Numpy is telling you that the shapes of the concatenated matrices should align. For example, it is possible to concatenate a 3x4 matrix with 3x5 matrix so that we get 3x9 matrix (we added dimension 1).
The problem here is that Numpy is telling you that the axis don't align. In my example, that would be trying to concatenate 3x4 matrix with 10x10 matrix. This is not possible as the shapes are not aligned.
This usually means that the you are trying to concatenate wrong things. If you are sure though, try using np.reshape function, which will change the shape of one of the matrices so that they can be concatenated.
As the traceback shows, np.append is actually using np.concatenate. Did you read (study) the docs for either function? Understand what they say about dimensions?
From the display [[current_pred]], converted to array will be (1,1,1) shape. Do you understand that?
current_batch[:,1:,:] is, as best I can tell from the small image (1,5,3)
You are asking to join on axis 1, which is 1 and 5, ok. But it's saying that the last dimension, axis 2, doesn't match. That 1 does not equal 3. Do you understand that?
List append as you do with test_predictions.append(current_pred) works well in an iteration.
np.append does not work well. Even when it works, it is slow. And here it doesn't work, because you aren't taking sufficient care to match dimensions.
I have two arrays A1(5,3) and A2(5,2) and want to create a new array A(5,5). I've try to use A=np.concatenate((A1,A2)) but it gives me the error
all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 3 and the array at index 1 has size 2
How can I solve this?
To use np.concatenate you should specify on which axis the concatenation is happening. In this case, you should use:
A = np.concatenate((A1, A2), axis=1)
Other way to solve this issue is to use horizontal stack:
A = np.hstack((A1, A2))
I have a 2d array of shape (512,512). I need to convert this to shape (512,512,3). All values of 2d dimension will be repeated on other two dims. How can I do this in python?
you can try using np dstack
it would work for your case
np.dstack([a,a,a])
I would use array[..., None].repeat(3, -1)
I have many numpy arrays of shape (Ni,227,227,3), where Ni of each array is different.
I want to join them and make array of shape (N1+N2+..+Nk,227,227,3) where k is the number of arrays.
I tried numpy.concatenate and numpy.append but they ask for same dimension in axis 0. I am also confused on what is axis 1 and axis 2 in my arrays.
So, the main problem here was with the one of the arrays of shape (0,) instead of (0,227,227,3).
np.concatenate(alist,axis=0) works.
I have two (or sometimes more) matrixes, which I want to combine to a tensor. The matrixes e.g. have the shape (100, 400) and when they are combined, they should have the dimensions (2, 100, 400).
How do I do that? I tried it the same way I created matrixes from vectors, but that didn't work:
tensor = numpy.concatenate(list_of_matrixes, axis=0)
Probably you want
tensor = np.array(list_of_matrices)
np.array([...]) just loves to combine the inputs into a new array along a new axis. In fact it takes some effort to prevent that.:)
To use concatenate you need to add an axis to your arrays. axis=0 means 'join on the current 1st axis', so it would produce a (200,400) array.
np.concatentate([arr1[None,...], arr2[None,...], axis=0)
would do the the trick, or more generally
np.concatenate([arr[None,...] for arr in list_arr], axis=0)
If you look at the code for dstack, hstack, vstack you'll see that they do this sort of dimension adjustment before passing the task to concatenate.
The np.array solution is easy, but the concatenate solution is a good learning opportunity.