We Keep Coding

how to add new elements to a numpy array

I created an array with size (256, 144, 3).
empty_windows = np.empty((256, 144, 3))
Then I want to append new elements into the array with:
for i in range(256):
for j in range(144):
empty_windows[i, j] = np.append(empty_windows[i, j], np.asarray(some_new_array)).reshape(3, )
But it doesnt work as I get the error msg:
ValueError: cannot reshape array of size 6 into shape (3,)
Is there a way of doing it? Thank you.

I hope, it will help you understanding concatenate 3dim array
import numpy as np
empty_windows = np.empty((256, 144, 3))
random_arr = np.random.randint(0, 100, size=(256, 144, 3)) # it's dimension should be same
np.concatenate([empty_windows, random_arr], axis=2) # it can concatenate into an array axis=2 defines 3rd dimension

np.empty and np.append are dangerous functions to use. They are not clones of the the empty list [] and list.append.
empty_windows = np.empty((256, 144, 3))
has made a (256,144,3) shape array with float values - they are unpredictable, but more than likely not what you want. Look at that array, or a smaller example to see for yourself. Also read, and if necessary reread, the np.empty docs. np.zeros is safer.
With scalar i,j,
empty_windows[i, j]
is a (3,) shape array, or slot.
When you np.append it with another (3,) shape, the result is a (6,) shape, with the first 3 value being those "random" values originally in empty_window. The error tells you quite clearly that it can't put a (6,) shape array into a slot that only holds (3,).
Your goal isn't clear, but you can't grow a (n,m,3) shape array to (n,m,6) by doing this kind of "row" by "row" append.
You can set the "row" with new values, as in:
empty_windows[i, j] = np.asarray(some_new_array)).reshape(3, )

Multiply multidimensional numpy array by 1-D array

I have a multidimensional array and a set of scale factors that I want to apply along the first axis:
>>> data.shape, scale_factors.shape
((22, 20, 2048, 2048), (22,))
>>> data * scale_factors
ValueError: operands could not be broadcast together with shapes (22,20,2048,2048) (22,)
I can do this with apply_along_axis, but is there a vectorized way to do this? I found a similar question, but the solution is specific to a 1-D * 2-D operation. The "data" ndarray will not always be the same shape, and won't even always have the same number of dimensions. But the length of the 1-D scale_factors will always be the same as axis 0 of data.

You can try reshape the data into 2D, then broadcast scale_factor to 2D, and reshape back:
(data.reshape(data.shape[0], -1) * scale_factors[:,None]).reshape(data.shape)
Or, you can swap the 0-th axis to the last so you can broadcast:
(data.swapaxes(0,-1) * scale_factors).swapaxes(0,-1)

data * scale_factors.reshape([-1]+[1]*(len(data.shape)-1))

data * scale_factors[:,None,None,None]

Numpy: How to multiply (N,N) and (N,N,M,M) numpy arrays?

I want to multiply two numpy arrays. One numpy array is given by matrix of shape (10, 10) and the other is given by a matrix of matrices, i.e. shape (10, 10, 256, 256).
I now simply want to multiply each matrix in the second matrix of matrices with the corresponding component in the first matrix. For instance, the matrix at position (0, 0) in the second matrix shall be multiplied by the value at position (0, 0) in the first matrix.
Intuitively, this is not really complicated, but numpy does not seem to support that. Or at least I am not smart enough to make it work. The ValueError that is thrown says:
ValueError: operands could not be broadcast together with shapes (10,10) (10,10,256,256)
Can anybody of you help me please? How can I achieve what I want in a numpyy way.

You can use the NumPy einsum function, e.g., (using zeros arrays as dummies in this example):
import numpy as np
x = np.zeros((10, 10))
y = np.zeros((10, 10, 256, 256))
z = np.einsum("ij,ijkm->km", x, y)
print(z.shape)
(256, 256)
See here for a nice description of einsum's usage.

python difference between array(10,1) array(10,)

I'm trying to load MNIST dataset into arrays.
When I use
(X_train, y_train), (X_test, y_test)= mnist.load_data()
I get an array y_test(10000,) but I want it to be in the shape of (10000,1).
What is the difference between array(10000,1) and array(10000,)?
How can I convert the first array to the second array?

Your first Array with shape (10000,) is a 1-Dimensional np.ndarray.
Since the shape attribute of numpy Arrays is a Tuple and a tuple of length 1 needs a trailing comma the shape is (10000,) and not (10000) (which would be an int). So currently your data looks like this:
import numpy as np
a = np.arange(5) # >>> array([0, 1, 2, 3, 4]
print(a.shape) # >>> (5,)
What you want is an 2-Dimensional array with shape of (10000, 1).
Adding a dimension of length 1 doesn't require any additional data, it is basically and "empty" dimension. To add an dimension to an existing array you can use either np.expand_dims() or np.reshape().
Using np.expand_dims:
import numpy as np
b = np.array(np.arange(5)) # >>> array([0, 1, 2, 3, 4])
b = np.expand_dims(b, axis=1) # >>> array([[0],[1],[2],[3],[4]])
The function was specifically made for the purpose of adding empty dimensions to arrays. The axis keyword specifies which position the newly added dimension will occupy.
Using np.reshape:
import numpy as np
a = np.arange(5)
X_test_reshaped = np.reshape(a, shape=[-1, 1]) # >>> array([[0],[1],[2],[3],[4]])
The shape=[-1, 1] specifies how the new shape should look like after the reshape operation. The -1 itself will be replaced by the shape that 'fits the data' by numpy internally.
Reshape is a more powerful function than expand_dims and can be used in many different ways. You can read more on other uses of it in the numpy docs. numpy.reshape()

An array with a size of (10,1) is a 2D array containing empty columns.
An array with a size of (10,) is a 1D array.
To convert (10,1) to (10,), you can simply collapse the columns. For example, we take the x array, which has x.shape = (10,1). now using x[:,] you can collapse the columns and x[:,].shape = (10,).
To convert (10,) to (10,1), you can add a dimension by using np.newaxis. So, after import numpy as np, assuming we are using numpy arrays here. Take a y array for example, which has y.shape = (10,). Using y[:, np.newaxis], you can a new array with the shape of (10,1).

How to get these shapes to line up for a numpy matrix

I'm trying to input vectors into a numpy matrix by doing:
eigvec[:,i] = null
However I keep getting the error:
ValueError: could not broadcast input array from shape (20,1) into shape (20)
I've tried using flatten and reshape, but nothing seems to work

The shapes in the error message are a good clue.
In [161]: x = np.zeros((10,10))
In [162]: x[:,1] = np.ones((1,10)) # or x[:,1] = np.ones(10)
In [163]: x[:,1] = np.ones((10,1))
...
ValueError: could not broadcast input array from shape (10,1) into shape (10)
In [166]: x[:,1].shape
Out[166]: (10,)
In [167]: x[:,[1]].shape
Out[167]: (10, 1)
In [168]: x[:,[1]] = np.ones((10,1))
When the shape of the destination matches the shape of the new value, the copy works. It also works in some cases where the new value can be 'broadcasted' to fit. But it does not try more general reshaping. Also note that indexing with a scalar reduces the dimension.

I can guess that
eigvec[:,i] = null.flat
would work (however, null.flatten() should work too). In fact, it looks like NumPy complains because of you are assigning a pseudo-1D array (shape (20, 1)) to a 1D array which is considered to be oriented differently (shape (1, 20), if you wish).
Another solution would be:
eigvec[:,i] = null.T
where you properly transpose the "vector" null.
The fundamental point here is that NumPy has "broadcasting" rules for converting between arrays with different numbers of dimensions. In the case of conversions between 2D and 1D, a 1D array of size n is broadcast into a 2D array of shape (1, n) (and not (n, 1)). More generally, missing dimensions are added to the left of the original dimensions.
The observed error message basically said that shapes (20,) and (20, 1) are not compatible: this is because (20,) becomes (1, 20) (and not (20, 1)). In fact, one is a column matrix, while the other is a row matrix.