I have some numpy arrays which its elements are pixels of 28*28 images like this:
25 of these arrays are in one array in shape of (25,28,28) or (5,5,28,28). Is there any efficient way to stack them to have one image: 5*5 of 28*28 images.
I tried np.reshape to (140,140) array and plt.imgshow. But the output was a messed image.
"I tried np.reshape to (140,140)..." That will work if you first transpose the input appropriately.
Suppose the input x has shape (5, 5, 28, 28). To get the array y with shape (140, 140) that contains the images arranged the way you want, you can do:
xshp = x.shp
y = x.transpose((0, 2, 1, 3)).reshape((xshp[0]*xshp[2], xshp[1]*xshp[3]))
If x always has shape (5, 5, 28, 28), you can hardcode the constant 140:
y = x.transpose((0, 2, 1, 3)).reshape((140, 140))
For example, here I create x with shape (5, 5, 28, 28) where each 28x28 image is a constant. The constants are chosen randomly. The tranposed, reshaped array y is plotted, and you can see that all the constant blocks are arranged correctly.
In [148]: rng = np.random.default_rng()
In [149]: x = np.repeat(rng.integers(0, 256, size=(5, 5)), 28*28, axis=-1).reshape((5, 5, 28, 28))
In [150]: y = x.transpose((0, 2, 1, 3)).reshape((140, 140))
In [151]: imshow(y)
Related
I have a 2D array of shape (10, 3) and an image represented as a 3D array of shape (480, 640, 3). I'd like to perform a difference between each pixel and each element of the 2D array, to get a final result of shape (10, 480, 640, 3).
For now, my code looks like this:
arr_2d = np.random.rand(10, 3)
arr_3d = np.random.rand(480, 640, 3)
res = np.ones_like(arr_3d)
res = np.tile(res, (10, 1, 1, 1))
for i in range(10):
res[i] = arr_3d - arr_2d[i]
My question is if there's a way to do this without the for loop, only using numpy operations.
You can try broadcasting with np.array like this
arr_2d = arr_2d.reshape(-1,1,1,3)
arr_3d = arr_3d.reshape((-1,*arr_3d.shape))
res = arr_3d - arr_2d
This should give the same result as your original code
A n-dimensional array which is initialised as
features=np.empty(shape=(100,5,2), dtype=float)
and I am trying to add 3D array into it as
features[i,:] = features_next
features_next has shape (2,5,2).
However, it shows error,
ValueError: could not broadcast input array from shape (2,5,2) into shape (5,2).
here is the piece of code :
features=np.empty(shape=((historical*2),5,2), dtype=float)
i = 0
while i < 50:
state = self.getDictState(state_index)
asks = state['asks']
bids = state['bids']
features_next = self.getNormalisedFeature(
bids=bids,
asks=asks,
state_index=state_index,
qty=qty,
price=price,
size=size,
normalize=normalize,
levels=levels
)
'''if i == 0:
features = np.array(features_next)
else:
features = np.vstack((features, features_next))'''
features[i,:] = features_next
state_index = (state_index - 1)
return features
Note : I am trying to replace commented 'if condition' with features[i,:] = features_next to make the code execution bit faster.
Its pretty simple, just one point to make features[i,:] has shape (5, 2) and feature_next has shape (2, 5, 2). I want to say that both are compatible shapes. But broadcasting is done on a smaller shape over a larger shape. SO there is error since you are doing revere. Also look up on broadcasting on numpy docs.
Next, this one I think will do
This does not directly solve your problem, but has some ideas about what you can try, like you can reshape your array before going in loop using features.shape = (50, 2, 5, 2).
import numpy as np
features=np.empty(shape=(100,5,2), dtype=float)
features_next = np.random.random((2, 5, 2))
features.shape = ((50, 2, 5, 2))
features[:] = features_next
features.shape = (100, 5, 2)
I am working with Keras and the provided MNIST data set. I believe the dataset is a numpy array. I have reshaped it as follows:
X_train = X_train.reshape(X_train.shape[0], 1, 28, 28)
X_test = X_test.reshape(X_test.shape[0], 1, 28, 28)
This gives a (60000, 1, 28, 28) numpy array. This can be read as there are 60000 28 x 28 images. I want to extract every single 28 x 28 image and apply some sort of function f to it. I have tried the following:
f = lambda a, _: print a.shape
np.apply_over_axes(f, data, [2,3])
But I am unsure exactly the second axis parameter comes into play though...
I have also tried:
f = lambda a: print a.shape
np.apply_along_axis(f, 0, data)
But the shape is always (60000,) instead of what I would expect (1, 28, 28). How do I get each subimage?
There is no performance gained by using np.apply_along_axis, np.vectorize, etc. Just use a loop:
import numpy as np
s = (4,1,28,28)
a = np.zeros(s)
for img in a[:,0]:
print(img.shape)
# (28, 28)
# (28, 28)
# (28, 28)
# (28, 28)
This lambda doesn't make sense:
lambda a, _: print a.shape
it's equivalent to
def foo(a, x):
return print a.shape
print a.shape prints something, and returns nothing, maybe even an error.
lambda a,x: a.shape is better, returning the shape of a, and ignoring the x argument.
If the size 1 dimension is in the way, why not just omit it?
X_train = X_train.reshape(X_train.shape[0], 28, 28)
or remove it
X_train[:,0,...]
np.squeeze(X_train)
But what's the point of the apply_over? Just to find the shape of a set of submatrices?
In [304]: X = np.ones((6,1,2,3))
In [305]: [x.shape for x in X]
Out[305]: [(1, 2, 3), (1, 2, 3), (1, 2, 3), (1, 2, 3), (1, 2, 3), (1, 2, 3)]
or
[x.shape for x in X[:,0]]
to remove the 2nd dimension, getting just the shape of the last 2.
This apply_along_axis, iterates on the last 3 dim, passing a 1d array to the lambda. So in effect it is returning X[:,0,i,j].shape.
In [308]: np.apply_along_axis(lambda a: a.shape, 0, X)
Out[308]:
array([[[[6, 6, 6],
[6, 6, 6]]]])
Generally iterations like this aren't needed. And when used, are slow compared to 'full-array' ones.
I have a numpy array v with shape (1000, 68), v is supposed to padding to 100 dimension with 0s. As a result, the v's shape will be (1000, 100)
I tried to use the following approaches:
t = np.lib.pad(v, (16, 16), 'minimum') # numpy method
t = sequence.pad_sequences(v, maxlen = 100, padding = 'post') # Keras text processing method
Above two methods returned the t with correct shape (1000, 100), but each array t[n] (n from 0 to 99) is a zero vector [0, 0, 0, ....0]
Following numpy.pad documentation, I tried
np.pad(v, [(0,0), (16,16)], 'constant')
with the expected result: 16 columns of zeros added on the left, and 16 on the right.
Just working on a CNN and am stuck on a tensor algorithm.
I want to be able to iterate through a list, or tuple, of dimensions and choose a range of elements of X (a multi dimensional array) from that dimension, while leaving the other dimensions alone.
x = np.random.random((10,3,32,32)) #some multi dimensional array
dims = [2,3] #aka the 32s
#for a dimension in dims
#I want the array of numbers from i:i+window in that dimension
#something like
arr1 = x.index(i:i+3,axis = dim[0])
#returns shape 10,3,3,32
arr2 = arr1.index(i:i+3,axis = dim[1])
#returns shape 10,3,3,3
np.take should work for you (read its docs)
In [237]: x=np.ones((10,3,32,32),int)
In [238]: dims=[2,3]
In [239]: arr1=x.take(range(1,1+3), axis=dims[0])
In [240]: arr1.shape
Out[240]: (10, 3, 3, 32)
In [241]: arr2=x.take(range(1,1+3), axis=dims[1])
In [242]: arr2.shape
Out[242]: (10, 3, 32, 3)
You can try slicing with
arr1 = x[:,:,i:i+3,:]
and
arr2 = arr1[:,:,:,i:i+3]
Shape is then
>>> x[:,:,i:i+3,:].shape
(10, 3, 3, 32)