Just working on a CNN and am stuck on a tensor algorithm.
I want to be able to iterate through a list, or tuple, of dimensions and choose a range of elements of X (a multi dimensional array) from that dimension, while leaving the other dimensions alone.
x = np.random.random((10,3,32,32)) #some multi dimensional array
dims = [2,3] #aka the 32s
#for a dimension in dims
#I want the array of numbers from i:i+window in that dimension
#something like
arr1 = x.index(i:i+3,axis = dim[0])
#returns shape 10,3,3,32
arr2 = arr1.index(i:i+3,axis = dim[1])
#returns shape 10,3,3,3
np.take should work for you (read its docs)
In [237]: x=np.ones((10,3,32,32),int)
In [238]: dims=[2,3]
In [239]: arr1=x.take(range(1,1+3), axis=dims[0])
In [240]: arr1.shape
Out[240]: (10, 3, 3, 32)
In [241]: arr2=x.take(range(1,1+3), axis=dims[1])
In [242]: arr2.shape
Out[242]: (10, 3, 32, 3)
You can try slicing with
arr1 = x[:,:,i:i+3,:]
and
arr2 = arr1[:,:,:,i:i+3]
Shape is then
>>> x[:,:,i:i+3,:].shape
(10, 3, 3, 32)
Related
I have two numpy arrays, one with shape let's say (10, 5, 200), and another one with the shape (1, 200), how can I stack them so I get as a result an array of dimensions (10, 6, 200)? Basically by stacking it to each 2-d array iterating along the first dimension
a = np.random.random((10, 5, 200))
b = np.zeros((1, 200))
I'v tried with hstack and vstack but I get an error in incorrect number of axis
Let's say:
a = np.random.random((10, 5, 200))
b = np.zeros((1, 200))
Let's look at the volume (number of elements) of each array:
The volume of a is 10*5*200 = 10000.
The volume of an array with (10,6,200) is 10*5*200=1200.
That is you want to create an array that has 2000 more elements.
However, the volume of b is 1*200 = 200.
This means a and b can't be stacked.
As hpaulj mentioned in the comments, one way is to define an numpy array and then fill it:
result = np.empty((a.shape[0], a.shape[1] + b.shape[0], a.shape[2]))
result[:, :a.shape[1], :] = a
result[:, a.shape[1]:, :] = b
A n-dimensional array which is initialised as
features=np.empty(shape=(100,5,2), dtype=float)
and I am trying to add 3D array into it as
features[i,:] = features_next
features_next has shape (2,5,2).
However, it shows error,
ValueError: could not broadcast input array from shape (2,5,2) into shape (5,2).
here is the piece of code :
features=np.empty(shape=((historical*2),5,2), dtype=float)
i = 0
while i < 50:
state = self.getDictState(state_index)
asks = state['asks']
bids = state['bids']
features_next = self.getNormalisedFeature(
bids=bids,
asks=asks,
state_index=state_index,
qty=qty,
price=price,
size=size,
normalize=normalize,
levels=levels
)
'''if i == 0:
features = np.array(features_next)
else:
features = np.vstack((features, features_next))'''
features[i,:] = features_next
state_index = (state_index - 1)
return features
Note : I am trying to replace commented 'if condition' with features[i,:] = features_next to make the code execution bit faster.
Its pretty simple, just one point to make features[i,:] has shape (5, 2) and feature_next has shape (2, 5, 2). I want to say that both are compatible shapes. But broadcasting is done on a smaller shape over a larger shape. SO there is error since you are doing revere. Also look up on broadcasting on numpy docs.
Next, this one I think will do
This does not directly solve your problem, but has some ideas about what you can try, like you can reshape your array before going in loop using features.shape = (50, 2, 5, 2).
import numpy as np
features=np.empty(shape=(100,5,2), dtype=float)
features_next = np.random.random((2, 5, 2))
features.shape = ((50, 2, 5, 2))
features[:] = features_next
features.shape = (100, 5, 2)
The question is on how to join two arrays in this case more efficiently- There's a numpy array one of shape (N, M, 1) and array two of shape (M,F). It's required to join the second array with the first, to create an array of the shape (N, M, F+1). The elements of the second array will be broadcast along N.
One solution is copying array 2 to have size of the first (along all dims but one) and then concatenate. But this if the copying can be done as a broadcast during the join/concat it would use much lesser memory.
Any suggestions on how to make this more efficient?
The setup:
import numpy as np
arr1 = np.random.randint(0,10,(5,10))
arr1 = np.expand_dims(arr1, axis=-1) #(5,10, 1)
arr2 = np.random.randint(0,4,(10,15))
arr2 = np.expand_dims(arr2, axis=0) #(1, 10, 15)
arr2_2 = arr2
for i in range(len(arr1)-1):
arr2_2 = np.concatenate([arr2_2, arr2],axis=0)
arr2_2.shape #(5, 10, 15)
np.concatenate([arr1, arr2_2],axis=-1) # (5, 10, 16) -> correct end result
Joining arr1 and arr2 to get
try this
>>> a = np.random.randint(0, 10, (5, 10))
>>> b = np.random.randint(0, 4, (10, 15))
>>> c = np.dstack((a[:, :, np.newaxis], np.broadcast_to(b, (a.shape[0], *b.shape))))
>>> a.shape, b.shape, c.shape
((5, 10), (10, 15), (5, 10, 16)))
I have some numpy arrays which its elements are pixels of 28*28 images like this:
25 of these arrays are in one array in shape of (25,28,28) or (5,5,28,28). Is there any efficient way to stack them to have one image: 5*5 of 28*28 images.
I tried np.reshape to (140,140) array and plt.imgshow. But the output was a messed image.
"I tried np.reshape to (140,140)..." That will work if you first transpose the input appropriately.
Suppose the input x has shape (5, 5, 28, 28). To get the array y with shape (140, 140) that contains the images arranged the way you want, you can do:
xshp = x.shp
y = x.transpose((0, 2, 1, 3)).reshape((xshp[0]*xshp[2], xshp[1]*xshp[3]))
If x always has shape (5, 5, 28, 28), you can hardcode the constant 140:
y = x.transpose((0, 2, 1, 3)).reshape((140, 140))
For example, here I create x with shape (5, 5, 28, 28) where each 28x28 image is a constant. The constants are chosen randomly. The tranposed, reshaped array y is plotted, and you can see that all the constant blocks are arranged correctly.
In [148]: rng = np.random.default_rng()
In [149]: x = np.repeat(rng.integers(0, 256, size=(5, 5)), 28*28, axis=-1).reshape((5, 5, 28, 28))
In [150]: y = x.transpose((0, 2, 1, 3)).reshape((140, 140))
In [151]: imshow(y)
i have 3 numpy arrays which store image data of shape (4,100,100).
arr1= np.load(r'C:\Users\x\Desktop\py\output\a1.npy')
arr2= np.load(r'C:\Users\x\Desktop\py\output\a2.npy')
arr3= np.load(r'C:\Users\x\Desktop\py\output\a3.npy')
I want to merge all 3 arrays into 1 array.
I have tried in this way:
merg_arr = np.zeros((len(arr1)+len(arr2)+len(arr3), 4,100,100), dtype=input_img.dtype)
now this make an array of the required length but I don't know how to copy all the data in this array. may be using a loop?
This will do the trick:
merge_arr = np.concatenate([arr1, arr2, arr3], axis=0)
np.stack arranges arrays along a new dimension. Their dimensions (except for the first) need to match.
Demo:
arr1 = np.empty((60, 4, 10, 10))
arr2 = np.empty((14, 4, 10, 10))
arr3 = np.empty((6, 4, 10, 10))
merge_arr = np.concatenate([arr1, arr2, arr3], axis=0)
print(merge_arr.shape) # (80, 4, 10, 10)