I have a numpy array that looks like as follows:
[ 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
2. 2. 2. 2. 2. 2. 2. 2. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3.
3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 4. 4. 4. 4. 4. 4.
4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 5. 5.
5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5.
5. 5. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6.
6. 6. 6. 6. 6. 6. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 2. 2. 2. 2. 2. 2. 2. 2.
2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 3. 3. 3. 3.
3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3.
4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4.
4. 4. 4. 4. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5.
5. 5. 5. 5. 5. 5. 5. 5. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6.
6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 2. 2.
2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
2. 2. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3.
3. 3. 3. 3. 3. 3. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4.
4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 5. 5. 5. 5. 5. 5. 5. 5.
5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 6. 6. 6. 6.
6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6.
1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
2. 2. 2. 2. 2. 2. 2. 2. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3.
3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 4. 4. 4. 4. 4. 4.
4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 4. 5. 5.
5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5. 5.
5. 5. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6. 6.
6. 6. 6. 6. 6. 6.]
Now I need to re-arrange the array without any spliting or slicing such that all of the 1s, 2s,---6s are grouped together. What would be the best way? The numbers change after every 22 elements. So the new array should have 88 entries of each number.
Part b of the question:
If I have an extended form of the above array with shape (528, 32768) and the values in the array are now completely random and I need to group values at the indices of 1s, 2s, 3s....... together with no sorting; what would be the best way?
This may not be the fastest solution:
array( sorted( a ) )
This is your array
import numpy as np
arr = ((np.arange(24) % 6 + 1)[:, np.newaxis] * np.ones(22)).ravel()
Or this
arr = ((np.arange(6)[:, np.newaxis] * np.ones(22))[np.newaxis]
* np.ones([4, 1, 1])).ravel() + 1
Now try
arr.reshape(-1, 6, 22).transpose(1, 0, 2).reshape(6, -1)
This is a different view on the array up until the transpose. On the second reshape numpy has to copy.
Here is a way to rearrange data coming out of a 4-channel spectrometer with data specification described in the comments:
# data vectors stacked (4 * 132)
data = np.random.randn(528, 32768)
# data vectors divided into original 4 blocks
data.reshape(4, 132, 32768)
# data vectors subdivided into 6 * 22 = 132
data.reshape(4, 6, 22, 32768)
As you see, dividing the data into the appropriate compartments works with a single (the last) reshape. The rearrangement is done with transpose as follows:
# the following operation switches dimensions 0 and 1
# yielding a vector of shape (6, 4, 22, 32768)
data.reshape(4, 6, 22, 32768).transpose(1, 0, 2, 3)
# stack all 4 outputs of 22 lines together into blocks of 88 lines
data.reshape(4, 6, 22, 32768).transpose(1, 0, 2, 3).reshape(6, 88, 32768)
# concatenate everything back to get a reordered matrix
data.reshape(4, 6, 22, 32768).transpose(1, 0, 2, 3).reshape(6 * 88, 32768)
Related
I have the following np.array():
[[55.3 1. 2. 2. 2. 2. ]
[55.5 1. 2. 0. 2. 2. ]
[54.9 2. 2. 2. 2. 2. ]
[47.9 2. 2. 2. 0. 0. ]
[57. 1. 2. 2. 0. 2. ]
[56.6 1. 2. 2. 2. 2. ]
[54.7 1. 2. 2. 2. nan]
[51.4 2. 2. 2. 2. 2. ]
[55.3 2. 2. 2. 2. nan]]
And I would Like to get the following one :
[[1. 2. 2. 2. 2. ]
[1. 2. 0. 2. 2. ]
[2. 2. 2. 2. 2. ]
[2. 2. 2. 0. 0. ]
[1. 2. 2. 0. 2. ]
[1. 2. 2. 2. 2. ]
[1. 2. 2. 2. nan]
[2. 2. 2. 2. 2. ]
[2. 2. 2. 2. nan]]
I did try :
MyArray[1:]#But this delete the first line
np.delete(MyArray, 0, 1) #Where I don't understand the output
[[ 2. 2. 2. 2. 2.]
[ 1. 2. 2. 2. 2.]
[ 1. 2. 0. 2. 2.]
[ 2. 2. 2. 2. 2.]
[ 2. 2. 2. 0. 0.]
[ 1. 2. 2. 0. 2.]
[ 1. 2. 2. 2. 2.]
[ 1. 2. 2. 2. nan]
[ 2. 2. 2. 2. 2.]
[ 2. 2. 2. 2. nan]]
You made a bit of a mistake using np.delete,
The np.delete arguments are array,list of indexes to be deleted, axis. By using the below snippet you get the output you want.
arr=np.delete(arr,[0],1)
The problem you created was, you passed integer instead of a list, which is why it isn't giving correct output.
You could try: new_array = [i[1:] for i in MyArray]
Try MyArray[:,1:]
I think you can get rid of column 0 with this
It should be straight forward with
new_array = MyArray[:, 1:]
See this link for explanation and examples.
Or this link
I am looking to merge NumPy array elements in a list into a single NumPy array. How can I do this?
This is how the list containing arrays is structured and the code I tried:
import numpy as np
baked_quad_vertices = []
A = (1,2,3,4,5,
1,2,3,4,5,
1,2,3,4,5,
1,2,3,4,5)
A = np.array(A, dtype=np.float32)
B = (1,2,3,4,5,
1,2,3,4,5,
1,2,3,4,5,
1,2,3,4,5)
B = np.array(B, dtype=np.float32)
baked_quad_vertices.append(A)
baked_quad_vertices.append(B)
Z = baked_quad_vertices
Z = np.vstack(Z)
print(Z)
I get:
[[1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5.]
[1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5.]]
I want:
[1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5.
1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5.]
Optimally I'd want:
[1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5.
1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5., dtype=np.float32]
To get the result you want, try using np.hstack instead of np.vstack.
Editor's note: original answer below is referring to Revision 1 of the question:
This looks wrong as each numpy array is still separated, what does the ... mean?
In fact, when you print an array it looks just like that. The output of np.vstack returns an array so you should have an array. Try printing:
print(type(baked_quad_vertices[chunk_count]))
In python, how to create a matrix or 2D array of N x N such that :
[A] Each Row has non-duplicate integers from 0 : N-1
And [B] Each Column has non-duplicate integers from 0:N-1
Example :
[[1 0 2]
[2 1 0]
[0 2 1]]
So I had a bit of a tinker with this question, this code seems to work
import numpy as np
N = 10
row = np.arange(N)
result = np.zeros((N, N))
for i in row:
result[i] = np.roll(row, i)
print(result)
output:
[[0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
[9. 0. 1. 2. 3. 4. 5. 6. 7. 8.]
[8. 9. 0. 1. 2. 3. 4. 5. 6. 7.]
[7. 8. 9. 0. 1. 2. 3. 4. 5. 6.]
[6. 7. 8. 9. 0. 1. 2. 3. 4. 5.]
[5. 6. 7. 8. 9. 0. 1. 2. 3. 4.]
[4. 5. 6. 7. 8. 9. 0. 1. 2. 3.]
[3. 4. 5. 6. 7. 8. 9. 0. 1. 2.]
[2. 3. 4. 5. 6. 7. 8. 9. 0. 1.]
[1. 2. 3. 4. 5. 6. 7. 8. 9. 0.]]
Ask away if you have any questions.
I have a NumPy array with missing values. I want to impute the mean of the nearest values vertically.
import numpy as np
arr = np.random.randint(0, 10, (10, 4)).astype(float)
arr[2, 0] = np.nan
arr[4, 3] = np.nan
arr[0, 2] = np.nan
print(arr)
[[ 5. 7. nan 4.] # should be 4
[ 2. 6. 4. 9.]
[nan 2. 5. 5.] # should be 4.5
[ 7. 0. 3. 8.]
[ 6. 4. 3. nan] # should be 4
[ 8. 1. 2. 0.]
[ 0. 0. 1. 1.]
[ 1. 2. 6. 6.]
[ 8. 1. 9. 7.]
[ 3. 5. 8. 8.]]
If you are open to using Pandas, pd.DataFrame.interpolate is easy to use. Set limit_direction if "interpolating" values at ends of array:
df = pd.DataFrame(arr).interpolate(limit_direction='both')
df.to_numpy() # back to a numpy array if needed (if using v0.24.0 or above)
Output:
array([[5. , 7. , 4. , 4. ],
[2. , 6. , 4. , 9. ],
[4.5, 2. , 5. , 5. ],
[7. , 0. , 3. , 8. ],
[6. , 4. , 3. , 4. ],
[8. , 1. , 2. , 0. ],
[0. , 0. , 1. , 1. ],
[1. , 2. , 6. , 6. ],
[8. , 1. , 9. , 7. ],
[3. , 5. , 8. , 8. ]])
import numpy as np
arr = np.random.randint(0, 10, (10, 4)).astype(float)
arr[2, 0] = np.nan
arr[4, 3] = np.nan
arr[0, 2] = np.nan
print(arr)
[[ 5. 7. nan 4.]
[ 2. 6. 4. 9.]
[nan 2. 5. 5.]
[ 7. 0. 3. 8.]
[ 6. 4. 3. nan]
[ 8. 1. 2. 0.]
[ 0. 0. 1. 1.]
[ 1. 2. 6. 6.]
[ 8. 1. 9. 7.]
[ 3. 5. 8. 8.]]
for x, y in np.argwhere(np.isnan(arr)):
sample = arr[np.maximum(x - 1, 0):np.minimum(x + 2, 20), y]
arr[x, y] = np.mean(sample[np.logical_not(np.isnan(sample))])
print(arr)
[[5. 7. 4. 4. ] # 3rd value here is mean(4)
[2. 6. 4. 9. ]
[4.5 2. 5. 5. ] # first value here is mean(2, 7)
[7. 0. 3. 8. ]
[6. 4. 3. 4. ] # 4th value here is mean(8, 0)
[8. 1. 2. 0. ]
[0. 0. 1. 1. ]
[1. 2. 6. 6. ]
[8. 1. 9. 7. ]
[3. 5. 8. 8. ]]
I have the following arrays:
from mxnet import nd
A=nd.array([[1,1,1,1],[2,2,2,2]])
B=nd.array([[11,11,11,11],[22,22,22,22]])
Y=nd.array([[91,91,91,91],[92,92,92,92]])
Imagine that each list whithin each array corresponds to a client.
So [1,1,1,1] is the result of operation A to client 1 and [2,2,2,2] is the result of operation A to client 2.
Then I have another array with a diferent operation that is applied to all the clients. [11,11,11,11] is the result of operation B to client 1 and so on.
And I need to get the following result:
D=nd.array( [ [[1,1,1,1],[11,11,11,11]],[[2,2,2,2],[22,22,22,22]] ])
list([D,Y])
This returns:
[
[[[ 1. 1. 1. 1.]
[11. 11. 11. 11.]]
[[ 2. 2. 2. 2.]
[22. 22. 22. 22.]]]
<NDArray 2x2x4 #cpu(0)>,
[[91. 91. 91. 91.]
[92. 92. 92. 92.]]
<NDArray 2x4 #cpu(0)>]
As you can see, the operations (A and B) are grouped for each client.
I tried:
list([list(zip(A,B)),Y])
And I get:
[[(
[1. 1. 1. 1.]
<NDArray 4 #cpu(0)>,
[11. 11. 11. 11.]
<NDArray 4 #cpu(0)>), (
[2. 2. 2. 2.]
<NDArray 4 #cpu(0)>,
[22. 22. 22. 22.]
<NDArray 4 #cpu(0)>)],
[[91. 91. 91. 91.]
[92. 92. 92. 92.]]
<NDArray 2x4 #cpu(0)>]
Which is not what I need. Plus, arrays A and B are really big, so I don't want to use a loop or something that will take too long.
Thanks.
this is typically an operation you can do with an mxnet.ndarray.concat, yet you need to expand the dimension of the concatenated items before the concat so that they stay in separate arrays.
This command will get exactly the output you ask for:
C = nd.concat(A.expand_dims(axis=1), B.expand_dims(axis=1), dim=1)
print(C)
which returns:
[[[ 1. 1. 1. 1.]
[11. 11. 11. 11.]]
[[ 2. 2. 2. 2.]
[22. 22. 22. 22.]]]
<NDArray 2x2x4 #cpu(0)>