So, I want to slice my 3d array to skip the first 2 arrays and then return the next two arrays. And I want the slice to keep following this pattern, alternating skipping 2 and giving 2 arrays etc.. I have found a solution, but I was wondering if there is a more elegant way to go about this? Preferably without having to reshape?
arr = np.arange(1, 251).reshape((10, 5, 5))
sliced_array = np.concatenate((arr[2::4], arr[3::4]), axis=1).ravel().reshape((4, 5, 5))
You can use boolean indexing using a mask that repeats [False, False, True, True, ...]:
import numpy as np
arr = np.arange(1, 251).reshape((10, 5, 5))
mask = np.arange(arr.shape[0]) % 4 >= 2
out = arr[mask]
out:
array([[[ 51, 52, 53, 54, 55],
[ 56, 57, 58, 59, 60],
[ 61, 62, 63, 64, 65],
[ 66, 67, 68, 69, 70],
[ 71, 72, 73, 74, 75]],
[[ 76, 77, 78, 79, 80],
[ 81, 82, 83, 84, 85],
[ 86, 87, 88, 89, 90],
[ 91, 92, 93, 94, 95],
[ 96, 97, 98, 99, 100]],
[[151, 152, 153, 154, 155],
[156, 157, 158, 159, 160],
[161, 162, 163, 164, 165],
[166, 167, 168, 169, 170],
[171, 172, 173, 174, 175]],
[[176, 177, 178, 179, 180],
[181, 182, 183, 184, 185],
[186, 187, 188, 189, 190],
[191, 192, 193, 194, 195],
[196, 197, 198, 199, 200]]])
Since you want to select, and skip, the same numbers, reshaping works.
For a 1d array:
In [97]: np.arange(10).reshape(5,2)[1::2]
Out[97]:
array([[2, 3],
[6, 7]])
which can then be ravelled.
Generalizing to more dimensions:
In [98]: x = np.arange(100).reshape(10,10)
In [99]: x.reshape(5,2,10)[1::2,...].reshape(-1,10)
Out[99]:
array([[20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
[60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
[70, 71, 72, 73, 74, 75, 76, 77, 78, 79]])
I won't go on to 3d because the display will be longer, but it should be straight forward.
Related
I have one numpy array that looks like this:
array([ 0, 1, 2, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16,
18, 19, 20, 22, 27, 28, 29, 32, 33, 34, 36, 37, 38,
39, 42, 43, 44, 45, 47, 48, 51, 52, 54, 55, 56, 60,
65, 66, 67, 68, 69, 70, 71, 73, 74, 75, 77, 78, 80,
81, 82, 83, 84, 85, 86, 87, 88, 89, 92, 94, 95, 97,
98, 100, 101, 102, 105, 106, 108, 109, 113, 114, 117, 118, 119,
121, 123, 124, 126, 127, 128, 129, 131, 132, 133, 134, 135, 137,
138, 141, 142, 143, 144, 145, 147, 148, 149, 152, 154, 156, 157,
159, 160, 161, 163, 165, 166, 167, 168, 169, 170, 172, 176, 177,
179, 180, 182, 183, 185, 186, 187, 188, 191, 192, 194, 196, 197,
199, 200, 201, 202, 204, 205, 206, 207, 208])
I'm able to convert this to a set using set() no problem
However, I have another numpy array that looks like:
array([[ 2],
[ 4],
[ 10],
[ 10],
[ 12],
[ 13],
[ 14],
[ 16],
[ 19],
[ 21],
[ 21],
[ 22],
[ 29],
[209]])
When I try to use set() this gives me an error: TypeError: unhashable type: 'numpy.ndarray'
How can I convert my second numpy array to look like the first array and so I will be able to use set()?
For reference my second array is converted from a PySpark dataframe column using:
np.array(data2.select('row_num').collect())
And both arrays are used with set() in:
count = sorted(set(range(data1)) - set(np.array(data2.select('row_num').collect())))
As mentioned, use ravel to return a contiguous flattened array.
import numpy as np
arr = np.array(
[[2], [4], [10], [10], [12], [13], [14], [16], [19], [21], [21], [22], [29], [209]]
)
print(set(arr.ravel()))
Outputs:
{2, 4, 10, 12, 13, 14, 16, 209, 19, 21, 22, 29}
This is somewhat equivalent to doing a reshape with a single dimension being the array size:
print(set(arr.reshape(arr.size)))
I have a numpy array of shape (20,512,512).
I want to get a list of arrays of length 20 matching the array at points (:,100,100) (:,105,100), (:,110,100)
and so on.
I understand i need to use map for it, but how do I do it exactly?
many thanks
Yuval
you could achieve this by index slicing.
here's an example for a 1D array.
a = np.arange(200)
a[100::5] # from index 100 to end with increments of 5
>>> array([100, 105, 110, 115, 120, 125, 130, 135, 140, 145, 150, 155, 160,
165, 170, 175, 180, 185, 190, 195])
for your case
shape = [20,512,512]
arr = np.random.randint(0, 100, shape) # creating an array with the required shape
arr[:, 100::5, 100] # index slicing
>>> array([[88, 74, 45, ..., 72, 33, 63],
[88, 26, 53, ..., 47, 78, 16],
[26, 54, 85, ..., 89, 81, 66],
...,
[76, 1, 11, ..., 3, 74, 7],
[93, 34, 84, ..., 84, 73, 79],
[77, 10, 61, ..., 4, 21, 19]])
arr[:, 100::5, 100].shape
>>> (20, 83)
I do not understand why the two cases below give different results.
I want to reorder over axis 1 (0-based counting), and only take the elements with index 0 on axis 3.
(Changing (1,2,0) to [1,2,0] or np.array((1,2,0)) makes no difference.)
>> w # (3,3,5,5) input
array([[[[206, 172, 9, 2, 43],
[232, 101, 85, 251, 150],
[247, 99, 6, 88, 100],
[250, 124, 244, 2, 73],
[ 49, 23, 227, 3, 125]],
[[110, 162, 246, 123, 110],
[ 67, 197, 87, 230, 29],
[110, 51, 79, 136, 155],
[ 86, 62, 121, 18, 113],
[ 59, 197, 149, 112, 172]],
[[198, 231, 137, 2, 238],
[ 47, 97, 94, 102, 206],
[ 1, 232, 189, 173, 75],
[207, 171, 40, 23, 102],
[243, 232, 13, 109, 26]]],
[[[114, 218, 50, 173, 95],
[ 92, 29, 170, 247, 42],
[ 75, 251, 65, 246, 231],
[151, 210, 79, 27, 175],
[105, 55, 224, 79, 4]],
[[172, 230, 0, 115, 38],
[ 10, 165, 169, 230, 163],
[159, 142, 15, 134, 124],
[ 91, 161, 19, 103, 214],
[102, 168, 181, 20, 75]],
[[ 78, 65, 245, 29, 155],
[ 40, 108, 198, 180, 231],
[202, 47, 60, 156, 183],
[210, 74, 18, 113, 148],
[231, 177, 240, 15, 200]]],
[[[ 28, 40, 169, 249, 218],
[ 96, 205, 3, 38, 106],
[229, 129, 78, 113, 13],
[243, 170, 186, 35, 74],
[111, 224, 132, 184, 23]],
[[ 21, 181, 126, 5, 42],
[135, 93, 133, 166, 111],
[ 85, 85, 31, 220, 124],
[ 61, 5, 94, 216, 135],
[ 4, 225, 204, 128, 115]],
[[ 63, 23, 122, 146, 140],
[245, 139, 76, 173, 12],
[ 31, 195, 239, 188, 254],
[253, 231, 187, 22, 15],
[ 59, 40, 61, 185, 216]]]], dtype=uint16)
>> w[:,(1,2,0),:,0:1] # case 1 without squeeze
array([[[[110],
[ 67],
[110],
[ 86],
[ 59]],
[[198],
[ 47],
[ 1],
[207],
[243]],
[[206],
[232],
[247],
[250],
[ 49]]],
[[[172],
[ 10],
[159],
[ 91],
[102]],
[[ 78],
[ 40],
[202],
[210],
[231]],
[[114],
[ 92],
[ 75],
[151],
[105]]],
[[[ 21],
[135],
[ 85],
[ 61],
[ 4]],
[[ 63],
[245],
[ 31],
[253],
[ 59]],
[[ 28],
[ 96],
[229],
[243],
[111]]]], dtype=uint16)
>> w[:,(1,2,0),:,0:1].squeeze() # case 1 with squeeze, for readability and comparability
array([[[110, 67, 110, 86, 59],
[198, 47, 1, 207, 243],
[206, 232, 247, 250, 49]],
[[172, 10, 159, 91, 102],
[ 78, 40, 202, 210, 231],
[114, 92, 75, 151, 105]],
[[ 21, 135, 85, 61, 4],
[ 63, 245, 31, 253, 59],
[ 28, 96, 229, 243, 111]]], dtype=uint16)
>> w[:,(1,2,0),:,0] # case 2: 0 index instead of length-1 slice 0:1
array([[[110, 67, 110, 86, 59],
[172, 10, 159, 91, 102],
[ 21, 135, 85, 61, 4]],
[[198, 47, 1, 207, 243],
[ 78, 40, 202, 210, 231],
[ 63, 245, 31, 253, 59]],
[[206, 232, 247, 250, 49],
[114, 92, 75, 151, 105],
[ 28, 96, 229, 243, 111]]], dtype=uint16)
I have a nested numpy array (dtype=object), it contains 333 arrays that increase consistently from size 52x1 to size 52x333
I would like to effectively extract and concatenate these arrays so that I have a single 52x55611 array
I imagine this may be straightforward but my attempts using numpy.reshape have been unsuccesful
If you want to stack them along the second axis, you can use numpy.hstack.
list_of_arrays = [ array_1, ..., array_n] #all these arrays have same shape[0]
big_array = np.hstack( list_of_arrays)
if I have understood you correctly, you could use numpy.concatenate.
>>> import numpy as np
>>> a = np.array([range(52)])
>>> b = np.array([range(52,104), range(104, 156)])
>>> np.concatenate((a,b))
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38,
39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51],
[ 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,
78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103],
[104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,
130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,
143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155]])
>>>
I'm using FFmpeg to decode a video, and am piping the RGB24 raw data into python.
So the format of the binary data is:
RGBRGBRGBRGB...
I need to convert this into a (640, 360, 3) numpy array, and was wondering if I could use reshape for this and, especially, how.
If rgb is a bytearray with 3 * 360 * 640 bytes, all you need is :
np.array(rgb).reshape(640, 360, 3)
As an example:
>>> import random
>>> import numpy as np
>>> bytearray(random.getrandbits(8) for _ in range(3 * 4 * 4))
bytearray(b'{)jg\xba\xbe&\xd1\xb9\xdd\xf9#\xadL?GV\xca\x19\xfb\xbd\xad\xc2C\xa8,+\x8aEGpo\x04\x89=e\xc3\xef\x17H#\x90]\xd5^\x94~/')
>>> rgb = bytearray(random.getrandbits(8) for _ in range(3 * 4 * 4))
>>> np.array(rgb)
array([112, 68, 7, 41, 175, 109, 124, 111, 116, 6, 124, 168, 146,
60, 125, 133, 1, 74, 251, 194, 79, 14, 72, 236, 188, 56,
52, 145, 125, 236, 86, 108, 235, 9, 215, 49, 190, 16, 90,
9, 114, 43, 214, 65, 132, 128, 145, 214], dtype=uint8)
>>> np.array(rgb).reshape(4,4,3)
array([[[112, 68, 7],
[ 41, 175, 109],
[124, 111, 116],
[ 6, 124, 168]],
[[146, 60, 125],
[133, 1, 74],
[251, 194, 79],
[ 14, 72, 236]],
[[188, 56, 52],
[145, 125, 236],
[ 86, 108, 235],
[ 9, 215, 49]],
[[190, 16, 90],
[ 9, 114, 43],
[214, 65, 132],
[128, 145, 214]]], dtype=uint8)
You might want to look at existing numpy and scipy methods for image processing. misc.imread could be interesting.