We Keep Coding

Getting each column in a 3d numpy array

I converted an image from RBG to CieLab, now I need to use the value of the cielab to calculate some equations.
I have been trying to get the value of each column in the array. For example if I have:
List =
[[[ 65 234 169]
[203 191 245]
[ 36 58 196]
[207 208 143]
[251 208 187]]
[[ 79 69 237]
[ 13 124 42]
[104 165 82]
[170 178 178]
[ 66 42 210]]
[[ 40 163 219]
[142 37 140]
[ 75 205 143]
[246 30 221]
[ 16 98 102]]]
How can I get it to give me the values of each columns like:
1st_column =
65
203
36
207
251
79
13
104
170
66
40
142
75
246
16
Thank you.

Try:
>>> m[:, :, 0]
array([[ 65, 203, 36, 207, 251],
[ 79, 13, 104, 170, 66],
[ 40, 142, 75, 246, 16]])
As suggested by #mozway, you can use the ellipsis syntax: m[..., 0].
To know more, read How do you use the ellipsis slicing syntax in Python?
You can also flatten your array:
>>> m[:, :, 0].flatten()
array([ 65, 203, 36, 207, 251, 79, 13, 104, 170, 66, 40, 142, 75, 246, 16])

KeyError: class 'numpy.object_' while downloading image dataset using imread

I am trying to download images from URLs using imread. After downloading about 700 images, I see KeyError: class 'numpy.object_' . I am really not familiar with numpy and Conda libraries. Any help would be appreciated
for i in range(len(classes)):
if not os.path.exists(saved_dirs[i]):
os.mkdir()
saved_dir = saved_dirs[i]
for url in urls[i]:
# print(url)
img = io.imread(url)
saved_path = os.path.join(saved_dir, url[-20:])
io.imsave(saved_path, img)
Trigger URL: https://requestor-proxy.figure-eight.com/figure_eight_datasets/open-images/test/f54ecb040198098c.jpg
Printing trigger image:
[array([[[ 70, 141, 143],
[ 71, 142, 146],
[ 67, 141, 144],
...,
[242, 253, 255],
[242, 253, 255],
[242, 253, 255]],
[[ 61, 135, 136],
[ 64, 135, 137],
[ 63, 134, 138],
...,
[242, 253, 255],
[242, 253, 255],
[242, 253, 255]],
[[ 57, 133, 133],
[ 63, 137, 138],
[ 65, 136, 138],
...,
[242, 253, 255],
[242, 253, 255],
[242, 253, 255]],
...,
[[246, 244, 255],
[244, 242, 253],
[244, 242, 253],
...,
[ 21, 43, 100],
[ 20, 40, 101],
[ 22, 42, 103]],
[[244, 243, 251],
[243, 241, 252],
[242, 240, 251],
...,
[ 26, 49, 103],
[ 25, 45, 104],
[ 25, 45, 106]],
[[244, 243, 251],
[243, 242, 250],
[243, 242, 250],
...,
[ 27, 48, 103],
[ 26, 45, 104],
[ 26, 44, 106]]], dtype=uint8)
array(<PIL.MpoImagePlugin.MpoImageFile image mode=RGB size=3872x2592 at 0x17D8F7434E0>,
dtype=object)]
The ouput:
KeyError Traceback (most recent call last)
<ipython-input-22-b76e704a99f8> in <module>
8 img = io.imread(url)
9 saved_path = os.path.join(saved_dir, url[-20:])
---> 10 io.imsave(saved_path, img)
~\Anaconda3\lib\site-packages\skimage\io\_io.py in imsave(fname, arr, plugin, **plugin_args)
137 if fname.lower().endswith(('.tiff', '.tif')):
138 plugin = 'tifffile'
--> 139 if is_low_contrast(arr):
140 warn('%s is a low contrast image' % fname)
141 if arr.dtype == bool:
~\Anaconda3\lib\site-packages\skimage\exposure\exposure.py in is_low_contrast(image, fraction_threshold, lower_percentile, upper_percentile, method)
501 image = rgb2gray(image)
502
--> 503 dlimits = dtype_limits(image, clip_negative=False)
504 limits = np.percentile(image, [lower_percentile, upper_percentile])
505 ratio = (limits[1] - limits[0]) / (dlimits[1] - dlimits[0])
~\Anaconda3\lib\site-packages\skimage\util\dtype.py in dtype_limits(image, clip_negative)
55 warn('The default of `clip_negative` in `skimage.util.dtype_limits` '
56 'will change to `False` in version 0.15.')
---> 57 imin, imax = dtype_range[image.dtype.type]
58 if clip_negative:
59 imin = 0
KeyError: <class 'numpy.object_'>

Edit
I reproduced your results with:
from skimage import io
url = "https://requestor-proxy.figure-eight.com/figure_eight_datasets/open-images/test/f54ecb040198098c.jpg"
img = io.imread(url)
print(img.shape)
Turns out, this actually returns two images or layers of the same image, where img[0] is the actual image you want, and img[1] is some faulty reading in Pillow (the module skimage is using to read images).
Check this issue out.
For now, some quick workaround should be fine.
if img.shape[0] == 2:
img = img[0]
Original
Could you show the URL that triggers this error? It could be your formatting on url[-20:] that adds some funky extension. Also, I'd recommend just printing img and img.shape and img.dtype to have a better idea of what's going on.

Time series with matrix

I have a mat extension data which I want to separate every seconds values. My matrix is (7,5,2500) time series 3 dimensional matrix which want to get the values of (7,5,1) ...(7,5,2500) separately and save it
for example
array([155, 33, 129,167,189,63,35
161, 218, 6,58,36,25,3
89,63,36,25,78,95,21
78,52,36,56,25,15,68
]],
[215, 142, 235,
143, 249, 164],
[221, 71, 229,
56, 91, 120],
[236, 4, 177,
171, 105, 40])
for getting every part of this data for example this matrix
[215, 142, 235,
143, 249, 164]
what should I do?

a = [[155, 33, 129, 161, 218, 6],
[215, 142, 235, 143, 249, 164],
[221, 71, 229, 56, 91, 120],
[236, 4, 177, 171, 105, 40]]
print(a[1])

Assuming you have your data saved in a numpy array you could use slicing to extract the sub-matrices you need. Here is an example with a (3,5,3) matrix (but the example could be applied to any dimension):
A = numpy.array([[[1,1,1],
[2,2,2],
[3,3,3],
[4,4,4],
[5,5,5]],
[[11,11,11],
[21,21,21],
[31,31,31],
[41,41,41],
[51,51,51]],
[[12,12,12],
[22,22,22],
[32,32,32],
[42,42,42],
[52,52,52]]]
sub_matrix_1 = A[:,:,0]
print (sub_matrix_1)
Will produce:
[[ 1 2 3 4 5]
[11 21 31 41 51]
[12 22 32 42 52]]
EDIT: it is also possible to iterate over the array to get the 3rd dimension array:
for i in range(A.shape[-1]):
print (A[:,:,i])
# Your submatrix is A[:,:,i], you can directly manipulate it

How to use numpy to flip this array?

Suppose I have an array like this:
a = array([[[ 29, 29, 27],
[ 36, 38, 40],
[ 86, 88, 89]],
[[200, 200, 198],
[199, 199, 197]
[194, 194, 194]]])
and I want to flip the 3rd element from left to right in the list-of-lists so it will become like this:
b = array([[[ 29, 29, 89], # 27 became 89
[ 36, 38, 40],
[ 86, 88, 27]], # 89 became 27
[[200, 200, 194], # 198 became 194
[199, 199, 197],
[194, 194, 198]]]) # 194 became 198
I looked up the NumPy manual but I still cannot figure out a solution. .flip and .fliplr look suitable in this case, but how do I use them?

Index the array to select the sub-array, using:
> a[:,:,-1]
array([[198, 197, 194],
[ 27, 40, 89]])
This selects the last element along the 3rd dimension of a. The sub-array is of shape (2,3). Then reverse the selection using:
a[:,:,-1][:,::-1]
The second slice, [:,::-1], takes everything along the first dimension as-is ([:]), and all of the elements along the second dimension, but reversed ([::-1]). The slice syntax is basically saying start at the first element, go the last element ([:]), but do it in the reverse order ([::-1]). You could pseudo-code write it as [start here : end here : use this step size]. The the -1 tells it walk backwards.
And assign it to the first slice of the original array. This updates/overwrites the original value of a
a[:,:,-1] = a[:,:,-1][:,::-1]
> a
array([[[ 29, 29, 89],
[ 36, 38, 40],
[ 86, 88, 27]],
[[200, 200, 194],
[199, 199, 197],
[194, 194, 198]]])

Product of two equals numpy arrays are different

I'm facing very strange problem with arrays in python and numpy. First of all what Im trying to archive is :
1) Get an MxN matrix from KxTxN matrix
2) Transpose this matrix and calculate product of this transposed matrix and the original one
What I get is some what strange, here comes the code :
First of all, I have read an image with help of cv2, and got K by T by 3 matrix (a field of RGB points), then I'm cutting a small window form it and reshaping this window to M by N matrix :
def clipSubwindowFromImage(img, i, j, winSize):
winI = img[i - winSize: i + winSize + 1, j - winSize : j + winSize + 1, : ]
res = np.vstack((winI[:,::3,:].reshape(winI.shape[1],3), winI[:,1::3,:].reshape(winI.shape[1],3), winI[:,2::3,:].reshape(winI.shape[1],3)))
return res
so far so god, say we had winSize = 1, i = 1, j = 1 and got a 9x3 matrix as a result: this matrix :
>> subWin = clipSubwindowFromImage(background12x12b, 1, 1, 1)
>> [[201 199 187]
[216 219 198]
[226 228 207]
[243 241 228]
[240 244 221]
[233 235 213]
[239 238 220]
[238 240 216]
[233 235 211]]
Then I just want to get the product in question, like this :
>>r1 = subWin.T.dot(subWin)
>>[[197 234 89]
[234 65 163]
[ 89 163 105]]
Well, it's not right, the right result should be :
>>[[477125 479466 438361]
[479466 481857 440483]
[438361 440483 402793]]
But if I initialize subWin manually like this :
>>subWin = np.array([[201, 199, 187], [216, 219, 198], [226, 228, 207], [243, 241, 228], [240, 244, 221], [233, 235, 213],[239, 238, 220], [238, 240, 216],[233, 235, 211]])
I get right result.
I can't get it, subWin is the SAME array in both cases (I checked it). Any ideas?

As #Aguy said, your problem comes from the data-type of your array. The dot product of a uint8 array with an other uint8 array gives an array that is also uint8 hence the data-type is overflowed in your case. Here's an example that shows the effect of overflow on your values:
import numpy as np
a = np.array([[201, 199, 187], [216, 219, 198], [226, 228, 207], [243, 241, 228], [240, 244, 221], [233, 235, 213],[239, 238, 220], [238, 240, 216],[233, 235, 211]])
b = a.T.dot(a)
print b.dtype
print b
print "overflowed uint8 :"
print b.astype(np.uint8)
Gives:
>>> int64
>>> [[477125 479466 438361]
>>> [479466 481857 440483]
>>> [438361 440483 402793]]
>>> overflowed uint8 :
>>> [[197 234 89]
>>> [234 65 163]
>>> [ 89 163 105]]
Just change the data-type of one array to something more suitable in your dot product and you're good to go :
r1 = subWin.T.dot(subWin.astype(np.uint32))