This question already has answers here:
Vectorized NumPy linspace for multiple start and stop values
(4 answers)
Closed 5 years ago.
I have a dataset that contains a multidimensional array of shape (2400, 2).
I want to be able to take each of these 2400 rows, and modify them to be a range from the start and end points (the two elements in each of the 2400 rows). The range is always the same length (in my case, a length of 60).
For example, if I have something like this:
array([[ 78, 82],
[ 90, 94],
[ 102, 106]])
My output should be something like this:
array([[ 78, 79, 80, 81, 82],
[ 90, 91, 92, 93, 94],
[ 102, 103, 104, 105, 106]])
The only way I have been able to do this is with a for loop, but I am trying to avoid looping through each row as the dataset can get very large.
Thanks!
Since by necessity all of the aranges need to be equally long, we can create an arange along the first entry and then replicate it for the others.
For example:
x = np.array([[78, 82],
[90, 94],
[102, 106]])
>>> x[:, :1] + np.arange(0, 1 + x[0, 1] - x[0, 0])
# array([[ 78, 79, 80, 81],
# [ 90, 91, 92, 93],
# [102, 103, 104, 105]])
If the difference between the second column and first column is always 4, then you can extract the first column and add an array of [0,1,2,3,4] to it:
arr = np.array([[ 78, 82],
[ 90, 94],
[ 102, 106]])
arr[:,:1] + np.arange(5)
Out[331]:
array([[ 78, 79, 80, 81, 82],
[ 90, 91, 92, 93, 94],
[102, 103, 104, 105, 106]])
Related
I would like to ask a question related to the color detection in an image in Python 3.9. I would like to detect black color at the specific x axis in the image and extract the location of the black area in the image. Could you help me how to extract the location of the black area in an image?
I can load the image and visualize it by the following code.
from PIL import Image
im =Image.open('c97dae05fd9244d4d10305b8066246psXmnlRxU59D0kJ0-0.jpeg')
im.show()
import numpy
imarray = numpy.array(im)
print(imarray)
Image.fromarray(imarray)
Some background context before I give you a solution.
Images are 3D arrays meaning the index 1 for example will be 2 arrays deep so something like this: [[[]]].
For example using numpy you can check the shape of such array using imarray.shape which should return something like (3888, 5184, 3). With this information you can see that there are 3888 arrays that contain 5184 arrays that contain 3 values.
You can maybe see a pattern occur. RGB is red, green, blue (3 values) and the third array is 3 values long which is where you will find your RGB values.
Lets take a image I had in my computer as example. I loaded it using pillow and made it into a numpy array, just like you did.
from Pill import Image
import numpy
im = Image.open('G0626494.jpg')
imarray = numpy.array(im)
Lets now print imarray:
This should print some columns and rows and RGB values, normally some from the beginning, middle and end.
array([[[ 66, 143, 185],
[ 64, 141, 183],
[ 60, 137, 179],
...,
[ 58, 96, 75],
[ 56, 94, 73],
[ 57, 95, 74]],
[[ 65, 142, 184],
[ 63, 140, 182],
[ 60, 137, 179],
...,
[ 57, 95, 74],
[ 56, 94, 73],
[ 57, 95, 74]],
[[ 63, 140, 182],
[ 62, 139, 181],
[ 59, 136, 178],
...,
[ 57, 95, 74],
[ 57, 95, 74],
[ 58, 96, 75]],
...,
[[ 80, 167, 195],
[ 80, 167, 195],
[ 82, 167, 196],
...,
[130, 190, 162],
[129, 189, 161],
[129, 189, 161]],
[[ 82, 169, 197],
[ 81, 168, 196],
[ 83, 168, 197],
...,
[128, 188, 160],
[128, 188, 160],
[128, 188, 160]],
[[ 82, 169, 197],
[ 81, 168, 196],
[ 83, 168, 197],
...,
[127, 187, 159],
[127, 187, 159],
[127, 187, 159]]], dtype=uint8)
As you can see those 3 values on the 3rd array are the RGB values of the image. How do you get them?
Lets take the first pixel as example. If you know basic array manipulation in python you probably will know this but since we know the shape is 3 this means the first pixel will be under imarray[0][0].
If we print this value we indeed get the RGB of the first pixel:
array([ 66, 143, 185], dtype=uint8)
Now how do you cycle through them. I challenge you to try this based on the past knowledge or try along this tutorial.
So if we have 2 arrays to get the RGB of the first pixel we need 2 loops right? So we can do it like this:
for row in imarray:
for column in row:
print(column)
You will now be able to see a bunch of RGB values get printed.
So lets check for black pixels now. We know black is 0, 0, 0 so we can check if any pixel is 0, 0, 0 and print it.
for row in imarray:
for column in row:
if column[0] == 0 and column[1] == 0 and column[2] == 0:
print(column)
You can also check for colors next to black but not 0, 0, 0 if you check if the column values are all the same but less then 50 for example.
I might be having a brain-freeze here but how do you iterate over the last n slices of length d of a list of indeterminate length and still get the last slice [-d:]?
What I tried:
In [37]: x = list(range(100))
In [38]: window = 15
In [39]: d = window // 3
In [40]: for i in range(0, window, d):
...: print(i, x[-i-d:-i])
...:
Output:
0 []
5 [90, 91, 92, 93, 94]
10 [85, 86, 87, 88, 89]
What I want:
0 [95, 96, 97, 98, 99]
5 [90, 91, 92, 93, 94]
10 [85, 86, 87, 88, 89]
The problem of course, is that in the last iteration, -i is zero not None. I'm sure there's an easier way than this:
In [42]: for i in range(0, window, d):
...: if i == 0:
...: print(i, x[-i-d:])
...: else:
...: print(i, x[-i-d:-i])
...:
or this:
In [44]: import numpy as np
In [45]: np.array(x)[-window:].reshape((3,-1))
Out[45]:
array([[85, 86, 87, 88, 89],
[90, 91, 92, 93, 94],
[95, 96, 97, 98, 99]])
As bool(0) is False you can use to take None for other bound, and you'll get all the slice :
for i in range(0, window, d):
print(i, x[-i - d:-i or None])
0 [95, 96, 97, 98, 99]
5 [90, 91, 92, 93, 94]
10 [85, 86, 87, 88, 89]
I have a mat extension data which I want to separate every seconds values. My matrix is (7,5,2500) time series 3 dimensional matrix which want to get the values of (7,5,1) ...(7,5,2500) separately and save it
for example
array([155, 33, 129,167,189,63,35
161, 218, 6,58,36,25,3
89,63,36,25,78,95,21
78,52,36,56,25,15,68
]],
[215, 142, 235,
143, 249, 164],
[221, 71, 229,
56, 91, 120],
[236, 4, 177,
171, 105, 40])
for getting every part of this data for example this matrix
[215, 142, 235,
143, 249, 164]
what should I do?
a = [[155, 33, 129, 161, 218, 6],
[215, 142, 235, 143, 249, 164],
[221, 71, 229, 56, 91, 120],
[236, 4, 177, 171, 105, 40]]
print(a[1])
Assuming you have your data saved in a numpy array you could use slicing to extract the sub-matrices you need. Here is an example with a (3,5,3) matrix (but the example could be applied to any dimension):
A = numpy.array([[[1,1,1],
[2,2,2],
[3,3,3],
[4,4,4],
[5,5,5]],
[[11,11,11],
[21,21,21],
[31,31,31],
[41,41,41],
[51,51,51]],
[[12,12,12],
[22,22,22],
[32,32,32],
[42,42,42],
[52,52,52]]]
sub_matrix_1 = A[:,:,0]
print (sub_matrix_1)
Will produce:
[[ 1 2 3 4 5]
[11 21 31 41 51]
[12 22 32 42 52]]
EDIT: it is also possible to iterate over the array to get the 3rd dimension array:
for i in range(A.shape[-1]):
print (A[:,:,i])
# Your submatrix is A[:,:,i], you can directly manipulate it
Suppose I have an array like this:
a = array([[[ 29, 29, 27],
[ 36, 38, 40],
[ 86, 88, 89]],
[[200, 200, 198],
[199, 199, 197]
[194, 194, 194]]])
and I want to flip the 3rd element from left to right in the list-of-lists so it will become like this:
b = array([[[ 29, 29, 89], # 27 became 89
[ 36, 38, 40],
[ 86, 88, 27]], # 89 became 27
[[200, 200, 194], # 198 became 194
[199, 199, 197],
[194, 194, 198]]]) # 194 became 198
I looked up the NumPy manual but I still cannot figure out a solution. .flip and .fliplr look suitable in this case, but how do I use them?
Index the array to select the sub-array, using:
> a[:,:,-1]
array([[198, 197, 194],
[ 27, 40, 89]])
This selects the last element along the 3rd dimension of a. The sub-array is of shape (2,3). Then reverse the selection using:
a[:,:,-1][:,::-1]
The second slice, [:,::-1], takes everything along the first dimension as-is ([:]), and all of the elements along the second dimension, but reversed ([::-1]). The slice syntax is basically saying start at the first element, go the last element ([:]), but do it in the reverse order ([::-1]). You could pseudo-code write it as [start here : end here : use this step size]. The the -1 tells it walk backwards.
And assign it to the first slice of the original array. This updates/overwrites the original value of a
a[:,:,-1] = a[:,:,-1][:,::-1]
> a
array([[[ 29, 29, 89],
[ 36, 38, 40],
[ 86, 88, 27]],
[[200, 200, 194],
[199, 199, 197],
[194, 194, 198]]])
I have a series of numpy array, i need to save these numpy array in a loop as a raw binary float32 (without any header information) which need to be read from FORTRAN.
import numpy as np
f=open('test.bin','wb+')
for i in range(0,10):
np_data=np.random.rand(10,5)
fortran_data=np.asfortranarray(np_data,'float32')
fortran_data.tofile(f)
f.close()
Is this the correct way so that I can read this binary file created in python from FORTRAN correctly. Your suggestions will be highly apprecitaed
The code you wrote is almost right, but the .tofile method always write the vector in C order. I don't know why the np.asfortranarray() avoids this when writing in the binary file, but I tested and unfortunately we need to transpose the matrix before writing to correct read in Fortran without any other concern (this means in Fortran you can give the actual matrix dimension without needing any transpose).
The code below is to illustrate with a 3D matrix (which I ussually need to use) what I am saying:
a = np.arange(1,10*3*4+1)
b = a.reshape(10,12,order='F')
array([[ 1, 11, 21, 31, 41, 51, 61, 71, 81, 91, 101, 111],
[ 2, 12, 22, 32, 42, 52, 62, 72, 82, 92, 102, 112],
[ 3, 13, 23, 33, 43, 53, 63, 73, 83, 93, 103, 113],
[ 4, 14, 24, 34, 44, 54, 64, 74, 84, 94, 104, 114],
[ 5, 15, 25, 35, 45, 55, 65, 75, 85, 95, 105, 115],
[ 6, 16, 26, 36, 46, 56, 66, 76, 86, 96, 106, 116],
[ 7, 17, 27, 37, 47, 57, 67, 77, 87, 97, 107, 117],
[ 8, 18, 28, 38, 48, 58, 68, 78, 88, 98, 108, 118],
[ 9, 19, 29, 39, 49, 59, 69, 79, 89, 99, 109, 119],
[ 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120]])
b is already in Fortran order
c=b.reshape(10,3,4, order='F')
print(c[:,:,0])
[[ 1 11 21]
[ 2 12 22]
[ 3 13 23]
[ 4 14 24]
[ 5 15 25]
[ 6 16 26]
[ 7 17 27]
[ 8 18 28]
[ 9 19 29]
[10 20 30]]
Then I save the matrix c in a binary file:
c.T.tofile('test_c.bin')
So, using this Fortran code I am able to read the binary data in the correct order I created the c matrix in Python:
PROGRAM read_saved_python
IMPLICIT NONE
INTEGER(KIND=8),ALLOCATABLE :: matrix(:,:,:)
INTEGER :: Nx, Ny, Nz
Nx = 10
Ny = 3
Nz = 4
ALLOCATE(matrix(Nx, Ny, Nz))
OPEN(33, FILE="/home/victor/test_c.bin",&
FORM="UNFORMATTED", STATUS="UNKNOWN", ACTION="READ", ACCESS='STREAM')
READ(33) matrix
write(*,*) matrix(:,1,1)
CLOSE(33)
DEALLOCATE(matrix)
END PROGRAM read_saved_python
Notice in Fortran the indexes start in 1 and the print shows in column order (in this case: print the first column, the second and then the third). If you don't transpose the matrix here c.T.tofile('test_c.bin') when reading in Fortran you'll notice that the matrix is not as you wanted, even if you use function np.asfortranarray as you did ( I even tried np.asfortranarray(c).T.tofile('/home/victor/teste_d.bin') (just to make sure) but the matrix is written in c order in the binary file.
You will need the meta data of the array to read it in FORTRAN. This website (https://scipy.github.io/old-wiki/pages/Cookbook/InputOutput.html) has some information on using libnpy to write and an example code fex.f95 to read the binary file.