I'm facing very strange problem with arrays in python and numpy. First of all what Im trying to archive is :
1) Get an MxN matrix from KxTxN matrix
2) Transpose this matrix and calculate product of this transposed matrix and the original one
What I get is some what strange, here comes the code :
First of all, I have read an image with help of cv2, and got K by T by 3 matrix (a field of RGB points), then I'm cutting a small window form it and reshaping this window to M by N matrix :
def clipSubwindowFromImage(img, i, j, winSize):
winI = img[i - winSize: i + winSize + 1, j - winSize : j + winSize + 1, : ]
res = np.vstack((winI[:,::3,:].reshape(winI.shape[1],3), winI[:,1::3,:].reshape(winI.shape[1],3), winI[:,2::3,:].reshape(winI.shape[1],3)))
return res
so far so god, say we had winSize = 1, i = 1, j = 1 and got a 9x3 matrix as a result: this matrix :
>> subWin = clipSubwindowFromImage(background12x12b, 1, 1, 1)
>> [[201 199 187]
[216 219 198]
[226 228 207]
[243 241 228]
[240 244 221]
[233 235 213]
[239 238 220]
[238 240 216]
[233 235 211]]
Then I just want to get the product in question, like this :
>>r1 = subWin.T.dot(subWin)
>>[[197 234 89]
[234 65 163]
[ 89 163 105]]
Well, it's not right, the right result should be :
>>[[477125 479466 438361]
[479466 481857 440483]
[438361 440483 402793]]
But if I initialize subWin manually like this :
>>subWin = np.array([[201, 199, 187], [216, 219, 198], [226, 228, 207], [243, 241, 228], [240, 244, 221], [233, 235, 213],[239, 238, 220], [238, 240, 216],[233, 235, 211]])
I get right result.
I can't get it, subWin is the SAME array in both cases (I checked it). Any ideas?
As #Aguy said, your problem comes from the data-type of your array. The dot product of a uint8 array with an other uint8 array gives an array that is also uint8 hence the data-type is overflowed in your case. Here's an example that shows the effect of overflow on your values:
import numpy as np
a = np.array([[201, 199, 187], [216, 219, 198], [226, 228, 207], [243, 241, 228], [240, 244, 221], [233, 235, 213],[239, 238, 220], [238, 240, 216],[233, 235, 211]])
b = a.T.dot(a)
print b.dtype
print b
print "overflowed uint8 :"
print b.astype(np.uint8)
Gives:
>>> int64
>>> [[477125 479466 438361]
>>> [479466 481857 440483]
>>> [438361 440483 402793]]
>>> overflowed uint8 :
>>> [[197 234 89]
>>> [234 65 163]
>>> [ 89 163 105]]
Just change the data-type of one array to something more suitable in your dot product and you're good to go :
r1 = subWin.T.dot(subWin.astype(np.uint32))
Related
I have array of vertices from an stl file which i converted to a 2D numpy array. Here's some of it as an example:
print(vertices.shape)
(67748, 3)
I need to turn these into a 3D binary array where each element = 1 where the index is given by the vertices array.
Minimal reproducible example (expected output) using a 5 x 3 vertices array instead of 67748 x 3:
verts = np.array([[ 77, 239, 83],
[100, 237, 88],
[100, 149, 94],
[100, 220, 128],
[100, 145, 86]])
voxels = np.zeros((256,256,256)).astype(int)
voxels[77,239,83] = 1
voxels[100,149,94] = 1
voxels[100,237,88] = 1
voxels[100,220,128] = 1
voxels[100,145, 86] = 1
You can use np.put and np.ravel_multi_index
np.put(voxel_array,
np.ravel_multi_index(vertices.T,
voxel_array.shape),
1)
And then:
np.where(voxel_array)
Out[]:
(array([ 77, 100, 100, 100, 100], dtype=int64),
array([239, 145, 149, 220, 237], dtype=int64),
array([ 83, 86, 94, 128, 88], dtype=int64))
Answer from Michael Szczesny in the comments:
for x, y, z in vertices: voxel_array[x, y, z] = 1
This works just as well as Daniel F's but is easier to understand. Both take 0.0 seconds to run on 256 x 265 x 265 array
I try to sum together three positive arrays, however, the result yields an array that has negative values. How is this possible?
#Example of an image
img=np.array(([[[246, 240, 243],[240, 239, 239],
[243, 242, 244]],[[ 241, 240, 240],
[241, 243, 246],[ 239, 239, 239]],
[[249, 249, 250],[ 33, 33, 34],
[249, 249, 249]],[[ 33, 33, 33],
[250, 250, 249],[ 34, 34, 34]]]), dtype=np.uint8)
#Creating three positive arrays from image
#Image type converted to np.int16 as otherwise values remain between 0-255
R=abs((img[:,:,0].astype(np.int16)-255)**2)
G=abs((img[:,:,1].astype(np.int16)-255)**2)
B=abs((img[:,:,2].astype(np.int16)-255)**2)
print(R, G, B)
[[ 81 225 144]
[ 196 196 256]
[ 36 16252 36]
[16252 25 16695]] [[ 225 256 169]
[ 225 144 256]
[ 36 16252 36]
[16252 25 16695]] [[ 144 256 121]
[ 225 81 256]
[ 25 16695 36]
[16252 36 16695]]
#Adding three positive arrays together
R+G+B
array([[ 450, 737, 434],
[ 646, 421, 768],
[ 97, -16337, 108],
[-16780, 86, -15451]], dtype=int16)
I thought it had something to do with the abs() function I am applying, however, the results separately clearly show they are referenced correctly and positive?
I have some code here:
import numpy as np
import matplotlib.pyplot as plt
#height (cm)
X = np.array([[147, 150, 153, 158, 163, 165, 168, 170, 173, 175, 178, 180, 183]])
print(X.T)
print("=======================")
print(X)
Can anyone explain me what is T mean and the differences between X and X.T?
The .T is an attribute of numpy array, that transposes the array.
The significant differences are seen in the shape and strides attributes:
In [64]: X = np.array([[1,2,3,4]])
In [65]: X
Out[65]: array([[1, 2, 3, 4]])
In [66]: X.T
Out[66]:
array([[1],
[2],
[3],
[4]])
In [67]: X.shape
Out[67]: (1, 4)
In [68]: X.T.shape
Out[68]: (4, 1)
In [69]: X.strides
Out[69]: (32, 8)
In [70]: X.T.strides
Out[70]: (8, 32)
The .T attribute returns the array transpose. Note that in your case, you declared almost a vector (a real vector would be one dimensional, but in your case you have two dimensions because of the double brackets [[]] use in the definition of x):
import numpy as np
x = np.array([[147, 150, 153, 158, 163, 165, 168, 170, 173, 175, 178, 180]])
print("Line vector:")
print(x)
print("Column vector:")
print(x.T)
Line vector:
[[147 150 153 158 163 165 168 170 173 175 178 180]]
Column vector:
[[147]
[150]
[153]
[158]
[163]
[165]
[168]
[170]
[173]
[175]
[178]
[180]]
Note that in this case, there is no need to use double square brackets [[]]. If you define x with single brackets, the transpose is not different (because there is no dimension to transpose with):
import numpy as np
x = np.array([147, 150, 153, 158, 163, 165, 168, 170, 173, 175, 178, 180])
print("Line vector:")
print(x)
print("Column vector:")
print(x.T)
Line vector:
[147 150 153 158 163 165 168 170 173 175 178 180]
Column vector:
[147 150 153 158 163 165 168 170 173 175 178 180]
I have a mat extension data which I want to separate every seconds values. My matrix is (7,5,2500) time series 3 dimensional matrix which want to get the values of (7,5,1) ...(7,5,2500) separately and save it
for example
array([155, 33, 129,167,189,63,35
161, 218, 6,58,36,25,3
89,63,36,25,78,95,21
78,52,36,56,25,15,68
]],
[215, 142, 235,
143, 249, 164],
[221, 71, 229,
56, 91, 120],
[236, 4, 177,
171, 105, 40])
for getting every part of this data for example this matrix
[215, 142, 235,
143, 249, 164]
what should I do?
a = [[155, 33, 129, 161, 218, 6],
[215, 142, 235, 143, 249, 164],
[221, 71, 229, 56, 91, 120],
[236, 4, 177, 171, 105, 40]]
print(a[1])
Assuming you have your data saved in a numpy array you could use slicing to extract the sub-matrices you need. Here is an example with a (3,5,3) matrix (but the example could be applied to any dimension):
A = numpy.array([[[1,1,1],
[2,2,2],
[3,3,3],
[4,4,4],
[5,5,5]],
[[11,11,11],
[21,21,21],
[31,31,31],
[41,41,41],
[51,51,51]],
[[12,12,12],
[22,22,22],
[32,32,32],
[42,42,42],
[52,52,52]]]
sub_matrix_1 = A[:,:,0]
print (sub_matrix_1)
Will produce:
[[ 1 2 3 4 5]
[11 21 31 41 51]
[12 22 32 42 52]]
EDIT: it is also possible to iterate over the array to get the 3rd dimension array:
for i in range(A.shape[-1]):
print (A[:,:,i])
# Your submatrix is A[:,:,i], you can directly manipulate it
I have an array (n,m) :
[255 100 255]
[100 255 100]
[255 100 255]
I need to create a new array like that where neigboring values are tested an if North, East, South, West are ALL equal to 100 my value is set to 100 :
[255 100 255]
[100 100 100]
[255 100 255]
I have a simple solution that loops on 1:n and 1:m but it is obviously very slow and I am wondering if there is way to do it faster.
I found several links that are talking about sliding window to compute average but i don't see how i can keep track of my indexes to create the new array.
Using strides for an efficient moving average filter
Thanks in advance for your inputs.
Assuming A as the input array, here's one approach using slicing and boolean indexing -
# Get west, north, east & south elements for [1:-1,1:-1] region of input array
W = A[1:-1,:-2]
N = A[:-2,1:-1]
E = A[1:-1,2:]
S = A[2:,1:-1]
# Check if all four arrays have 100 for that same element in that region
mask = (W == 100) & (N == 100) & (E == 100) & (S == 100)
# Use the mask to set corresponding elements in a copy version as 100s
out = A.copy()
out[1:-1,1:-1][mask] = 100
Sample run -
In [90]: A
Out[90]:
array([[220, 93, 205, 82, 23, 210, 22],
[133, 228, 100, 27, 210, 186, 246],
[196, 100, 73, 100, 86, 100, 53],
[195, 131, 100, 142, 100, 214, 100],
[247, 73, 117, 116, 24, 100, 50]])
In [91]: W = A[1:-1,:-2]
...: N = A[:-2,1:-1]
...: E = A[1:-1,2:]
...: S = A[2:,1:-1]
...: mask = (W == 100) & (N == 100) & (E == 100) & (S == 100)
...:
...: out = A.copy()
...: out[1:-1,1:-1][mask] = 100
...:
In [92]: out
Out[92]:
array([[220, 93, 205, 82, 23, 210, 22],
[133, 228, 100, 27, 210, 186, 246],
[196, 100, 100, 100, 86, 100, 53],
[195, 131, 100, 142, 100, 100, 100],
[247, 73, 117, 116, 24, 100, 50]])
Such problems are seen largely in signal-processing/image-processing domain. So, you can use 2D convolution too for an alternative solution, like so -
from scipy import signal
from scipy import ndimage
# Use a structuring elements with north, west, east and south elements as 1s
strel = ndimage.generate_binary_structure(2, 1)
# 2D Convolve to get 4s at places that are surrounded by 1s
mask = signal.convolve2d((A==100).astype(int),strel,'same')==4
# Use the mask to set corresponding elements in a copy version as 100
out = A.copy()
out[mask] = 100
Sample run -
In [119]: A
Out[119]:
array([[108, 184, 0, 176, 131, 86, 201],
[ 22, 47, 100, 78, 151, 196, 221],
[185, 100, 142, 100, 121, 100, 24],
[201, 101, 100, 138, 100, 20, 100],
[127, 227, 217, 19, 206, 100, 43]])
In [120]: strel = ndimage.generate_binary_structure(2, 1)
...: mask = signal.convolve2d((A==100).astype(int),strel,'same')==4
...:
...: out = A.copy()
...: out[mask] = 100
...:
In [121]: out
Out[121]:
array([[108, 184, 0, 176, 131, 86, 201],
[ 22, 47, 100, 78, 151, 196, 221],
[185, 100, 100, 100, 121, 100, 24],
[201, 101, 100, 138, 100, 100, 100],
[127, 227, 217, 19, 206, 100, 43]])
A more straight-forward approach would be with ndimage.binary_closing, which is exactly the intended operation of closing here. So, another alternative way to get the mask would be -
strel = ndimage.generate_binary_structure(2, 1)
mask = ndimage.binary_closing(A==100, structure=strel)