I want to find an efficient way to rotate a 4x4 image patches from a larger image by angles that are multiples of 15. I am currently extracting a 6x6 patch e.g. patch=img[x-3:x+3,y-3:y+3] and then running scipy.ndimage.interpolation.rotate(patch,-15*o,reshape=False)[1:5,1:5]. However, I essentially need to do this at ever location (x,y) in the image. I have a "stacked" version of the image with an array of size (m,n,6,6) where m and n are the dimensions of the original image. Even if run interpolation.rotate on the stacked version, it looks like it internally simply does it iteratively and it takes a long time.
Since I only need to do this at fixed angles, I am trying to pre-compute some constants and vectorize the implementation so that I can process them all at once. I have tried digging into the implementation of SciPy rotate but it did not help much.
Is there a sensible way to do this?
Related
I'm working on a perspective transform application involving transforming 3D points to 2D camera pixels. It is a purely mathematical model, because I'm preparing to use it on hardware that I don't really have access to (so I'm making up focal length and offset values for the intrinsic camera matrix).
When I do the mapping, depending on the xyz location of the camera, I get huge differences in where my transformed image is, and I have to make the matrix where I'm inputting the pixels really large. (I'm mapping an image of 1000x1000 pixels to an image of about 600x600 pixels, but its located around 6000, so I have to make my output matrix 7000x7000, which takes a long time to plt.imshow. I have no use for the actual location of the pixels, because I'm only concerned with what the remapped image looks like.
I was wondering how people dealt with this issue:
I can think of just cropping the image down to the area that is non-zero (where my pixels are actually mapped too? Like seen in:
How to crop a numpy 2d array to non-zero values?
but that still requires me to use space and time to alot a 7000x7000 destination matrix
I am using opencv warpPerspective() function to warp the found countour in the image to find contour i am using findContours().
This is shown in this image:
but the warpPerspective() function takes "more time" to warp to full image is there any alternative to this function to warp the object in image to full image as shown in figure.
OR will traversing help?but this would be difficult to do so that i can reduce the time the warpPerspective() function takes.
You can try to work on rotation and translation matrices (or roto-translational matrix, a combination of both), which can warp image as you wish. The function warpPerspective() utilizes similar approach, so you will basically will have an opportunity to look inside the function.
The approach is:
You calculate the matrix, then multiply the height and width of
the original image to find dimensions of the output image.
Go through all pixels in the original image and multiply their
(x,y) coordinates to the matrix R
(rotation/translation/roto-translation matrix) to get the
coordinates on the output image (xo,yo).
On every calculated coordinate (xo,yo) assign value from the
corresponding original image coordinate (x,y).
Interpolate using median filter/bilinear/bicubic/etc. method as
sometimes there may be empty points left on the output image
However, if you work in Python your implementation may work even slower than warpPerspective(), so you may consider C++. Another thing is that OpenCV uses C++ compiler and I am pretty sure that implementation of warpPerspective() in OpenCV is very efficient.
So, I think that you can go around warpPerspective(), however, I am not sure if you can do it faster than in OpenCV without any boosts (like GPU, powerful CPU etc.) :)
Good luck!
(Taking a chance at not posting an example here but I think the question is general enough that one is not necessary.)
I am using skimage.transform.warp to warp a 200x2000 image given 500 source and destination control points calculated with skimage.transform.PiecewiseAffineTransform. When I run this on a single image, it takes about 3 seconds. Is this a reasonable runtime for this calculation in everyone's experience?
The reason I ask is that I have potentially hundreds of images of the same dimensions that I want to apply the same inverse transform to but this will take waaaaay too long. If I use Python's multiprocessing module, the calculation hangs and never completes.
What I would like to do is run warp on a single image and then calculate a polynomial that defines the value of each pixel in the warped image given the values of all 400000 pixels in the input image. Mathematically:
f'(x,y) = a0_0*f(0,0) + a0_1*f(0,1) + ... + a200_1999*f(200,1999) + a200_2000*f(200,2000)
Does anyone have a recommendation as to how would I go about doing this or something similar or implementing something faster?
Thank you!
I ran into a similar issue when I had to correct some images from a spectroscopic camera. I ended up using sp.ndimage.map_coordinates. You have to build a function that transforms your source point coordinates into destination coordinates (dummy function in the example below). I understand from the question this transformation is the same for a bunch of images, and that you already have this function.
Then you generate a full grid of coordinates, and map_coordinates will map your original image onto these new coordinates trough spline interpolation.
from scipy.ndimage import map_coordinates
# stack of 10 images
imgs=np.random.normal(size=[10,200,2000])
x, y = np.arange(imgs.shape[1]), np.arange(imgs.shape[2])
ini_coord=np.meshgrid(x, y)
# dummy function transforms source points into destination points
def dummy(ini_coord):
return [0.9*x.T for x in ini_coord]
out_coord=dummy(ini_coord)
import time
tt=time.clock()
out_img=np.zeros(imgs.shape)
for i, img in enumerate(imgs):
out_img[i]=map_coordinates(img, out_coord, mode='nearest')
print('{:3f} s'.format(time.clock()-tt))
This runs in less than 1 sec. on my computer
How can I obtain average direction of the pattern shown in the figure below. It is the direction of the red arrow relative to the yellow (horizontal) line. Any ideas for an approach? I couldn't figure out a way to approach. This is a spatio-temporal image created from a video. Thank you.
Here is my original image:
The simplest approach would be to compute the gradient vector (x derivative and y derivative) and find its direction at each pixel (atan2(y,x)). The average orientation is what you want, not the average direction (will cancel out). So apply modulus pi, then average across the image.
The best way to compute image gradients is through the Gaussian gradients.
The structure tensor is the more robust way of accomplishing this. In short, it computes local averages of the gradient vector to reduce the effect of noise. It does this by computing the outer product of the gradient vector with itself, which produces a symmetric matrix. The individual components of this matrix can then be locally averaged (i.e. apply a smoothing filter). This is similar to computing the angle of the vectors, doubling the angles to make vectors in opposite directions equal, then averaging them.
Note that you can apply either of these solutions in 3D (you can think of the video data as 2D + time = 3D). That way, you compute both the speed and the direction of motion within the 2D frame, rather than just the speed along the direction in which you extracted the 2D image you show in the question. The image gradient and the concept of the structure tensor easily extend to 3D. This 3D approach is similar to the approach by Lucas-Kanade for optical flow. If you follow that link to Wikipedia, you'll see it uses the structure tensor in 2D, and adds gradients along the time dimension.
Might be useful to try Fourier transform.
In your case you should get two vertical lines in the middle of the transformed image corresponding to the information when traveling vertically in the image.
On the other hand there shouldn't be a horizontal line since when traveling horizontally in the image there is little information (little change)
For example you can use this online site to play with fourier transforms:
https://www.ejectamenta.com/Fourifier-fullscreen/
It might sound like the problem remains the same but in fact it is much easier now.
The 2D pattern is converted into dominant lines which are quite easy to find in the transformed image.
For example you can search for the strongest pixels in the image and simply determine if they are more likely to be horizontal line or a vertical line or determine the angle of the dominant line. Then rotate by 90 degrees.
For example see this image of wood grain and the resulting transformed image:
And don't worry about the two lines. The image is symmetric so just ignore 3/4 of the image and look in 1 quarter.
I recommend giving the Hough transform a go, it is available in OpenCv. The Hough transform maps lines to angles, and might be useful in your case.
I was surprised when I started learning numpy that there are N dimensional arrays. I'm a programmer and all I thought that nobody ever use more than 2D array. Actually I can't even think beyond a 2D array. I don't know how think about 3D, 4D, 5D arrays or more. I don't know where to use them.
Can you please give me examples of where 3D, 4D, 5D ... etc arrays are used? And if one used numpy.sum(array, axis=5) for a 5D array would what happen?
A few simple examples are:
A n x m 2D array of p-vectors represented as an n x m x p 3D matrix, as might result from computing the gradient of an image
A 3D grid of values, such as a volumetric texture
These can even be combined in the case of a gradient of a volume in which case you get a 4D matrix
Staying with the graphics paradigm, adding time adds an extra dimension, so a time-variant 3D gradient texture would be 5D
numpy.sum(array, axis=5) is not valid for a 5D-array (as axes are numbered starting at 0)
Practical applications are hard to come up with but I can give you a simple example for 3D.
Imagine taking a 3D world (a game or simulation for example) and splitting it into equally sized cubes. Each cube could contain a specific value of some kind (a good example is temperature for climate modelling). The matrix can then be used for further operations (simple ones like calculating its Transpose, its Determinant etc...).
I recently had an assignment which involved modelling fluid dynamics in a 2D space. I could have easily extended it to work in 3D and this would have required me to use a 3D matrix instead.
You may wish to also further extend matrices to cater for time, which would make them 4D. In the end, it really boils down to the specific problem you are dealing with.
As an end note however, 2D matrices are still used for 3D graphics (You use a 4x4 augmented matrix).
There are so many examples... The way you are trying to represent it is probably wrong, let's take a simple example:
You have boxes and a box stores N items in it. You can store up to 100 items in each box.
You've organized the boxes in shelves. A shelf allows you to store M boxes. You can identify each box by a index.
All the shelves are in a warehouse with 3 floors. So you can identify any shelf using 3 numbers: the row, the column and the floor.
A box is then identified by: row, column, floor and the index in the shelf.
An item is identified by: row, column, floor, index in shelf, index in box.
Basically, one way (not the best one...) to model this problem would be to use a 5D array.
For example, a 3D array could be used to represent a movie, that is a 2D image that changes with time.
For a given time, the first two axes would give the coordinate of a pixel in the image, and the corresponding value would give the color of this pixel, or a grey scale level. The third axis would then represent time. For each time slot, you have a complete image.
In this example, numpy.sum(array, axis=2) would integrate the exposure in a given pixel. If you think about a film taken in low light conditions, you could think of doing something like that to be able to see anything.
They are very applicable in scientific computing. Right now, for instance, I am running simulations which output data in a 4D array: specifically
| Time | x-position | y-position | z-position |.
Almost every modern spatial simulation will use multidimensional arrays, along with programming for computer games.