(Taking a chance at not posting an example here but I think the question is general enough that one is not necessary.)
I am using skimage.transform.warp to warp a 200x2000 image given 500 source and destination control points calculated with skimage.transform.PiecewiseAffineTransform. When I run this on a single image, it takes about 3 seconds. Is this a reasonable runtime for this calculation in everyone's experience?
The reason I ask is that I have potentially hundreds of images of the same dimensions that I want to apply the same inverse transform to but this will take waaaaay too long. If I use Python's multiprocessing module, the calculation hangs and never completes.
What I would like to do is run warp on a single image and then calculate a polynomial that defines the value of each pixel in the warped image given the values of all 400000 pixels in the input image. Mathematically:
f'(x,y) = a0_0*f(0,0) + a0_1*f(0,1) + ... + a200_1999*f(200,1999) + a200_2000*f(200,2000)
Does anyone have a recommendation as to how would I go about doing this or something similar or implementing something faster?
Thank you!
I ran into a similar issue when I had to correct some images from a spectroscopic camera. I ended up using sp.ndimage.map_coordinates. You have to build a function that transforms your source point coordinates into destination coordinates (dummy function in the example below). I understand from the question this transformation is the same for a bunch of images, and that you already have this function.
Then you generate a full grid of coordinates, and map_coordinates will map your original image onto these new coordinates trough spline interpolation.
from scipy.ndimage import map_coordinates
# stack of 10 images
imgs=np.random.normal(size=[10,200,2000])
x, y = np.arange(imgs.shape[1]), np.arange(imgs.shape[2])
ini_coord=np.meshgrid(x, y)
# dummy function transforms source points into destination points
def dummy(ini_coord):
return [0.9*x.T for x in ini_coord]
out_coord=dummy(ini_coord)
import time
tt=time.clock()
out_img=np.zeros(imgs.shape)
for i, img in enumerate(imgs):
out_img[i]=map_coordinates(img, out_coord, mode='nearest')
print('{:3f} s'.format(time.clock()-tt))
This runs in less than 1 sec. on my computer
Related
Outline
I'm trying to warp an image (of a spectral peak in a time series, but this is not important) by generating a polynomial based on some 'centroid' data that is associated with the image (the peak at each time step) and augmenting the polynomial. These original and augmented polynomials make up my 'source' and 'destination' points, respectively, that I am trying to warp the image with, using skimage.transform.warp().
The goal of this warping is to produce two warped images (i.e. repeat the process twice). These images would then be positively correlated with one another, or negatively correlated if one of the two warped images were to be horizontally flipped (again, not that important here).
Here is an example output for comparison:
(Note that the polynomial augmentation is performed by adding/subtracting noise at each polynomial peak/trough, proportional to the magnitude (pixel) at each point, then generating a new polynomial of the same order through these augmented points, with additional fixed points in place to prevent the augmented polynomial from inverting).
Code Snippet
I achieve this in code by creating a GeometricTransform and applying this to warp() as an inverse_map, as follows:
from skimage import transform
# Create the transformation object using the source and destination (N, 2) arrays in reverse order
# (as there is no explicit way to do an inverse polynomial transformation).
t = transform.estimate_transform('polynomial', src=destination, dst=source, order=4) # order = num_poly_degrees - 1
# Warp the original image using the transformation object
warped_image = transform.warp(image, t, order=0, mode='constant', cval=float('nan'))
Problems
I have two main problems with the resulting warp:
There are white spaces left behind due to the image warp. I know that this can be solved by changing the mode within transform.warp() from 'constant' to 'reflect', for example. However, that would repeat existing data, which is related to my next problem...
Assuming I have implemented the warp correctly, it seems to have raised the 'zig-zag' feature seen at time step 60 to ~50 (red circles). My goal with the warping is to horizontally warp the images so that each feature remains within its own time step (perhaps give-or-take a very small amount), but their 'pixel' position (x-axis) is augmented. This is also why I am unsure about using 'reflect' or another mode within transform.warp(), as this would artificially add more data, which would cause problems later in my pipeline where I compare pairs of warped images to see how they are correlated (relating back to my second paragraph in Outline).
My Attempts
I have tried using RANSAC, as used in this question which also uses a polynomial transformation: Robustly estimate Polynomial geometric transformation with scikit-image and RANSAC in order to improve the warping. I had hoped that this method would only leave behind smaller white spaces, then I would be satisfied with switching to another mode within transform.warp(), however, this does not fix either of my issues as the performance was about the same.
I have also looked into using a piecewise affine transformation and Delaunay triangulation (using cv2) as a means of both preserving the correct image dimensions (without repeating data), and having minimal y-component warping. The results do solve the two stated problems, however the warping effect is almost imperceptible, and I am not sure if I should continue down this path by adding more triangles and trying more separated source and destination points (though this line of thought may require another post).
Summary
I would like a way to warp my images horizontally using a polynomial transformation (any other suggestions for a transformation method are also welcome!), which does its best to preserve the image's features within their original time steps.
Thank you for your time.
Edit
Here is a link to a shared google drive directory contain a .py file and data necessary to run an example of this process.
I want to find an efficient way to rotate a 4x4 image patches from a larger image by angles that are multiples of 15. I am currently extracting a 6x6 patch e.g. patch=img[x-3:x+3,y-3:y+3] and then running scipy.ndimage.interpolation.rotate(patch,-15*o,reshape=False)[1:5,1:5]. However, I essentially need to do this at ever location (x,y) in the image. I have a "stacked" version of the image with an array of size (m,n,6,6) where m and n are the dimensions of the original image. Even if run interpolation.rotate on the stacked version, it looks like it internally simply does it iteratively and it takes a long time.
Since I only need to do this at fixed angles, I am trying to pre-compute some constants and vectorize the implementation so that I can process them all at once. I have tried digging into the implementation of SciPy rotate but it did not help much.
Is there a sensible way to do this?
My question is whether I can optimally determine the distance between the source and the center of rotation and the distance between the center of rotation and the detector array for a given image and projection geometry.
By optimal I mean that the number of zero entries of the measurement vector is minimized.
In the following code snippet I used the Astra toolbox with which we simulate our 2D tomography.
from skimage.io import imread
from astra import creators, optomo
import numpy as np
# load some 400x400 pixel image
image = imread("/path/to/image.png, as_gray=True)"
# create geometries and projector
# proj_geom = astra_create_proj_geom('fanflat', det_width, det_count, angles, source_origin, origin_det)
proj_geom = creators.create_proj_geom('fanflat', 1.0, 400, np.linspace(0,np.pi,180), 1500.0, 500.0);
vol_geom = creators.create_vol_geom(400,400)
proj_id = creators.create_projector('line_fanflat', proj_geom, vol_geom)
# create forward projection
# fp is our measurement vector
W = optomo.OpTomo(proj_id)
fp = W*image
In my example if I use np.linspace(0,np.pi,180) the number of zero-entries of fp are 1108, if I use np.linspace(0,np.pi/180,180) instead the number increases to 5133 which makes me believe that the values 1500.0 and 500.0 are not well chosen.
Generally speaking, those numbers are chosen due to experimental constrains and not algorithmic ones. In many settings these values are fixed, but lets ignore those, as you seem to have the flexibility.
In an experimental scan, what do you want?
If you are looking for high resolution you want the "magnification" DSD/DSO to be the highest, thus putting the detector far, and the object close to the source. This comes with problems though. A far detector requires higher scanning times for the same SNR (due to scatter and other phenomena that will make your X-rays not go straight). And not only that, the bigger the mag, the more likely you are to have huge parts of the object completely outside your detector range, as detectors are not that big (in mm).
So the common scanning strategy to set these up is 1) put the detector as far as you can allow with your strict scanning time. 2) put the object as close to the source as you can, but always making sure its entire width fits in the detector.
Often compromises can be done, particularly if you know what is the smallest feature you want to see (allow 3 or 4 pixels to properly see it).
However, algorithmically speaking? its irrelevant. I can't speak for ASTRA, but likely not even the computational time will be affected, as pixels that have zeroes are because they are out of the field of view and therefore simply not computed, at all.
Now, for your simple toy example, if you completely ignore all physics, there is a way:
1.- use simple trigonometry to compute what ratios of distances you need to make sure all the object is in the detector.
2.- create a fully white image and go changing the sizes iteratively until the first couple of pixels in the outside part of the detector become zero.
I am using opencv warpPerspective() function to warp the found countour in the image to find contour i am using findContours().
This is shown in this image:
but the warpPerspective() function takes "more time" to warp to full image is there any alternative to this function to warp the object in image to full image as shown in figure.
OR will traversing help?but this would be difficult to do so that i can reduce the time the warpPerspective() function takes.
You can try to work on rotation and translation matrices (or roto-translational matrix, a combination of both), which can warp image as you wish. The function warpPerspective() utilizes similar approach, so you will basically will have an opportunity to look inside the function.
The approach is:
You calculate the matrix, then multiply the height and width of
the original image to find dimensions of the output image.
Go through all pixels in the original image and multiply their
(x,y) coordinates to the matrix R
(rotation/translation/roto-translation matrix) to get the
coordinates on the output image (xo,yo).
On every calculated coordinate (xo,yo) assign value from the
corresponding original image coordinate (x,y).
Interpolate using median filter/bilinear/bicubic/etc. method as
sometimes there may be empty points left on the output image
However, if you work in Python your implementation may work even slower than warpPerspective(), so you may consider C++. Another thing is that OpenCV uses C++ compiler and I am pretty sure that implementation of warpPerspective() in OpenCV is very efficient.
So, I think that you can go around warpPerspective(), however, I am not sure if you can do it faster than in OpenCV without any boosts (like GPU, powerful CPU etc.) :)
Good luck!
I have a series of images which serve as my raw data which I am trying to prepare for publication. These images have a series of white specks randomly throughout which I would like to replace with the average of some surrounding pixels.
I cannot post images, but the following code should produce a PNG that approximates the issue that I'm trying to correct:
import numpy as np
from scipy.misc import imsave
random_array = np.random.random_sample((512,512))
random_array[random_array < 0.999] *= 0.25
imsave('white_specs.png', random_array)
While this should produce an image with a similar distribution of the specks present in my raw data, my images do not have specks uniform in intensity, and some of the specks are more than a single pixel in size (though none of them are more than 2). Additionally, there are spots on my image that I do not want to alter that were intentionally saturated during data acquisition for the purpose of clarity when presented: these spots are approximately 10 pixels in diameter.
In principle, I could write something to look for pixels whose value exceeds a certain threshold then check them against the average of their nearest neighbors. However, I assume what I'm ultimately trying to achieve is not an uncommon action in image processing, and I very much suspect that there is some SciPy functionality that will do this without having to reinvent the wheel. My issue is that I am not familiar enough with the formal aspects/vocabulary of image processing to really know what I should be looking for. Can someone point me in the right direction?
You could simply try a median filter with a small kernel size,
from scipy.ndimage import median_filter
filtered_array = median_filter(random_array, size=3)
which will remove the specks without noticeably changing the original image.
A median filter is well suited for such tasks since it will better preserve features in your original image with high spatial frequency, when compared for instance to a simple moving average filter.
By the way, if your images are experimental (i.e. noisy) applying a non-aggressive median filter (such as the one above) never hurts as it allows to attenuate the noise as well.