I'm trying to register two 3d volumes. An attempt at this can found here. The code first generates two different volumes, both containing exactly one sphere of radius 4. I'm then trying to align them using the default translation parameter map. However, as can be seen in the last line (and from the plots if running locally), the result volume is not aligned with the fixed volume at all. When attempting the same procedure, in 2d this time, the resulting image does appear to be correctly aligned with the fixed image, as can be seen here. Am I using the SimpleElastix API incorrectly? I looked through the Github repo of SimpleElastix, but I could not find any examples of 3d image registration (at least not using volumes generated in Python and then converting them to ITK images).
Code from 3d example:
vol1 = np.zeros((50, 50, 50))
for x in range(vol.shape[0]):
for y in range(vol.shape[1]):
for z in range(vol.shape[2]):
vol1[x, y, z] = np.linalg.norm(np.subtract([x, y, z], [5, 3, 2])) < 4
vol2 = np.zeros((50, 50, 50))
for x in range(vol.shape[0]):
for y in range(vol.shape[1]):
for z in range(vol.shape[2]):
vol1[x, y, z] = np.linalg.norm(np.subtract([x, y, z], [20, 30, 10])) < 4
img_a = sitk.GetImageFromArray(vol1)
img_b = sitk.GetImageFromArray(vol2)
parameterMap = sitk.GetDefaultParameterMap('translation')
itk_filter = sitk.ElastixImageFilter()
itk_filter.LogToConsoleOn()
itk_filter.SetFixedImage(img_a)
itk_filter.SetMovingImage(img_b)
itk_filter.SetParameterMap(parameterMap)
itk_filter.Execute()
result_vol = sitk.GetArrayFromImage(itk_filter.GetResultImage())
np.max(np.abs(vol1 - result_vol))
Code from 2d example:
vol1 = np.zeros((50, 50))
for x in range(vol1.shape[0]):
for y in range(vol1.shape[1]):
vol1[x, y] = np.linalg.norm(np.subtract([x, y], [20, 20])) < 4
vol2 = np.zeros((50, 50))
for x in range(vol2.shape[0]):
for y in range(vol2.shape[1]):
vol2[x, y] = np.linalg.norm(np.subtract([x, y], [4, 5])) < 4
img_a = sitk.GetImageFromArray(vol1)
img_b = sitk.GetImageFromArray(vol2)
parameterMap = sitk.GetDefaultParameterMap('translation')
itk_filter = sitk.ElastixImageFilter()
itk_filter.LogToConsoleOn()
itk_filter.SetFixedImage(img_a)
itk_filter.SetMovingImage(img_b)
itk_filter.SetParameterMap(parameterMap)
itk_filter.Execute()
result_vol = sitk.GetArrayFromImage(itk_filter.GetResultImage())
np.max(np.abs(vol1 - result_vol))
The registration probably fails because the spheres do not overlap. You could try to smooth the images a bit more using e.g. sitk.DiscreteGaussian(). However, in this artificial example it is highly likely that simply moving the spheres closer to each other would result in a good registration.
Note it is difficult to register binary images as there is very little gradient information in these kinds of images: Gradients are non-zero at the border only, and zero everywhere else because of the regions are flat intensity-wise.
Related
I try to run a 9x9 pixel kernel across a large satellite image with a custom filter. One satellite scene has ~ 40 GB and to fit it into my RAM, I'm using xarrays options to chunk my dataset with dask.
My filter includes a check if the kernel is complete (i.e. not missing data at the edge of the image). In that case a NaN is returned to prevent a potential bias (and I don't really care about the edges). I now realized, that this introduces not only NaNs at the edges of the image (expected behaviour), but also along the edges of each chunk, because the chunks don't overlap. dask provides options to create chunks with an overlap, but are there any comparable capabilities in xarray? I found this issue, but it doesn't seem like there has been any progress in this regard.
Some sample code (shortened version of my original code):
import numpy as np
import numba
import math
import xarray as xr
#numba.jit("f4[:,:](f4[:,:],i4)", nopython = True)
def water_anomaly_filter(input_arr, window_size = 9):
# check if window size is odd
if window_size%2 == 0:
raise ValueError("Window size must be odd!")
# prepare an output array with NaNs and the same dtype as the input
output_arr = np.zeros_like(input_arr)
output_arr[:] = np.nan
# calculate how many pixels in x and y direction around the center pixel
# are in the kernel
pix_dist = math.floor(window_size/2-0.5)
# create a dummy weight matrix
weights = np.ones((window_size, window_size))
# get the shape of the input array
xn,yn = input_arr.shape
# iterate over the x axis
for x in range(xn):
# determine limits of the kernel in x direction
xmin = max(0, x - pix_dist)
xmax = min(xn, x + pix_dist+1)
# iterate over the y axis
for y in range(yn):
# determine limits of the kernel in y direction
ymin = max(0, y - pix_dist)
ymax = min(yn, y + pix_dist+1)
# extract data values inside the kernel
kernel = input_arr[xmin:xmax, ymin:ymax]
# if the kernel is complete (i.e. not at image edge...) and it
# is not all NaN
if kernel.shape == weights.shape and not np.isnan(kernel).all():
# apply the filter. In this example simply keep the original
# value
output_arr[x,y] = input_arr[x,y]
return output_arr
def run_water_anomaly_filter_xr(xds, var_prefix = "band",
window_size = 9):
variables = [x for x in list(xds.variables) if x.startswith(var_prefix)]
for var in variables[:2]:
xds[var].values = water_anomaly_filter(xds[var].values,
window_size = window_size)
return xds
def create_test_nc():
data = np.random.randn(1000, 1000).astype(np.float32)
rows = np.arange(54, 55, 0.001)
cols = np.arange(10, 11, 0.001)
ds = xr.Dataset(
data_vars=dict(
band_1=(["x", "y"], data)
),
coords=dict(
lon=(["x"], rows),
lat=(["y"], cols),
),
attrs=dict(description="Testdata"),
)
ds.to_netcdf("test.nc")
if __name__ == "__main__":
# if required, create test data
create_test_nc()
# import data
with xr.open_dataset("test.nc",
chunks = {"x": 50,
"y": 50},
) as xds:
xds_2 = xr.map_blocks(run_water_anomaly_filter_xr,
xds,
template = xds).compute()
xds_2["band_1"][:200,:200].plot()
This yields:
enter image description here
You can clearly see the rows and columns of NaNs along the edges of each chunk.
I'm happy for any suggestions. I would love to get the overlapping chunks (or any other solution) within xarray, but I'm also open for other solutions.
You can use Dask's map_blocks as follows:
arr = dask.array.map_overlap(
water_anomaly_filter, xds.band_1.data, dtype='f4', depth=4, window_size=9
).compute()
da = xr.DataArray(arr, dims=xds.band_1.dims, coords=xds.band_1.coords)
Note that you will likely want to tune depth and window_size for your specific application.
I'm exploring neural networks, and I want to model some pictures with neural network. Picture is a function that maps pixel coordinates to color, so I make my network also with 2 input variables (x, y) and 1 (shade) to 3 (R, G, B) output coordinates. For example, like this:
import torch.nn as nn
net = nn.Sequential(
nn.Linear(2, 2),
nn.Sigmoid(),
nn.Linear(2, 1),
)
Now, I plot it like this:
import matplotlib.pyplot as plt
import numpy as np
def draw_image1(f):
image = []
y = 1
delta = 0.005
while y > 0:
x = 0
row = []
while x < 1:
row.append(f(x, y))
x += delta
image.append(row)
y -= delta
plt.imshow(image, extent=[0, 1, 0, 1], cmap='winter')
plt.draw()
draw_image1(lambda x, y: net(torch.Tensor([x, y])).item())
But it looks ugly and is slow because it uses Python lists instead of numpy arrays or tensors.
I have another version of code that draws images from functions, which looks better and is 100x faster:
def draw_image2(f):
x = np.linspace(0, 1, num = 200)
y = np.linspace(0, 1, num = 200)
X, Y = np.meshgrid(x, y)
image = f(X, Y)
plt.imshow(image, extent=[0, 1, 0, 1], cmap='winter')
plt.draw()
It works for functions that use numpy operations (like lambda x: x + y), but when I plug in my net in the same way as for previous function (draw_image2(lambda x, y: net(torch.Tensor([x, y])).item())), I get RuntimeError: mat1 and mat2 shapes cannot be multiplied (400x200 and 2x2), which I understand as my neural net complaining that it wants to be fed data in smaller pieces.
Is there any proper way to plot pytorch neural network output?
To feed a whole batch into nn.Linear(i, o), the input typically has the shape (b, i) where b is the size of the batch. If we take a look at the documentation you can actually use additional "batch"-dimensions in between. Actually since pytorch was primarily made for deep learning that is based on stochastic gradietn descent, pretty much all modules of pytorch require you to have at least one batch dimension.
So you could easily modify your second plotting function to something like:
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
net = nn.Sequential(
nn.Linear(2, 2),
nn.Sigmoid(),
nn.Linear(2, 1),
)
def draw_image2(f):
device = torch.device('cpu') # or use your gpu alternatively
with torch.no_grad(): # disable building evaluation graph if you don't need it
x = torch.linspace(0, 1, 200)
y = torch.linspace(0, 1, 200)
X, Y = torch.meshgrid(x, y)
# the data dimension should be the last (2), as per documentation
inp = torch.stack([X, Y], dim=2).to(device) # shape = (200, 200, 2)
image = f(inp) # shape = (200, 200, 1)
image = image[..., 0].detach().cpu() # shape (200, 200)
plt.imshow(image, extent=[0, 1, 0, 1], cmap='winter')
plt.show()
return image
draw_image2(net)
Note that the with torch.no_grad() is not necessary for it to work, but it will save you some time. Depending on your network architecture it might also be worth to set your network to eval mode (net.eval()) first. Finally the .to(device)/.cpu() is also not necessary if you're not using your GPU.
I have used interp2 in Matlab, such as the following code, that is part of #rayryeng's answer in: Three dimensional (3D) matrix interpolation in Matlab:
d = size(volume_image)
[X,Y] = meshgrid(1:1/scaleCoeff(2):d(2), 1:1/scaleCoeff(1):d(1));
for ind = z
%Interpolate each slice via interp2
M2D(:,:,ind) = interp2(volume_image(:,:,ind), X, Y);
end
Example of Dimensions:
The image size is 512x512 and the number of slices is 133. So:
volume_image(rows, columns, slices in 3D dimenson) : 512x512x133 in 3D dimenson
X: 288x288
Y: 288x288
scaleCoeff(2): 0.5625
scaleCoeff(1): 0.5625
z = 1 up to 133 ,hence z: 1x133
ind: 1 up to 133
M2D(:,:,ind) finally is 288x288x133 in 3D dimenson
Aslo, Matlabs syntax for size: (rows, columns, slices in 3rd dimenson) and Python syntax for size: (slices in 3rd dim, rows, columns).
However, after convert the Matlab code to Python code occurred an error, ValueError: Invalid length for input z for non rectangular grid:
for ind in range(0, len(z)+1):
M2D[ind, :, :] = interpolate.interp2d(X, Y, volume_image[ind, :, :]) # ValueError: Invalid length for input z for non rectangular grid
What is wrong? Thank you so much.
In MATLAB, interp2 has as arguments:
result = interp2(input_x, input_y, input_z, output_x, output_y)
You are using only the latter 3 arguments, the first two are assumed to be input_x = 1:size(input_z,2) and input_y = 1:size(input_z,1).
In Python, scipy.interpolate.interp2 is quite different: it takes the first 3 input arguments of the MATLAB function, and returns an object that you can call to get interpolated values:
f = scipy.interpolate.interp2(input_x, input_y, input_z)
result = f(output_x, output_y)
Following the example from the documentation, I get to something like this:
from scipy import interpolate
x = np.arange(0, volume_image.shape[2])
y = np.arange(0, volume_image.shape[1])
f = interpolate.interp2d(x, y, volume_image[ind, :, :])
xnew = np.arange(0, volume_image.shape[2], 1/scaleCoeff[0])
ynew = np.arange(0, volume_image.shape[1], 1/scaleCoeff[1])
M2D[ind, :, :] = f(xnew, ynew)
[Code not tested, please let me know if there are errors.]
You might be interested in scipy.ndimage.zoom. If you are interpolating from one regular grid to another, it is much faster and easier to use than scipy.interpolate.interp2d.
See this answer for an example:
https://stackoverflow.com/a/16984081/1295595
You'd probably want something like:
import scipy.ndimage as ndimage
M2D = ndimage.zoom(volume_image, (1, scaleCoeff[0], scaleCoeff[1])
I need to create a tridimensional map. I would create an x axis with values between 0 and 255 with step 0.5. The same thing with the y axis.
And then I would assign a value for each coordinate (for example at the point (10.5,10)).
Matrix is not the solution because I can't decide values in the x and y axes.
Can you help me?
EDIT: I try to explain better the question. This is a piece of my code:
img = cv2.imread('Lena256.bmp',0)
M = cv2.getRotationMatrix2D((cols/2,rows/2),angle,1)
img_rotate = cv2.warpAffine(img,M,(cols,rows))
Then I locate some point in the img_rotate: for example p=(10,10). I want to map "p" to the corresponding point in the original image. To do that I have written this code:
T = np.zeros((rows,cols))
T[10][10] = 1
M_INV = cv2.getRotationMatrix2D((cols/2,rows/2),-angle,1)
T = cv2.warpAffine(T,M_INV,(cols,rows))
In this way it works. But if I locate a point with no integer coordinates (in the img_rotate), for example (10.5,10), I should to create a matrix T with double dimensions where I could assign values 0, 0.5, 1, ecc in order to identify point (10.5,10). And then I could apply the inverse rotation.
I hope to be enough clear
You could use a dictionary:
d = {
(10.5, 10): 23,
(0,0): 42,
(-100,999): 15,
#etc
}
Then you can access the value at some coordinates (x,y) by doing d[x,y]. (or d.get((x,y), default_value_goes_here) if you're not sure whether that coordinate exists in the collection yet)
you can use np.meshgrid for that
import numpy as np
x_ = np.linspace(0., .5, 255)
y_ = np.linspace(1., .5, 255)
# You can make what you want in z ex:
z_ = np.linspace(3., 4., 30)
x, y, z = np.meshgrid(x_, y_, z_, indexing='ij')
I'm am working on a project at the moment where I am trying to create a Hilbert curve using the Python Imaging Library. I have created a function which will generate new coordinates for the curve through each iteration and place them into various lists which then I want to be able to move, rotate and scale. I was wondering if anyone could give me some tips or a way to do this as I am completely clueless. Still working on the a lot of the code.
#! usr/bin/python
import Image, ImageDraw
import math
# Set the starting shape
img = Image.new('RGB', (1000, 1000))
draw = ImageDraw.Draw(img)
curve_X = [0, 0, 1, 1]
curve_Y = [0, 1, 1, 0]
combinedCurve = zip(curve_X, curve_Y)
draw.line((combinedCurve), fill=(220, 255, 250))
iterations = 5
# Start the loop
for i in range(0, iterations):
# Make 4 copies of the curve
copy1_X = list(curve_X)
copy1_Y = list(curve_Y)
copy2_X = list(curve_X)
copy2_Y = list(curve_Y)
copy3_X = list(curve_X)
copy3_Y = list(curve_Y)
copy4_X = list(curve_X)
copy4_Y = list(curve_Y)
# For copy 1, rotate it by 90 degree clockwise
# Then move it to the bottom left
# For copy 2, move it to the top left
# For copy 3, move it to the top right
# For copy 4, rotate it by 90 degrees anticlockwise
# Then move it to the bottom right
# Finally, combine all the copies into a big list
combinedCurve_X = copy1_X + copy2_X + copy3_X + copy4_X
combinedCurve_Y = copy1_Y + copy2_Y + copy3_Y + copy4_Y
# Make the initial curve equal to the combined one
curve_X = combinedCurve_X[:]
curve_Y = combinedCurve_Y[:]
# Repeat the loop
# Scale it to fit the canvas
curve_X = [x * xSize for x in curve_X]
curve_Y = [y * ySize for y in curve_Y]
# Draw it with something that connects the dots
curveCoordinates = zip(curve_X, curve_Y)
draw.line((curveCoordinates), fill=(255, 255, 255))
img2=img.rotate(180)
img2.show()
Here is a solution working on matrices (which makes sense for this type of calculations, and in the end, 2D coordinates are matrices with 1 column!),
Scaling is pretty easy, just have to multiply each element of the matrix by the scale factor:
scaled = copy.deepcopy(original)
for i in range(len(scaled[0])):
scaled[0][i]=scaled[0][i]*scaleFactor
scaled[1][i]=scaled[1][i]*scaleFactor
Moving is pretty easy to, all you have to do is to add the offset to each element of the matrix, here's a method using matrix multiplication:
import numpy as np
# Matrix multiplication
def mult(matrix1,matrix2):
# Matrix multiplication
if len(matrix1[0]) != len(matrix2):
# Check matrix dimensions
print 'Matrices must be m*n and n*p to multiply!'
else:
# Multiply if correct dimensions
new_matrix = np.zeros(len(matrix1),len(matrix2[0]))
for i in range(len(matrix1)):
for j in range(len(matrix2[0])):
for k in range(len(matrix2)):
new_matrix[i][j] += matrix1[i][k]*matrix2[k][j]
return new_matrix
Then create your translation matrix
import numpy as np
TranMatrix = np.zeros((3,3))
TranMatrix[0][0]=1
TranMatrix[0][2]=Tx
TranMatrix[1][1]=1
TranMatrix[1][2]=Ty
TranMatrix[2][2]=1
translated=mult(TranMatrix, original)
And finally, rotation is a tiny bit trickier (do you know your angle of rotation?):
import numpy as np
RotMatrix = np.zeros((3,3))
RotMatrix[0][0]=cos(Theta)
RotMatrix[0][1]=-1*sin(Theta)
RotMatrix[1][0]=sin(Theta)
RotMatrix[1][1]=cos(Theta)
RotMatrix[2][2]=1
rotated=mult(RotMatrix, original)
Some further reading on what I've done:
http://en.wikipedia.org/wiki/Transformation_matrix#Affine_transformations
http://en.wikipedia.org/wiki/Homogeneous_coordinates
http://www.essentialmath.com/tutorial.htm (concerning all the algebra transformations)
So basically, it should work if you insert those operations inside your code, multiplying your vectors by the rotation / translation matrices
EDIT
I just found this Python library that seems to provide all type of transformations: http://toblerity.org/shapely/index.html