I'm currently working on a volume rendering project in python where I use a compositing ray casting function to produce an image, given a 3D volume consisting of voxels. The function (which I show below) works correctly, but has a very long runtime. Do you guys have tips on how to make this function faster? The code is Python 3.6.8 and uses various numpy arrays.
def render_compositing(self, view_matrix: np.ndarray, volume: Volume, image_size: int, image: np.ndarray):
# Clear the image
self.clear_image()
# U, V, View vectors. See documentation in parent's class
u_vector = view_matrix[0:3]
v_vector = view_matrix[4:7]
view_vector = view_matrix[8:11]
# Center of the image. Image is squared
image_center = image_size / 2
# Center of the volume (3-dimensional)
volume_center = [volume.dim_x / 2, volume.dim_y / 2, volume.dim_z / 2]
# Define a step size to make the loop faster
step = 2 if self.interactive_mode else 1
for i in range(0, image_size, step):
for j in range(0, image_size, step):
sum_color = TFColor(0, 0, 0, 0)
for k in range(0, image_size, step):
# Get the voxel coordinate X
voxel_coordinate_x = u_vector[0] * (i - image_center) + v_vector[0] * (j - image_center) + \
view_vector[0] * (k - image_center) + volume_center[0]
# Get the voxel coordinate Y
voxel_coordinate_y = u_vector[1] * (i - image_center) + v_vector[1] * (j - image_center) + \
view_vector[1] * (k - image_center) + volume_center[1]
# Get the voxel coordinate Y
voxel_coordinate_z = u_vector[2] * (i - image_center) + v_vector[2] * (j - image_center) + \
view_vector[2] * (k - image_center) + volume_center[2]
color = self.tfunc.get_color(
get_voxel(volume, voxel_coordinate_x, voxel_coordinate_y, voxel_coordinate_z))
sum_color.r = color.a * color.r + (1 - color.a) * sum_color.r
sum_color.g = color.a * color.g + (1 - color.a) * sum_color.g
sum_color.b = color.a * color.b + (1 - color.a) * sum_color.b
sum_color.a = color.a + (1 - color.a) * sum_color.a
red = sum_color.r
green = sum_color.g
blue = sum_color.b
alpha = sum_color.a
# Compute the color value (0...255)
red = math.floor(red * 255) if red < 255 else 255
green = math.floor(green * 255) if green < 255 else 255
blue = math.floor(blue * 255) if blue < 255 else 255
alpha = math.floor(alpha * 255) if alpha < 255 else 255
# Assign color to the pixel i, j
image[(j * image_size + i) * 4] = red
image[(j * image_size + i) * 4 + 1] = green
image[(j * image_size + i) * 4 + 2] = blue
image[(j * image_size + i) * 4 + 3] = alpha
I don't understand why you want to use python for this code. Isn't using a shader the better approach if you are concerned about speed?
Anyways here are few things that can be done in the current code.
voxel coordinates can be calculated using a numpy. you can make a 3 channel 2d image and compute the x,y,z coordinates for an entire slice(k) in a single shot.
Above step can be further optimized by storing an image of x,y,z coordinated of first slice(k=0) and a constant view_directionstep (step_size). Now every other slice can be simply calculated by (XYZ#k=0) + kstep_size.
Use early ray termination by thresholding alpha value to 0.999 or 0.99. This does not look like much but gives a lot of speed gain.
Related
I watched some tutorials and tried to create a Perlin noise generator in python.
It takes in a tuple for the number of vectors in the x and y directions and a scale for the distance in pixels between the arrays, then calculates the dot product between each pixel and each of the 4 arrays surrounding it, It then interpolates them bilinearly to get the pixel's value.
here's the code:
from PIL import Image
import numpy as np
scale = 16
size = np.array([8, 8])
vectors = []
for i in range(size[0]):
for j in range(size[1]):
rand = np.random.rand() * 2 * np.pi
vectors.append(np.array([np.cos(rand), np.sin(rand)]))
interpolated_map = np.zeros(size * scale)
def interpolate(x1, x2, w):
t = (w % scale) / scale
return (x2 - x1) * t + x1
def dot_product(a, b):
return a[0] * b[0] + a[1] * b[1]
for i in range(size[1] * scale):
for j in range(size[0] * scale):
dot_products = []
for m in range(4):
corner_vector_x = round(i / scale) + (m % 2)
corner_vector_y = round(j / scale) + int(m / 2)
x = i - corner_vector_x * scale
y = j - corner_vector_y * scale
if corner_vector_x >= size[0]:
corner_vector_x = 0
if corner_vector_y >= size[1]:
corner_vector_y = 0
corner_vector = vectors[corner_vector_x + corner_vector_y * (size[0])]
distance_vector = np.array([x, y])
dot_products.append(dot_product(corner_vector, distance_vector))
x1 = interpolate(dot_products[0], dot_products[1], i)
x2 = interpolate(dot_products[2], dot_products[3], i)
interpolated_map[i][j] = (interpolate(x1, x2, j) / 2 + 1) * 255
img = Image.fromarray(interpolated_map)
img.show()
I'm getting this image:
but I should be getting this:
I don't know what's going wrong, I've tried watching multiple different tutorials, reading a bunch of different articles, but the result is always the same.
I am implementing Bilinear Interpolation to resize image. The function for bilinear interpolation and resizing is as follows:
def bl_resize(original_img, new_h, new_w):
old_h, old_w, c = original_img.shape
resized = np.ones((new_h, new_w, c))
w_scale_factor = (old_w - 1) / (new_w - 1) if new_h != 0 else 0
h_scale_factor = (old_h - 1) / (new_h - 1) if new_w != 0 else 0
for i in range(new_h):
for j in range(new_w):
for k in range(c):
x = i * h_scale_factor
y = j * w_scale_factor
x_floor = math.floor(x)
x_ceil = min( old_h - 1, math.ceil(x))
y_floor = math.floor(y)
y_ceil = min(old_w - 1, math.ceil(y))
if (x_ceil == x_floor) and (y_ceil == y_floor):
q = original_img[int(x), int(y), k]
else:
v1 = original_img[x_floor, y_floor, k]
v2 = original_img[x_ceil, y_floor, k]
v3 = original_img[x_floor, y_ceil, k]
v4 = original_img[x_ceil, y_ceil, k]
q1 = v1 * (x_ceil - x) + v2 * (x - x_floor)
q2 = v3 * (x_ceil - x) + v4 * (x - x_floor)
q = q1 * (y_ceil - y) + q2 * (y - y_floor)
resized[i,j,k] = q
return resized.astype(np.uint8)
I am using x_ceil = min( old_h - 1, math.ceil(x)) and y_ceil = min(old_w - 1, math.ceil(y)) to avoid access to index larger the the dimensions of the original image array. Without it I would get index out of range error for the last index in both dimensions.
The resized image using this code contains a black grid on it. Here are some output images. The first image is of a shrunken version of the original image and the second one is that of the enlarged one!
EDIT: I have identified what is exactly causing the problem, but I don't understand why it is causing a problem. Changing the scale factor for both the dimensions from (old/new) to (old - 1)/(new - 1) lead to grid free results. I want to understand how the scale factor values can create a problem.
Well, after doing some debugging I figured out the reason. The black grid is obtained because of incorrectly assigned zero values to pixels where either x or y take integer values, that results in q = 0.
I have documented everything here: https://meghal-darji.medium.com/implementing-bilinear-interpolation-for-image-resizing-357cbb2c2722#f91e-235aaa8634b8
I have reached to this bilinear interpolation code (added here), but I would like to improve this code to 3D, meaning update it to work with an RGB image (3D, instead of only 2D).
If you have any suggestions of how I can to that I would love to know.
This was the one dimension linear interpolation:
import math
def linear1D_resize(in_array, size):
"""
`in_array` is the input array.
`size` is the desired size.
"""
ratio = (len(in_array) - 1) / (size - 1)
out_array = []
for i in range(size):
low = math.floor(ratio * i)
high = math.ceil(ratio * i)
weight = ratio * i - low
a = in_array[low]
b = in_array[high]
out_array.append(a * (1 - weight) + b * weight)
return out_array
And this for the 2D:
import math
def bilinear_resize(image, height, width):
"""
`image` is a 2-D numpy array
`height` and `width` are the desired spatial dimension of the new 2-D array.
"""
img_height, img_width = image.shape[:2]
resized = np.empty([height, width])
x_ratio = float(img_width - 1) / (width - 1) if width > 1 else 0
y_ratio = float(img_height - 1) / (height - 1) if height > 1 else 0
for i in range(height):
for j in range(width):
x_l, y_l = math.floor(x_ratio * j), math.floor(y_ratio * i)
x_h, y_h = math.ceil(x_ratio * j), math.ceil(y_ratio * i)
x_weight = (x_ratio * j) - x_l
y_weight = (y_ratio * i) - y_l
a = image[y_l, x_l]
b = image[y_l, x_h]
c = image[y_h, x_l]
d = image[y_h, x_h]
pixel = a * (1 - x_weight) * (1 - y_weight) + b * x_weight * (1 - y_weight) + c * y_weight * (1 - x_weight) + d * x_weight * y_weight
resized[i][j] = pixel # pixel is the scalar with the value comptued by the interpolation
return resized
Check out some of the scipy ndimage interpolate functions. They will do what you're looking for and are 'using numpy'.
They are also very functional, fast and have been tested many times.
Richard
I am trying to colour MandelBrot using HSV values and the PIL Library.
Even after multiple tries fiddling with HSV values, I could not achieve the desired effect.
here is what I currently have
Here is the desired effect
This is the code that I am trying, It could also be beneficial if you could add some tips to optimise the below code to compute the set faster, I am new to python
from PIL import Image
import random
import math
from decimal import Decimal
# Size of the Image Canvas
HEIGHT = 500
ZOOM = 0.0
Y_PAN = 0.0
# Range of the Complex Plane
MIN_X = -2.0 + ZOOM
MAX_X = 2.0 - ZOOM
MAX_Y = 2.0 + Y_PAN - ZOOM
MIN_Y = -2.0 + Y_PAN + ZOOM
DATA = []
def map_to_scale_d(x, in_min, in_max, out_min, out_max):
# returns float
return float((x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min)
def map_to_scale(x, in_min, in_max, out_min, out_max):
# returns int
return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min
# Max iterations till Zn
ITER = 200
# loop to traverse every single point in Canvas
for y in xrange(HEIGHT):
for x in xrange(HEIGHT):
# convert to complex plane scale
a = map_to_scale_d(x, 0, HEIGHT, MIN_X, MAX_X)
b = map_to_scale_d(y, 0, HEIGHT, MAX_Y, MIN_Y)
# original values
_a = a
_b = b
counter = 0
# start the iteration at (a,b) in complex plane
# calculate z^2 + c
while(counter < ITER):
aa = a * a - b * b
bb = 2 * a * b
a = aa + _a
b = bb + _b
if((abs(aa + bb)) > 4):
break
counter = counter + 1
# initialise color
h = 0
s = map_to_scale(counter, 0, ITER, 0, 100)
v = map_to_scale(counter, 0, ITER, 0, 100)
if(counter == ITER):
h = 0
s = 0
v = 0
# convert to 8-bit
h = map_to_scale(h, 0, 360, 0, 255)
s = map_to_scale(s, 0, 100, 0, 255)
v = map_to_scale(v, 0, 100, 0, 255)
DATA.append((h, s, v))
img = Image.new('HSV', (HEIGHT, HEIGHT))
img.putdata(DATA)
img.show()
img.convert('RGB').save('test.png')
I am trying to enhance my image by first converting RGB color space to YUV color space and do histogram equalization to Y value. However, the output image does not look good.
For histogram equalization, I use the method found on Wikipedia.
Here is the input image:
Here is the output image:
I really don't know where the problem is, can anyone help me or give me some hint?
Below is my code,
import cv2
import numpy as np
img = cv2.imread('/Users/simon/Documents/DIP/Homework_3/input4.bmp')
shape = img.shape
Y_origin_hist = [0] * 256
U_origin = [[0 for i in range(0, shape[1])] for j in range(0, shape[0])]
V_origin = [[0 for i in range(0, shape[1])] for j in range(0, shape[0])]
Y_hist = [0] * 256
# Read RGB value and calculate YUV value
for i in range(0, shape[0]) :
for j in range(0, shape[1]) :
px = img[i,j]
y = int(0.299 * px[2] + 0.587 * px[1] + 0.114 * px[0])
u = int(-0.169 * px[2] - 0.331 * px[1] + 0.5 * px[0]) + 128
v = int(0.5 * px[2] - 0.419 * px[1] - 0.081 * px[0]) + 128
Y_origin_hist[y] = Y_origin_hist[y] + 1
U_origin[i][j] = u
V_origin[i][j] = v
# Histogram equalization
for i in range(0, 256) :
Y_hist[i] = int(((sum(Y_origin_hist[0:i]) - min(Y_origin_hist) - 1) * 255) / ((shape[0] * shape[1]) - 1))
# Write back to RGB value
for i in range(0, shape[0]) :
for j in range(0, shape[1]) :
px = img[i,j]
px[0] = int(Y_hist[px[0]] + 1.77216 * (U_origin[i][j] - 128) + 0.00099 * (V_origin[i][j] - 128))
px[1] = int(Y_hist[px[1]] - 0.3437 * (U_origin[i][j] - 128) - 0.71417 * (V_origin[i][j] - 128))
px[2] = int(Y_hist[px[2]] - 0.00093 * (U_origin[i][j] - 128) + 1.401687 * (V_origin[i][j] - 128))
cv2.imwrite('/Users/simon/Documents/DIP/Homework_3/output4.bmp', img)
For OpenCV in C++ the + and - operators are overloaded and they automatically prevent overflows. However, this is not the case when using Python. For this reason you should use cv2.add() and cv2.subtract() when doing math to get the same results that you would get using C++.