I have the following skeleton:
From this image I'd like to eliminate lines that are not part of loops.
I imagine this as a process in which ends of lines are found (marked with red dots) and the lines are gobbled up until there's a point where they branch (marked with blue dots).
I haven't found an operation for this in OpenCV or Scikit-Image.
Is there a name for such a transform? Is there a way to implement it in Python that would work efficiently?
I've also uploaded the image here in case the above image doesn't load correctly.
I haven't found a good way to do this in Python using existing libraries (though I hope someone is able to point me one), nor the name of this.
So I've decided to call this the Fuse Transform, since the action of the algorithm is similar to burning the lines away like fuses until they split.
I've implemented the Fuse Transform below as a Cython function for efficiency.
The algorithm requires a single O(N) time in the size of the matrix to sweep to identify seed cells (those cells that are at the start of a fuse) and then O(N) time in the total length of the fuses to eliminate the lines in question.
The Fuse Transform Algorithm
%%cython -a --cplus
import numpy as np
import cv2
import skimage.morphology as skm
import cython
from libcpp.queue cimport queue
cimport numpy as np
#cython.boundscheck(False)
#cython.wraparound(False)
#cython.nonecheck(False)
#cython.cdivision(True)
#Richard's Fuse Transform
#https://stackoverflow.com/a/51738867/752843
cpdef void FuseTransform(unsigned char [:, :] image):
# set the variable extension types
cdef int c, x, y, nx, ny, width, height, neighbours
cdef queue[int] q
# grab the image dimensions
height = image.shape[0]
width = image.shape[1]
cdef int dx[8]
cdef int dy[8]
#Offsets to neighbouring cells
dx[:] = [-1,-1,0,1,1,1,0,-1]
dy[:] = [0,-1,-1,-1,0,1,1,1]
#Find seed cells: those with only one neighbour
for y in range(1, height-1):
for x in range(1, width-1):
if image[y,x]==0: #Seed cells cannot be blank cells
continue
neighbours = 0
for n in range(0,8): #Looks at all neighbours
nx = x+dx[n]
ny = y+dy[n]
if image[ny,nx]>0: #This neighbour has a value
neighbours += 1
if neighbours==1: #Was there only one neighbour?
q.push(y*width+x) #If so, this is a seed cell
#Starting with the seed cells, gobble up the lines
while not q.empty():
c = q.front()
q.pop()
y = c//width #Convert flat index into 2D x-y index
x = c%width
image[y,x] = 0 #Gobble up this part of the fuse
neighbour = -1 #No neighbours yet
for n in range(0,8): #Look at all neighbours
nx = x+dx[n] #Find coordinates of neighbour cells
ny = y+dy[n]
#If the neighbour would be off the side of the matrix, ignore it
if nx<0 or ny<0 or nx==width or ny==height:
continue
if image[ny,nx]>0: #Is the neighbouring cell active?
if neighbour!=-1: #If we've already found an active neighbour
neighbour=-1 #Then pretend we found no neighbours
break #And stop looking. This is the end of the fuse.
else: #Otherwise, make a note of the neighbour's index.
neighbour = ny*width+nx
if neighbour!=-1: #If there was only one neighbour
q.push(neighbour) #Continue burning the fuse
#Read in image
img = cv2.imread('part.jpg')
ShowImage('Original',img,'bgr')
#Convert image to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
#Apply Otsu's method to eliminate pixels of intermediate colour
ret, thresh = cv2.threshold(gray,0,255,cv2.THRESH_OTSU)
#Apply the Fuse Transform
skh_dilated = skelhuman.copy()
FuseTransform(skh_dilated)
Input
Result
In the following algorithm first I normalize the image pixels to have values zeros and ones. Then I examine the 8-connected neighbors of a non-zero pixel by applying a 3x3 unnormalized box-filter. If we multiply (pixelwise) the filter output from the input image, we get all the non-zero pixels, this time, their values tell us how many 8-connected neighbors they have plus 1. So, here the center pixel counts itself as its neighbor.
Red is the center pixel. Yellow are its 8-connected neighborhood.
We should eliminate the result pixel values less than 3.
The code will make things clearer. It may not be very efficient. I didn't try to dig into Richard's code. May be he's doing a similar thing efficiently.
import cv2
import numpy as np
im = cv2.imread('USqDW.png', 0)
# set max pixel value to 1
s = np.uint8(im > 0)
count = 0
i = 0
while count != np.sum(s):
# non-zero pixel count
count = np.sum(s)
# examine 3x3 neighborhood of each pixel
filt = cv2.boxFilter(s, -1, (3, 3), normalize=False)
# if the center pixel of 3x3 neighborhood is zero, we are not interested in it
s = s*filt
# now we have pixels where the center pixel of 3x3 neighborhood is non-zero
# if a pixels' 8-connectivity is less than 2 we can remove it
# threshold is 3 here because the boxfilter also counted the center pixel
s[s < 3] = 0
# set max pixel value to 1
s[s > 0] = 1
i = i + 1
After pruning:
Related
I need to write function that convolving an image with a kernel.
In other words -The function receives an image with a single color channel (ie a two-dimensional list(for example - [[1,1,1],[1,1,1],[1,1,1]]) and a kernel (also a two-dimensional list), and returns an image of the same size as the original image, with each pixel in the new image calculated by running the kernel on it.
That is: identify the pixel [image [row] [column with the main input in the kernel matrix, and sum the values of its neighbors (including
The pixel itself) double the corresponding input for them in the kernel.
When calculating a value for an x-pixel that is on the image boundaries, pixel values that are outside the image boundaries should be considered.
The source seemed to have the same value as the pixel x.
For example- for input:
image = [[0,128,255]]
kernel =[[1/9 ,1/9 ,1/9],[1/9 ,1/9 ,1/9] ,[1/9 ,1/9 ,1/9]]
output: [[14,128,241]]
The function starts at about zero and will place it in the center of a kernel-sized matrix, along with the adjacent values that are within the boundaries of this matrix.
In the example, this is a 3 * 3 matrix and therefore the matrix we will receive after entering the values is-[[0,0,0],[0,128,0],[0,0,0]].
After we have entered the corresponding values we will multiply the enter matrix by the kernel matrix (respectively so that pixels in the same coordinates between the two matrices will be multiplied by each other) and sum it all together then enter the result in the image size list instead of the value 0.
And then do the same with the next value- 128 and so on.
Eventually, we will return a new image with the new pixels we calculated as I presented.
Another explanation-
https://towardsdatascience.com/types-of-convolution-kernels-simplified-f040cb307c37
According to the instructions I received I can not use a numpy.
def new_image(image,kernel):
new_image= copy.deepcopy(image)
rows = len(image)
columns = len(image[0])
kernel_h = len(kernel)
kernel_w = len(kernel[0])
for i in range(rows):
for j in range(columns):
sum = 0
h = (-1 * (kernel_h // 2))
w = (-1 * (kernel_w // 2))
for m in range(kernel_h):
for n in range(kernel_w):
if 0 <= j+w < columns:
sum += round(kernel[m][n] * new_image[i][j+h])
if j + h < 0 or j + h >= columns:
sum += round(kernel[m][n] * new_image[i][j])
h+=1
w+=1
new_image[i][j] = sum
return new_image
This is what I wrote until now, but it does not work as required, meaning it does not return the image as required.
Output-[[42, 131, 239]]
instead of- [[14,128,241]]
Input=[[0,128,255]
I have no idea how to fix it, i would appreciate help.
Using matplotlib(or if there exists anything else), i want to populate a scatterplot image by using a grey scale image as its distribution. I have found many resource to create heat maps from images but not the other way around.
The input image will be like this one.
I think I understand what you're going for, but I'm not certain. I also don't really understand what this would be used for so I'm extra uncertain about this answer, but here goes:
So by loading the image we can evaluate each pixel position and its intensity. We can use that intensity as a "fitness" value and probabilistically add it to our plot so that we can get some of that "density" of points that you want to see. I picked a really simple equation as a decider (I just cubed the value), but feel free to replace that with whatever you want.
import cv2
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import random
# select func
def selection(value):
return value**3 >= random.randint(0, 255**3);
# populate the sample
def populate(img):
# get res
h, w = img.shape;
# go through and populate
sx = [];
sy = [];
for y in range(0, h):
for x in range(0, w):
val = img[y, x];
# use intensity to decide if it gets in
# replace with what you want this function to look like
if selection(val):
sx.append(x);
sy.append(h - y); # opencv is top-left origin
return sx, sy;
# I'm using opencv to pull the image into code, use whatever you like
# matplotlib can also do something similar, but I'm not familiar with its format
img = cv2.imread("circ.png");
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY);
# lets take a sample
sx, sy = populate(img);
# find the bigger square size
h, w = img.shape;
side = None;
if h > w:
side = h;
else:
side = w;
# make a square graph
fig, ax = plt.subplots();
ax.scatter(sx, sy, s = 4);
ax.set_xlim((0, side));
ax.set_ylim((0, side));
x0,x1 = ax.get_xlim();
y0,y1 = ax.get_ylim();
ax.set_aspect(abs(x1-x0)/abs(y1-y0));
fig.savefig("out.png", dpi=600);
plt.show();
Feel free to replace opencv with whatever image library you're comfortable with. I'm pretty sure matplotlib can open images as well, but openCV is what I'm most familiar with so I used that.
As far as I can tell, you're trying to generate random coordinates that follow a distribution described by a grayscale image: the brighter each point, the more likely that point's coordinates will be generated. Your problem can thus be solved by a rejection sampler, as follows.
Assume you know the width and height of the image in pixels, call them w and h.
Generate two random numbers: one in the interval [0, w), and [0, h). These are the x and y coordinates, respectively.
Get the pixel at the given coordinates x and y in the image. This can be done using interpolation, but describing interpolation techniques is beyond the scope of this answer. For this reason, we will use only the nearest pixel ("nearest neighbor") in the image: take the pixel at coordinate floor(x) and floor(y) (and step 1 devolves to generating random integers). Convert the pixel somehow to a number p in the interval [0, 1]; in this answer we will assume black is 0 and white is 1, to simplify matters.
With probability p, return the point (x, y). Otherwise, go to step 1.
Roughly speaking, the time complexity of this algorithm depends on the numbers of "bright points" the input image has, compared to the number of "dark points". In general, the "brighter" the image, the higher the acceptance rate (and the faster the algorithm runs).
I have made a workflow code to detect the edges of a flame in an image. I could get the edge line. It consists of many pixel points stored in an array (data in my code). Now based on the data, I would like to calculate the length of the edge. The idea is to calculate the distance between every point in data and sum them all to get the length. I really stuck in making that. Please help me, many thanks.
Here is a processed image:
Here is the original image that converted to the processed image, I put in the code is to compare the result:
import cv2
import matplotlib.pyplot as plt
if __name__ == '__main__':
path = '1897_1.jpg' #processed image
pic = cv2.imread(path)
original = cv2.imread('1897_2.jpg') #original image
img2 = cv2.flip(original, 1)
b,g,r = cv2.split(pic)
img4 = cv2.flip(b, 1)
h,w = img4.shape
data = []
th_val = 20
for i in range(h):
for j in range(w):
val = img4[i, j]
if (val >= th_val):
data.append(j)
break
b1 = range(len(data))
b2 = len(data)
result = [b2]
print (b2)
plt.figure(figsize = (10, 8))
plt.subplot(121)
plt.imshow(img4)
plt.plot(data, b1)
plt.axis('off');
plt.subplot(122)
plt.plot(data, b1)
plt.imshow(img2)
plt.axis('off')
I came up with a very simple solution, it is far from optimal, but it works for this example, and it is a good starting point. Unfortunately, this solution is not optimal for the blue chanell, where the curve is not smooth, but it works for green and red chanells.
data contains width coordinates of the first red pixel overcoming threshold. So, all first pixels are separated by 1 pixel step on vertical axes and data[i+1] - data[i] on horizontal axes. These two values can be considered as two cathetus of the squeare triangle, and the hypothenuse is the distance we want to calculate. So, here is the solution:
length = 0
for i in range(0,len(data)-1):
cathetus = data[i+1]-data[i]
hypothenuse = (cathetus**2 + 1**2)**1/2
length += hypothenuse
print(length)
Update
I have came up with two solutions: a hardcoded one and one released in the form of the function. Let us start with the first one: mean is a rather good approximator for the signal + noise. In the situation, when you do not have very strong noise or missing data, you may use this approach. In the example below we select points with x in [1,2,3] then we calculate mean y for these points and assign mean to coordinate x=2. Next we select points x in [2,3,4] and so on. As a result, we obtain mean_data list with y coordinates and mean_x with x coordinates. We can calculate length with the approach described above. You may also increase the power of smoothing by averaging over 4 and more points from data.
mean_data = []
mean_x = range(1,len(data)-1)
for i in range(0,len(data)-2):
mean_d = (data[i] + data[i+1] + data[i+2])/3
mean_data.append(mean_d)
Another approach is to use smoothing tools from scipy package. One of them is described below. When calculating the length you will have to adjust to new x axes xnew.
from scipy.interpolate import spline
import numpy as np
#transform to np.arrays initial data
b1_ = np.array(b1)
data_ = np.array(data)
# create new x with more data points
xnew = np.linspace(b1_.min(),b1_.max(),50) #50 is a number of points in between
smoothed_data = spline(b1_,data_,xnew)
I am new in python programming, so forgive me if my questions are too basic. I've been helped a lot by this forum before and thanks to you guys for all your contributions.
This time I have a set of 12,000 image data which I am performing singular value decomposition (svd) on and calculating their mean. Some of the images have pixels with very high positive or negative values which I don't want to use during computation, so I used
numpy.ma.masked_array to exclude them from both svd and mean computation. And some images are smaller than others and they were padded with zeros values to make all images to have the same (pixel) dimension. But I also don't want the 'zero paddings' to be used during computation, so I used numpy.ma.masked_array to exclude them from both svd and mean calculation.
Here are some example images:
The problem is that when I perform both svd and mean calculation, the masked values (array elements) are not excluded during computation. I have tried all that I know to resolve this without success. Below are the steps that I took.
from numpy.linalg import svd
import numpy as np
from numpy.ma import masked_array
n, x, y = images.data.shape
Z = []
meanimage = []
for icount in range(n):
image = images[icount,:,:] # current image
# creating a mask for too positively or negatively high values
mask = (np.abs(image) > 2).astype(int);
yindex = 0; xindex = 0;
# --- creating a mask for zero padded values
for i in range(y/2): # get the index of the first none zero pixel
if image[i,x/2] != 0:
yindex = i
break
for i in range(x/2): # get the index of the first none zero pixel
if image[y/2,i] != 0:
xindex = i
break
mask[:yindex,:] = 1;mask[-yindex:,:] = 1;
mask[:xindex,:] = 1;mask[-xindex:,:] = 1;
# ---
image = masked_array(images[icount,:,:], mask)
Z.append(image.ravel()) # accummulating matrix for svd computation
meanimage.append(image) # accummulating matrix for for mean computation
# calc. SVD
u,s,v = svd(masked_array(Z))
#calc. mean image
meanimage = masked_array(meanimage).mean(axis=0)
bimage = np.dot(np.dot(u[:,:2],np.diag(s[:2])),np.transpose(v)[:2,:])
eigenimage = bimage[2,:].reshape(x, y)
The final results - eigenimage and meanimage - that I get does not exclude the masked values from computation. I don't know what I did wrong. Please, I need some ideas that will help me to resolve this.
Above are some samples of the images (beams) data that I am working with.
The final images that I get after computation for the eigenimage and meanimage are :
Eigen (beam) image (with SVD)
Mean (beam) image (masked_array mean)
From the above figures, both the eigenimage and meanimage loses a lot of side lobes information which are not desired.
But I was expecting the final eigen images to be like
The masked_array mean actually excludes masked pixels ('zero paddings') from mean computation. I confirmed this by comparing this result with the one calculated without a mask and noticed a remarkable difference, which confirms that numpy.ma.masked_array mean works perfectly for my case.
On the SVD Eigen image:
The problem was with the transposing v (np.transpose(v)). I found out from documentation (1) that numpy.linalg.svd returns a transpose of v, so I just needed to perform the dot product without transposing v.
bimage = np.dot(np.dot(u[:,:2],np.diag(s[:2])),v[:2,:])
I have a range image of a scene. I traverse the image and calculate the average change in depth under the detection window. The detection windows changes size based on the average depth of the surrounding pixels of the current location. I accumulate the average change to produce a simple response image.
Most of the time is spent in the for loop, it is taking about 40+s for a 512x52 image on my machine. I was hoping for some speed up. Is there a more efficient/faster way to traverse the image? Is there a better pythonic/numpy/scipy way to visit each pixel? Or shall I go learn cython?
EDIT: I have reduced running time to about 18s by using scipy.misc.imread() instead of skimage.io.imread(). Not sure what the difference is, I will try to investigate.
Here is a simplified version of the code:
import matplotlib.pylab as plt
import numpy as np
from skimage.io import imread
from skimage.transform import integral_image, integrate
import time
def intersect(a, b):
'''Determine the intersection of two rectangles'''
rect = (0,0,0,0)
r0 = max(a[0],b[0])
c0 = max(a[1],b[1])
r1 = min(a[2],b[2])
c1 = min(a[3],b[3])
# Do we have a valid intersection?
if r1 > r0 and c1 > c0:
rect = (r0,c0,r1,c1)
return rect
# Setup data
depth_src = imread("test.jpg", as_grey=True)
depth_intg = integral_image(depth_src) # integrate to find sum depth in region
depth_pts = integral_image(depth_src > 0) # integrate to find num points which have depth
boundary = (0,0,depth_src.shape[0]-1,depth_src.shape[1]-1) # rectangle to intersect with
# Image to accumulate response
out_img = np.zeros(depth_src.shape)
# Average dimensions of bbox/detection window per unit length of depth
model = (0.602,2.044) # width, height
start_time = time.time()
for (r,c), junk in np.ndenumerate(depth_src):
# Find points around current pixel
r0, c0, r1, c1 = intersect((r-1, c-1, r+1, c+1), boundary)
# Calculate average of depth of points around current pixel
scale = integrate(depth_intg, r0, c0, r1, c1) * 255 / 9.0
# Based on average depth, create the detection window
r0 = r - (model[0] * scale/2)
c0 = c - (model[1] * scale/2)
r1 = r + (model[0] * scale/2)
c1 = c + (model[1] * scale/2)
# Used scale optimised detection window to extract features
r0, c0, r1, c1 = intersect((r0,c0,r1,c1), boundary)
depth_count = integrate(depth_pts,r0,c0,r1,c1)
if depth_count:
depth_sum = integrate(depth_intg,r0,c0,r1,c1)
avg_change = depth_sum / depth_count
# Accumulate response
out_img[r0:r1,c0:c1] += avg_change
print time.time() - start_time, " seconds"
plt.imshow(out_img)
plt.gray()
plt.show()
Michael, interesting question. It seems that the main performance problem you have is that each pixel in the image has two integrate() functions computed on it, one of size 3x3 and the other of a size which is not known in advance. Calculating individual integrals in this way is extremely inefficient, regardless of what numpy functions you use; it's an algorithmic issue, not an implementation issue. Consider an image of size NN. You can calculate all integrals of any size KK in that image using only approximately 4*NN operations, not (as one might naively expect) NNKK. The way you do that is first calculate an image of sliding sums over a window K in each row, and then sliding sums over the result in each column. Updating each sliding sum to move to the next pixel requires only adding the newest pixel in the current window and subtracting the oldest pixel in the previous window, thus two operations per pixel regardless of window size. We do have to do that twice (for rows and columns), therefore 4 operations per pixel.
I am not sure if there is a sliding window sum built into numpy, but this answer suggests a couple of ways to do it, using stride tricks: https://stackoverflow.com/a/12713297/1828289. You can certainly accomplish the same with one loop over columns and one loop over rows (taking slices to extract a row/column).
Example:
# img is a 2D ndarray
# K is the size of sums to calculate using sliding window
row_sums = numpy.zeros_like(img)
for i in range( img.shape[0] ):
if i > K:
row_sums[i,:] = row_sums[i-1,:] - img[i-K-1,:] + img[i,:]
elif i > 1:
row_sums[i,:] = row_sums[i-1,:] + img[i,:]
else: # i == 0
row_sums[i,:] = img[i,:]
col_sums = numpy.zeros_like(img)
for j in range( img.shape[1] ):
if j > K:
col_sums[:,j] = col_sums[:,j-1] - row_sums[:,j-K-1] + row_sums[:,j]
elif j > 1:
col_sums[:,j] = col_sums[:,j-1] + row_sums[:,j]
else: # j == 0
col_sums[:,j] = row_sums[:,j]
# here col_sums[i,j] should be equal to numpy.sum(img[i-K:i, j-K:j]) if i >=K and j >= K
# first K rows and columns in col_sums contain partial sums and can be ignored
How do you best apply that to your case? I think you might want to pre-compute the integrals for 3x3 (average depth) and also for several larger sizes, and use the value of the 3x3 to select one of the larger sizes for the detection window (assuming I understand the intent of your algorithm). The range of larger sizes you need might be limited, or artificially limiting it might still work acceptably well, just pick the nearest size. Calculating all integrals together using sliding sums is so much more efficient that I am almost certain it is worth calculating them for a lot of sizes you would never use at a particular pixel, especially if some of the sizes are large.
P.S. This is a minor addition, but you may want to avoid calling intersect() for every pixel: either (a) only process pixels which are farther from the edge than the max integral size, or (b) add margins to the image of the max integral size on all sides, filling the margins with either zeros or nans, or (c) (best approach) use slices to take care of this automatically: a slice index outside the boundary of an ndarray is automatically limited to the boundary, except of course negative indexes are wrapped around.
EDIT: added example of sliding window sums