I can compute the SLIC boundaries using skimage as follows:
def compute_superpixels(frame, num_pixels=100, std=5, iter_max=10,
connectivity=False, compactness=10.0):
return slic(frame, n_segments=num_pixels, sigma=std, max_iter=iter_max,
enforce_connectivity=connectivity, compactness=compactness)
Now, what I would like to do is get the index of pixels which form the boundary of each label. So my idea was to get all pixels belonging to a given segment and then check which pixels have a change in all two directions
def boundary_pixels(segments, index):
# Get all pixels having a given index
x, y = np.where(segments == index)
right = x + 1
# check we are in bounds
right_mask = right < segments.shape[0]
down = y + 1
down_mask = down < segments.shape[1]
left = x - 1
left_mask = left >= 0
up = y - 1
up_mask = up >= 0
neighbors_1 = np.union1d(right_n, down_n)
neighbors_2 = np.union1d(left_n, up_n)
neighbors = np.union1d(neighbors_1, neighbors_2)
# Not neighbours to ourselves
neighbors = np.delete(neighbors, np.where(neighbors == i))
However, with this all I managed to do was to get the neighbours in the 4 directions of a given label. Can someone suggest some way to actually get all pixels on the border of the label.
I found an answer to my own question. The mark_boundaries in the skimage.segmentation package does exactly what I needed.
Usage:
processed = mark_boundaries(frame, segments==some_segment)
Here frame is he current image frame and segments is the label array. some_segment is the label integer index whose boundaries we are interested in.
You can make use of the find_contours function available in skimage.measure module to find the co-ordinates of the pixels along the boundary. An example is available at find_contours.. Next, you can change for change in both directions as needed.
Related
I have an image (saved as a variable called canny_image) and it looks like this after preprocessing.
I am basically trying to find the distance between the first two vertical lines. I tried using the hough_line function from skimage, but it's unable to find the first line, so I thought it might be easier to solve this manually.
I am basically trying to solve this by going through each row in the image until I get to the first pixel with a value of 255, (the lines have a value of 255, while everything else is zero), and then I store the location of that pixel in an array. And I take the mode of the values in the array as the x location of the first line. I'll do the same for the 2nd line by using the first x-value as a starting point.
def find_lines(canny_image):
threshold = 255
for y in range(canny_image.shape[0]):
for x in range(canny_image.shape[1]):
if canny_image[x, y] == threshold:
return x
This is the code I wrote to get the x-location of the first line, however, I'm not getting the desired output. Any help on how to solve this will be much appreciated. Thanks!
Perhaps try something like this
# Returns an array of lines x positions in an image
def find_line_x_positions(image, lines_to_detect: int, buffer_zone: int):
threshold = 255
(height, width) = image.shape
x_position_sums = np.zeros((lines_to_detect, 2), np.double) # For each line, store x_pos sum[i,0] and point count[i,1]
for y in range(height):
buffer = 0
line_index = 0
for x in range(width):
if buffer > 0:
buffer -= 1
if (image[y, x] >= threshold) and (buffer == 0):
buffer = buffer_zone
x_position_sums[line_index, 0] += x
x_position_sums[line_index, 1] += 1
line_index += 1
if ((line_index) == lines_to_detect):
break
# Divide the x position sums by the point counts to get the average x position for each line
results = x_position_sums[np.all(x_position_sums,axis=1)]
results = np.divide(results[:,0],results[:,1])
return results
You can also try OpenCV's HoughLines() function which is simpler to implement than scikit lib. When I tested OpenCV implementation out, it seems to have a hard time finding vertical lines(within 10 degrees from vertical) but you can solve this by rotating your image X degrees and look for lines within that range of rotation.
I have a set of data like these:
And I want to recognize this 2 kind of corner shape, is there any way? I wrote s snippet but it sucks.
In this way I am trying to find desc corner (from top to down) and to find asc corner I am "rotating" the dataset and I apply the same algorithm for desc corners.
...
n_rows = len(matrix)
edge_closing_left_total = 0
j=0
for i in range(n_rows):
edge_closing_left = True
current_value = matrix[i][j]
if current_value >= 230:
current_row = i
start = j+1
end = j+1+3
for k in range(start, end):
if current_row+2 < n_rows:
if matrix[current_row][k]<230 and matrix[current_row+1][k]<230 or matrix[current_row+1][k]<230 and matrix[current_row+2][k]<=230:
current_row+=1
else:
edge_closing_left = False
break
if edge_closing_left:
edge_closing_left_total+=1
return edge_closing_left_total
Here the csv dataset file.
So you need to do template/pattern matching. This could be achieved with correlation. The following code should demonstrate how to do it using scipy:
import pandas as pd
from scipy.signal import correlate2d
img = pd.read_csv('matrix.csv', header=None).to_numpy()
norm = img - img.mean() # subtract mean to normalize
edge = norm[26:36, :] # the edge template, adjust if needed
corr = correlate2d(norm, edge, mode='valid')
auto_corr = correlate2d(edge, edge, mode='valid')
corr /= auto_corr # normalize correlation so that 1 means perfect correlation
corr_cutoff = .9 # 1 is pixel-perfect match
print(f'Found edge template {(corr > corr_cutoff).sum()} times in image when taking a similarity cut-off of {corr_cutoff}')
# Found edge template 2 times in image when taking a similarity cut-off of 0.9
The original image (transposed to make it wider than high):
The edge template looks like this:
The correlation map looks like this (note: this is actually only 1 px wide but blown up for visualization):
The thresholded correlation map with cut-off 0.9 (giving 2 results as expected, choose threshold as needed):
I have a template in the form of a GeoPandas GeoDataFrame with several shapely geometries that correspond to areas on a page.
For this process, the pages are photographed and I need the geometries in my template with the corresponding areas on the photographed page. Right now, my "projection" of the template to the bounding box of the photographed page does not align with the photographed image. I'm also pretty sure there is a better way to do this.
I've created a simplified example to illustrate:
I have this document, with both axes scaled to 1.
I created template that matches the objects on the image
shapes = [
('line1', LineString([(0.146, 0.216), (0.476 , 0.216)])),
('line2', LineString([(0.498, 0.871), (0.838, 0.871)]))
]
shapes = gpd.GeoDataFrame(shapes, columns=['name', 'geometry'], geometry='geometry')
Plotting that on top of the image we can verify that the shapes closely align with the shapes in the image.
plt.imshow(img, extent=[0,1,1,0], aspect=1.4142)
ax = plt.gca()
shapes.plot(ax=ax)
ax.set_aspect(1.4142)
I take a photograph of the same document get the bounding box of the page.
photo = load_image('example_photo.jpg')
bbox = Polygon([(0.847429096698761, 0.047594085335731506),
(0.9442346692085266, 0.8787651062011719),
(0.05563090369105339, 0.8789013028144836),
(0.12740036845207214, 0.06380380690097809),
(0.847429096698761, 0.047594085335731506)])
plt.figure(figsize=(6,6))
plt.imshow(photo, extent=[0,1,1,0], aspect=1.4142)
plt.plot(*bbox.boundary.xy)
The problem is in the next step as I try to remap or project the original template into the shape of the bounding box. This is what I have tried, but I'm sure this isn't the most efficient way to do this. Also it doesn't work.
Summary of the method below is:
figure out which edge is top, bottom, right, left and orient them.
map each point into the new shape finding the intersection of the lines meeting each opposite side of the bounding box at proportionaly the same place as in the unit square.
def get_edges(bbox, visualize=True):
'''Takes shapely polygon with exactly 5 points'''
# check for 5 points
if len(bbox.boundary.coords) != 5:
raise('Polygon must have 5 points (4 sides)')
#find top/bottom edge
x, y = bbox.boundary.xy
# remove last point which must be the same as the first
x = np.array(x)[:-1]
y = np.array(y)[:-1]
# sort by y values. Higher is closer to top, lower closer to bottom
y_sorted = np.argsort(y)
# get the index of the top and bottom lines
top_points_idx = y_sorted[-2:]
bot_points_idx = y_sorted[:2]
# order the top point coords left to right
top_point_order = top_points_idx[
np.argsort(x[top_points_idx])
]
bot_point_order = bot_points_idx[
np.argsort(x[bot_points_idx])
]
top_points = np.array(bbox.boundary.coords)[top_point_order]
bot_points = np.array(bbox.boundary.coords)[bot_point_order]
left_points = LineString([bot_points[0], top_points[0]])
right_points = LineString([bot_points[1], top_points[1]])
top_points = LineString(top_points)
bot_points = LineString(bot_points)
return top_points, bot_points, left_points, right_points
def project_unit_square_point(point, bbox, visualize=False):
# check for 5 points
if len(bbox.boundary.coords) != 5:
raise('Polygon must have 5 points (4 sides)')
# use the position as the portion of the each side
x_scale, y_scale = point.coords[0]
top, bot, left, right = get_edges(bbox)
# find proportional intersections on edges
top_point = top.interpolate(x_scale*top.length)
bot_point = bot.interpolate(x_scale*bot.length)
left_point = left.interpolate(y_scale*left.length)
right_point = right.interpolate(y_scale*right.length)
# connect edge points
vline = LineString([top_point, bot_point])
hline = LineString([left_point, right_point])
# new point is intersection of vline and hline
new_point = vline.intersection(hline)
return new_point
def project_unit_square_geom(geom, bbox):
new_points = []
for point in geom.coords:
new_points.append(project_unit_square_point(Point(point), bbox))
new_geom = LineString(new_points)
return new_geom
# project geoms onto form
projected_shapes = []
for shape in shapes.geometry:
projected_shapes.append(
project_unit_square_geom(shape, bbox)
)
# create a new df for the mapped shapes
projected_shapes = gpd.GeoSeries(projected_shapes, name='geometry')
projected_shapes = gpd.GeoDataFrame({'name': shapes['name'],
'geometry': projected_shapes},
geometry='geometry')
Then when I visualize the result I get this:
plt.figure(figsize=(6,6))
plt.imshow(photo, extent=[0,1,1,0])
plt.plot(*bbox.boundary.xy)
ax = plt.gca()
projected_shapes.plot(ax=ax)
ax.set_aspect(1.4142)
Close but not close enough. Obvioulsy my approach is not working. How can I map the template shapes onto the new shape defined by the bounding box?
Here are the original images to work with.
I am trying to identify connected regions of pixels in an image stack. Since it is a stack, the input is quite large (on the order of 10 million pixels, although only about 1 million are bright), and looping through it pixel by pixel is extremely slow. Nonetheless, my first attempt works as follows:
1) Check if current pixel is bright, if so go to 2. If not go to 4.
2) Obtain slice of image containing all 26 neighbors. Set all neighbors that have not yet been scanned to be dark. Set all neighbors that fall outside of the image (e.g. indices of -1) to be dark. Set current pixel to dark.
3) If all pixels in slice are dark, assign current pixel a new label. If exactly one pixel in slice is bright, set current pixel label to be equal to this bright pixel's label. If more than one pixel is bright, set current pixel to lowest of bright pixel labels, and record the equivalence of these labels.
4) Advance one row. Once all rows in the column are scanned, advance one column. Once every pixel is in the slice is scanned, advance one slice in the stack. Once the entire image is scanned, advance to 5.
5) Loop through the list of equivalencies, reassigning labels in the output.
I don't think the processing in steps 2 and 3 are much slower than other, similar algorithms, but I could be wrong. I think the major bottleneck is simply looping through each pixel. I know the bottleneck is not #5 because I have never been patient enough to reach that step. Even just looping over the image indices with a print statement is quite slow. I know that with the same data I can do this analysis quickly, so I suspect there must be a smarter way to do this with a very different approach. I have read something vague about performing dilation and using intersections, but I can't imagine what the mechanism they were alluding to was.
Is there something fast and clever out there, or is this basically the only way to do it? How is the MATLAB algorithm so much faster?
for i in xrange(0,nPix):
for j in xrange(0,nPix):
for k in xrange(0,nFrames):
if img[i,j,k] != 0: #If the current pixel is bright
#Construct an array of already scanned
#neighbors. Pixels are scanned first in z, then y,
#then x.
#Construct indices allowing for periodic boundary
ir = np.array(range(i-1,i+2)).reshape(-1,1,1)%nPix
jr = np.array(range(j-1,j+2)).reshape(1,-1,1)%nPix
kr = np.array(range(k-1,k+2)).reshape(1,1,-1)%nFrames
pastNeighbors = img[ir,jr,kr] #Includes i-1:i+1
#Set all indices outside image and "future neighbors" to 0
pastNeighbors[2,:,:] = 0
pastNeighbors[1,2,:] = 0
pastNeighbors[1,1,2] = 0
pastNeighbors[1,1,1] = 0 #Pixel itself; not a neighbor
#Eliminate periodic boundary
if i-1 < 0: pastNeighbors[0,:,:] = 0
if i+1 == nPix: pastNeighbors[2,:,:] = 0
if j-1 < 0: pastNeighbors[:,0,:] = 0
if j+1 == nPix: pastNeighbors[:,2,:] = 0
if k-1 < 0: pastNeighbors[:,:,0] = 0
if k+1 == nFrames: pastNeighbors[:,:,2] = 0
if np.count_nonzero(pastNeighbors) == 0: #If all dark neighbors
label = label + 1 #Assign new label to pixel
out[i,j,k] = label
elif np.count_nonzero(pastNeighbors) == 1: #One bright neighbor
relX, relY, relZ = np.nonzero(pastNeighbors)
relX = relX - 1
relY = relY - 1
relZ = relZ - 1
out[i,j,k] = out[i + relX, j + relY, k + relZ]
else: #Mutliple bright neighbors
relX, relY, relZ = np.nonzero(pastNeighbors)
relX = relX - 1
relY = relY - 1
relZ = relZ - 1
equiv = out[i + relX, j + relY, k + relZ]
out[i, j, k] = np.min(equiv) #Assign one of the
#multiple labels
for i in xrange(0,equiv.size):
equivList = np.append(equivList, [[np.min(equiv),equiv[i]]], axis = 0)
The input is a spectrum with colorful (sorry) vertical lines on a black background. Given the approximate x coordinate of that band (as marked by X), I want to find the width of that band.
I am unfamiliar with image processing. Please direct me to the correct method of image processing and a Python image processing package that can do the same.
I am thinking PIL, OpenCV gave me an impression of being overkill for this particular application.
What if I want to make this an expert system that can classify them in the future?
I'll give a complete minimal working example (as suggested by sega_sai). I don't have access to your original image, but you'll see it doesn't really matter! The peak distributions found by the code below are:
Mean values at: 26.2840960523 80.8255092125
import Image
from scipy import *
from scipy.optimize import leastsq
# Load the picture with PIL, process if needed
pic = asarray(Image.open("band2.png"))
# Average the pixel values along vertical axis
pic_avg = pic.mean(axis=2)
projection = pic_avg.sum(axis=0)
# Set the min value to zero for a nice fit
projection /= projection.mean()
projection -= projection.min()
# Fit function, two gaussians, adjust as needed
def fitfunc(p,x):
return p[0]*exp(-(x-p[1])**2/(2.0*p[2]**2)) + \
p[3]*exp(-(x-p[4])**2/(2.0*p[5]**2))
errfunc = lambda p, x, y: fitfunc(p,x)-y
# Use scipy to fit, p0 is inital guess
p0 = array([0,20,1,0,75,10])
X = xrange(len(projection))
p1, success = leastsq(errfunc, p0, args=(X,projection))
Y = fitfunc(p1,X)
# Output the result
print "Mean values at: ", p1[1], p1[4]
# Plot the result
from pylab import *
subplot(211)
imshow(pic)
subplot(223)
plot(projection)
subplot(224)
plot(X,Y,'r',lw=5)
show()
Below is a simple thresholding method to find the lines and their width, it should work quite reliably for any number of lines. The yellow and black image below was processed using this script, the red/black plot illustrates the found lines using parameters of threshold = 0.3, min_line_width = 5)
The script averages the rows of an image, and then determines the basic start and end positions of each line based on a threshold (which you can set between 0 and 1), and a minimum line width (in pixels). By using thresholding and minimum line width you can easily filter your input images to get the lines out of them. The first function find_lines returns all the lines in an image as a list of tuples containing the start, end, center, and width of each line. The second function find_closest_band_width is called with the specified x_position, and returns the width of the closest line to this position (assuming you want distance to centre for each line). As the lines are saturated (255 cut-off per channel), their cross-sections are not far from a uniform distribution, so I don't believe trying to fit any kind of distribution is really going to help too much, just unnecessarily complicates.
import Image, ImageStat
def find_lines(image_file, threshold, min_line_width):
im = Image.open(image_file)
width, height = im.size
hist = []
lines = []
start = end = 0
for x in xrange(width):
column = im.crop((x, 0, x + 1, height))
stat = ImageStat.Stat(column)
## normalises by 2 * 255 as in your example the colour is yellow
## if your images start using white lines change this to 3 * 255
hist.append(sum(stat.sum) / (height * 2 * 255))
for index, value in enumerate(hist):
if value > threshold and end >= start:
start = index
if value < threshold and end < start:
if index - start < min_line_width:
start = 0
else:
end = index
center = start + (end - start) / 2.0
width = end - start
lines.append((start, end, center, width))
return lines
def find_closest_band_width(x_position, lines):
distances = [((value[2] - x_position) ** 2) for value in lines]
index = distances.index(min(distances))
return lines[index][3]
## set your threshold, and min_line_width for finding lines
lines = find_lines("8IxWA_sample.png", 0.7, 4)
## sets x_position to 59th pixel
print 'width of nearest line:', find_closest_band_width(59, lines)
I don't think that you need anything fancy for you particular task.
I would just use PIL + scipy. That should be enough.
Because you essentially need to take your image, make a 1D-projection of it
and then fit a Gaussian or something like that to it. The information about the approximate location of the band should be used a first guess for the fitter.