I am trying to extract some plot lines from the image below. As you can see one line is quite thin. My idea was to remove the noise around the lines (grid lines and text).
So far I came up with this code to remove the grid:
import numpy as np
import cv2
gray = cv2.imread('test.png')
edges = cv2.Canny(gray,50,150,apertureSize = 3)
lines = cv2.HoughLinesP(image=edges,rho=0.01,theta=np.pi/90, threshold=100,lines=np.array([]), minLineLength=100,maxLineGap=80)
a,b,c = lines.shape
for i in range(a):
cv2.line(gray, (lines[i][0][0], lines[i][0][1]), (lines[i][0][2], lines[i][0][3]), (255, 255, 255), 3, cv2.LINE_AA)
cv2.imwrite('result.png',gray)
After that i get the following result:
As you can see, there is still some noise around the plots and they are a bit cut off (doesn't have to be perfectly). Has anyone a better solution or some tips how i can improve this? Maybe remove the words first? Maybe detect the lines directly instead of removing the grid etc.?
You can segment both plots using the HSV color space and looking for the blue and orange color. This results on a pretty clean binary mask. Let's check out the code:
# Imports:
import numpy as np
import cv2
# Set image path
path = "D://opencvImages//"
fileName = "graphs.png"
# Reading an image in default mode:
inputImage = readImage(path + fileName)
# BGR to HSV:
hsvImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2HSV)
The first portion of the script converts the BGR image to the HSV color space. Next you need the color values to apply a simple binary thresholding. The tricky part is to get correct HSV values. For the blue graph, a proper Hue threshold seems to be from [85, 179] while leaving the rest of the channels opened, from [0, 255]. For the orange color, a possible Hue range could be [11, 30]. I create a list with these two thresholds:
# Array with HSV values:
hsvValues = []
# Blue range:
lowThreshold = [85, 0, 0]
highThreshold = [179, 255, 255]
# Into the list:
hsvValues.append((lowThreshold, highThreshold))
# Orange range:
lowThreshold = [11, 0, 0]
highThreshold = [30, 255, 255]
# Into the list:
hsvValues.append((lowThreshold, highThreshold))
Now, let's create the mask. Just iterate over the HSV list and apply the thresholding using the cv2.inRange function:
# Create mask:
for i in range(len(hsvValues)):
# Get current thresholds:
currentLowThres = np.array(hsvValues[i][0])
currentHighThres = np.array(hsvValues[i][2])
# Apply HSV threshold:
hsvMask = cv2.inRange(hsvImage, currentLowThres, currentHighThres)
cv2.imshow("Hsv Mask", hsvMask)
cv2.waitKey(0)
These are the two masks:
Blue plot:
Orange plot:
Now, do you want to create actual line models using this information? That's another problem. I'd be cautious to use Hough's line transform/detection. Although the masks are pretty clean, Hough's line parameter tuning is notoriously capricious (i.e., difficult and non-scalable) if you attempt to run not-so similar images through the algorithm, Additionally, I guess you could be more interested in multiple lines segments per plot instead of one continuous line, so I'd be on the lookout for a more ad-hoc approach.
Related
I am new to deep learning but have succeeded in semantic segmentation of the image I am trying to get the pixel count of each class in the label. As an example in the image I want to get the pixel count of the carpet, or the chandelier or the light stand. How do I go about? Thanks any suggestions will help.
Edit: In what format the regions are returned? Do you have only the final image or the regions are given as contours? If you have them as contours (list of coordinates), you can apply findContourArea directly on that structure.
If you can receive/sample the regions one by one in an image (but do not have the contour), you can sequentially paint each of the colors/classes in a clear image, either convert it to grayscale or directly paint it in grayscale or binary, or binarize with threshold; then numberPixels = len(cv2.findNonZero(bwImage)). cv2.findContour and cv2.contourArea should do the same.
Instead of rendering each class in a separate image, if your program receives only the final segmentation and not per-class contours, you can filter/mask the regions by color ranges on that image. I built that and it seemed to do the job, 14861 pixels for the pink carpet:
import cv2
import numpy as np
# rgb 229, 0, 178 # the purple carpet in RGB (sampled with IrfanView)
# b,g,r = 178, 0, 229 # cv2 uses BGR
class_color = [178, 0, 229]
multiclassImage = cv2.imread("segmented.png")
cv2.imshow("MULTI", multiclassImage)
filteredImage = multiclassImage.copy()
low = np.array(class_color);
mask = cv2.inRange(filteredImage, low, low)
filteredImage[mask == 0] = [0, 0, 0]
filteredImage[mask != 0] = [255,255,255]
cv2.imshow("FILTER", filteredImage)
# numberPixelsFancier = len(cv2.findNonZero(filteredImage[...,0]))
# That also works and returns 14861 - without conversion, taking one color channel
bwImage = cv2.cvtColor(filteredImage, cv2.COLOR_BGR2GRAY)
cv2.imshow("BW", bwImage)
numberPixels = len(cv2.findNonZero(bwImage))
print(numberPixels)
cv2.waitKey(0)
If you don't have the values of the colors given or/and can't control them, you can use numpy.unique(): https://numpy.org/doc/stable/reference/generated/numpy.unique.html and it will return the unique colors, then they could be applied in the algorithm above.
Edit 2: BTW, another way to compute or verify such counts is by calculating histograms. That's with IrfanView on the black-white image:
I'm currently trying to apply an activation heatmap to a photo.
Currently, I have the original photo, as well as a mask of probabilities. I multiply the probabilities by 255 and then round down to the nearest integer. I'm then using cv2.applyColorMap with COLORMAP.JET to apply the colormap to the image with an opacity of 25%.
img_cv2 = cv2.cvtColor(np_img, cv2.COLOR_RGB2BGR)
heatmapshow = np.uint8(np.floor(mask * 255))
colormap = cv2.COLORMAP_JET
heatmapshow = cv2.applyColorMap(np.uint8(heatmapshow - 255), colormap)
heatmap_opacity = 0.25
image_opacity = 1.0 - heatmap_opacity
heatmap_arr = cv2.addWeighted(heatmapshow, heatmap_opacity, img_cv2, image_opacity, 0)
This current code successfully produces a heatmap. However, I'd like to be able to make two changes.
Keep the opacity at 25% For all values above a certain threshold (Likely > 0, but I'd prefer more flexibility), but then when the mask is below that threshold, reduce the opacity to 0% for those cells. In other words, if there is very little activation, I want to preserve the color of the original image.
If possible I'd also like to be able to specify a custom colormap, since the native ones are pretty limited, though I might be able to get away without this if I can do the custom opacity thing.
I read on Stackoverflow that you can possibly trick cv2 into not overlaying any color with NaN values, but also read that only works for floats and not ints, which complicates things since I'm using int8. I'm also concerned that this functionality could change in the future as I don't believe this is intentional design purposefully built into cv2.
Does anyone have a good way of accomplishing these goals? Thanks!
With regard to your second question:
Here is how to create a simple custom two color gradient color map in Python/OpenCV.
Input:
import cv2
import numpy as np
# load image as grayscale
img = cv2.imread('lena_gray.png', cv2.IMREAD_GRAYSCALE)
# convert to 3 equal channels
img = cv2.merge((img, img, img))
# create 1 pixel red image
red = np.full((1, 1, 3), (0,0,255), np.uint8)
# create 1 pixel blue image
blue = np.full((1, 1, 3), (255,0,0), np.uint8)
# append the two images
lut = np.concatenate((red, blue), axis=0)
# resize lut to 256 values
lut = cv2.resize(lut, (1,256), interpolation=cv2.INTER_LINEAR)
# apply lut
result = cv2.LUT(img, lut)
# save result
cv2.imwrite('lena_red_blue_lut_mapped.png', result)
# display result
cv2.imshow('RESULT', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result of colormap applied to image:
With regard to your first question:
You are blending the heat map image with the original image using a constant "opacity" value. You can replace the single opacity value with an image. You just have to do the addWeighted manually as heatmap * opacity_img + original * (1-opacity_img) where your opacity image is float in the range 0 to 1. Then clip and convert back to uint8. If your opacity image is binary, then you can use cv2.bitWiseAnd() in place of multiply.
I am trying to write an algorithm to systematically determine how many different "curves" are in an image. Example Image. I'm specifically interested in the white lines here, so I've used a color threshold to mask the rest of the image and only get the white pixels. These lines represent a path run by a player (wide receivers in the NFL), so I'm interested in the x and y coordinates that the path represents - and each "curve" represents a different path that the player took (or "route"). All curves should start on or behind the blue line.
However, while I can get just the white pixels, I can't figure out how to systematically identify the separate curves. In this example image, there are 8 white curves (or routes) present. I've identified those curves in this image. I tried edge detection, and then using scipy ndimage to get the number of connected components, but because the curves overlap it counts them as connected and only gives me 3 labeled components for this image as opposed to eight. Here's what the edge detection output looks like. Is there a better way to go about this? Here is my sample code.
import cv2
from skimage.morphology import skeletonize
import numpy as np
from scipy import ndimage
#Read in image
image = cv2.imread('example_image.jpeg')
#Color boundary to get white pixels
lower_white = np.array([230, 230, 230])
upper_white = np.array([255, 255, 255])
#mask image for white pixels
mask = cv2.inRange(image, lower_white, upper_white)
c_pixels = cv2.bitwise_and(image, image, mask=mask)
#make pixels from 0 to 1 form to use in skeletonize
c_pixels = c_pixels.clip(0,1)
ske_c = skeletonize(c_pixels[:,:,1]).astype(np.uint8)
#Edge Detection
inputImage =ske_c*255
edges = cv2.Canny(inputImage,100,200,apertureSize = 7)
#Show edges
cv2.imshow('edges', edges)
cv2.waitKey(0)
cv2.destroyAllWindows()
#Find number of components
# smooth the image (to remove small objects); set the threshold
edgesf = ndimage.gaussian_filter(edges, 1)
T = 50 # set threshold by hand to avoid installing `mahotas` or
# `scipy.stsci.image` dependencies that have threshold() functions
# find connected components
labeled, nr_objects = ndimage.label(edgesf > T) # `dna[:,:,0]>T` for red-dot case
print("Number of objects is %d " % nr_objects)
I need to add space between two lines by using OpenCV or PIL.
If the lines vary "sufficiently" in their length, then the following approach might be useful:
Inverse binarize the image using cv2.threshold.
Dilate the image with a horizontal line kernel using cv2.dilate to emphasize the lines.
Sum all pixels row-wise using np.sum, and calculate the absolute differences between the rows using np.diff.
There will be "steps" in the differences between the rows, which resemble the step between the lines. Set up a threshold and find the proper indices using np.where.
Insert white lines in the original image before the found indices using np.insert. In the below example, the index was chosen manually. Work has to be done to properly automatize this: Exclude "steps" to "background", find "steps" between multiple lines.
Here comes a code snippet:
import cv2
from matplotlib import pyplot as plt
import numpy as np
from skimage import io # Only needed for web grabbing images, use cv2.imread for local images
# Read and binarize image
image = cv2.cvtColor(io.imread('https://i.stack.imgur.com/56g7s.jpg'), cv2.COLOR_RGB2GRAY)
_, image_bin = cv2.threshold(image, 128, 255, cv2.THRESH_BINARY_INV)
# Dilate rows by using horizontal line as kernel
kernel = np.ones((1, 51), np.uint8)
image_dil = cv2.dilate(image_bin, kernel)
# Sum pixels row-wise, and calculate absolute differences between the rows
row_sum = np.sum(image_dil / 255, 1, dtype=np.int32)
row_sum_diff = np.abs(np.diff(row_sum))
# Just for visualization: Summed row-wise pixels
plt.plot(row_sum)
plt.show()
# Find "steps" in the differences between the rows
step_thr = 100
step_idx = np.where(row_sum_diff > step_thr)[0]
# Insert n lines before desired index; simple hard-coding here, more work needs to be done for multiple lines
n_lines = 5
image_mod = np.insert(image, step_idx[1] + 1, 255 * np.ones((n_lines, image.shape[1]), np.uint8), axis=0)
# Result visualization
cv2.imshow('image', image)
cv2.imshow('image_dil', image_dil)
cv2.imshow('image_mod', image_mod)
cv2.waitKey(0)
cv2.destroyAllWindows()
The dilated, inverse binarized image:
The visualization of the "steps":
The final output with n = 5 inserted white lines:
As you can see, the result isn't perfect, but that's due to the original image. In the corresponding row, you have parts of the first and second line. So, a proper distinction between these two isn't possible. One might add a very small morphological closing to the output to get rid of these artifacts.
Hope that helps!
I am a complete novice to image processing, and I am guessing this is quite easy to do, but I just don't know the terminology.
Basically, I have a black and white image, I simply want to apply a colored overlay to the image, so that I have got the image overlayed with blue green red and yellow like the images shown below (which actually I can't show because I don't have enough reputation to do so - grrrrrr). Imagine I have a physical image, and a green/red/blue/yellow overlay, which I place on top of the image.
Ideally, I would like to do this using Python PIL but I would be just as happy to do it using ImageMagik, but either way, I need to be able to script the process as I have 100 or so images that I need to carry out the process on.
EDIT: As mentioned by Matt in the comments, this functionality is now available in skimage.color.label2rgb.
In the latest development version, we've also introduced a saturation parameter, which allows you to add overlays to color images.
Here's a code snippet that shows how to use scikit-image to overlay colors on a grey-level image. The idea is to convert both images to the HSV color space, and then to replace the hue and saturation values of the grey-level image with those of the color mask.
from skimage import data, color, io, img_as_float
import numpy as np
import matplotlib.pyplot as plt
alpha = 0.6
img = img_as_float(data.camera())
rows, cols = img.shape
# Construct a colour image to superimpose
color_mask = np.zeros((rows, cols, 3))
color_mask[30:140, 30:140] = [1, 0, 0] # Red block
color_mask[170:270, 40:120] = [0, 1, 0] # Green block
color_mask[200:350, 200:350] = [0, 0, 1] # Blue block
# Construct RGB version of grey-level image
img_color = np.dstack((img, img, img))
# Convert the input image and color mask to Hue Saturation Value (HSV)
# colorspace
img_hsv = color.rgb2hsv(img_color)
color_mask_hsv = color.rgb2hsv(color_mask)
# Replace the hue and saturation of the original image
# with that of the color mask
img_hsv[..., 0] = color_mask_hsv[..., 0]
img_hsv[..., 1] = color_mask_hsv[..., 1] * alpha
img_masked = color.hsv2rgb(img_hsv)
# Display the output
f, (ax0, ax1, ax2) = plt.subplots(1, 3,
subplot_kw={'xticks': [], 'yticks': []})
ax0.imshow(img, cmap=plt.cm.gray)
ax1.imshow(color_mask)
ax2.imshow(img_masked)
plt.show()
Here's the output:
I ended up finding an answer to this using PIL, basically creating a new image with a block colour, and then compositing the original image, with this new image, using a mask that defines a transparent alpha layer. Code below (adapted to convert every image in a folder called data, outputting into a folder called output):
from PIL import Image
import os
dataFiles = os.listdir('data/')
for filename in dataFiles:
#strip off the file extension
name = os.path.splitext(filename)[0]
bw = Image.open('data/%s' %(filename,))
#create the coloured overlays
red = Image.new('RGB',bw.size,(255,0,0))
green = Image.new('RGB',bw.size,(0,255,0))
blue = Image.new('RGB',bw.size,(0,0,255))
yellow = Image.new('RGB',bw.size,(255,255,0))
#create a mask using RGBA to define an alpha channel to make the overlay transparent
mask = Image.new('RGBA',bw.size,(0,0,0,123))
Image.composite(bw,red,mask).convert('RGB').save('output/%sr.bmp' % (name,))
Image.composite(bw,green,mask).convert('RGB').save('output/%sg.bmp' % (name,))
Image.composite(bw,blue,mask).convert('RGB').save('output/%sb.bmp' % (name,))
Image.composite(bw,yellow,mask).convert('RGB').save('output/%sy.bmp' % (name,))
Can't post the output images unfortunately due to lack of rep.
See my gist https://gist.github.com/Puriney/8f89b43d96ddcaf0f560150d2ff8297e
Core function via opencv is described as below.
def mask_color_img(img, mask, color=[0, 255, 255], alpha=0.3):
'''
img: cv2 image
mask: bool or np.where
color: BGR triplet [_, _, _]. Default: [0, 255, 255] is yellow.
alpha: float [0, 1].
Ref: http://www.pyimagesearch.com/2016/03/07/transparent-overlays-with-opencv/
'''
out = img.copy()
img_layer = img.copy()
img_layer[mask] = color
out = cv2.addWeighted(img_layer, alpha, out, 1 - alpha, 0, out)
return(out)
Add colored and transparent overlay on either RGB or gray image can work: