Related
I am in the initial stages of writing a Rubik's cube solver and am stuck at the following challenge:
Using the following image-processing code gives me the following image:
import cv2 as cv
import glob
import numpy as np
for img in glob.glob("captured_images/*.jpg"):
image = cv.imread(img)
copy = image.copy()
grey = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
decrease_noise = cv.fastNlMeansDenoising(grey, 10, 15, 7, 21)
blurred = cv.GaussianBlur(decrease_noise, (3, 3), 0)
canny = cv.Canny(blurred, 20, 40)
thresh = cv.threshold(canny, 0, 255, cv.THRESH_OTSU + cv.THRESH_BINARY)[1]
contours = cv.findContours(thresh, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for c in contours:
# obtain the bounding rectangle coordinates for each square
x, y, w, h = cv.boundingRect(c)
# With the bounding rectangle coordinates we draw the green bounding boxes
cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)
cv.imshow('copy', copy)
cv.waitKey(0)
cv.destroyAllWindows()
There are numerous bound rectangles highlighted. Trying to filter out only the squares using this code:
contour_list = []
for contour in contours:
approx = cv.approxPolyDP(contour, 0.01 * cv.arcLength(contour, True), True)
area = cv.contourArea(contour)
if len(approx) == 4:
(x, y, w, h) = cv.boundingRect(approx)
if (float(w)/h) == 1:
cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)
contour_list.append(contour)
doesn't work as the squares aren't precise enough to fit the definition of "all sides of square are equal".
I though retaking the images against a white background might help to more easily find the relevant squares, however modifying the original image to a cube with a white background and using the original code causes only the larger cube to be recognised as a square:
My question is three-fold:
1a) How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:
There must be four corners
All four lines must be roughly the same length
All four corners must be roughly 90 degrees
1b) In the second image with the white background, how can I select everything outside the bound rectangle and convert that white background to black, which helps greatly in correctly detecting the appropriate squares?
1c) In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?
Any help in gaining some clearer understanding is much appreciated! :)
How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:
Your code only accepts contours that are exactly square. You need to have a "squaredness" factor and then determine some acceptable threshold.
The "squaredness" factor is h/w if w > h else w/h. The closer that value to one, the more square the rectangle is. Then you can accept only rectangles with a factor of .9 or higher (or whatever works best).
In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?
The contour finding algorithm that OpenCV uses is actually:
Suzuki, S. and Abe, K., Topological Structural Analysis of Digitized Binary Images by Border Following. CVGIP 30 1, pp 32-46 (1985)
In your case, the algorithm might just have picked up the contours just fine, but you have set the RETR_EXTERNAL flag, which will cause OpenCV to only report the outermost contours. Try changing it to RETR_LIST.
Find the OpenCV docs with regards to contour finding here: https://docs.opencv.org/master/d9/d8b/tutorial_py_contours_hierarchy.html
I loaded the image and tried to draw a red point in an image
img=mpimg.imread('./images/im00001.jpg')
red = [0,0,255]
# Change one pixel
img[ 0.,-26.10911452,0. ]=red
imgplot = plt.imshow(img)
but the following error occurred
ValueError: assignment destination is read-only
What you are doing actually changes your image.
To draw points on the image as it is being shown, you can show the image in a matplotlib figure and then plot points on it. You can use pyplot.plot() function to plot points, or pyplot.scatter() function to plot an array of points.
image = mpimg.imread("road.jpg")
pts = np.array([[330,620],[950,620],[692,450],[587,450]])
plt.imshow(image)
plt.plot(640, 570, "og", markersize=10) # og:shorthand for green circle
plt.scatter(pts[:, 0], pts[:, 1], marker="x", color="red", s=200)
plt.show()
You're on the right track. You can change a pixel's property using Numpy splicing
img[x,y] = [B,G,R]
So for example, to change a pixel at (50,50) to red, you can do
img[50,50] = [0,0,255]
Here we change a single pixel to red (it's pretty tiny)
import cv2
import numpy as np
width = 100
height = 100
# Make empty black image of size (100,100)
img = np.zeros((height, width, 3), np.uint8)
red = [0,0,255]
# Change pixel (50,50) to red
img[50,50] = red
cv2.imshow('img', img)
cv2.waitKey(0)
An alternative method is to use cv2.circle() to draw your point inplace.
The function header is
cv2.circle(image, (x, y), radius, (B,G,R), thickness)
Using this, we obtain the same result
cv2.circle(img, (50,50), 1, red, -1)
mpimg indicates that you are using matplotlib to read the image.
Here are few points to remember to work with images using matplotlib:
matplotlib stores image data into Numpy arrays. So, type(img) will return <class 'numpy.ndarray'>. (Ref 1)
The shape of the ndarray represents the height, width and number of bands of the image.
Each inner list represents a pixel. For RGB image inner list length is 3. For RGBA Image inner list length is 4. Each value of the list stores floating point data between 0.0 to 1.0. Each value represents value of R(Red), G(Green), B(Blue) and A(Alpha / transparency) of the pixel.
For RGB image, to set a pixel to red color the pixel should be assigned: [1, 0, 0]
For RGBA image, to set a pixel to red color the pixel should be assigned: [1, 0, 0, 1]
In matplotlib, the Figure's size is fixed, and the contents are stretched/squeezed/interpolated to fit the figure. So, after saving the image the resolution may change. (Ref 2)
According to these points, I have edited a RGBA image (png format) by putting a red dot in center of it.
Original image:
Edited image:
code.py:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
# dpi for the saved figure: https://stackoverflow.com/a/34769840/3129414
dpi = 80
# Set red pixel value for RGB image
red = [1, 0, 0]
img = mpimg.imread("minion.png")
height, width, bands = img.shape
# Update red pixel value for RGBA image
if bands == 4:
red = [1, 0, 0, 1]
# Update figure size based on image size
figsize = width / float(dpi), height / float(dpi)
# Create a figure of the right size with one axes that takes up the full figure
figure = plt.figure(figsize=figsize)
axes = figure.add_axes([0, 0, 1, 1])
# Hide spines, ticks, etc.
axes.axis('off')
# Draw a red dot at pixel (62,62) to (66, 66)
for i in range(62, 67):
for j in range(62, 67):
img[i][j] = red
# Draw the image
axes.imshow(img, interpolation='nearest')
figure.savefig("test.png", dpi=dpi, transparent=True)
References:
Matplotlib official
tutorial
Stackoverflow answer on saving image in same resolution as original
image
I have this image of an eye where I want to get the center of the pupil:
Original Image
I applied adaptive threshold as well as laplacian to the image using this code:
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('C:\Users\User\Documents\module4\input\left.jpg',0)
image = cv2.medianBlur(img,5)
th = cv2.adaptiveThreshold(image,255,cv2.ADAPTIVE_THRESH_MEAN_C,
cv2.THRESH_BINARY,11,2)
laplacian = cv2.Laplacian(th,cv2.CV_64F)
cv2.imshow('output', laplacian)
cv2.imwrite('C:\Users\User\Documents\module4\output\output.jpg', laplacian)
cv2.waitKey(0)
cv2.destroyAllWindows
and the resulting image looks like this: Resulting image by applying adaptive threshold
I want to draw a circle around the smaller inner circle and get its center. I've tried using contours and circular hough transform but it does not correctly detect any circles in the image.
Here is my code for Circular Hough Transform:
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('C:\Users\User\Documents\module4\output\output.jpg',0)
circles = cv2.HoughCircles(img,cv2.HOUGH_GRADIENT,1,20,param1=50,param2=30,minRadius=0,maxRadius=0)
circles = np.uint16(np.around(circles))
for i in circles[0,:]:
# draw the outer circle
cv2.circle(img,(i[0],i[1]),i[2],(255,255,0),2)
# draw the center of the circle
cv2.circle(img,(i[0],i[1]),2,(255,0,255),3)
cv2.imshow('detected circles',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
And here is the code for applying contour:
import cv2
import numpy as np
img = cv2.imread('C:\Users\User\Documents\module4\output\output.jpg',0)
_, contours,hierarchy = cv2.findContours(img, 1, 2)
cnt = contours[0]
(x,y),radius = cv2.minEnclosingCircle(cnt)
center = (int(x),int(y))
radius = int(radius)
img = cv2.circle(img,center,radius,(0,255,255),2)
cv2.imshow('contour', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
The resulting image of this code exactly looks like the image wherein I applied adaptive threshold. I would really appreciate it if anyone can help me solve my problem. I've been stuck with this for a while now. Also, if any of you guys can suggest a better way to detect the center of the pupil besides this method, I would also really appreciate it.
try to apply edge detection instead of shareholding after filtering of original image and then apply hough circle
My thought would be to use the Hough transform like you're doing. But another method might be template matching like this. This assumes you know the approximate radius of the pupil in the image, you can try to build a template.
import skimage
import numpy as np
import matplotlib.pyplot as plt
img = skimage.io.imread('Wjioe.jpg')
#just use grayscale, but you could make separate template for each r,g,b channel
img = np.mean(img, axis=2)
(M,N) = img.shape
mm = M-20
nn = N-20
template = np.zeros([mm,nn])
## Create template ##
#darkest inner circle (pupil)
(rr,cc) = skimage.draw.circle(mm/2,nn/2,4.5, shape=template.shape)
template[rr,cc]=-2
#iris (circle surrounding pupil)
(rr,cc) = skimage.draw.circle(mm/2,nn/2,8, shape=template.shape)
template[rr,cc] = -1
#Optional - pupil reflective spot (if centered)
(rr,cc) = skimage.draw.circle(mm/2,nn/2,1.5, shape=template.shape)
template[rr,cc] = 1
plt.imshow(template)
normccf = skimage.feature.match_template(img, template,pad_input=True)
#center pixel
(i,j) = np.unravel_index( np.argmax(normccf), normccf.shape)
plt.imshow(img)
plt.plot(j,i,'r*')
You're defining a 3 channel color for a gray-scale image. Based on my test it will only read the first value in that tuple. Because the first value in your other colors (in the middle code) starts with 255, it draws a full white circle and because the first value in your last color (in your last code) starts with 0, it draws a full black circle which you can't see.
Just change your color values to a 1 channel color (an int between 0 and 255) and you'll be fine.
I've read a lot about the Circular Hough transform on Stack Overflow, but I seem to be missing something. I wrote a program that is supposed to detect the circles of a "Bull's Eye" target. However, even after playing with the parameters, the algorithm is quite bad - it ignores most of the circles and one time it finds a circle but seems to "wander off". I've even tried applying an "Unsharp Mask" to no avail. I have added my code, the image I started with and the output. I hope someone can point me at the right direction.
import cv2
import cv2.cv as cv
import numpy as np
import math
# Load Image
img = cv2.imread('circles1.png',0)
# Apply Unsharp Mask
tmp = cv2.medianBlur(img,5)
img = cv2.addWeighted(img,1.5,tmp,-0.5,0)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
# Hough Transform
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,1,5,
param1=100,param2=100,minRadius=0,maxRadius=0)
circles = np.uint16(np.around(circles))
# Go over circles, eliminating the ones that are not cocentric enough
height, width = img.shape
center = (width/2,height/2)
for i in circles[0,:]:
# draw the outer circle
if math.sqrt((center[0]-i[0])**2 + (center[1]-i[1])**2) < 15:
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),1)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
Quick explanation: I load the image, apply Unsharp Mask, use the Hough Transfrom to detect circles, then draw the circles that are close to the center (I found that the other circles are false circles).
I tried playing with the parameters, and this is the best I got. I feel like this is a simple enough problem which has me buffled. I appriciate any help.
My input image:
My output image:
As I mentioned in my comment, you'll need to run successive iterations of cv2.HoughCircles for different range of radii to ensure that you get all of the circles. With the way the Circular Hough Transform works, specifying a minimum and maximum radius that has quite a large range will be inaccurate and will also be slow. They don't tell you this in the documentation, but for the Circular Hough Transform to work successfully, the following two things need to be valid:
maxRadius < 3*minRadius
maxRadius - minRadius < 100
With the above, blindly making the minimum radius very small and the maximum radius very large won't give you great results. Therefore, what you could do is start at... say...radius=1, then iterate up to radius=300 in steps of 20. Between each chunk of 20, run cv2.HoughCircles and update your image with these contours.
Doing this requires very little modification to your code. BTW, I removed the unsharp masking it because I was getting poor results with it. I also changed a couple of parameters in cv2.HoughCircles slightly to get this to work as best as possible given your situation:
import cv2
import cv2.cv as cv
import numpy as np
import math
# Load Image
img = cv2.imread('circles1.png',0)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
# Specify different radii
radii = np.arange(0,310,10)
# For each pair of radii...
for idx in range(len(radii)-1):
# Get the minimum and maximum radius
# Note you need to add 1 to each minimum
# as the maximum of the previous pair covers this new minimum
minRadius = radii[idx]+1
maxRadius = radii[idx+1]
# Hough Transform - Change here
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,1,5,
param1=25,param2=75,minRadius=minRadius,maxRadius=maxRadius)
# Skip if no circles are detected - Change here
if circles is None:
continue
circles = np.uint16(np.around(circles))
# Go over circles, eliminating the ones that are not cocentric enough
height, width = img.shape
center = (width/2,height/2)
for i in circles[0,:]:
# draw the outer circle
if math.sqrt((center[0]-i[0])**2 + (center[1]-i[1])**2) < 15:
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),1)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
I get this figure:
Unfortunately it isn't perfect as it doesn't detect all of the circles. You'll have to play around with the cv2.HoughCircles function until you get good results.
However, I wouldn't recommend using cv2.HoughCircles here. May I suggest using cv2.findContours instead? This finds all of the contours in the image. In this case, these will be the black circles. However, you need to reverse the image because cv2.findContours assumes non-zero pixels are object pixels, so we can subtract 255 from the image assuming a np.uint8 type:
# Make copy of original image
cimg2 = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
# Find contours
contours,_ = cv2.findContours(255 - img, cv2.RETR_LIST, cv.CV_CHAIN_APPROX_NONE)
# Draw all detected contours on image in green with a thickness of 1 pixel
cv2.drawContours(cimg2, contours, -1, color=(0,255,0), thickness=1)
# Show the image
cv2.imshow('detected circles', cimg2)
cv2.waitKey(0)
cv2.destroyAllWindows()
This is what I get:
I followed this tutorial from official documentation. I run their code:
import numpy as np
import cv2
im = cv2.imread('test.jpg')
imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(imgray,127,255,0)
contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(img, contours, -1, (0,255,0), 3)
That is ok: no errors, but nothing is displayed.I want to display the result they got as they showed it on the picture:
How can I display the result of the countours like that (just the left result or the right one) ?
I know I must use cv2.imshow(something) but how in this specific case ?
First off, that example only shows you how to draw contours with the simple approximation. Bear in mind that even if you draw the contours with the simple approximation, it will be visualized as having a blue contour drawn completely around the rectangle as seen in the left image. You will not be able to get the right image by simply drawing the contours onto the image. In addition, you want to compare two sets of contours - the simplified version on the right with its full representation on the left. Specifically, you need to replace the cv2.CHAIN_APPROX_SIMPLE flag with cv2.CHAIN_APPROX_NONE to get the full representation. Take a look at the OpenCV doc on findContours for more details: http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#findcontours
In addition, even though you draw the contours onto the image, it doesn't display the results. You'll need to call cv2.imshow for that. However, drawing the contours themselves will not show you the difference between the full and simplified version. The tutorial mentions that you need to draw circles at each contour point so we shouldn't use cv2.drawContours for this task. What you should do is extract out the contour points and draw circles at each point.
As such, create two images like so:
# Your code
import numpy as np
import cv2
im = cv2.imread('test.jpg')
imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(imgray,127,255,0)
## Step #1 - Detect contours using both methods on the same image
contours1, _ = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
contours2, _ = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
### Step #2 - Reshape to 2D matrices
contours1 = contours1[0].reshape(-1,2)
contours2 = contours2[0].reshape(-1,2)
### Step #3 - Draw the points as individual circles in the image
img1 = im.copy()
img2 = im.copy()
for (x, y) in contours1:
cv2.circle(img1, (x, y), 1, (255, 0, 0), 3)
for (x, y) in contours2:
cv2.circle(img2, (x, y), 1, (255, 0, 0), 3)
Take note that the above code is for OpenCV 2. For OpenCV 3, there is an additional output to cv2.findContours that is the first output which you can ignore in this case:
_, contours1, _ = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
_, contours2, _ = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
Now let's walk through the code slowly. The first part of the code is what you provided. Now we move onto what is new.
Step #1 - Detect contours using both methods
Using the thresholded image, we detect contours using both the full and simple approximations. This gets stored in two lists, contours1 and contours2.
Step #2 - Reshape to 2D matrices
The contours themselves get stored as a list of NumPy arrays. For the simple image provided, there should only be one contour detected, so extract out the first element of the list, then use numpy.reshape to reshape the 3D matrices into their 2D forms where each row is a (x, y) point.
Step #3 - Draw the points as individual circles in the image
The next step would be to take each (x, y) point from each set of contours and draw them on the image. We make two copies of the original image in colour form, then we use cv2.circle and iterate through each pair of (x, y) points for both sets of contours and populate two different images - one for each set of contours.
Now, to get the figure you see above, there are two ways you can do this:
Create an image that stores both of these results together side by side, then show this combined image.
Use matplotlib, combined with subplot and imshow so that you can display two images in one window.
I'll show you how to do it using both methods:
Method #1
Simply stack the two images side by side, then show the image after:
out = np.hstack([img1, img2])
# Now show the image
cv2.imshow('Output', out)
cv2.waitKey(0)
cv2.destroyAllWindows()
I stack them horizontally so that they are a combined image, then show this with cv2.imshow.
Method #2
You can use matplotlib:
import matplotlib.pyplot as plt
# Spawn a new figure
plt.figure()
# Show the first image on the left column
plt.subplot(1,2,1)
plt.imshow(img1[:,:,::-1])
# Turn off axis numbering
plt.axis('off')
# Show the second image on the right column
plt.subplot(1,2,2)
plt.imshow(img2[:,:,::-1])
# Turn off the axis numbering
plt.axis('off')
# Show the figure
plt.show()
This should display both images in separate subfigures within an overall figure window. If you take a look at how I'm calling imshow here, you'll see that I am swapping the RGB channels because OpenCV reads in images in BGR format. If you want to display images with matplotlib, you'll need to reverse the channels as the images are in RGB format (as they should be).
To address your question in your comments, you would take which contour structure you want (contours1 or contours2) and search the contour points. contours is a list of all possible contours, and within each contour is a 3D matrix that is shaped in a N x 1 x 2 format. N would be the total number of points that represent the contour. I'm going to remove the singleton second dimension so we can get this to be a N x 2 matrix. Also, let's use the full representation of the contours for now:
points = contours1[0].reshape(-1,2)
I am going to assume that your image only has one object, hence my indexing into contours1 with index 0. I unravel the matrix so that it becomes a single row vector, then reshape the matrix so that it becomes N x 2. Next, we can find the minimum point by:
min_x = np.argmin(points[:,0])
min_point = points[min_x,:]
np.argmin finds the location of the smallest value in an array that you supply. In this case, we want to operate along the x coordinate, or the columns. Once we find this location, we simply index into our 2D contour point array and extract out the contour point.
You should add cv2.imshow("Title", img) at the end of your code. It should look like this:
import numpy as np
import cv2
im = cv2.imread('test.jpg')
imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(imgray,127,255,0)
contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(im, contours, -1, (0,255,0), 3)
cv2.imshow("title", im)
cv2.waitKey()
Add these 2 lines at the end:
cv2.imshow("title", im)
cv2.waitKey()
Also, be aware that you have img instead of im in your last line.