I am doing pupil detection for my school project. It's my first time working with OpenCV and Python, using Python version 3.4.2 and OpenCV 3.1.0.
I am using the Raspberry Pi NoIR camera, and I am getting good images.
But i can't detect a pupil nicely (because of glint, lashes and shadows.
I refer to some code on the web and the following is part of that code.
...
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
image = frame.array
cv2.imshow("image", image)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
retval, thresholded = cv2.threshold(gray, 80, 255, 0)
cv2.imshow("threshold", thresholded)
closed = cv2.erode(cv2.dilate(thresholded, kernel, iterations=1), kernel, iterations=1)
#closed = cv2.morphologyEx(close, cv2.MORPH_CLOSE, kernel)
cv2.imshow("closed", closed)
thresholded, contours, hierarchy = cv2.findContours(closed, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
drawing = np.copy(image)
cv2.drawContours(drawing, contours, -1, (255, 0, 0), 2)
for contour in contours:
area = cv2.contourArea(contour)
bounding_box = cv2.boundingRect(contour)
extend = area / (bounding_box[2] * bounding_box[3])
# reject the contours with big extend
if extend > 0.8:
continue
# calculate countour center and draw a dot there
m = cv2.moments(contour)
if m['m00'] != 0:
center = (int(m['m10'] / m['m00']), int(m['m01'] / m['m00']))
cv2.circle(drawing, center, 3, (0, 255, 0), -1)
# fit an ellipse around the contour and draw it into the image
try:
ellipse = cv2.fitEllipse(contour)
cv2.ellipse(drawing, box=ellipse, color=(0, 255, 0))
except:
pass
# show the frame
cv2.imshow("Drawing", drawing)
...
Input image:
Output image:
How can I remove the parts of the image that are not related to the pupil, as shown above?
In addition to answers, any hints are also welcome.
There are several things you can do. How well they work depends on how much variation there is in the images you want to apply the algorithm on. You could make several assumptions and then discard all the candidates that do not meet them.
remove small detections
At first I would consider removing candidates that are too small by adding this line at the beginning of your loop:
if area < 100:
continue
The threshold was chosen randomly and worked well for this particular image. It removed almost all the false detection. Only the biggest one remains. But you have to check it against your other images and adapt it to your needs.
remove detections that are not round
Another assumption you can make is that pupils are usually round and you can remove every detection that is not 'round' enough. One simple measure of roundness is to look at the ratio of circumference to area.
circumference = cv2.arcLength(contour,True)
circularity = circumference ** 2 / (4*math.pi*area)
The circularity is about 2.72 for the shadow in the right side and 1.31 for the pupil.
improving roundness
You notice, that the contour of your pupil is not perfectly round because of reflections. You can improve this by computing the convex hull of the contours.
contour = cv2.convexHull(contour)
If you do that before computing the area and circumference you get circularity values of 1.01 and 1.37. (A perfect circle has a circularity of 1) This means the defect from the reflection was almost perfectly repaired. This might not be necessary in this case but could be useful in cases with more reflections.
Some years ago I realized a very similar project. To everything added above I can suggest one little trick.
As you can see you have two reflections from any light source. One on the surface of the eye and second reflection on the surface of the glass.
If you will remove the glass (if it's possible) you will have a very bright reflection almost in the center of pupil. In that case you can ignore all objects without this bright reflection. This also will help you to find a position of eye in the space near camera
Related
I'm writing a program that takes an image that has a grid of 4 by 4 letters somewhere in that image.
E.g.
1
I want to read these letters into my program and for that I'm using pytesseract for the OCR.
Before feeding the image to pytesseract I do some preprocessing with openCV to increase the odds of pytesseract working correctly.
This is the code I use for this:
import cv2
img = cv2.imread('my_image.png')
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_pre_processed = cv2.threshold(img_gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
And these are some sample outputs of img_pre_processed:
2 3 4
Since the letters in the grid are spaced apart pytesseract has a difficult time to read them when I give the entire image as input. So it would be helpful if I knew the coordinates of every letter, then I could edit the image in such a way that pytesseract can always recognise them.
I started to try and solve this problem on my own and the solution I'm coming up with might work but it's getting rather complicated. So I'm wondering if there is a better way to do it.
At the moment I'm using the cv2.findContours() function to get all the contours of the objects in the image. For every contour I calculate the center coordinates and the area of the box you would be able to draw around it. I then sort these by area to get the largest contours. Now here it starts to get more and more complicated. I can't just say take the biggest 16 contours, because there might be unwanted objects in the picture that have a bigger area than the 16 letters that I want. Also some letters like O, P, Q,... have 2 contours and their inner contour might even be bigger than another letters outer contour like the letter I for example.
E.g. This is an image with the 18 biggest contours marked with a green box. 5
So to continue with my way of attacking the problem I would have to write an algorithm that finds the contours that are most likely part of the grid while ignoring the contours that are unwanted and also the inner contours of letters that have 2 contours.
While this is possible, I'm wondering if there is be a better way of doing this.
Somebody told me that if you can filter the image in such a way that everything gets more blurry so that all the letters become blobs. That it might be possible to do a pattern detection with 4x4 grid of blobs. But I don't know how to do that or if that's possible.
So if somebody knows a better way to tackle this problem or if you know how to execute the plan of attack I mentioned earlier that would be most helpfull.
Thanks in advance!
You can simply filter the bounding rectangles by width and height. As this is a rule based approach, it may need more example images to fine tune the filter rules.
import cv2
# get bounding rectangles of contours
img = cv2.imread('img.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
bbox = [cv2.boundingRect(c) for c in contours]
# filter rectangles by width and height
for x, y, w, h in bbox:
if (4 < w < 200) and (30 < h < 200):
cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
cv2.imshow("img", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
I am trying to separate a cow from a depth image.
When I use contours it separates most of it but fails to separate the fence when the cow is leaning on it. (Note: it is ok that the head is being removed from the cow, the application I am using it on works better if the head is removed)
Here is the code I use to detect and remove contours. My idea is to remove them by size. It works when the cow is not touching the fence but in this case, doesn't work.
# Remove stuructures connected to the image border------------------------------------------------
# find contours in the image and initialize the mask that will be
cnts = cv2.findContours(BW3.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
mask = np.ones( BW3.shape[:2], dtype="uint8") * 255
# loop over the contours
for c in cnts:
if cv2.contourArea(c) > 250000:
# if the contour is bad, draw it on the mask
cv2.drawContours(mask, [c], -1, 0, -1)
BW3 = cv2.bitwise_and( BW3, BW3, mask=mask)
if cv2.contourArea(c) < 10000:
cv2.drawContours(mask, [c], -1, 0, -1)
BW3 = cv2.bitwise_and( BW3, BW3, mask=mask)
cv2.imshow('H_Black and white', BW3)
cv2.waitKey()
Is there any way to remove the fencing around the cow when it is touching? I have tried using HoughLinesP() with no luck, I am new to OpenCV so I could be going about it the wrong way. Another potential solution would be to crop the image, but the camera is in a slightly different location each time so cropping would have to be adjusted for each camera variation. Any advice would be appreciated.
Thank you
EDIT:
The purpose of separating the cow from the background is to use volumetric estimation to determine the weight of the animal. If effectively implemented this will be a cheaper solution than a standard scale. This is for a research project (The project will be open-sourced, not monetized).
The original input depth image is cropped before any other code is run (All images reflect this except for the first depth image in this post) To get the contours I change the depth picture to HSV. Then take the Hue image and change that to black and white before running cv2.findCountours
Here is a reconstruction of the depth values from the jet colormap (inverted values so it's easier to read visually):
I have a population of images of maps (4tb worth - 670K images total). The maps are made up of overlain images of different maps. The boundaries of each map signified by a red, blue, or grey boundry. The solution I'm trying to come up with a solution that answers these two questions:
Does the map contain red lines that are not on top of grey lines.
If the answer to 1 is yes, are there portions of the red square located inside of blue squares?
I aim to build either a CNN or a script that will answer these questions by:
Recognizing images with intersecting red/grey lines.
Recognizing images with red lines inside the blue lines.
My first step (and please feel free to criticize my approach) is to take a map and mask the blue lines therein.
#Inpaint - or Photoshop "Content Aware Fill"
im = cv2.imread('Sample2.png')
# Make mask of blue pixels - True where blue, False elsewhere
mask = np.all(im == (255, 0, 0), axis=-1)
# Inpaint white areas using Navier Strokes
result = cv2.inpaint(im,np.uint8(mask)*255,3,cv2.INPAINT_NS)
cv2.imwrite('result-NAVIER-STOKES.png',result)
This provides this output.
Next, I want to create an image containing a white background with the squares as black which I then hope to use to train a NN or write a script to identify if the squares intersect. In order to do that, I will need to further pre-process the image to look as close to this as possible (in cases where no grey square exists it would be only one square):
I've played around with masking to remove the noise and only isolate the grey and red rectangles but it seems like what I've written can only produce an output that isolates the red rectangle only:
import cv2
import numpy as np
import os
filename = 'output.PNG'
im = cv2.imread('output.PNG')
output_file = (os.path.splitext(filename)[0])
hsv = cv2.cvtColor(im, cv2.COLOR_BGR2HSV)
lower = (0,240,160)
upper = (120,255,255)
thresh = cv2.inRange(hsv, lower, upper)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (9,9))
clean = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15,15))
clean = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
# get external contours
contours = cv2.findContours(clean, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
result1 = im.copy()
result2 = im.copy()
for c in contours:
cv2.drawContours(result1,[c],0,(0,0,0),2)
# get rotated rectangle from contour
rot_rect = cv2.minAreaRect(c)
box = cv2.boxPoints(rot_rect)
box = np.int0(box)
# draw rotated rectangle on copy of img
cv2.drawContours(result2,[box],0,(0,0,0),2)
How can I isolate the grey square as well as the red square? I know it's a challenge because the map in the background is similar in coloration to the grey line. Is there a way to isolate the grey square an perhaps change its color to something other than grey to apply an additional mask?
Also if there's a better way to approach this problem, would love to hear thoughts and suggestions.
I recently decided to teach myself computer vision, so I only know the fundamentals of image processing. I have decided to attempt a hopefully simple exercise: detect the tiles in pictures similar to this one:
I am aware I can use the Hough transform to find the circles in each tile, and I'll use that to make the algorithm more robust later on, but at the moment I am struggling to find the contours of the tiles.
My first attempt has been to resize the image, blur it, make it grayscale and apply canny. Then, tiles are those contours which have four sides and a certain aspect ratio. This works fine for most tiles in most pictures, but in this one it seems to fail on the top one because of the white speaker in the background (it does not think it has four sides):
gray = cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY)
blurred_gray = cv2.GaussianBlur(gray, (5, 5), 0)
edged = cv2.Canny(blurred_gray, 75, 200)
cv2.imshow("Edged", edged)
cv2.waitKey(0)
I then discovered image pyramids and decided to give it a go by blurring my image and then substracting from it a blurred interpolated version from a higher level of the pyramid, and while the edges are more defined now, the rugosities of the background have been highlighted as well:
blurred_gray = cv2.GaussianBlur(
cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY), (5, 5), 0)
blurred_gray_recons = cv2.pyrUp(cv2.pyrDown(blurred_gray))
edged = blurred_gray - blurred_gray_recons
cv2.imshow('Edged', edged)
How could I reliably perform edge detection in these cases? Please note that the background won't always be of the same color.
I'm working with a little program which will use OpenCV + python's background subtraction to count cars. I'm fine with background subtraction, I already have a background image. But when I use cv2.findContours(fgmask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE), I get way too many contours. While I can do some basic filtering by checking contour area (cv2.contourArea(contour)), as shown in http://www.pyimagesearch.com/2015/05/25/basic-motion-detection-and-tracking-with-python-and-opencv/, not all cars are detected.
I also looked at cv2.groupRectangles(rectList, minNum[, eps]), but I can't seem to create a vector of rectangles (By the way, that's in http://docs.opencv.org/2.4/modules/objdetect/doc/cascade_classification.html, last function).
My code to find contours/draw rectangles:
contours, im2 = cv2.findContours(fgmask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
if MAXAREA >= cv2.contourArea(cnt) >= MINAREA:
rect = cv2.minAreaRect(cnt)
points = cv2.cv.BoxPoints(rect)
cv2.rectangle(img, (int(points[1][0]), int(points[1][1])), (int(points[3][0]), int(points[3][1])), (0, 255, 0), 2)
(MINAREA and MAXAREA are the maximum and minimum areas for the contour to be drawn)
My Question: How can I either group rectangles or use some criteria to draw the correct rectangles (and keep cars from not being recognized)?
Any help is highly appreciated.
As i have understood the question.You need to identify the rectangular shape(the car in the actual image) from the image.irrespective of the size.The minAreaRect function will fit a minimum area rectangle over all the contours given to it.This way u cannot identify the rectangle shape.But u can take this minimum rectangle as a template for each contours and match it with that particular contour.based on the matching score you can decide whether it is rectangular shape or not.One more approach you can try is How to detect simple geometric shapes using OpenCV.