I am doing a dice value recognition hobby project that I want to run on a Raspberry Pi. For now, I am just learning OpenCV as that seems like the hardest thing for me. I have gotten this far, where I have dilated, eroded and canny filtered out the dice. This has given me a hierarchy of contours. The image shows the bounding rectangles for the parent contours:
My question is: how would I proceed to count the pips? Is it better to do some template matching for face values, or should I mathematically test if a pip is in a valid position within the bounding box?
There could be multiple ways to do it:
Use hole filling and then morphological operator to filter circles.
Simpler approach would be using white pixel density (% of white pixels). Five dot would have higher white pixel density.
Use image moments (mathematical property which represents shape and structure of image) to train the neural network for different kinds of dice faces.
Reference:
Morphology
http://blogs.mathworks.com/pick/2008/05/23/detecting-circles-in-an-image/
As Sivam Kalra Said, there are many valid approaches.
I would go with template matching, as it should be robust and relatively easy to implement.
using your green regions in the canny image, copy each found die face from the original grayscale image into a smaller search image. The search image should be slightly larger than a die face, and larger than your 6 pattern images.
optionally normalize the search image
use cvMatchTemplate with each of the 6 possible dice patterns (I recommend the CV_TM_SQDIFF_NORMED algorithm, but test which works best)
find and store the global minimum in the result image for each of the 6 matches
rotate the search image in ~2° steps from 0° to 90°, and repeat the template match for each step
the dice pattern with the lowest minimum over all steps is the correct one.
contour hierechy could be a good and very easy option, but you need a perpendicular vision.
so you can do it with contours but fitting circles with som threshold
(sorry about my apalling english)
Related
I have a task of identifying the number of bacterial colonies on a relatively diverse set of top-down photos of a Petri dish located on a table. The basic process is the following:
detect the Petri dish on the image, crop everything outside of it;
apply binary thresholding which should result in a black background and white colonies or clusters thereof;
use simple blob detector or watershed to identify the colonies, highlight them on the source image and output their count.
Input example 1 Input example 2 Input example 3: an edge case
Problem 1
The table around the Petri dish isn't smooth and contains spots so I usually use Hough transform to detect the dish and remove everything outside of it. The problem is that there are light reflections near the edge of the Petri dish represented as rings with their radius on par with that of the dish edge, as well as other reflections that obscure the view of the colonies and affect the thresholding applied. So I need reliable code for detecting the innermost circle that has roughly the same centre as the outer border of the Petri dish and doesn't contain any further reflections, i.e. cropping at the outer border is sub-optimal.
Cropping attempt
fig. 1. Grab first detected circle with a radius within the range of [int(image.shape[1]/4),int(image.shape[1]/2)] from circle Hough Transform, use a mask and crop to [x+r:x-r,y+r:y-r]
Problem 2.1
The colonies have a colour usually close to the colour of the background (the agar) and in different areas these can overlap (e.g. colony colour in a section A has the same colour as the background in a section B). This renders the method of general thresholding useless. Different photos having different brightness is an issue as well in the context of the binary method and its rigid parameters - for some images a param of (184,255) is useful while on others only a setting as low as (120,255) results in something half-usable.
Gaussian blur, pyrmeanshift, binary threshold
fig 2.1. Gaussian blur (3,3) + pyrmeanshift (6,27) + binary threshold (205,255)
Problem 2.2
The bacterial colonies have round shapes which sometimes form clusters of overlapping circles so the simple blob detector tends to ignore those. The algorithm is supposed to detect the cluster and identify how many colonies (circles) are in it. To tackle this, I've tried Euclidean distance transform coupled with watershed as an alternative to simple blob detector but this needs to be fed a clean image not containing anything other than the colonies themselves, so a robust threshold algorithm is required for removing all the light reflections and eliminating the background's (agar's) gradient. There are also many spots on the Petri dish usually smaller in size than the colonies and not really round - these should be ignored by the detector algorithm. I've heard of adaptive thresholding used for overcoming the problem of a varied background but this tends to convert non-colony small spots on the dish into full-fledged circles which isn't very optimal.
Adaptive threshold - Gaussian method
fig 2.2. Adaptive threshold (Gaussian C)
An attempt at detecting colonies on input example 2
fig 3. A failed attempt at using watershed with distance transform, demonstrating that this algorithm requires a well cleaned-up and properly thresholded input
I'm interested to know whether this task is feasible in the context of a varied collection of photos taken in different lighting conditions as well as different colonies having different sizes and colours. If so, what ways are there to approach this?
I'm currently working on my first assignment in image processing (using OpenCV in Python). My assignment is to calculate a precise score (to tenths of a point) of one to several shooting holes in an image uploaded by a user. One of the requirements is to transform the uploaded shooting target image to be from "birds-eye view" for further processing. For that I have decided that I need to find center coordinates of numbers (7 & 8) to select them as my 4 quadrilateral.
Unfortunately, there are several limitations that need to be taken into account.
Limitations:
resolution of the processed shooting target image can vary
the image can be taken in different lighting conditions
the image processed by this part of my algorithm will always be taken under an angle (extreme angles will be automatically rejected)
the image can be slightly rotated (+/- 10 degrees)
the shooting target can be just a part of the image
the image can be only of the center black part of the target, meaning the user doesn't have to take a photo of the whole shooting target (but there always has to be the center black part on it)
this algorithm can take a maximum of 2000ms runtime
What I have tried so far:
Template matching
here I quickly realized that it was unusable since the numbers could be slightly rotated and a different scale
Feature matching
I have tried all of the different feature matching types (SIFT, SURF, ORB...)
unfortunately, the numbers do not have that specific set of features so they matched a quite lot of false positives, but I could possibly filter them by adding shape matching, etc..
the biggest blocker was runtime, the runtime of only a single number feature matching took around 5000ms (even after optimizations) (on MacBook PRO 2017)
Optical character recognition
I mostly tried using pytesseract library
even after thresholding the image to inverted binary (so the text of numbers 7 and 8 is black and the background white) it failed to recognize them
I also tried several ways of preprocessing the image and I played a lot with the tesseract config parameter but it didn't seem to help whatsoever
Contour detection
I have easily detected all of the wanted numbers (7 & 8) as single contours but failed to filter out all of the false positives (since the image can be in different resolutions and also there are two types of targets with different sizes of the numbers I couldn't simply threshold the contour by its width, height or area)
After I would detect the numbers as contours I wanted to extract them as some ROI and then I would use OCR on them (but since there were so many false positives this would take a lot of time)
I also tried filtering them by using cv2.matchShapes function on both contours and cropped template / ROI but it seemed really unreliable
Example processed images:
high resolution version here
high resolution version here
high resolution version here
high resolution version here
high resolution version here
high resolution version here
As of right now, I'm lost on how to progress about this. I have tried everything I could think of. I would be immensely happy if any of you image recognition experts gave me any kind of advice or even better a usable code example to help me solve my problem.
Thank you all in advance.
Find the black disk by adaptive binarization and contour (possibly blur to erase the inner features);
Fit an ellipse to the outline, as accurate as possible;
Find at least one edge of the square (Hough lines);
Classify the edge as one of NWSE (according to angle);
Use the ellipse and the line information to reconstruct the perspective transformation (it is an homography);
Apply the inverse homography to straighten the image and obtain the exact target center and axis;
Again by adaptive binarization, find the bullet holes (center/radius);
Rate the holes after their distance to the center, relative to the back disk radius.
If the marking scheme is variable, detect the circles (Hough circles, using the known center, or detect peaks in an oblique profile starting from the center).
If necessary, you could OCR the digits, but it seems that the score is implicitly starting at one in the outer ring.
i need some advice in a computer vision projekt that i am working on. I am trying to extract a corner in the image below. The edge im searching for is marked yellow in the right image. The edge detection is always failing because the edge is too blurred in the middle.
I run this process with opencv and python.
I started to remove the white dots with a threshold method. After that a big median blur (31-53). After that a adaptive Threshod method to seperate the areas left and right from the corners. But the sepearation is always bad because the edge is barely visible.
Is there some other way to extract this edge or do i have to try with a better camera?
Thanks for your help.
First do you have other dataset? because it is hard to discuss it just from 1 input.
Couple things that you can do.
The best is you change the camera of imaging technique to have a better and clear edge.
When it is hard to do so. Try model-based fitting.If you image is repeatable in all class. I can observe some circles on the right and 2 sharp straight-line edges on the left. Your wanted red soft edge circle is in the middle of those 2 apparent features. That can be considered as a model. then you can always use some other technique for the pixel in-between those 2 region(because they are easy to detect) . Those technique includes but not limit to histogram equalization, high pass filter or even wavelet transform.
The Wost way is to use parameter fitting to do. What you want to segment is sth not a strong edge and sth not a smooth plane. So you can tweak the canny edge detect to find those edge which is not so strong. I do not support this method. If you really no choice and no other image, then you can try it.
Last way is to use deep learning based method to train and auto segment this part out. This method might work. but it needs you to have hundred if not thousands of dataset and labels.
Regards
Shenghai Yuan
I am trying to extract the tiles ( Letters ) placed on a Scrabble Board. The goal is to identify / read all possible words present on the board.
An example image -
Ideally, I would like to find the four corners of the scrabble Board, and apply perspective transform, for further processing.
After Perspective transform -
The algorithm that I am using is as follows -
Apply Adaptive thresholding to the gray scale image of the Scrabble Board.
Dilate / Close the image, find the largest contour in the given image, then find the convex hull, and completely fill the area enclosed by the convex hull.
Find the boundary points ( contour ) of the resultant image, then apply Contour approximation to get the corner points, then apply perspective transform
Corner Points found -
This approach works with images like these. But, as you can see, many square boards have a base, which is curved at the top and the bottom. Sometimes, the base is a big circular board. And with these images my approach fails. Example images and outputs -
Board with Circular base:
Points found using above approach:
I can post more such problematic images, but this image should give you an idea about the problem that I am dealing with. My question is -
How do I find the rectangular board when a circular board is also present in the image?
Some points I would like to state -
I tried using hough lines to detect the lines in the image, find the largest vertical line(s), and then find their intersections to detect the corner points. Unfortunately, because of the tiles, all lines seem to be distorted / disconnected, and hence my attempts have failed.
I have also tried to apply contour approximation to all the contours found in the image ( I was assuming that the large rectangle, too, would be a contour ), but that approach failed as well.
I have implemented the solution in openCV-python. Since the approach is what matters here, and the question was becoming a tad too long, I didn't post the relevant code.
I am willing to share more such problematic images as well, if it is required.
Thank you!
EDIT1
#Silencer's answer has been mighty helpful to me for identifying letters in the image, but I want to accurately find the placement of the words in the image. Hence, I feel identifying the rows and columns is necessary, and I can do that only when a perspective transform is applied to the board.
I wrote an answer on MSER text detection:
Trying to Plot OpenCV's MSER regions using matplotlib
The code generate the following results on your images.
You can have a try.
I think #silencer has already given quite promising solution.
But to perform perspective transform as you have mentioned that you have already tried with hough lines to find the largest rectangle but it fails because for tiles present.
Given you have large image data set may be more than 1000 images, you can also give a shot to Deep learning based approach where you can train a model with images as input and corresponding rectangle boundary points coordinate as outputs.
I have an image of analog voltmeter:
I'd like to detect the needle and then find it's direction/angle.
I'm using python and opencv and have tried contours but haven't managed to solve it so far:
#I'll insert my code attempts here soon!
Hough transform should work well for this kind of image.
Needle is the longest and the most powerful straight object, and proper threshold choice could give good result.
You can also remove some thin segments of scale and make needle thinner with morphological operation erosion
Here is what you can do:
Simple thresholding to detect all the dark patterns
Erase all the small patterns
Erase the patterns that touch the border.
Geodesic diameter computation in order to detect the longest pattern.
PCA in order to determine the orientation.