I am working with an image containing a lot of small objects formed of hexagons, which are roughly inside a rectangular figure.
See image here:
There are also areas of noise outside this rectangle with the same pixel intensity, which I want to disregard with future functions. I have 2 questions regarding this:
How can I create a segmentation/ROI to only consider the objects/shapes inside that rectangular figure? I tried using Canny and contouring, as well as methods to try and create bounding boxes, but in each of them I always segment the individual objects directly in the entire image, and I can't eliminate the outside noise as a preliminary step.
How can I identify the number of white hexagons inside the larger rectangle? My original idea was to find the area of each of the individual objects I would obtain inside the rectangle (using contouring), sort from smallest to lowest (so the smallest area would correspond to a single hexagon), and then divide all the areas by the hexagonal area to get the number, which I could sum together. Is there an easier way to do this?
Related
I'm currently trying to write a program that can automatically extract data from some graphs in multiple scanned documents. Mainly by using opencv I would like to detect some features of the graphs in order to convert them into usable data. In the left graph I'm looking for the height of the circle sectors and in the right graph the distance from the center to the points where the dotted lines intersect with the gray area. In both cases I would like to convert these values into numeric data for further usage.
What follows is a step by step plan of how I think my algorithm will work:
Align the image based on the big dotted lines. This way I can ensure that the graphs in all the scanned images will have the exact same positions. After all, it is possible that some images will be slightly tilted or moved in comparison with other images, due to the manual scanning process. Basically I want the coordinate of a pixel in one image to correspond to the exact same pixel in another image.
We now know that the coordinates of the graph centers and the angles for the circle sectors are identical for all images now. For each circle sector, filter the darker pixels from the lighter ones. This is done using the openCV inRange function.
Search for the best fitting segment over the darker pixels in the left graph and search for the best fitting triangle in the right graph. This is done by global optimization.
Return the radius of the optimal segment and return the edge lengths of the optimal triangle. Now we have values that we can use as data.
I have more or less figured out how to do every step, except the first one. I have no clue on how I would go about aligning my images. Does someone might have an idea or a strategy on how to achieve this alignment?
Step 1: canny, it give you perfect long edge. If this is the only part you dont understand, here is the answer. You can adjust the parameter to get the best result. The first will be idea for both line and pie circle. But if you only keen to find pie. change the parameter accordingly to get my 2nd image
The red denotes the doted line. sample from opencv directly
Step 2: local area enhancement/segmentation to find both circles (from image 1 parameter with houghcircle param2 set to 110)
Step 3: Segment the pie out(all the way to the edge of image) and find the median line
Step 4: OCR on the test image pies and find the distance of none-background color along the median line.
Step 5: generate list out and send to csv or sth
Need to get rectangular shapes from a noisy color segmented image.
The problem is that sometimes the object isn't uniformly the correct color causing holes in the image, or sometimes reflection of the object in the background cause noise/false positive for the color segmentation.
The object could be in any position of the image and of any unknown rectangular size, the holes can occur anywhere inside the object and the noise could occur on any side of the object.
The only known constant is that the object is rectangular in shape.
Whats the best way to filter out that noise to the left of the object and get a bounding box around the object?
Using erosion would remove the detail from the bottom of the object and would cause the size of the bounding box to be wrong
I can't comment because of my rep, but I think you could try to analyse the colored image using other color spaces. Create a upper and a lower bound of the color you want until it selects the object, leaving you with less noise, which you can filter with erode/dilate/opening/closing.
For example, in my project I wanted to found a bounding box of a color-changing green rectangle, so I went and tried a lot of diferent color spaces with a lot of diferent upper/lower bounds until I finally got something worthy. Here is a nice read of what I'm talking about : Docs
You can also try filtering the object by área, after dilating it (you dilate first so the closer points connect to one another, while the more distant ones, which are the noise, don't, creating a big rectangle with lots of noise, but then you filter by a big área).
One method is to take histogram projection on both the horizontal and vertical axes, and select the intersection of ranges that have high projections.
The projections are just totals of object pixels in each row and each column. When you are looking for only one rectangle, the values indicated the probablity of the row/column belonging to the rectangle.
I am trying to extract the tiles ( Letters ) placed on a Scrabble Board. The goal is to identify / read all possible words present on the board.
An example image -
Ideally, I would like to find the four corners of the scrabble Board, and apply perspective transform, for further processing.
After Perspective transform -
The algorithm that I am using is as follows -
Apply Adaptive thresholding to the gray scale image of the Scrabble Board.
Dilate / Close the image, find the largest contour in the given image, then find the convex hull, and completely fill the area enclosed by the convex hull.
Find the boundary points ( contour ) of the resultant image, then apply Contour approximation to get the corner points, then apply perspective transform
Corner Points found -
This approach works with images like these. But, as you can see, many square boards have a base, which is curved at the top and the bottom. Sometimes, the base is a big circular board. And with these images my approach fails. Example images and outputs -
Board with Circular base:
Points found using above approach:
I can post more such problematic images, but this image should give you an idea about the problem that I am dealing with. My question is -
How do I find the rectangular board when a circular board is also present in the image?
Some points I would like to state -
I tried using hough lines to detect the lines in the image, find the largest vertical line(s), and then find their intersections to detect the corner points. Unfortunately, because of the tiles, all lines seem to be distorted / disconnected, and hence my attempts have failed.
I have also tried to apply contour approximation to all the contours found in the image ( I was assuming that the large rectangle, too, would be a contour ), but that approach failed as well.
I have implemented the solution in openCV-python. Since the approach is what matters here, and the question was becoming a tad too long, I didn't post the relevant code.
I am willing to share more such problematic images as well, if it is required.
Thank you!
EDIT1
#Silencer's answer has been mighty helpful to me for identifying letters in the image, but I want to accurately find the placement of the words in the image. Hence, I feel identifying the rows and columns is necessary, and I can do that only when a perspective transform is applied to the board.
I wrote an answer on MSER text detection:
Trying to Plot OpenCV's MSER regions using matplotlib
The code generate the following results on your images.
You can have a try.
I think #silencer has already given quite promising solution.
But to perform perspective transform as you have mentioned that you have already tried with hough lines to find the largest rectangle but it fails because for tiles present.
Given you have large image data set may be more than 1000 images, you can also give a shot to Deep learning based approach where you can train a model with images as input and corresponding rectangle boundary points coordinate as outputs.
I am trying to get the RGB average values for the area within the box. However, I cannot use things such as a contour, since some of the edges are cut off. I also cannot use fixed (x,y) points, since the "boxes" in other images are located in different position and may cut off certain edges. Is there a way to automatically determine what is "inside" the "box" as well as getting the average RGB values.
I am attempting to write a program to handle the finding the intersections of the lines outlining a rectangular object, (eg. Computer screen). I attempted to use hough lines originally, however, due to variable lighting conditions as well as content appearing on the screen, the lines that are drawn are not always the outline of the screen. Additionally, there may be a large amount of other random objects in the frame.
My next approach was to use contours which always seem to outline both the screen as well as being able to handle the variable content encapsulated within it. How do I go about using the contours to approximate a line?
I used
print len(contours)
and received a contour fairly long contour length > 200 consistently.
Feel free to comment asking for clarity.
EDIT
The green "lines" are the contours found by findContours. I am primarily interested in the contour surrounding the screen content. How can I use those contours to find a line approximating those contours and then find the point of intersection of the two lines? This is from a webcam stream so conditions, angle, and distance may not stay constant.
A first step might be to use the size of the contours to filter out those you are not interested in since the smaller contours usually correspond to stuff on the screen.
Also, the findContour method can be used to return the contours in a hierarchy of nested contours. This will tell you which contour is contained within another and allow you to get the outer-most, second outer-most one etc. If you are trying to get the screen, then it would have to be a large contour that is possibly the second largest and nested just below the contour for the monitor.
After obtaining the potential contours, which are just list of points, you can do a robust fitting for a single rectangle using just these points either by RANSAC (with a model for rectangles, not homography), or hough transform modified for this case.