We Keep Coding

Obtain complete pattern of shapes with OpenCV

I'm working in a script using different OpenCV operations for processing an image with solar panels in a house roof. My original image is the following:
After processing the image, I get the edges of the panels as follows:
It can be seen how some rectangles are broken due to reflection of the Sun in the picture.
I would like to know if it's possible to fix those broken rectangles, maybe by using the pattern of those which are not broken.
My code is the following:
# Load image
color_image = cv2.imread("google6.jpg")
cv2.imshow("Original", color_image)
# Convert to gray
img = cv2.cvtColor(color_image, cv2.COLOR_BGR2GRAY)
# Apply various filters
img = cv2.GaussianBlur(img, (5, 5), 0)
img = cv2.medianBlur(img, 5)
img = img & 0x88 # 0x88
img = cv2.fastNlMeansDenoising(img, h=10)
# Invert to binary
ret, thresh = cv2.threshold(img, 127, 255, 1)
# Perform morphological erosion
kernel = np.ones((5, 5),np.uint8)
erosion = cv2.morphologyEx(thresh, cv2.MORPH_ERODE, kernel, iterations=2)
# Invert image and blur it
ret, thresh1 = cv2.threshold(erosion, 127, 255, 1)
blur = cv2.blur(thresh1, (10, 10))
# Perform another threshold on blurred image to get the central portion of the edge
ret, thresh2 = cv2.threshold(blur, 145, 255, 0)
# Perform morphological erosion to thin the edge by ellipse structuring element
kernel1 = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
contour = cv2.morphologyEx(thresh2, cv2.MORPH_ERODE, kernel1, iterations=2)
# Get edges
final = cv2.Canny(contour, 249, 250)
cv2.imshow("final", final)
I have tried to modify all the filters I'm using in order to reduce as much as possible the effect of the Sun in the original picture, but that is as far as I have been able to go.
I'm in general happy with the result of all those filters (although any advice is welcome), so I'd like to work on the black/white imaged I showed, which is already smooth enough for the post-processing I need to do.
Thansk!

The pattern is not broken in the original image, so it being broken in your binarized result must mean your binarization is not optimal.
You apply threshold() to binarize the image, and then Canny() to the binary image. The problems here are:
Thresholding removes a lot of information, this should always be the last step of any processing pipeline. Anything you lose here, you've lost for good.
Canny() should be applied to a gray-scale image, not a binary image.
The Canny edge detector is an edge detector, but you want to detect lines, not edges. See here for the difference.
So, I suggest starting from scratch.
The Laplacian of Gaussian is a very simple line detector. I took these steps:
Read in image, convert to grayscale.
Apply Laplacian of Gaussian with sigma = 2.
Invert (negate) the result and then set negative values to 0.
This is the output:
From here, it should be relatively straight-forward to identify the grid pattern.
I don't post code because I used MATLAB for this, but you can accomplish the same result in Python with OpenCV, here is a demo for applying the Laplacian of Gaussian in OpenCV.
This is Python + OpenCV code to replicate the above:
import cv2
color_image = cv2.imread("/Users/cris/Downloads/L3RVh.jpg")
img = cv2.cvtColor(color_image, cv2.COLOR_BGR2GRAY)
out = cv2.GaussianBlur(img, (0, 0), 2) # Note! Specify size of Gaussian by the sigma, not the kernel size
out = cv2.Laplacian(out, cv2.CV_32F)
_, out = cv2.threshold(-out, 0, 1e9, cv2.THRESH_TOZERO)
However, it looks like OpenCV doesn't linearize (apply gamma correction) when converting from BGR to gray, as the conversion function does that I used when creating the image above. I think this gamma correction might have improved the results a bit by reducing the response to the roof tiles.

how to draw rectangle on Brain tumor using open cv?

I want to draw rectangle or bounding box in a green color on my brain tumor image dataset to highlight the tumor in image. What would be shortest method for it? I do not want segmentation of tumor.
I have tried openev draw rectangle function but it does not work. Also I have found Active Contour and Canny algorithm but this also does not work.
import cv2
import numpy as np
image = cv2.imread('12.png')
cv2.imshow('Input Image', image)
cv2.waitKey(0)
gray =cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Canny Edges edged = cv2.Canny(gray, 30, 200)
# cv2.imshow('Canny Edges', edged)
# cv2.waitKey(0)
_, contours, hierarchy = cv2.findContours(edged, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cv2.imshow('Canny Edges after Contouring',edged)
cv2.waitKey(0)
print("Number of contours found = " +str(len(contours)))
cv2.drawContours(image, contours, -1, (0, 255, 0),3)
cv2.imshow('Contours', image)
cv2.waitKey(0)
cv2.destroyAllWindows()
I expect the the tumor part should be in a box from whole image.Rest of the image area should be the same as my actual brain image.Just one green box around the tumor part through simple code not whole algorithm.

Based on the comment to this answer I updated my answer, and I think that this is still a Deep Learning problem, but of a different kind.
If you need to segment the tumor along its contour, one way to do it may be Mask R-CNN(1) (you can read the original paper published in 2017). This method allows image segmentation along their contours. Here you can find a sample implementation of the method.
As far as I can see from the image, you have to deal with grayscale images, and it may be difficult to segment tumors from the rest of the scene by simple methods, here I agree with #BahramdunAdil. Therefore, I suspect that the training might need a lot of data and time.
Good luck!
(1) He, Kaiming, et al. "Mask r-cnn." Proceedings of the IEEE international conference on computer vision. 2017.

Characters detection on a non-uniform background image

I am trying to do characters detection, have to draw a box around them, then crop and then feed to a neural network for recognition. Everything is working but before I was using sets of characters on a single color background image and segmentation was easily done.
However with real photos I have different lighting conditions and really struggle to find the contours.
After applying some adaptive thresholding I managed to get the folowing results, but starting from that I really can't figure how to properly proceed and detect each character. I can detect half of the characters easily, but not all of them. Probably because they are surrounded by lots of small irrelevant contours.
I have a feeling there is one step left but I can't figure which one.
Find Countours is capable of finding only about half of the characters.
For now, in short, im doing:
im_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
im_gray = cv2.GaussianBlur(im_gray, (5, 5), 0)
_, th1 = cv2.threshold(im_gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
cim, ctrs, hier = cv2.findContours(th1.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
and
th2 = cv2.adaptiveThreshold(im_gray,255,cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY,11,2)
Images below - original image and some variations of intermediate results.
Original picture:
After some thresholding:
After some thresholding:
Inverse thresholding:
So the question is - what is the step/steps after to segment the characters?

You can perform difference of gaussians. The idea is to blur the image with two different kernels and subtract their respective results:
Code:
im = cv2.imread(img, 0)
#--- it is better to take bigger kernel sizes to remove smaller edges ---
kernel1 = 15
kernel2 = 31
blur1 = cv2.GaussianBlur(im,(kernel1, kernel1), 0)
blur2 = cv2.GaussianBlur(im,(kernel2, kernel2), 0)
cv2.imshow('Difference of Gaussians',blur2 - blur1)
Result:

pupil detection in OpenCV & Python

I am doing pupil detection for my school project. It's my first time working with OpenCV and Python, using Python version 3.4.2 and OpenCV 3.1.0.
I am using the Raspberry Pi NoIR camera, and I am getting good images.
But i can't detect a pupil nicely (because of glint, lashes and shadows.
I refer to some code on the web and the following is part of that code.
...
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
image = frame.array
cv2.imshow("image", image)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
retval, thresholded = cv2.threshold(gray, 80, 255, 0)
cv2.imshow("threshold", thresholded)
closed = cv2.erode(cv2.dilate(thresholded, kernel, iterations=1), kernel, iterations=1)
#closed = cv2.morphologyEx(close, cv2.MORPH_CLOSE, kernel)
cv2.imshow("closed", closed)
thresholded, contours, hierarchy = cv2.findContours(closed, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
drawing = np.copy(image)
cv2.drawContours(drawing, contours, -1, (255, 0, 0), 2)
for contour in contours:
area = cv2.contourArea(contour)
bounding_box = cv2.boundingRect(contour)
extend = area / (bounding_box[2] * bounding_box[3])
# reject the contours with big extend
if extend > 0.8:
continue
# calculate countour center and draw a dot there
m = cv2.moments(contour)
if m['m00'] != 0:
center = (int(m['m10'] / m['m00']), int(m['m01'] / m['m00']))
cv2.circle(drawing, center, 3, (0, 255, 0), -1)
# fit an ellipse around the contour and draw it into the image
try:
ellipse = cv2.fitEllipse(contour)
cv2.ellipse(drawing, box=ellipse, color=(0, 255, 0))
except:
pass
# show the frame
cv2.imshow("Drawing", drawing)
...
Input image:
Output image:
How can I remove the parts of the image that are not related to the pupil, as shown above?
In addition to answers, any hints are also welcome.

There are several things you can do. How well they work depends on how much variation there is in the images you want to apply the algorithm on. You could make several assumptions and then discard all the candidates that do not meet them.
remove small detections
At first I would consider removing candidates that are too small by adding this line at the beginning of your loop:
if area < 100:
continue
The threshold was chosen randomly and worked well for this particular image. It removed almost all the false detection. Only the biggest one remains. But you have to check it against your other images and adapt it to your needs.
remove detections that are not round
Another assumption you can make is that pupils are usually round and you can remove every detection that is not 'round' enough. One simple measure of roundness is to look at the ratio of circumference to area.
circumference = cv2.arcLength(contour,True)
circularity = circumference ** 2 / (4*math.pi*area)
The circularity is about 2.72 for the shadow in the right side and 1.31 for the pupil.
improving roundness
You notice, that the contour of your pupil is not perfectly round because of reflections. You can improve this by computing the convex hull of the contours.
contour = cv2.convexHull(contour)
If you do that before computing the area and circumference you get circularity values of 1.01 and 1.37. (A perfect circle has a circularity of 1) This means the defect from the reflection was almost perfectly repaired. This might not be necessary in this case but could be useful in cases with more reflections.

Some years ago I realized a very similar project. To everything added above I can suggest one little trick.
As you can see you have two reflections from any light source. One on the surface of the eye and second reflection on the surface of the glass.
If you will remove the glass (if it's possible) you will have a very bright reflection almost in the center of pupil. In that case you can ignore all objects without this bright reflection. This also will help you to find a position of eye in the space near camera

Python OpenCV detect circles from black and white mask

I have created a black and white mask using various OpenCV filters. There are four circles that are clearly visible
I am trying to outline these circles using HoughCircles, but it gives many false positives and generally bad results:
circles = cv2.HoughCircles(combined, cv.CV_HOUGH_GRADIENT, 1, 300, np.array([]), 10, 30, 60, 300)
How can I properly detect the circular shapes in the black and white image?
Here is runnable code that can be used with the black and white image:
import numpy as np
import cv2
import cv
image = cv2.imread("image.png")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
circles = cv2.HoughCircles(gray, cv.CV_HOUGH_GRADIENT, 1, 300, np.array([]), 10, 30, 60, 300)
if circles is not None:
circles = np.uint16(np.around(circles))
for i in circles[0,:]:
cv2.circle(image, (i[0], i[1]), i[2], (0, 255, 0), 1)
cv2.circle(image, (i[0], i[1]), 2, (0, 0, 255), 3)
cv2.imshow("thing", image)
cv2.waitKey(0)
cv2.destroyAllWindows()

First of all, if I'm not mistaken, Hough Transform for circles expects a hollow circle, not a full one. This means you need to extract only the boundary/perimeter of the circles before applying the Hough Transform. OpenCV's findContours and arcLength functions will help you find the perimeters.
Second of all, unfortunately, in my experience Hough Transform is very sensitive to variations of the circle shape, meaning that if the shape you're trying to detect is "almost" a circle, it might not be able to detect it.
My advice is you should try to make your objects "rounder" by applying the Closing morphological operation on your binary image, with a disk-shaped structuring element. Then extract the perimeter of the objects in the image, and only then apply the Hough Transform. Hopefully, this will give you good enough results.
Alternatively, you can try to detect circles using the RANSAC algorithm. Here is an implementation for detecting lines, but you can adjust it for circles - just choose 3 points randomly (instead of 2), and define a circle that goes through them. Next, find all the points that lie near that circle (those are called the inlier points). These will be the points that satisfy the inequality:
where (x,y) is the point, (x0,y0) is the center of the circle, r is its radius, and margin is a parameter you'll have to tune.
The rest of the algorithm is the same.
Good luck!