We Keep Coding

How can I connect a gap in a white frame on black background with python opencv?

I developed a frame detection algorithm in python using opencv that detects a metallic frame in a rgb image. The output is a white frame on a black background. However, if the frame is partly covered, it contains gaps. How can I fill the gaps, preferably using opencv?
Example output of my algorithm:
Frame with gap
Desired output:
Frame without gap
Edit:
This is how I detect the frame currently.
I have an rgb image of a crop canopy and a frame on top of it:
Crop canopy image
My code detects bright parts in the image and removes noise such that only the frame should be left (which is the largest brigth object in the image). Below is a reproducible example when setting image path to the example image above:
# import packages
import numpy as np
import cv2
kernel_size = 10
brightness_thres_low = 175
brightness_thres_high = 255
erode_it_1 = 6
dilate_it_1 = 25
erode_it_2 = 20
# read image
img = cv2.imread(image_path)
# find bright pixel values that indicate frame and set to bw
thres = cv2.threshold(cv2.cvtColor(img, cv2.COLOR_BGR2GRAY), brightness_thres_low, brightness_thres_high, cv2.THRESH_BINARY_INV)[1]
thres = np.invert(thres)
# adapt image to only contain frame
kernel = np.ones((kernel_size, kernel_size), np.uint8)
img_adapted = cv2.erode(thres,kernel, iterations = erode_it_1)
img_adapted = cv2.dilate(img_adapted,kernel, iterations = dilate_it_1)
img_adapted = cv2.erode(img_adapted, kernel, iterations = erode_it_2)
If the frame is fully in the image, it gets detected fine. But as you can see in the example rgb image, part of the frame is covered by a plant. This results in an incomplete detection of the frame. I would like to fill this gap in the frame in a generic way such that it can be applied to any gap of any processed rgb image.

Here is some code you can start with:
import numpy as np
import cv2
kernel_size = 10
erode = 2
thres_low = 175
thres_high = 255
img = cv2.imread("img.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, thres = cv2.threshold(gray, thres_low, thres_high, cv2.THRESH_BINARY)
kernel = np.ones((kernel_size, kernel_size), np.uint8)
thres = cv2.erode(thres, kernel, iterations=erode) # erode fine contours
contours, _ = cv2.findContours(thres, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) # find contours
all_contours = np.vstack(contours) # concatenate all contours
hull = cv2.convexHull(all_contours) # get convex hull of all contours
peri = cv2.arcLength(hull, True) # get perimeter length of hull for the next step
approx = cv2.approxPolyDP(hull, 0.05 * peri, True) # simplify hull
cv2.drawContours(img, [approx], 0, (0, 0, 255), 10) # draw the contour
cv2.imwrite("out.png", img)
Output image:

OpenCV Contour Detection in color

I'm trying to detect the optic disc in an image of the back of the eye using OpenCV and findContour, then fitEllipse, but my program isn't detecting the optic disc at all. How do I fix this? Code and images are below
import cv2
import numpy as np
from sklearn.linear_model import LinearRegression
import math
from decimal import Decimal
def find_elongation(image):
img = cv2.imread(image,0)
ret,thresh = cv2.threshold(img,127,255,0)
contour,hierarchy = cv2.findContours(thresh, 1, 2)
contours = []
for i in range(len(contour)):
if len(contour[i])>=5:
contours.append(contour[i])
cnt = contours[0]
k = cv2.isContourConvex(cnt)
ellipse = cv2.fitEllipse(cnt)
im = cv2.ellipse(img,ellipse,(0,255,0),2)
(x,y),(ma,Ma),angle = cv2.fitEllipse(cnt)
return Ma/ma
print(find_elongation('eye.png'))
print(find_elongation('eye2.png'))
print(find_elongation('eye3.png'))
Image (one of them):
I'm trying to get the brightly colored circle in the middle:
Thanks for the help!

I have developed a piece of code to implement what you have asked. It mainly uses de Value channel of the HSV color space followed by some morphological operations.
import cv2
import numpy as np
import matplotlib.pyplot as plt
# Read the image
img = cv2.imread('so.png')
# Transform the image to HSV color-space and keep only the value channel
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)
# Threshold the iamge with the 95% of the brightest possible pixels
maxVal = np.max(v)
per = 0.95
_, th = cv2.threshold(v, maxVal*per, 255, cv2.THRESH_BINARY)
# Erode the image and find the connected components
th = cv2.erode(th, np.ones((2,2), np.uint8))
n, conComp, stats, centroids = cv2.connectedComponentsWithStats(th)
# Obtain the sizes of the connectedComponents skipping the background
sizes = stats[1:,-1]
# Obtain the number of the connectedComponent with biggest size
nComp = np.argmax(sizes) + 1
# Isolate the connectedComponent and apply closing
out = np.zeros((img.shape[0], img.shape[1]), np.uint8)
out[conComp==nComp] = 1
out = cv2.morphologyEx(out, cv2.MORPH_CLOSE, np.ones((10,10)))
# Apply gradient to mask to obtain the border of the ellipse
out = cv2.morphologyEx(out, cv2.MORPH_GRADIENT, np.ones((2,2)))
# Join the border of the ellipse with the image to display it
img[out==1] = (0,0,0)
plt.imshow(cv2.cvtColor(img,cv2.COLOR_BGR2RGB))
plt.show()
I attach the output I have obtained with the picture you posted:

Is there a function similar to OpenCV findContours that detects curves and replaces points with a spline?

I am trying to take the below image, trace the white shape, and export the resulting path to pdf. The problem I have is that findContours seeming only finds points along the edge of the shape. Is there a solution out there, similar to findContours, that detects curves in a shape and replaces its points with a spline wherever there is a curve? If I use scipy.interpolate it ignores straight lines and turns the entire contour into one big curved shape, which is no good either. I need something that does both things.
import numpy as np
import cv2
from scipy.interpolate import splprep, splev
from pyx import *
import matplotlib.pyplot as plt
#read in image file
original = cv2.imread('test.jpg')
#blur the image to smooth edges
im = cv2.medianBlur(original,5)
#threshold the image
imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(imgray,170,255,cv2.THRESH_BINARY)
#findContours
im2, contours, hierarchy = cv2.findContours(thresh,cv2.RETR_EXTERNAL,cv2.CHAIN_\
APPROX_SIMPLE)
#drawContours
cv2.drawContours(original, [approx], -1, (0,255,0), 3)
cv2.imshow("Imageee", original)
cv2.waitKey(0)

Except using cv2.findContours with flag cv2.CHAIN_APPROX_SIMPLE to approx the contours, we can do it manually.
use cv2.findContours with flag cv2.CHAIN_APPROX_NONE to find contours.
use cv2.arcLength to calculate the contour length.
use cv2.approxPoolyDP to approx the contour manually with epsilon = eps * arclen.
Here is one of the results when eps=0.005:
More results:
#!/usr/bin/python3
# 2018.01.04 13:01:24 CST
# 2018.01.04 14:42:58 CST
import cv2
import numpy as np
import os
img = cv2.imread("test.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,threshed = cv2.threshold(gray,170,255,cv2.THRESH_BINARY)
# find contours without approx
cnts = cv2.findContours(threshed,cv2.RETR_LIST,cv2.CHAIN_APPROX_NONE)[-2]
# get the max-area contour
cnt = sorted(cnts, key=cv2.contourArea)[-1]
# calc arclentgh
arclen = cv2.arcLength(cnt, True)
# do approx
eps = 0.0005
epsilon = arclen * eps
approx = cv2.approxPolyDP(cnt, epsilon, True)
# draw the result
canvas = img.copy()
for pt in approx:
cv2.circle(canvas, (pt[0][0], pt[0][1]), 7, (0,255,0), -1)
cv2.drawContours(canvas, [approx], -1, (0,0,255), 2, cv2.LINE_AA)
# save
cv2.imwrite("result.png", canvas)

I think your problem actually consists of two issues.
The first issue is to extract the contour, which you can achieve using teh findContour function:
import numpy as np
print cv2.__version__
rMaskgray = cv2.imread('test.jpg', 0)
(thresh, binRed) = cv2.threshold(rMaskgray, 200, 255, cv2.THRESH_BINARY)
_, Rcontours, hier_r = cv2.findContours(binRed,cv2.RETR_CCOMP,cv2.CHAIN_APPROX_SIMPLE)
r_areas = [cv2.contourArea(c) for c in Rcontours]
max_rarea = np.argmax(r_areas)
CntExternalMask = np.ones(binRed.shape[:2], dtype="uint8") * 255
contour= Rcontours[max_rarea]
cv2.drawContours(CntExternalMask,[contour],-1,0,1)
print "These are the contour points:"
print c
print
print "shape: ", c.shape
for p in contour:
print p[0][0]
cv2.circle(CntExternalMask, (p[0][0], p[0][1]), 5, (0,255,0), -1)
cv2.imwrite("contour.jpg", CntExternalMask)
cv2.imshow("Contour image", CntExternalMask)
cv2.waitKey(0)
If you execute the program, the contour points are printed as a list of point coordinates.
The contour approximation method you choose influences the interpolation which is actually used (and the number of points found), as described here. I have added small dots at the points found with the approximation method cv2.CHAIN_APPROX_SIMPLE. You see that the straight lines are already approximated.
I may not fully have understood your second step, though. You want to omit some of those points, replacing point lists partially by splines. There might be different way to do this, depending on your final intention. Do you just want to replace the straight lines? If you replace curved parts, what is the margin of error you are allowing?

# import the necessary packages
import numpy as np
import argparse
import glob
import cv2
#For saving pdf
def save_pdf(imagename):
import img2pdf
# opening from filename
with open("output.pdf","wb") as f:
f.write(img2pdf.convert(imagename))
#for fouind biggest contours
def bigercnt(contours):
max_area=0
cnt=[]
for ii in contours:
area=cv2.contourArea(ii)
if area>max_area:
cnt = ii
return cnt
#STARTING
print ("Reading img.jpg file")
# load the image, convert it to grayscale, and blur it slightly
image = cv2.imread('img.jpg')
image = cv2.resize(image, (0,0), fx=0.5, fy=0.5)
print ("Converting it gray scale")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
print ("Bluring")
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
print ("Looking for edges" )
# apply Canny edge detection using a wide threshold, tight
# threshold, and automatically determined threshold
tight = cv2.Canny(blurred, 255, 250)
print ("Looking for contours")
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10, 10))
close = cv2.morphologyEx(tight, cv2.MORPH_CLOSE, kernel)
_,contours, hierarchy = cv2.findContours( close.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
print("Looking for big contour")
cnt = bigercnt(contours)
print ("Cropping found contour")
x,y,w,h = cv2.boundingRect(cnt)
croped_image = image[y:y+h,x:x+w]
img2 = np.zeros((h,w,4),np.uint8)
print ("Taking only pixels in countour and creating png")
for i in range(h):
for j in range(w):
#print (x+j, y+i)
#print cv2.pointPolygonTest(cnt, (x+j, y+i), False)
if cv2.pointPolygonTest(cnt, (x+j, y+i), False)==1:
#print True
img2[i,j] = [croped_image[i, j][0],croped_image[i, j][1],croped_image[i, j][2],255]
else:
img2[i,j] = [255,255,255,0]
print ("Showing output image")
# Show the output image
#cv2.imshow('croped', croped_image)
cv2.imshow('output', img2)
params = list()
params.append(cv2.IMWRITE_PNG_COMPRESSION)
params.append(8)
print ("Saving output image")
cv2.imwrite("output.png",img2,params)
print ("Finish:converted")
cv2.waitKey(0)
cv2.destroyAllWindows()

How to get rid of transparent background usign OpenCV 3 and Python 2.7?

I'm trying to remove the transparent background (the excess whitespace which is not visible here) from the last image. It looks like this:
The code which I'm using is as follows:
import cv2
import numpy as np
import os
from matplotlib import pyplot as plt
##Change directory to desktop
os.chdir("/home/meh/Desktop/")
##Reading the image
img_gray_scale = cv2.imread('img2.jpg',0)
img_colored = cv2.imread('img2.jpg',1)
###CONTOURS FOR IMAGE SEGMENTAITON####
##Gray scale image must be used
ret, thresh = cv2.threshold(img_gray_scale,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
im2, contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
####Extracting just the ROI
###First argument img is the source of image
###Second is the countours which should be passed as python list
###Third is index of contours (to draw all contours pass -1)
####remaining are color and thickness
mask2 = cv2.drawContours(thresh, contours, 0, (255,0,0), -1)
masked_data = cv2.bitwise_and(img_gray_scale,img_gray_scale, mask = mask2)
b,g,r = cv2.split(img_colored)
rgba = [b,g,r, thresh]
dst = cv2.merge(rgba,4)
cv2.imwrite('phone_original_without_background.png',dst)
dst = cv2.cvtColor(dst,cv2.COLOR_BGR2GRAY)
cv2.imwrite('phone_grayscale_without_background.png',dst)
My question is, how do I remove the transparent background and just keep the phone's image?

I tried your code and it seems to do nothing. Assuming that you want to crop out all the outer color pixels, here's my solution
Get all point of interest:
height,width = img_gray_scale.shape
fg = []
for col in range(width):
for row in range(height):
if thresh[row][col] < 255:
fg.append((col,row))
Get the minimal rectangle:
rotatedRect = cv2.minAreaRect(np.array(fg))
Use warpAffine to crop out the region of interest:
def subimage2(image, rotatedRect):
center, rotatedRect, angle = rotatedRect
width,height = int(shape[0]),int(shape[1])
# convert angle to radian and build affine transformation mat
theta = angle * np.pi/180
cosine,sine = np.cos(theta), np.sin(theta)
mapping = np.array([[cosine, sine, -center[0]+width/2],
[-sine, cosine, -center[1]+height/2]])
# write output
return cv2.warpAffine(image,mapping,(width,height))
cropped = subimage2(dst,rotatedRect)
And here's what we get

How to create vector polygon objects after watershed segmentation

After watershed segmentation using openCV-python to segment objects , I would like to get vector polygon objects (objects inside the blue circle) but I don't know how to do it in opencv-python. I attached the python code of the watershed segmentation and the image.
How to create vector polygon objects
import cv2
import numpy as np
import scipy.misc
import scipy.ndimage as snd
# image is read and is converted to a numpy array
img = cv2.imread('D:/exam_watershed/Example_2_medicine/Medicine_create_poly/medicine.jpg')
# image is convereted to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# binary thresholding is done using the threshold
# from Otsu's method
ret1,thresh1 = cv2.threshold(gray,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# foreground pixels are determined by
# performing erosion
fore_ground = cv2.erode(thresh1,None,iterations = 3)
bgt = cv2.dilate(thresh1,None,iterations = 3)
ret,back_ground = cv2.threshold(bgt,1,100,1)
# marker is determined by adding foreground and background pixels
marker = cv2.add(fore_ground,back_ground)
# converting marker to 32 int
marker32 = np.int32(marker)
cv2.watershed(img,marker32)
m = cv2.convertScaleAbs(marker32) #the output is converted to unit8 image
ret3,thresh3 = cv2.threshold(gray,0,255,\
cv2.THRESH_BINARY+cv2.THRESH_OTSU)
_, contours1, _= cv2.findContours(thresh3,cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
b = cv2.drawContours(img, contours, -1, (0,255,0), thickness=1, lineType=8)

You are close, just need a few more lines after finding the contours:
polys = []
for cont in contours1:
approx_curve = cv2.approxPolyDP(cont, 3, False)
polys.append(approx_curve)
cv2.drawContours(img, polys, -1, (0, 255, 0), thickness=1, lineType=8)
cv2.imshow("medicine polygons", img)
cv2.waitKey()
The doc on approxPolyDP.