I have an image with a text.
This is the image:
What I'm trying to do is to straighten the text, using perspective transformation.
The red dots in the corners are the detected boundaries.
This is more or less my code (hard-coded, for simplicity):
old_pts=np.float32([[2,41],[37,965],[1389,1121],[1389,0]])
bor=cv2.boundingRect(old_pts) #bounding_rect
ul=[bor[0], bor[1]] #upper left
ur=[bor[0], bor[1]+bor[3]] #upper right
br=[bor[0]+bor[2],bor[1]+bor[3]] #bottom right
bl=[bor[0]+bor[2],bor[1]] #bottom left
new_pts=np.float32([ul,ur,br,bl]) #new pts=[[2,0],[2,1122],[1390,1122],[1390,0]]
M = cv2.getPerspectiveTransform(old_pts,new_pts)
transformed_img = cv2.warpPerspective(new_img,M,(bor[3],bor[2])) #bor[3] and bor[4] are the bounding rect height&width.
transforemed_img=transformed_img.astype(int)
cv2.imwrite('transformed.png',transformed_img)
Now, the result I'm getting is this:
Why am I not getting a nice, straightened rectangle??
Any help will be appreciated!
Take a look at your points:
You have :
old_pts=np.float32([[2,41],[37,965],[1389,1121],[1389,0]])
and:
new_pts=np.float32([ul,ur,br,bl]) #new pts=[[2,0],[2,1122],[1390,1122],[1390,0]]
But, OpenCV manages POINTS as (x,y) values, yours are in (y,x)... I know this is confusing, since the matrix manipulation is done with (y,x) notation... The thing is that OpenCV sees the matrix manipulation as the rows and columns like a matrix, but points are seen as Cartesian coordinates...
In conclusion, try flipping the axes for the points and check the results.
Related
I'm trying to develop some code to crop multiple areas within an image. The only info I have is the x/y pixel coordinates of the top left corner of each crop, and the pixel length of each side (this number is uniform as the crops are all perfect squares.
For example, within one image I may have multiple features needing cropping. Here, the first two figures are the x/y coords of the top left corner, and the third is the length of each of the four sides (r) x, y, r
Currently trying to achieve this in skimage, but not getting very far. Sorry if this is unclear, please feel free to ask more Qs.
Ta,
Rhod
Just use Numpy slicing. So if you have:
X = 10
Y = 20
R = 15
your extracted ROI is:
ROI = im[Y:Y+R, X:X+R]
I have one image. I want to get the coordinates in order so that the image can be redrawn. The image may be complex. I want to get sampled corner coordinates to redraw the outline of the image.
Attempt 1: corner function
I tried to detect the corner using corner() in matlab but it is not giving exact point. The corners are lying outside. The original image is attached below.
My code for corner detection is:
It=rgb2gray(I);
Itt=im2bw(It);
Itt1=corner(Itt);
imshow(Itt);
hold on
plot(Itt1(:,1), Itt2(:,2), 'r*');
The output of corner detection is attached below:
Problem with corner: If you zoom the image, you will find that the some corners don't lie on boundaries. So, please suggest some efficient and good working method.
Attempt 2: bwtraceboundaries
I also tried using bwtraceboundaries and corners to order the corner in terms of bwtraceboundaries output but the corner is not being exactly detected.
Question: Can you please suggest how can I detect the corner or is there any other method to extract the sampled corner points from the image so that the outline can be redrawn?
You can use bwboundaries to achieve what you want. It returns all the sorted boundary positions as a cell array:
B = bwboundaries(Itt);
imshow(Itt);
hold on
B = B{2}; % select the desired boundary (the second in this case)
plot(B(:,2), B(:,1), 'b-'); % draw a connected line
If you want to obtain only the corner points, you can filter them out manually by checking if the point lies in the middle of the next and previous point as follows:
iBremove = []; % redundant indexes in B
for k=2:length(B)-1 % loop over all the points, except the first and last one
Bk = (B(k-1, :) + B(k+1, :))/2; % interpolate the previous and next point
if all(Bk == B(k, :)) % check if point k is redundant
iBremove = [iBremove k]; % add to the list of indexes to remove
end
end
B(iBremove, :) = []; % remove the redundant points out of B
plot(B(:,2), B(:,1), 'r*-'); % draw a connected line
This is the continuation of my previous question. I now have an image like this
Here the corners are detected. Now I am trying to estimate the dimensions of the bigger box while smaller black box dimensions are known.
Can anyone guide me what is the best way to estimate the dimensions of the box? I can do it with simple Euclidean distance but I don't know if it is the correct way or not. Or even if it is the correct way then from a list of tuples (coordinates) how can I find distances like A-B or A-D or G-H but not like A-C or A-F?
The sequence has to be preserved in order to get correct dimensions. Also I have two boxes here so when I create list of corners coordinates then it should contain all coordinates from A-J and I don't know which coordinates belong to which box. So how can I preserve that for two different boxes because I want to run this code for more similar images.
Note: The corners in this image is not a single point but a set of points so I clustered the set of the corner and average them to get a single (x,y) coordinate for each corner.
I have tried my best to explain my questions. Will be extremely glad to have some answers :) Thanks.
For the
How can I find distances like A-B or A-D or G-H but not like A-C or
A-F
part
Here's a quick code, not efficient for images with lots of corners, but for your case it's OK. The idea is to start from the dilated edge image you got in your other question (with only the big box, but the idea is the same for the image where there is also the small box)
then for every possible combination of corners, you look at a few points on an imaginary line between them, and then you check if these points actually fall on a real line in the image.
import cv2
import numpy as np
#getting intermediate points on the line between point1 and point2
#for example, calling this function with (p1,p2,3) will return the point
#on the line between p1 and p2, at 1/3 distance from p2
def get_intermediate_point(p1,p2,ratio):
return [p1[0]+(p2[0]-p1[0])/ratio,p1[1]+(p2[1]-p1[1])/ratio]
#open dilated edge images
img=cv2.imread(dilated_edges,0)
#corners you got from your segmentation and other question
corners=[[29,94],[102,21],[184,52],[183,547],[101,576],[27,509]]
nb_corners=len(corners)
#intermediate points between corners you are going to test
ratios=[2,4,6,8] #in this example, the middle point, the quarter point, etc
nb_ratios=len(ratios)
#list which will contain all connected corners
connected_corners=[]
#double loop for going through all possible corners
for i in range(nb_corners-1):
for j in range(i+1,nb_corners):
cpt=0
c1=corners[i]; c2=corners[j]
#testing every intermediate points between the selected corners
for ratio in ratios:
p=get_intermediate_point(c1,c2,ratio)
#checking if these points fall on a white pixel in the image
if img[p[0],p[1]]==255:
cpt+=1
#if enough of the intermediate points fall on a white pixel
if cpt>=int(nb_ratios*0.75):
#then we assume that the 2 corners are indeed connected by a line
connected_corners.append([i,j])
print(connected_corners)
In general you cannot, since any reconstruction is only up to scale.
Basically, given a calibrated camera and 6 2D-points (6x2=12) you want to find 6 3D points + scale = 6x3+1=19. There aren't enough equations.
In order to do so, you will have to make some assumptions and insert them into the equations.
Form example:
The box edges are perpendicular to each other (which means that every 2 neighboring points share at least one coordinate value).
You need to assume that you know the height of the bottom points, i.e. they are on the same plane as your calibration box (this will give you the Z of the visible bottom points).
Hopefully, these constraints are enough to given you less equations that unknown and you can solve the linear equation set.
I am using OpenCV HoughlinesP to find horizontal and vertical lines. It is not finding any lines most of the time. Even when it finds a lines it is not even close to actual image.
import cv2
import numpy as np
img = cv2.imread('image_with_edges.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
flag,b = cv2.threshold(gray,0,255,cv2.THRESH_OTSU)
element = cv2.getStructuringElement(cv2.MORPH_CROSS,(1,1))
cv2.erode(b,element)
edges = cv2.Canny(b,10,100,apertureSize = 3)
lines = cv2.HoughLinesP(edges,1,np.pi/2,275, minLineLength = 100, maxLineGap = 200)[0].tolist()
for x1,y1,x2,y2 in lines:
for index, (x3,y3,x4,y4) in enumerate(lines):
if y1==y2 and y3==y4: # Horizontal Lines
diff = abs(y1-y3)
elif x1==x2 and x3==x4: # Vertical Lines
diff = abs(x1-x3)
else:
diff = 0
if diff < 10 and diff is not 0:
del lines[index]
gridsize = (len(lines) - 2) / 2
cv2.line(img,(x1,y1),(x2,y2),(0,0,255),2)
cv2.imwrite('houghlines3.jpg',img)
Input Image:
Output Image: (see the Red Line):
#ljetibo Try this with:
c_6.jpg
There's quite a bit wrong here so I'll just start from the beginning.
Ok, first thing you do after opening an image is tresholding. I recommend strongly that you have another look at the OpenCV manual on tresholding and the exact meaning of the treshold methods.
The manual mentions that
cv2.threshold(src, thresh, maxval, type[, dst]) → retval, dst
the special value THRESH_OTSU may be combined with one of the above
values. In this case, the function determines the optimal threshold
value using the Otsu’s algorithm and uses it instead of the specified
thresh .
I know it's a bit confusing because you don't actully combine THRESH_OTSU with any of the other methods (THRESH_BINARY etc...), unfortunately that manual can be like that. What this method actually does is it assumes that there's a "foreground" and a "background" that follow a bi-modal histogram and then applies the THRESH_BINARY I believe.
Imagine this as if you're taking an image of a cathedral or a high building mid day. On a sunny day the sky will be very bright and blue, and the cathedral/building will be quite a bit darker. This means the group of pixels belonging to the sky will all have high brightness values, that is will be on the right side of the histogram, and the pixels belonging to the church will be darker, that is to the middle and left side of the histogram.
Otsu uses this to try and guess the right "cutoff" point, called thresh. For your image Otsu's alg. supposes that all that white on the side of the map is the background, and the map itself the foreground. Therefore your image after thresholding looks like this:
After this point it's not hard to guess what goes wrong. But let's go on, What you're trying to achieve is, I believe, something like this:
flag,b = cv2.threshold(gray,160,255,cv2.THRESH_BINARY)
Then you go on, and try to erode the image. I'm not sure why you're doing this, was your intention to "bold" the lines, or was your intention to remove noise. In any case you never assigned the result of erosion to something. Numpy arrays, which is the way images are represented, are mutable but it's not the way the syntax works:
cv2.erode(src, kernel, [optionalOptions] ) → dst
So you have to write:
b = cv2.erode(b,element)
Ok, now for the element and how the erosion works. Erosion drags a kernel over an image. Kernel is a simple matrix with 1's and 0's in it. One of the elements of that matrix, usually centre one, is called an anchor. An anchor is the element that will be replaced at the end of the operation. When you created
cv2.getStructuringElement(cv2.MORPH_CROSS, (1, 1))
what you created is actually a 1x1 matrix (1 column, 1 row). This makes erosion completely useless.
What erosion does, is firstly retrieves all the values of pixel brightness from the original image where the kernel element, overlapping the image segment, has a "1". Then it finds a minimal value of retrieved pixels and replaces the anchor with that value.
What this means, in your case, is that you drag [1] matrix over the image, compare if the source image pixel brightness is larger, equal or smaller than itself and then you replace it with itself.
If your intention was to remove "noise", then it's probably better to use a rectangular kernel over the image. Think of it this way, "noise" is that thing that "doesn't fit in" with the surroundings. So if you compare your centre pixel with it's surroundings and you find it doesn't fit, it's most likely noise.
Additionally, I've said it replaces the anchor with the minimal value retrieved by the kernel. Numerically, minimal value is 0, which is coincidentally how black is represented in the image. This means that in your case of a predominantly white image, erosion would "bloat up" the black pixels. Erosion would replace the 255 valued white pixels with 0 valued black pixels if they're in the reach of the kernel. In any case it shouldn't be of a shape (1,1), ever.
>>> cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
array([[0, 1, 0],
[1, 1, 1],
[0, 1, 0]], dtype=uint8)
If we erode the second image with a 3x3 rectangular kernel we get the image bellow.
Ok, now we got that out of the way, next thing you do is you find edges using Canny edge detection. The image you get from that is:
Ok, now we look for EXACTLY vertical and EXACTLY horizontal lines ONLY. Of course there are no such lines apart from the meridian on the left of the image (is that what it's called?) and the end image you get after you did it right would be this:
Now since you never described your exact idea, and my best guess is that you want the parallels and meridians, you'll have more luck on maps with lesser scale because those aren't lines to begin with, they are curves. Additionally, is there a specific reason to get a Probability Hough done? The "regular" Hough doesn't suffice?
Sorry for the too-long post, hope it helps a bit.
Text here was added as a request for clarification from the OP Nov. 24th. because there's no way to fit the answer into a char limited comment.
I'd suggest OP asks a new question more specific to the detection of curves because you are dealing with curves op, not horizontal and vertical lines.
There are several ways to detect curves but none of them are easy. In the order of simplest-to-implement to hardest:
Use RANSAC algorithm. Develop a formula describing the nature of the long. and lat. lines depending on the map in question. I.e. latitude curves will almost be a perfect straight lines on the map when you're near the equator, with the equator being the perfectly straight line, but will be very curved, resembling circle segments, when you're at high latitudes (near the poles). SciPy already has RANSAC implemented as a class all you have to do is find and the programatically define the model you want to try to fit to the curves. Of course there's the ever-usefull 4dummies text here. This is the easiest because all you have to do is the math.
A bit harder to do would be to create a rectangular grid and then try to use cv findHomography to warp the grid into place on the image. For various geometric transformations you can do to the grid you can check out OpenCv manual. This is sort of a hack-ish approach and might work worse than 1. because it depends on the fact that you can re-create a grid with enough details and objects on it that cv can identify the structures on the image you're trying to warp it to. This one requires you to do similar math to 1. and just a bit of coding to compose the end solution out of several different functions.
To actually do it. There are mathematically neat ways of describing curves as a list of tangent lines on the curve. You can try to fit a bunch of shorter HoughLines to your image or image segment and then try to group all found lines and determine, by assuming that they're tangents to a curve, if they really follow a curve of the desired shape or are they random. See this paper on this matter. Out of all approaches this one is the hardest because it requires a quite a bit of solo-coding and some math about the method.
There could be easier ways, I've never actually had to deal with curve detection before. Maybe there are tricks to do it easier, I don't know. If you ask a new question, one that hasn't been closed as an answer already you might have more people notice it. Do make sure to ask a full and complete question on the exact topic you're interested in. People won't usually spend so much time writing on such a broad topic.
To show you what you can do with just Hough transform check out bellow:
import cv2
import numpy as np
def draw_lines(hough, image, nlines):
n_x, n_y=image.shape
#convert to color image so that you can see the lines
draw_im = cv2.cvtColor(image, cv2.COLOR_GRAY2BGR)
for (rho, theta) in hough[0][:nlines]:
try:
x0 = np.cos(theta)*rho
y0 = np.sin(theta)*rho
pt1 = ( int(x0 + (n_x+n_y)*(-np.sin(theta))),
int(y0 + (n_x+n_y)*np.cos(theta)) )
pt2 = ( int(x0 - (n_x+n_y)*(-np.sin(theta))),
int(y0 - (n_x+n_y)*np.cos(theta)) )
alph = np.arctan( (pt2[1]-pt1[1])/( pt2[0]-pt1[0]) )
alphdeg = alph*180/np.pi
#OpenCv uses weird angle system, see: http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html
if abs( np.cos( alph - 180 )) > 0.8: #0.995:
cv2.line(draw_im, pt1, pt2, (255,0,0), 2)
if rho>0 and abs( np.cos( alphdeg - 90)) > 0.7:
cv2.line(draw_im, pt1, pt2, (0,0,255), 2)
except:
pass
cv2.imwrite("/home/dino/Desktop/3HoughLines.png", draw_im,
[cv2.IMWRITE_PNG_COMPRESSION, 12])
img = cv2.imread('a.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
flag,b = cv2.threshold(gray,160,255,cv2.THRESH_BINARY)
cv2.imwrite("1tresh.jpg", b)
element = np.ones((3,3))
b = cv2.erode(b,element)
cv2.imwrite("2erodedtresh.jpg", b)
edges = cv2.Canny(b,10,100,apertureSize = 3)
cv2.imwrite("3Canny.jpg", edges)
hough = cv2.HoughLines(edges, 1, np.pi/180, 200)
draw_lines(hough, b, 100)
As you can see from the image bellow, straight lines are only longitudes. Latitudes are not as straight therefore for each latitude you have several detected lines that behave like tangents on the line. Blue drawn lines are drawn by the if abs( np.cos( alph - 180 )) > 0.8: while the red drawn lines are drawn by rho>0 and abs( np.cos( alphdeg - 90)) > 0.7 condition. Pay close attention when comparing the original image with the image with lines drawn on it. The resemblance is uncanny (heh, get it?) but because they're not lines a lot of it only looks like junk. (especially that highest detected latitude line that seems like it's too "angled" but in reality those lines make a perfect tangent to the latitude line on its thickest point, just as hough algorithm demands it). Acknowledge that there are limitations to detecting curves with a line detection algorithm
Problem
I am working on a project where I need to get the bounding boxes of dumbell like shapes. However, I need the fewest points possible, and the boxes need to fit the shapes at all corners. Here's an Image I made to test: Blurry, cracked, dumbell shape
I don't care about the gaps going into the shape, I just want to clean it up, and straighten the edges so that I can get the contours of a shape like this: Cleaned up
I have been attempting to threshold() it out, getting the contours of it using findContours() and then using approxPolyDP() to simplify the crazy amount of points the contours end up being. So, after fiddling with this for about three days now, how can I simply get either:
Two boxes specifying the ends of the dumbell and a rectangle in the middle, or
One contour with the twelve points for all the corners
The second option would be preferred since that really is my ultimate goal: getting the points that are at those corners.
A few things to note:
I am using OpenCV for Python
There will generally be many of these shapes of all sizes all over the input image
They will have only horizontal or vertical positioning. No strange 27 degree angles...
What I need:
I really don't need someone to write the code for me, I just need some method or algorithm in order to get this done, preferably with some simple examples.
My Code
Here is my overly clean code with functions I don't even use but figure I would use them eventually:
import cv2
import numpy as np
class traceImage():
def __init__(self, imageLocation):
self.threshNum = 127
self.im = cv2.imread(imageLocation)
self.imOrig = self.im
self.imGray = cv2.cvtColor(self.im, cv2.COLOR_BGR2GRAY)
self.ret, self.imThresh = cv2.threshold(self.imGray, self.threshNum, 255, 0)
self.contours, self.hierarchy = cv2.findContours(self.imThresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
def createGray(self):
self.imGray = cv2.cvtColor(self.im, cv2.COLOR_BGR2GRAY)
def adjustThresh(self, threshNum):
self.ret, self.imThresh = cv2.threshold(self.imGray, threshNum, 255, 0)
def getContours(self):
self.contours, self.hierarchy = cv2.findContours(self.imThresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
def approximatePoly(self, percent):
i=0
for shape in self.contours:
shape = cv2.approxPolyDP(shape, percent*cv2.arcLength(shape, True), True)
self.contours[i] = shape
i+=1
def drawContours(self, blobWidth, color=(255,255,255)):
cv2.drawContours(self.im, self.contours, -1, color, blobWidth)
def newWindow(self, name):
cv2.namedWindow(name)
def showImage(self, window):
cv2.imshow(window, self.im)
def display(self):
while True:
cv2.waitKey()
def displayUntil(self, key):
while True:
pressed = cv2.waitKey()
if pressed == key:
break
if __name__ == "__main__":
blobWidth = 30
ti = traceImage("dumbell.png")
ti.approximatePoly(0.01)
for thresh in range(127,256):
ti.adjustThresh(thresh)
ti.getContours()
ti.drawContours(blobWidth)
ti.showImage("Image")
ti.displayUntil(10)
ti.createGray()
ti.adjustThresh(127)
ti.getContours()
ti.approximatePoly(0.0099)
ti.drawContours(2, (0,255,0))
ti.showImage("Image")
ti.display()
Code Explanation
I know I might not be doing some things right here, but hey, I'm proud of it :)
So, the idea is that there are very often holes and gaps in these dumbells and so I figured that if I iterated through all the threshold values from 127 to 255 and drew the contours onto the image with large enough thickness, the thickness of drawing the contours would fill in any small enough holes, and I could use the new, blobby image to get the edges and then scale the sides back down to size. That was my thinking. There's got to be another, beter way though...
Summary
I want to end up with 12 points; one for each corner of the shape.
EDIT:
After trying out some erosion and dilation, it seems that the best option would be to slice the contours at certain points and then use bounding boxes around the sliced shapes to get the right boxy corners, and then doing some calculations to rejoin the boxes into one shape. A rather interesting challenge...
EDIT 2:
I discovered something that works well! I made my own line detection system, that only detects horizontal or vertical lines, and then on a detected line/contour edge, the program draws a black line that extends across the whole image, thus effectively slicing the image at the straight lines of the contours. Once it does that, it gets new contours of the sliced up boxes, draws bounding boxes around the pieces and then uses dilation to close the gaps. So far, it works well on large shapes, but when the shapes are small, it tends to lose a bit of the shape.
So, after fiddling with erosion, dilation, closing, opening, and looking at straight contours, I have figured out a solution that works. Thank you #Ante and #a.alsram! Your two ideas combined helped me get to my solution. So here's how it works.
Method
The program iterates over each contour, and over every pair of points in the contour, looking for point pairs that lie on the same axis and calculating the distance between them. If the distance is greater than an adjustable threshold, the program decides that those points are considered an edge on the shape. Then the program uses that edge, and draws a black line along the whole contour, thus cutting the contour at that edge. Then the program redetermines contours and since the shape was cut. These pieces that were cut off are know their own contours, which then are bounded by bounding boxes. and finally, all shapes are dilated and eroded (close) to rejoin the boxes that were cut off.
This method can be done several times, but each time there is a little bit of accuracy loss. But it works for what I need and certainly was a fun challenge! Thanks for your help guys!
natebot13
Maybe simple solution can help. If there is a threshold length to close a gaps,
it is possible to split image in a grid with cell lengths >= threshold, and use
cells that have something inside. With that there will be only horizontal and
vertical lines, and by taking a care about grid to follow original horizontal
and vertical lines it will cover main line features.
Update
Take a look on mathematical morphology. I think closing operation with structuring element (2*k+1)x(2*k+1) pixels can do what you are looking for.
Algorithm should take threshold parameter k, and performs dilation and than erosion. That means change image so that for each white pixel set all neighbours on distance k ((2*k+1)x(2*k+1) box) to the white, and than change image so that for each black pixel set neighbours on distance k to the black.
It is enough to do operations on boundary pixels.