We Keep Coding

Keep imgae resolution after undistortion process - camera calibration

I am working on camera calibration using opencv in python. I’ve already done calibration using cv2.calibrateCamera and got the camera matrix and distortion coefficients. I also have evaluated the validity of camera matrix; in other words, the estimated focal lens is very close to the sensor’s focal lens in the datasheet (I know the pixel size and the focal lens in mm from the datasheet). I should mention that in order to undistort new images, I follow the instructions below; as I NEED to keep all source pixels in the undistorted images.
alpha = 1. # to keep all pixels
scale = 1. # to change output image size
w,h = 200,200 # original image size captured by camera
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(camera_matrix, dist_coefs, (w,h), alpha, (int(scale2*w), int(scale2*h))))
mapx, mapy = cv2.initUndistortRectifyMap(camera_matrix, dist_coefs, None, newcameramtx, (w,h), 5)
dst = cv2.remap(img, mapx, mapy, cv2.INTER_CUBIC)
x_, y_, w_, h_ = roi
dst_cropped = dst[y_:y_+h_, x_:x_+w_]
And now the issues and my questions:
The source images are suffering high positive radial distortions, and dst images resulted from undistorsion process are satisfying and seems the positive radial distortion is already canceled, at lease visually. Because of alpha = 1. I also have all source pixels in the dst image. However, the roi is really small and it crops a region in the middle of imae. I could say that dst_cropped only contains the pixels close to the center of dst. According to the links below:
cv2.getOptimalNewCameraMatrix returns ROI of [0,0,0,0] on some data sets
https://answers.opencv.org/question/28438/undistortion-at-far-edges-of-image/?answer=180493#post-id-180493
I found that the probable issue might be because of my dataset; then I tried to balance the dataset to have more images having chessboard close to the image boundaries. I repeated the calibration and the obtained results are very close to the first trial, however still the same effect is presented in the dst_cropped images. I tried to play with alpha parameter as well, but any number less than 1. does not keep all source pixels in dst image. Considering all above information, it seems that I'm obliged to keep using dst images instead of dst_cropped ones; then another issue arises from dst size which is the same as source image (w,h). It is clear that because of alpha=1. the dst contains all source pixels as well as zero pixels, but my question is that how can keep the resolution as before. If I don't make a mistake, seems all points are mapped and then the resulted image is scaled down to fit (w,h). Then, my question is that how can I force the calibration to KEEP resolution as before? For example, if some points are mappted to (-100,-100) or (300,300) the dst should be [400,400] and not [200,200]. How to expand images instead of scaling down?
Thanks in advance for your helps or advice,

homography orientation wrong

im trying to use opencv's cv.findChessboardCorners, cv.findHomographyand cv.warpPerspective to find a chessboard in my video feed and display it with the original perspective.
My problem is that the result is sometimes inverted. the chessboard is displayed correctly and the perspective is also correct, but the homography is upside down (you can see this in the 2. image, the color of the drawn keypoints are not correct)
I think this is because the chessboard has no orientation, i.e its the same if i flip it.But i don't know how to fix that since it seems inherent for a chessboard.
Why is this happening (seemingly at "random")?
Is it possible to set the orientation of the
warpedPerspective, so that its always "one way"?
What can i do to make the homography always register in one direction? (i can assume that its never actually going to be upside down in the actual camera feed)
the first one is correct, second one is moved slightly and the orientation is flipped
Here is a part of the code i wrote:
# checkerboard is a uint8 array that i created, corner_count is the tuple with patternSize
_, self.cb_corners = cv.findChessboardCorners(checkerboard, corner_count)
found_board, corners = cv.findChessboardCorners(
image=frame,
patternSize=self.corner_count,
flags=cv.CALIB_CB_FAST_CHECK +
cv.CALIB_CB_NORMALIZE_IMAGE +
cv.CALIB_CB_ADAPTIVE_THRESH)
if found_board:
retval, mask = cv.findHomography(self.cb_corners, corners)
dst = cv.warpPerspective(frame, self.homography, (1920, 1080))
cv.imshow("dst", dst)

How to extract circular text from embossed object

I have an object, there are 2 code on it. text printed on it. The text is curve. half of text is in the top side, and another half is in bottom side of object. Here is my sample image
I am using OPENCV, and Deep learning approaches and tessract to OCR it's code.
I logical approach(not Deep approach) I first used HoughCircles() andlogPloar() to align text in line then used tessract such this example sample code. But because of distortion in aligned text, tesseract fail to OCR it's text.
In Deep approach I cant find fine a optimum solution for curve text OCR in tensorflow or torch. There are many sources for text detection not recognition.
Regards,John

why not transform the circular text to linear? Similar to this De-skew characters in binary image just a bit more complicated. So detect (or manually select) the center of circle and convert the image to unrotated one ...
So create new image that has dimensions 6.28*max_radius , 2*max_radius and copy pixels using polar unwraping ... simply convert target pixel position into polar coordinates and convert that to Cartesian source pixel position.
I do not code in Python nor OpenCV but here is a simple C++ example of this:
//---------------------------------------------------------------------------
picture pic0,pic1; // pic0 - original input image,pic1 output
//---------------------------------------------------------------------------
void ExtractCircularText(int x0,int y0) // pic0 -> pic1 center = (x0,y0)
{
int x,y,xx,yy,RR;
float fx,fy,r,a,R;
// resize target image
x= -x0; y= -y0; a=sqrt((x*x)+(y*y)); R=a;
x=pic0.xs-x0; y= -y0; a=sqrt((x*x)+(y*y)); if (R<a) R=a;
x= -x0; y=pic0.ys-y0; a=sqrt((x*x)+(y*y)); if (R<a) R=a;
x=pic0.xs-x0; y=pic0.ys-y0; a=sqrt((x*x)+(y*y)); if (R<a) R=a;
R=ceil(R); RR=R;
pic1.resize((628*RR)/100,RR<<1);
for (yy=0;yy<pic1.ys;yy++)
for (xx=0;xx<pic1.xs;xx++)
{
// pic1 position xx,yy -> polar coordinates a,r
a=xx; a/=R; r=yy;
// a,r -> pic0 position
fx=r*cos(a); x=x0+fx;
fy=r*sin(a); y=y0+fy;
// copy pixel
if ((x>=0)&&(x<pic0.xs))
if ((y>=0)&&(y<pic0.ys))
{
pic1.p[ yy][pic1.xs-1-xx]=pic0.p[y][x]; // 2 mirrors as the text is not uniformly oriented
pic1.p[pic1.ys-1-yy][ xx]=pic0.p[y][x];
}
}
pic1.save("out.png");
}
//---------------------------------------------------------------------------
I use my own picture class for images so some members are:
xs,ys is size of image in pixels
p[y][x].dd is pixel at (x,y) position as 32 bit integer type
clear(color) clears entire image with color
resize(xs,ys) resizes image to new resolution
And finally the resulting image:
I made a 2 copies of the un rotated image (hence 2*max_radius height) so I can copy image in 2 modes to made both orientations of the text readable (as they are mirrored to each other)
Text will be more straight if you chose the center (x0,y0)more precisely I did just click it by mouse on the center of the circle but I doubt the center of text has the same center as that circle/disc. After some clicking this is the best center I could found:
The result suggest that none of the two texts nor disc has the same center ...
The quality of input image is not good you should improve it before doing this (maybe even binarization is a good idea) also storing it as JPG is not a good idea as its lossy compression adding more noise to it. Take a look at these:
Enhancing dynamic range and normalizing illumination
OCR and character similarity
PS. The center could be computed geometrically from selected text (arc) simply find most distant points on it (edges) and point on the middle between them on the arc. From that you can compute arc center and radius... or even fit it ...

The black dot is a perfect feature for centering, and the polar unwarping seems to work fine, the deformation of the characters is negligible.
The failure of Tesserac might be explained by the low image quality (blur).

OpenCV Image Aligment using ORB

I need to precisely align two images. To do that I am using Enhanced Correlation Coefficient (ECC). Which gives me great results except for images that are rotated a lot. For example if the Reference image (base image) and tested image (that I want to align) are rotated by 90 degrees ECC method doesn't work which is right according to the documentation of findTransformECC() which says
Note that if images undergo strong displacements/rotations, an initial transformation that roughly aligns the images is necessary (e.g., a simple euclidean/similarity transform that allows for the images showing the same image content approximately).
So I have to use feature point based alignment method to do some rough alignment. I tried both SIFT and ORB and I am facing same problem with both. It works fine for some images and for others the resulting transformation is shifted or rotated on wrong side.
These are input images:
I thought that the problem is caused by wrong matches but if I use just 10 keypoints with smaller distance it seems to me that all of them are good matches(I exactly the same result when I use 100 keypoints)
This is the result of matching:
This is the result:
If you compare the rotated image it is shifted to the right and upside down.
What am I missing?
This is my code:
# Initiate detector
orb = cv2.ORB_create()
# find the keypoints with ORB
kp_base = orb.detect(base_gray, None)
kp_test = orb.detect(test_gray, None)
# compute the descriptors with ORB
kp_base, des_base = orb.compute(base_gray, kp_base)
kp_test, des_test = orb.compute(test_gray, kp_test)
# Debug print
base_keypoints = cv2.drawKeypoints(base_gray, kp_base, color=(0, 0, 255), flags=0, outImage=base_gray)
test_keypoints = cv2.drawKeypoints(test_gray, kp_test, color=(0, 0, 255), flags=0, outImage=test_gray)
output.debug_show("Base image keypoints",base_keypoints, debug_mode=debug_mode,fxy=fxy,waitkey=True)
output.debug_show("Test image keypoints",test_keypoints, debug_mode=debug_mode,fxy=fxy,waitkey=True)
# find matches
# create BFMatcher object
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
# Match descriptors.
matches = bf.match(des_base, des_test)
# Sort them in the order of their distance.
matches = sorted(matches, key=lambda x: x.distance)
# Debug print - Draw first 10 matches.
number_of_matches = 10
matches_img = cv2.drawMatches(base_gray, kp_base, test_gray, kp_test, matches[:number_of_matches], flags=2, outImg=base_gray)
output.debug_show("Matches", matches_img, debug_mode=debug_mode,fxy=fxy,waitkey=True)
# calculate transformation matrix
base_keypoints = np.float32([kp_base[m.queryIdx].pt for m in matches[:number_of_matches]]).reshape(-1, 1, 2)
test_keypoints = np.float32([kp_test[m.trainIdx].pt for m in matches[:number_of_matches]]).reshape(-1, 1, 2)
# Calculate Homography
h, status = cv2.findHomography(base_keypoints, test_keypoints)
# Warp source image to destination based on homography
im_out = cv2.warpPerspective(test_gray, h, (base_gray.shape[1], base_gray.shape[0]))
output.debug_show("After rotation", im_out, debug_mode=debug_mode, fxy=fxy)

The answer to this problem is both mundane and irritating. Assuming this is the same issue as what I've encountered (I think it is):
Problem and Explanation
Images are saved by most cameras with EXIF tags that include an "Orientation" value. Beginning with OpenCV 3.2, this orientation tag is automatically read-in when an image is loaded with cv.imread(), and the image is oriented based on the tag (there are 8 possible orientations, which include 90* rotations, mirroring and flipping). Some image viewing applications (such as Image Viewer in Linux Mint Cinnamon, and Adobe Photoshop) will display images rotated in the direction of the EXIF Orientation tag. Other applications (such as QGIS and OpenCV < 3.2) ignore the tag. If your Image 1 has an orientation tag, and Image 2 has an orientation tag, and you perform the alignment with ORB (I haven't tried SIFT for this) in OpenCV, your aligned Image 2 will appear with the correct orientation (that of Image 1) when opened in an application that reads the EXIF Orientation tag. However, if you open both images in an application that ignores the EXIF Orientation tag, then they will not appear to have the same orientation. This problem becomes even more pronounced when 1 image has an orientation tag and the other does not.
One Possible Solution
Remove the EXIF Orientation tags prior to reading the images into OpenCV. Now, as of OpenCV 3.4 (maybe 3.3?) there is an option to load the images ignoring the tag, but when this is done, they are loaded as grayscale (1 channel), which is not helpful if you NEED color cv.imread('image.jpg',128) where 128 means "ignore orientation). So, I use pyexiv2 in python to remove the offending EXIF Orientation tag from my images:
import pyexiv2
image = path_to_image
imageMetadata = pyexiv2.ImageMetadata(image)
imageMetadata.read()
try:
del imageMetadata['Exif.Image.Orientation']
imageMetadata.write()
except:
continue

Finding shapes in an image using opencv

I'm trying to look for shapes in an image using OpenCV. I know the shapes I want to match (there are some shapes I don't know about, but I don't need to find them) and their orientations. I don't know their sizes (scale) and locations.
My current approach:
Detect contours
For each contour, calculate the maximum bounding box
Match each bounding box to one of the known shapes separately. In my real project, I'm scaling the region to the template size and calculating differences in Sobel gradient, but for this demo, I'm just using the aspect ratio.
Where this approach comes undone is where shapes touch. The contour detection picks up the two adjacent shapes as a single contour (single bounding box). The matching step will then obviously fail.
Is there a way to modify my approach to handle adjacent shapes separately? Also, is there a better way to perform step 3?
For example: (Es colored green, Ys colored blue)
Failed case: (unknown shape in red)
Source code:
import cv
import sys
E = cv.LoadImage('e.png')
E_ratio = float(E.width)/E.height
Y = cv.LoadImage('y.png')
Y_ratio = float(Y.width)/Y.height
EPSILON = 0.1
im = cv.LoadImage(sys.argv[1], cv.CV_LOAD_IMAGE_GRAYSCALE)
storage = cv.CreateMemStorage(0)
seq = cv.FindContours(im, storage, cv.CV_RETR_EXTERNAL,
cv.CV_CHAIN_APPROX_SIMPLE)
regions = []
while seq:
pts = [ pt for pt in seq ]
x, y = zip(*pts)
min_x, min_y = min(x), min(y)
width, height = max(x) - min_x + 1, max(y) - min_y + 1
regions.append((min_x, min_y, width, height))
seq = seq.h_next()
rgb = cv.LoadImage(sys.argv[1], cv.CV_LOAD_IMAGE_COLOR)
for x,y,width,height in regions:
pt1 = x,y
pt2 = x+width,y+height
if abs(float(width)/height - E_ratio) < EPSILON:
color = (0,255,0,0)
elif abs(float(width)/height - Y_ratio) < EPSILON:
color = (255,0,0,0)
else:
color = (0,0,255,0)
cv.Rectangle(rgb, pt1, pt2, color, 2)
cv.ShowImage('rgb', rgb)
cv.WaitKey(0)
e.png:
y.png:
good:
bad:
Before anybody asks, no, I'm not trying to break a captcha :) OCR per se isn't really relevant here: the actual shapes in my real project aren't characters -- I'm just lazy, and characters are the easiest thing to draw (and still get detected by trivial methods).

As your shapes can vary in size and ratio, you should look at scaling invariant descriptors. A bunch of such descriptors would be perfect for your application.
Process those descriptors on your test template and then use some kind of simple classification to extract them. It should give pretty good results with simple shapes as you show.
I used Zernike and Hu moments in the past, the latter being the most famous. You can find an example of implementation here : http://www.lengrand.fr/2011/11/classification-hu-and-zernike-moments-matlab/.
Another thing : Given your problem, you should look at OCR technologies (stands for optical character recognition : http://en.wikipedia.org/wiki/Optical_character_recognition ;)).
Hope this helps a bit.
Julien

Have you try Chamfer Matching or contour matching (correspondence) using CCH as descriptor.
Chamfer matching is using distance transform of target image and template contour. not exactly scale invariant but fast.
The latter is rather slow, as the complexity is at least quadratic for bipartite matching problem. on the other hand, this method is invariant to scale, rotation, and probably local distortion (for approximate matching, which IMHO is good for the bad example above).