We Keep Coding

OpenCV: Projecting 3D points from Homography Matrix

I am trying to project points on an image after estimating its pose as shown in this OpenCV tutorial, but for any textured "base image" instead of a chessboard image.
To match the base image with a frame from my webcam, I'm using ORB and FLANN and it is working perfectly. The 2D border around the image is correctly rendered with cv2.findHomography and cv2.perspectiveTransform(border_points, homography_matrix) to map the points and cv2.polylines to draw them.
Now I should be able to draw 3D axes on the image's surface using the homography matrix, but I'm stuck not knowing how I would do it nor finding how to do it anywhere. What am I missing here?
# target_pts and webcam_pts are arrays of 2D points generated by ORB
homography_matrix, _ = cv2.findHomography(target_pts, webcam_pts, cv2.RANSAC, 5.0)
_, rvecs, tvecs, _ = cv2.decomposeHomographyMat(homography_matrix, calibration_matrix)
axis = np.float32([[3,0,0], [0,3,0], [0,0,-3]]).reshape(-1,3)
# TODO: cv2.Rodrigues() for rvecs ?
axis_points, _ = cv2.projectPoints(axis, rvecs[0], tvecs[0], calibration_matrix, calibration_coefficients)
# Draw axis_points on image
# ...

Keep imgae resolution after undistortion process - camera calibration

I am working on camera calibration using opencv in python. I’ve already done calibration using cv2.calibrateCamera and got the camera matrix and distortion coefficients. I also have evaluated the validity of camera matrix; in other words, the estimated focal lens is very close to the sensor’s focal lens in the datasheet (I know the pixel size and the focal lens in mm from the datasheet). I should mention that in order to undistort new images, I follow the instructions below; as I NEED to keep all source pixels in the undistorted images.
alpha = 1. # to keep all pixels
scale = 1. # to change output image size
w,h = 200,200 # original image size captured by camera
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(camera_matrix, dist_coefs, (w,h), alpha, (int(scale2*w), int(scale2*h))))
mapx, mapy = cv2.initUndistortRectifyMap(camera_matrix, dist_coefs, None, newcameramtx, (w,h), 5)
dst = cv2.remap(img, mapx, mapy, cv2.INTER_CUBIC)
x_, y_, w_, h_ = roi
dst_cropped = dst[y_:y_+h_, x_:x_+w_]
And now the issues and my questions:
The source images are suffering high positive radial distortions, and dst images resulted from undistorsion process are satisfying and seems the positive radial distortion is already canceled, at lease visually. Because of alpha = 1. I also have all source pixels in the dst image. However, the roi is really small and it crops a region in the middle of imae. I could say that dst_cropped only contains the pixels close to the center of dst. According to the links below:
cv2.getOptimalNewCameraMatrix returns ROI of [0,0,0,0] on some data sets
https://answers.opencv.org/question/28438/undistortion-at-far-edges-of-image/?answer=180493#post-id-180493
I found that the probable issue might be because of my dataset; then I tried to balance the dataset to have more images having chessboard close to the image boundaries. I repeated the calibration and the obtained results are very close to the first trial, however still the same effect is presented in the dst_cropped images. I tried to play with alpha parameter as well, but any number less than 1. does not keep all source pixels in dst image. Considering all above information, it seems that I'm obliged to keep using dst images instead of dst_cropped ones; then another issue arises from dst size which is the same as source image (w,h). It is clear that because of alpha=1. the dst contains all source pixels as well as zero pixels, but my question is that how can keep the resolution as before. If I don't make a mistake, seems all points are mapped and then the resulted image is scaled down to fit (w,h). Then, my question is that how can I force the calibration to KEEP resolution as before? For example, if some points are mappted to (-100,-100) or (300,300) the dst should be [400,400] and not [200,200]. How to expand images instead of scaling down?
Thanks in advance for your helps or advice,

Fisheye calibration OpenCV Python

I am using a fisheye camera and I would like to calibrate it and correct its barrel distortion using OpenCV or any other library in Python. I have tried different methods using the classic chessboard pattern through different images.
I have been following this approach:
# Prepare object points, like (0,0,0), (1,0,0), (2,0,0), ...,(7,5,0)
object_p = np.zeros((self.__ny*self.__nx,3),np.float32)
object_p[:,:2] = np.mgrid[0:self.__nx,0:self.__ny].T.reshape(-1,2) # s,y coordinates
for image_filename in calibration_filenames:
# Read in each image
image = cv2.imread(image_filename)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # RGB is standard in matlibplot
# Convert to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
# Find the chessboard corners
ret, corners = cv2.findChessboardCorners(gray, (self.__nx, self.__ny), None)
# If found, draw corners
if ret == True:
# Store the corners found in the current image
object_points.append(object_p) # how it should look like
image_points.append(corners) # how it looks like
# Draw and display the corners
cv2.drawChessboardCorners(image, (self.__nx, self.__ny), corners, ret)
plt.figure()
plt.imshow(image)
plt.show()
# Do the calibration
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(object_points, image_points, shape, None, None)
And then this to correct my image:
cv2.undistort(image, mtx, dist, None, mtx)
The main problem is that, though correction performs good with some images, is very susceptible to camera location. A variation of cm in the camera will result in very different corrections (see below).
Any idea of how could I control this zoom effect to show maintain the grid area always in my corrected image?
I am aware that this question may be similar to the one in here.

Your calibration is poorly estimated: as you can see, straight lines in the scene grid are not transformed to straight lines in the (allegedly) undistorted image.
Once you have a better calibration, you can compute (using the inverse mapping) the location of the 4 sides of the undistorted image in the original ones. These define your usable image boundaries, and can guide the placement of the camera with respect to the scene.

Classification on the basis of shape and size

How to classify objects on the basis of shape and size using machine learning?
Say I've an circle and some small dotted squares in an image. The difference between the two is their shape and size. So given an image, how to distinguish between these objects and return the result.
In the actual problem those objects are hot spots in a solar PV folder, which are defected parts of it. I need to classify them.The I/P image is as:
[This is less or more a square type of hot spot:]
https://i.stack.imgur.com/4JL7E.png

This answer doesn't ellaborate machine learning or any approach using classifiers
The circles can be detected by the Hough Circle
Transform
from OpenCV cv2.HoughCircles():
Documentation for Hough Circles in OpenCV
Note: By using the radius you can tune the shape size of the circles you want to detect. And to be honest I didn't really get what dotted squares are, maybe you could show an exemplary image in your question.
If there are only two different kinds of objects in the image you probably don't even need a classifier, because the two classes are already being separated by the subsequent image processing (that though depends highly on your input images).
import cv2
import numpy as np
img = cv2.imread('opencv_logo.png',0)
img = cv2.medianBlur(img,5)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
circles = cv2.HoughCircles(img,cv2.HOUGH_GRADIENT,1,20,
param1=50,param2=30,minRadius=0,maxRadius=0)
circles = np.uint16(np.around(circles))
for i in circles[0,:]:
# draw the outer circle
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),2)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result of the code is as follows

Obtaining the equation of the straight line projecting onto a given pixel in an image with opencv

I am trying to use Opencv with Python 2.7 to obtain world coordinate informations from images. I correctly performed the calibration of the camera (see description below), and obtained the camera pose in world coordinate, including the coordinates of the (virtual) pinhole of the camera. What I now need is a function f(pixel) which takes a pixel [x,y] of the image as an input argument, and returns the equation of the straight line (in world coordinates) that projects onto the given pixel (it is the same as finding the world coordinates of a point on the straight line, since I already have the coordinates of the pinhole). Help will be greatly appreciated. Below are the description of the steps I already performed:
EDIT:
This question may appear to be a duplicate, but it is a fact that I haven't seen a detailed solution of this problem in this forum.
If my understanding is good, the camera axes system is parallel to the axes of the image, and its third axis is pointing toward the view of the camera. So, it is clear that the problem amounts to the following question, interesting by itself:
Where is located the centre of the image with respect to the camera axes system (i.e need one translation vector), and what is the physical length of one pixel ?
STEPS PERFORMED:
Playing with a chess board in front of the camera, I first obtained the intrinsic parameters matrix, and the distortion parameters (see the tutorial in opencv-python):
(ret, mtx, dist, rvecs, tvecs) = cv2.calibrateCamera(objpoints, imgpoints, grayImg.shape[::-1],None,None)
Then I obtained the camera pause (extrinsic parameter matrix) from one image of the chess board, using the following piece of code:
[ret, corners] = cv2.findChessboardCorners(grayImg, [7,7], None)
corners2 = cv2.cornerSubPix(grayImg, corners, (11,11), (-1,-1), criteria)
[ret, rvec, tvec] = cv2.solvePnP(objp, corners2, mtx, dist)
Rmat = cv2.Rodrigues(rvec)[0] #rotation matrix of the camera pause
cameraPinhole = np.dot(-Rmat.T, tvec) #location of the virtual pinhole of the camera with respect to world coordinates
Then I obtained the projection of the world coordinate axes (located at a corner of the chess board) on the image, and use them to plot theses axes on the image:
axes = chess_board_square_sz * np.float32([[3,0,0], [0,3,0], [0,0,-3]]).reshape(-1,3)
[axes_proj, jac] = cv2.projectPoints(axes, rvec, tvec, newcameramtx, dist)
img = draw_axes(img, corners2, axes_coor)
What should be the next steps ?