I have a camera with width and height resolution and camera parameters:
[[position_x, position_y, position_z],
[focal_point_x, focal_point_y, focal_point_z],
[view_up_x, view_up_y, view_up_z]]
Say I was given a point of the camera image plane with coordinates x,y ( 0,0 top left corner width-1, height-1 bottom right corner).
How to get world coordinates of my point x,y from such Camera, Image data?
I'll try to bring some highlights that I found in a old book (reference at end of this answer).
First of all, you can't map directly z-axis of the world coordinate from your (x,y) image coordinate, because it's a one to many problem: one pixel point can be at any z-axis line point. However there's a lot of studies and algorithm about estimating this z-axis with just one image perspective and others using two image perspectives (stereo vision).
But if you already have the word coordinate Z axis, you can compute the the X and Y using the above formula.
where:
(x,y) = image coordinates of some point
(X0,Y0,Z0) = camera word coordinates
(X,Y,Z) = word coordinates of some point
α = angle between x and X axis
θ = angle between z and Z axis
λ = focal lenght
I'm not an expert in robotics, but maybe this can clarify your path. I use the images from below book (page 313).
Reference book: FU, K. S.; GONZALEZ, R. C.; LEE, C. S. G.. Robotics: control, sensing, vision and intelligence. [s. L.]: Mcgraw - Hill, 1987.
Related
Im currently working on a robot application, where the robot should be "controlled" by the Azure Kinect.
Goal is to take a picture of the scene, then I want the robot to move to the position that I clicked in the picture.
Therefore i have to transform the image coordinates (u,v) to the robots base frame (x,y,z).
The camera was calibrated using OpenCV and Python, so that the camera matrix and the distortion coefficient are available. The camera will be static and won't move.
The pixel coordinates are gained from the transformed color depth image (i.e. the color image is transformed to the geometry of the depth camera). The calibration was also done with that images.
With given Depth-Map I was following this approach and calculated the coordinates:
z = depth_image[int(v), int(u)] #get depth value from u,v coordinates
x = z / fx * (u - cx)
y = z / fy * (v - cy)
The calculated values are w.r.t. the camera coordinate system.
I marked the cx and cy value in the image with OpenCV and w.r.t. that i measured the distance from that point to the origin of my world coordinate system.
I added these distances as an offset to the previous calculated x and y value.
(Is that right?)
Another approach was following this topic. It worked pretty well but only when the Z-Values of the objectPoints were really close together or were the same. There are deviations from 10 up to 30mm.
Following this tutorial i wasn't able to use it with the depth camera but it worked quite good with the color camera for x and y values, not for the z values.
Are there other methods to solve the problem or am I on the right track using these methods?
Maybe there is a way to multiply the u,v coordinates with a rotation and translation vector/matrices?
Should I use the transofrmed color to depth image as now, or it is bette to use only depth or color image?
I've extracted a Circle shaped mask from an image in OpenCV. I used the following code for the same:
H, W = img.shape
x, y = np.meshgrid(np.arange(W), np.arange(H))**
d2 = (x - xc)**2 + (y - yc)**2**
mask = d2 < r **2**
And, used the mask value to find the average color outside the circle.
outside = np.ma.masked_where(mask, img)**
average_color = outside.mean()**
I want to extract an Ellipse from an image in the same above process in OpenCV Python.
Thank You.
Drawing Ellipse
To draw the ellipse, we need to pass several arguments. One argument is the center location (x,y). Next argument is axes lengths (major axis length, minor axis length). angle is the angle of rotation of ellipse in anti-clockwise direction. startAngle and endAngle denotes the starting and ending of ellipse arc measured in clockwise direction from major axis. i.e. giving values 0 and 360 gives the full ellipse. For more details, check the documentation of cv2.ellipse(). Below example draws a half ellipse at the center of the image.
cv2.ellipse(img,(256,256),(100,50),0,0,180,255,-1)
Taken from Miki's Link in the Question Comments
Can anyone please explain if it is possible, and if so how, to work with cv2.getPerspectiveTransform().
I have 3d information about my image: I know the length of a,b and also the Different heights of c,d,e,f and g. I made the height different to get more 3d information but if it isn't needed that will be preferable.
Ultimately I need to know where the pink dot really is in the rectangle after implementing the transform on my [x,y] position I get from the camera feed.
If you denote by C,D,E,F the positions of the four corners of the black polygon in the original image (each of them is a 2D point), and C',D',E',F' the positions of the corresponding points in your target image (probably (0,0), (a, 0), (a, b), (0, b)), M = cv2.getPerspectiveTransform({C,D,E,F}, {C',D',E',F'}) is the perspective transformation from one polygon to the other.
Given the position G of the vertical projection of g onto the black polygon in the original image, you can compute its position in the target image as cv2.transform(G, M). This will return a point (x,y,z), where the last coordinate z is a normalizing term. This z is zero when your point would be "at infinity" in the target image. If z is not zero, the point you are looking for is (x/z, y/z).
If z is zero, your point is at infinity, in the direction of the support of vector (x, y) (think of the case where G would be at the intersection of the supporting lines of two opposite sides of the black polygon in the source image).
If you know that the heights of c,d,e,f,g are equal, these points are also coplanar, and the exact same method applies to c,d,e,f,g instead of C,D,E,F,G.
I am trying to rotate a vtk camera around its focal point. The aim being to 'orbit' the model.
I'm using the camera.SetPosiiton(x, y, z) call to set the camera location, and I know I can do the same at each update period in my render window.
The focal point has the location (0, 0, 0), and some other bounding box getting gives me my initial camera (x, y, z) location. The distance from the focal point (0, 0, 0) to the camera location 9x, y, z) describes the radius the of the sphere.
In my head, this essentially moving the camera in steps around the point (0, 0, 0) and I am presuming there is a maths function I could use to feed it my starting camera point, and work out my next camera location.
This should result in the model appearing to spin in space. My camera view is offset from all x, y, z, planes, making it a 3d problem, not a 2d problem. However, I do want my camera to remain the same distance from the model (focal point)
What I am trying to achieve is like this:- take a pencil (my model is long and narrow). Hold it in your finger tips at arms length, tip pointing to the ceiling. Tilt the pencil by ~30 degrees. This is the camera start position. Rotate the pencil body in your fingers, maintaining tilt angle, and the distance from your eye.
THis post looks helpful: Plotting a point on the edge of a sphere however, this assumes you know the radius to get to the initial x, y location.
Could some one point me towards the maths I need to do this, my maths is horribly rusty.
It seems what you want is to rotate a vector about an axis, this can be most easily done using a rotation matrix
So, if your desired axis of rotation is tilted 30 degrees from the z axis on the zx plane, your axis of rotation is (cos(pi/6),0,sin(pi/6)), increment the rotation angle, plug that into the rotation matrix to get matrix R, the new camera position vector will be R*(x,y,z)'
Start off with the points (+-1,0,0), (0,+-1,0), (0,0,+-1). These form two Pyramids with all the points on the unit sphere.
Now you can take the midpoints of each triangle, and project out so they lie on the unit sphere too. For each triangle this now gives you 3 new triangles, and you can repeat the process.
The alternative to the midpoint of the triangle is to take the midpoints of each side, and join them up. That gives 3 new points that can be projected out to the unit circle. This gives you 4 triangles for each sub division.
Repeat as many times as you need.
I have been working around extracting the time series from shapes based on distances to center of mass clockwise starting from angle 0 to 360.
My Implementation that arranges contour points based on their angle to the [1,0], vector might be good for some shapes but is not useful for shapes that has much articulation. Consider the following code:
im = Image.open(os.path.join(path,filename))
im = im.filter(ifilter.MedianFilter)
contim = im.filter(ifilter.CONTOUR)
contim = contim[1:-1,1:-1] # this is because borders are extracted here as contours
contpts = np.where(contim ==0)
contpts = np.vstack((contpts[0],contpts[1])) # Just need to arrange these points clockwise with respect to the center of mass of the shape
Can anyone direct me to how I can extract that feature from any shape where I can start from a point and keep going along the contour to extract all the distances to the center of mass of the shape.
For more information about the feature, please view this paper: "LB_Keogh Supports Exact Indexing of Shapes under Rotation Invariance with Arbitrary Representations and Distance Measures"
If I understood, there's a geometrical figure in a discretized plane, represented as a matrix. If the entry is 1, you're inside the figure. If it's 0, you're outside. He wants to determine de distance between the edge of the figure and the center of the figure for all points in the edge. He parametrized it with a polar coordinate system. The center of the figure is the origin and now he wants to get the distance to the border as a function of the angle. This is what he calls his "time series".
Is this correct?
If yes, couldn't you just:
1. determine the center of mass,
2. reposition the origin to match the center of mass.
3. start angle at 0
4. r = 0
5. for each angle in [0,1,...,360]
1. If you're in inside the figure, increase r until you reach the border.
2. If you're outside the figure, decrease r until you reach the border.
3. When you reach the border, d(angle) = r
It the figure have a more or less continuous border, this will follow the contour.
Would this work?