I have a point cloud (.ply) and a projection matrix,
I've rendered the view from the first camera using the projection matrix and got this result: (python & opencv)
This is the original view:
Question: How can I render only the points that are seen from the particular viewpoint of the camera, in order not to see the occluded points?
I thought about converting it to a mesh w/ some surface reconstruction algorithm, and working with the mesh, like generating an occlusion map. Any ideas?
Implicit Surface Octrees (https://www.cse.iitb.ac.in/~rhushabh/publications/icvgip10/icvgip10.pdf) can be used to reconstruct the surface and visualize point clouds. Recent advances in real-time point cloud rendering have been achieved with this method. An overview of developments in this area can be found in this article - https://trepo.tuni.fi/bitstream/handle/10024/117953/KiviPetrus.pdf?sequence=2&isAllowed=y. In it, you can also find other approaches to solving this problem.
After building the octree, you get the ability to drop non-rendered points and render the surface with texturing and shading.
An experimental method for drawing only points. Here I mean that you want to draw the frame once, so this method works asymptotically O (N) and in the worst case O (P * N), where P is the number of pixels on the screen (when the points are too far / close (depending from the implementation) and the rendering queue from far to near). To optimize and obtain stable asymptotics for some input data, it may be useful to sort by distance from the camera.
Convert the coordinates of the points to 2D screen space.
create Z-buffer
for each point
if the coordinate in Z-buffer is closer to the viewer than for this point - skip (continue)
draw a dot on the screen
instead of marking one pixel in the Z-buffer, draw a circle in it (possibly with a radial gradient) with a radius depending on the distance (something like a distance * eps, where eps - you can use the angle in radians between two projection points on the screen)
Profit!
Fast and easy, but I've never done that, so I don't know how well it works.
Translated by Google Translate
Related
I'd like to do calculations on the 3D positions on both end's of a rigid object (see spot where the children are usually sitting in image below). The geometrical situation of the rigid object corresponds to a seesaw. Rotation has to be possible on three axes and can be represented by a ball bearing, which initially is located at the middle of the rod.
The input to the desired function should consist of three rotations performed at the position of the ball bearing, three translations along the bearing and the initial 3D positions of both ends of the object.
The output needs to be the calculated new 3D positions of both ends.
Does anyone know a python library that does provide functionalities regarding this issue?
I've just found out that Open3D has implemented exactly what I was looking for. As it is working with point clouds, all that needs to be done is to create two points in 3D space, define a rotation matrix and the center (= ball bearing in this case). The function "rotate" then calculates the altered positions of the rotated points. This also works for translation.
# Rotation
pcd.rotate(r, center = (0,0,0))
# Translation
pcd.translate(t)
With r = rotation matrix (3x3) and t = translation matrix (3x1).
I have about 300,000 points defining my 3D surface. I would like to know if I dropped a infinitely stiff sheet onto my 3D surface, what the equation of that plane would be. I know I need to find the 3 points the sheet would rest on as that defines a plane, but I'm not sure how to find my 3 points out of the ~300,000. You can assume this 3D surface is very bumpy and that this sheet will most likely lie on 3 "hills".
Edit: Some more background knowledge. This is point cloud data for a scan of a 3D surface which is nearly flat. What I would like to know is how this object would rest if I flipped it over and put it on a completely flat surface. I realize that this surface may be able to rest on the table in various different ways depending on the density and thickness of the object but you can assume the number of ways is finite and I would like to know all of the different ways just in case.
Edit: After looking at some point cloud libraries I'm thinking of doing something like computing the curvature using a kd tree (using SciPy) and only looking at regions that have a negative curvature and then there should be 3+ regions with negative curvature so some combinatorics + iterations should give the correct 3 points for the plane(s).
I work on pose estimation of a 3d objects. I am using CAD model of that object to generate all the possible hypothesis of its pose.
I am using pyopengl to render the view of the object from a specific POV. Can anyone explain how to get a list of all the visible edges?
So I use face culling to eliminate the occluded faces, but I don't know how to pass the visible edges(indices and segments) to other python functions.
If there are any other approaches (not using OpenGL), I would really appreciate it.
So I want to get the drawn edges in the The rendered image:
I don't really want the image to be displayed.
In summary, I have a CAD model, and I want a function that can return the visible segments out of a specific POV.
Thanks
Face culling
This works only for single convex strict winding rule mesh without holes!
The idea is that sign of dot product of 2 vectors will tell you if the vectors are opposite or not. So if we have a normal pointing out and view direction their dot should be negative for faces turned towards camera/viewer.
As you do not want to render just select visible planar faces/edges you can do this on CPU side entirely. What you need is to have your mesh in form of planar faces (does not matter if triangles,quads or whatever) so let assume triangles (for more points you just add them to _face but for computation still use only v0,v1,v2) ... Each face should have the vertexes and normal.
struct _face
{
double v0[3],v1[3],v2[3],n[3];
};
List<_face> mesh;
Now the vertexes v0,v1,v2 you already have. All of them should be ordered in strict winding rule. That means if you look at any face from outside the points should form only CW (clockwise) loop (or only CCW (counter-clockwise) loop). To compute normal you simply exploit cross product which returns vector perpendicular to both operands:
n = cross(v1-v0,v2-v1) // cross product
n = n / |n| // optional normalize to unit vector
If you need the vector math see
Understanding 4x4 homogenous transform matrices
On the bottom is how to compute this... Also the whole answer you will need for the camera direction so read it...
Now if your mesh has strict winding rule than all the computed normals are pointing out of mesh (or inwards depends on your coordinate system, CW/CCW and order of operands in cross product). Let assume they all pointing out (if not just negate normal).
In case you do not have strict winding rule compute avg point of your mesh (sum all vertexes and divide by their count) this will be the center c of your object. Now just compute
dot(n,(v0+v1+v2)/3 - c)
and if not positive negate the n. This will repair your normals (you can also reverse the v0,v1,v2 to repair the mesh.
Now the camera and mesh usually has its own 4x4 transform matrix. one transfroms from mesh LCS (local coordinate system) to GCS ("world" global coordinate system) and the other from GCS to camera LCS (screen). We do not need projections for this as we do not render ... So what we need to do for each face is:
convert n to GCS
compute dot(n,camera_view_direction)
where camera_view_direction is GCS vector pointing in view direction. You can take it from direct camera matrix directly. It is usually the Z axis vector (in OpenGL Perspective view it is -Z). Beware camera matrix used for rendering is inverse matrix so if the case either compute inverse first or transpose it as we do not need the offset anyway ...
decide if face visible from the sign of #2
Again all the math is explained in the link above...
In case you do not have mesh matrix (does not have changing position or orientation) you can assume its matrix is unit one which means GCS = mesh LCS so no need for transformations.
In some cases there is no camera and only mesh matrix (I suspect your case) then it is similar you just ignore the camera transforms and use (0,0,-1) or (0,0,+1) as view direction.
Also see this:
Understanding lighting in OpenGL
It should shine some light on the normals topic.
I'm looking to draw outlines of 2D-closed irregular shapes with curved edges, and then compute both (a) the center of gravity (centroid) of the shape's area, and (b) the center of gravity of the shape's perimeter (i.e. the centroid of, e.g., a wire wrapped tightly around the outside of the shape). I have a late-beginner's level proficiency with Python and Matlab.
(a) and (b) are easy enough given a polygon's vertices: the centroid of a polygon's area is given by the equation here, and the centroid of a polygon's perimeter is just the average of all the line segments' midpoints weighted by the line segments' lengths. I've already written some functions to do this.
The trouble I'm having is getting/approximating those vertices from any of the ways I know how to draw a closed shape with curved edges. The best solution I've come up with so far is to use something like this matplotlib-based script to draw the curvy shape, and then call path.to_polygons(), which converts Path objects to polygons — but does so with surprisingly low resolution, such that the resulting approximation is quite poor (and too poor for my purposes — I'd like to compute those centroids fairly precisely).
So, I'm looking to either (i) find some way to increase the resolution of .to_polygons (about 10-fold), which would be satisfactory for my purposes, or (ii) try some new strategy entirely. One option would be to draw the shapes using something like Adobe Illustrator and then get an approximation of their vertices via some plugin or maybe an image processing toolbox (but I have no clue how to do either of those things). Another would be to draw the shape using some toolbox/library that already has built-in functions for finding the centroids of areas and perimeters (I've seen some having the former, but none with the latter). But I'm sure there are many other options out there that I haven't considered.
I have an arbitrary rectangular Cartesian grid divided into potentially 10^6 or so rectangular cells. (Arbitrary means that the $x$ grid is along points $x_1,...x_n$ and the same goes for the $y$ grid.) I would like to draw an arbitrary object on top of it (say a rotated rectangle, or a circle), and efficiently calculate what fraction of each cell is overlapped by the object: if the cell is entirely inside the bounds of the object, 1.0; if the cell is entirely outside, 0.0; if half of the cell is covered by the object, 0.5. If you displayed this as an image and scaled it where 1 is black and 0 is white, the result would look like an antialiased drawing of the black object.
My application for this question is in Python, and it seems like this capability might be provided by some existing graphics library. Is there a Python module that will test for the fractional intersection of a rectangle and an arbitrary object? Is there a Python library that can at least efficiently test if a point is inside an arbitrary object like a rotated rectangle?
You could use PyCairo, which has fast native routines to do its drawing. It's antialiased by default.
Implementing the drawing algorithms in Python would be very slow.
To find the area of a trapezoid resulting from a polygon-square intersection, you can follow the process described by Sean Barrett at https://nothings.org/gamedev/rasterize/
The shapely Python library can find the area of a trapezoid and perform point-in-object tests. However, for best performance this sounds like something that you'd want to write in C/C++ and provide numpy bindings.