I am troubleshooting a problem with my code that if the depth value of any primitive is not zero it will not render on the screen. I suspect that it gets clipped away.
Is there an easy pythonic way to set my clipping planes in pyglet ?
This is my code so far:
import pyglet
from pyglet.gl import *
import pywavefront
from camera import FirstPersonCamera
def drawloop(win,camera):
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
#glClearColor(255,255,255,255)
glLoadIdentity()
camera.draw()
pyglet.graphics.draw(2, pyglet.gl.GL_POINTS,
('v3f', (10.0, 15.0, 0.0, 30.0, 35.0, 150.0))
)
glPointSize(20.)
return pyglet.event.EVENT_HANDLED
def main():
win = pyglet.window.Window()
win.set_exclusive_mouse(True)
win.clear()
camera = FirstPersonCamera(win)
#win.event
def on_draw():
drawloop(win,camera)
def on_update(delta_time):
camera.update(delta_time)
pyglet.clock.schedule(on_update)
pyglet.app.run()
if __name__ == '__main__':
main()
I am using the FirstPersonCamera snippet from here:
https://gist.github.com/mr-linch/f6dacd2a069887a47fbc
I am troubleshooting a problem with my code that if the depth value of any primitive is not zero it will not render on the screen. I suspect that it gets clipped away.
You have to set up a projection matrix to solve the issue. Either set up an orthographic projection matrix or a perspective projection matrix.
The projection matrix describes the mapping from 3D points of the view on a scene, to 2D points on the viewport. It transforms from eye space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) by dividing with the w component of the clip coordinates. The NDC are in range (-1,-1,-1) to (1,1,1). Every geometry which is out of the clippspace is clipped.
At Orthographic Projection the coordinates in the view space are linearly mapped to clip space coordinates and the clip space coordinates are equal to the normalized device coordinates, because the w component is 1 (for a cartesian input coordinate).
The values for left, right, bottom, top, near and far define a box. All the geometry which is inside the volume of the box is "visible" on the viewport.
At Perspective Projection the projection matrix describes the mapping from 3D points in the world as they are seen from of a pinhole camera, to 2D points of the viewport. The eye space coordinates in the camera frustum (a truncated pyramid) are mapped to a cube (the normalized device coordinates).
To set a projection matrix the projection matrix stack has to be selected by glMatrixMode.
An orthographic projection can be set by glOrhto:
w, h = 640, 480 # default pyglet window size
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
glOrtho( -w/2, w/2, -h/2, h/2, -1000.0, 1000.0) # [near, far] = [-1000, 1000]
glMatrixMode(GL_MODELVIEW)
....
An perspective projection can be set by gluPerspective:
w, h = 640, 480 # default pyglet window size
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
gluPerspective( 90.0, 640.0/480, 0.1, 1000.0) # fov = 90 degrees; [near, far] = [0.1, 1000]
glMatrixMode(GL_MODELVIEW)
....
I recommend to use the following coordinates, to "see" the points in both of the above cases:
e.g.:
pyglet.graphics.draw(2, pyglet.gl.GL_POINTS,
('v3f', (-50.0, -20.0, -200.0, 40.0, 20.0, -250.0)))
glPointSize(20.0)
Related
I have a code in which the geometry of a box changes in each iteration within known XYZ bounds.
From a fixed, viewpoint and using the same camera, I want to capture a snapshot in each iteration to observe the behavior of my model after each run.
but looking at the results, it seems that the settings are not kept the same in different iterations:
geometry size in the below three pictures, respectively:
3.156122777 x 5.048651987 x 5.27256857
11.93332875 x 3.52520350 x 3.937217469
11.96147930 x 4.34533669 x 11.48175857
this is the visualization part of my program:
# Create a visualization window and add the objects to it
vis = o3d.visualization.Visualizer()
vis.create_window(visible=False)
vis.add_geometry(mesh)
vis.add_geometry(pcd)
vis.add_geometry(pcd1)
vis.add_geometry(pcd2)
vis.add_geometry(wireframe_mesh)
for i in range(len(lines)):
vis.add_geometry(lines[i])
# Set the camera extrinsic parameters
vis.get_view_control().set_zoom(6)
vis.get_view_control().set_lookat([4, 5, 3])
vis.get_view_control().set_front([-0.5, -0.5, +0.5])
vis.get_view_control().set_up([0.0, 0.0, 1.0])
vis.get_view_control().set_constant_z_far(300)
# Set the camera intrinsic parameters
cam_params = vis.get_view_control().convert_to_pinhole_camera_parameters()
cam_params.extrinsic = np.identity(4) # No extrinsic transformation
cam_params.intrinsic.set_intrinsics(640, 480, 525, 525, 320, 240) # Set the camera intrinsics
vis.get_view_control().convert_from_pinhole_camera_parameters(cam_params)
# Hide the visualization window and capture a screenshot of the window
vis.poll_events()
vis.update_renderer()
vis.capture_screen_image(f'{images_folder}{iterationName}.jpg')
# Close the visualization window
vis.destroy_window()
I am having difficulty spotting the issue in my code with regards to an OpenCV to OpenGL pseudo augmented reality program. I say "pseudo" because the constraint is taking images and mat objects from openCV and converting them to OpenGL.
I have working models rendered, have used the cv2.rodrigues method and camera calibration to come up with some translation and rotation vectors, I have inverted the matrix and taken the transpose for proper display in the openGL context. The issue I have stems from the fact that the model gets warped when I rotate my CharUcoBoard and begins to drift in the opposite direction when I move my camera. I assume it has something to do with the fact I am using outdated tech, on an unsupported platform (MacOS Big Sur, 11.6). Unfortunately I do not have time to switch platforms or utilize more abstracted software. Ergo, I am stuck with what I got.
I have images, but since this is a new account am unable to post them directly.
I have tried several methods, including hard coding in my calibrated camera parameters, modifying the initial identity matrix to coincide with my projected matrix and adding additional calibration images from all manner of angles, rotations and camera distances.
Ive tried using glOrtho and glMultMatrixf to no avail. Perhaps I missed something in my code.
The math part:
self.INVERSE_MATRIX = np.array([[1.0, 1.0, 1.0, 1.0],
[-1.0, -1.0, -1.0, -1.0],
[-1.0, -1.0, -1.0, -1.0],
[1.0, 1.0, 1.0, 1.0]])
def _boardHandle(self, image):
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
res = cv2.aruco.detectMarkers(gray, dictionary, parameters=arucoParams)
corners = res[0]
ids = res[1]
if ids is not None:
res2 = cv2.aruco.interpolateCornersCharuco(corners, ids, gray,
board)
charucoRetval = res2[0]
charucoCorners = res2[1]
charucoIds = res2[2]
# with camera calibration
retval, rvecs, tvecs = cv2.aruco.estimatePoseCharucoBoard(charucoCorners, charucoIds, board,
self.mtx, self.dist, self.rvecs, self.tvecs)
if retval:
rmtx = cv2.Rodrigues(rvecs)[0]
view_matrix = np.array([[rmtx[0][0], rmtx[0][1], rmtx[0][2], tvecs[0]],
[rmtx[1][0], rmtx[1][1], rmtx[1][2], tvecs[1]],
[rmtx[2][0], rmtx[2][1], rmtx[2][2], tvecs[2]],
[0.0, 0.0, 0.0, 1.0]], dtype=object)
view_matrix = view_matrix * self.INVERSE_MATRIX
view_matrix = np.transpose(view_matrix)
return view_matrix
return self.INVERSE_MATRIX
The rendering pipeline and frame grabbing:
def _draw_scene(self):
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
# get image from webcam
image = self.webcam.get_current_frame()
# convert image to OpenGL texture format
bg_image = cv2.flip(image, 0)
bg_image = Image.fromarray(bg_image)
ix = bg_image.size[0]
iy = bg_image.size[1]
bg_image = bg_image.tobytes("raw", "BGRX", 0, -1)
# create background texture
glBindTexture(GL_TEXTURE_2D, self.bgTexture)
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST)
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST)
glTexImage2D(GL_TEXTURE_2D, 0, 3, ix, iy, 0, GL_RGBA, GL_UNSIGNED_BYTE, bg_image)
# draw background
glBindTexture(GL_TEXTURE_2D, self.bgTexture)
glPushMatrix()
glTranslatef(0.0, 0.0, -10.0)
self._draw_background()
glPopMatrix()
# handle boards/pieces
image = self._boardHandle(image)
#drawing the other pieces
glutSwapBuffers()
The above blocks handle the camera calibration files, the initial creation of the modelview matrix, and the rendering of the chessboard. I had originally assumed that estimating the pose and then utilizing that in a modelview matrix would have given me an accurate rendering, which seems to be the case when the camera is just shy of centered above the board. Im relatively new at openGL however and may have missed something crucial.
I know display lists are outdated as well, but at this point have been unable to make the above work with VAO's, VBO's or EBO's. That will have to be a future project for me.
Any help/pointers or resources will be appreciated.
Am trying to render cube with tkinter frame opengl.
But I don't know where the problem lies the cube didn't show expect 2 lines.
Check my code
Pls can you help me write the code and do you have any PDF to teach me opengl I can't find much resources online
import tkinter as tk
from OpenGL.GL import *
from pyopengltk import
OpenGLFrame
cubeVertices =
((1,1,1),(1,1,-1),
(1,-1,-1),(1,-1,1),.
(-1,1,1),(-1,-1,-1),
(-1,-1,1),(-1,1,-1))
cubeEdges = ((0,1),.
(0,3),(0,4),(1,2),.
(1,7),(2,5),(2,3),.
(3,6),(4,6),(4,7),.
(5,6),(5,7))
classframe(OpenGLFrame):
def initgl(self):
glViewport(0,0,250,250)
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
glOrtho(0,self.width,self.height,0,-1,1)
glMatrixMode(GL_MODELVIEW)
glLoadIdentity()
def redraw(self):
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
glPushMatrix()
glRotatef(90,0.0,1.0,0.0)
glBegin(GL_LINES)
glColor3f(0.0,1.0,0.0)
for cubeEdge in cubeEdges:
for cubeVertex in cubeEdge:
glVertex3fv(cubeVertices[cubeVertex])
glEnd()
glPopMatrix()
root = tk.Tk()
app = frame(root,width=900, height=600)
app.pack(
fill=tk.BOTH,expand=tk.YES)
app.mainloop()
You have to change the projection matrix. Since the cube has a dimension of 2x2x2 and the projection is an orthographic projection in window space, the cube will cover just 4 pixels in the window.
Change the view space and increase the distance to the near and far plane. Note, the geometry has to be in between the near and far plane, else the geometry will be clipped. For instance:
glOrtho(-10, 10, -10, 10, -10, 10)
Anyway I recommend to use Perspective projection. The projection matrix defines a 3 dimensional space (clip space) which is projected on the 2 dimensional viewport. At Perspective projection, this space is a frustum (Viewing frustum). The matrix can be set by gluPerspective. For instance:
classframe(OpenGLFrame):
def initgl(self):
glViewport(0,0,250,250)
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
# glOrtho(-10, 10, -10, 10, -10, 10)
gluPerspective(90, self.width/self.height, 0.1, 10.0)
glMatrixMode(GL_MODELVIEW)
glLoadIdentity()
The value for the near plane and the far plane have to be greater than 0.0 and the far plane has to be greater then the near plane 0 < near < far. In the above example the near plane is 0.1 and the far plane 10. When you draw the geometry, the you have to ensure, that the geometry is in between then near and the far plane (in clip space respectively in the viewing volume), else the geometry is clipped.
Use gluLookAt to define a view matrix with a point of view (0, -3, 0) that has a certain distance to the origin of the world (0, 0, 0):
classframe(OpenGLFrame):
# [...]
def redraw(self):
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
# define view matrix
glLoadIdentity()
gluLookAt(0, -3, 0, 0, 0, 0, 0, 0, 1)
glPushMatrix()
glRotatef(90,0.0,1.0,0.0)
glBegin(GL_LINES)
glColor3f(0.0,1.0,0.0)
for cubeEdge in cubeEdges:
for cubeVertex in cubeEdge:
glVertex3fv(cubeVertices[cubeVertex])
glEnd()
glPopMatrix()
What exactly are eye space coordinates?
I've been meddling around with PyOpenGL and pygame, and I managed to create an FPS-style camera object. Now I want to add a crosshairs in the middle of the screen, and potentially expand to display statistics on the sides of the window.
I've already looked into this, and it seems like you have to do some weird stuff with OpenGL like disabling depth test and changing the projection matrix, and until now none of that actually renders anything, and reduces performance.
It seems to me that it should be very easy, as all I want is something that is over everything else, and doesn't ever move. Is there really no way to tell pygame to draw over OpenGL so I can just draw two lines in the middle of the screen?
No there is no specified way to do that. Do it in OpenGL it is not that complicate.
According to your previous questions, I assume you want to do it in immediate mode using glBegin - glEnd sequences.
In the following I assume that width is the width of the window and height its height. You have to disable the depth test and back up the current matrices by glPushMatrix/glPopMatrix. Load the Identity matrix for the model view matrix and setup an orthographic projection corresponding to the window size (glOrtho):
cross_size = 100
glMatrixMode(GL_MODELVIEW)
glPushMatrix()
glLoadIdentity()
glMatrixMode(GL_PROJECTION)
glPushMatrix()
glLoadIdentity()
glOrtho(0, width, height, 0, -1, 1)
glDisable(GL_DEPTH_TEST)
glColor3ub(128, 128, 128) # color of the crosshair
glBegin(GL_LINES)
glVertex2f(width/2 - cross_size/2, height/2)
glVertex2f(width/2 + cross_size/2, height/2)
glVertex2f(width/2, height/2 - cross_size/2)
glVertex2f(width/2, height/2 + cross_size/2)
glEnd()
glEnable(GL_DEPTH_TEST)
glMatrixMode(GL_PROJECTION)
glPopMatrix()
glMatrixMode(GL_MODELVIEW)
glPopMatrix()
Ensure that 2 dimensional texturing is disabled (glDisable(GL_TEXTURE_2D))
Overview:
I am trying to create a 3D application similar to this:
www.youtube.com/watch?v=h9kPI7_vhAU.
I am using OpenCV2.2, Python2.7 and pyOpenGL.
This can be achieved by this background maths and code snippet where x, y, z are the positions of the viewers eye (as grabbed from a webcam!)
Issue:
When I do this, the object (a cube) that I have rendered becomes stretched along the z axis (into the screen) and I'm not too sure why. It is likened to looking down a very tall skyscraper from above (as opposed to a cube). The cube's position changes very rapidly in the z direction as the z position of the eye changes. This is a frame of the result, it has been stretched!
Code (with bigD's edit):
def DrawGLScene():
#get some parameters for calculating the FRUSTUM
NEAR_CLIPPING_PLANE = 0.01
FAR_CLIPPING_PLANE = 2
window = glGetIntegerv(GL_VIEWPORT)
WINDOW_WIDTH = window[2]
WINDOW_HEIGHT= window[3]
#do facial detection and get eye co-ordinates
eye = getEye()
#clear window
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
#before any projection transformation command comes these 2 lines:
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
#transform projection to that of our eye
glFrustum(NEAR_CLIPPING_PLANE*(-WINDOW_WIDTH /2 - eye[0])/eye[2],
NEAR_CLIPPING_PLANE*( WINDOW_WIDTH /2 - eye[0])/eye[2],
NEAR_CLIPPING_PLANE*(-WINDOW_HEIGHT/2 - eye[1])/eye[2],
NEAR_CLIPPING_PLANE*( WINDOW_HEIGHT/2 - eye[1])/eye[2],
NEAR_CLIPPING_PLANE, FAR_CLIPPING_PLANE)
glMatrixMode(GL_MODELVIEW)
glLoadIdentity()
glTranslatef(-eye[0],-eye[1],-eye[2])
drawCube()
glutSwapBuffers()
an example of the data getEye() returns is:
[0.25,0.37,1] if viewers is has their face near lower left of screen and is 1m away
[-0.5,-0.1,0.5] if viewers is has their face near upper right of screen and is 0.5m away
The cube when drawn has height, width, depth of 2 and its centre at (0,0,0).
I will provide the full code if anyone wants to do a similar project and wants a kickstart or thinks that the issue lies somewhere else than code provided.
The reason why you're getting strange results is because of this:
glTranslatef(-eye[0],-eye[1],-eye[2])
This call should be made after
glMatrixMode(GL_MODELVIEW)
glLoadIdentity()
Because the projection matrix is ready as it is with your glFrustum call, if you multiply it by a translation matrix that won't make it a perspective projection matrix anymore. The modelview matrix has to describe all world AND camera transformations.
Also bear in mind that if the only transformation you do on your modelview matrix is a translation, then you will always be staring down the negative-Z axis.