We Keep Coding

Processing an image of a compass to determine the direction a player is facing

I am building a video game overlay that sends data back to the player to create a custom HUD, just for fun.
I am trying to read an image of a video game compass and determine the exact orientation of the compass to be a part of my HUD.
Example photo which shows the compass at the top of the screen:
(The circle currently facing ~170°, NOTE: The position of the compass is also fixed)
Example photo which shows the compass at the top of the screen:
Obviously, when I image process on the compass I will only be looking at the compass and not the whole screen.
This has been more challenging for me compared to previous computer vision aspects of my HUD. I have been trying to process the image using cv2 and from there use some object detection to find the "needle" of the compass.
I am struggling to get a triangle shape detection on either needle that will help me know my orientation.
The solution could be lower-tech and hackier, perhaps just searching for the pixel on the edge of the compass and determining that is the end of the needle.
One solution I do not think is viable is using object detection to find a picture of a compass facing true north and then calculating the rotation of the current compass. This is due to the fact that the background of the compass does not rotate only the needle does.
So far I have applied Hough Circle Transform as seen here:
https://opencv24-python-tutorials.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_houghcircles/py_houghcircles.html#hough-circles
Which has helped me get a circle around my compass as well as the middle of my compass. However, I cannot find a good solution for finding the facing of the needle compared to the middle of the compass.
I understand this is a pretty open-ended question but I am looking for any theoretical solutions that would help me implement a solution. Anything would help as this is a strange problem for me and I am struggling to think how to go about solving it.

In general I would suggest to look at a thin ring just beneath the border or your compass (This will give you lowest error). Either you could work on an image which is a polar transform of this ring or directly on that ring, looking for the center of gravity of the color red. This center of gravity with respect to the center of your compass should give you the angle. Most likely you don't even need the polar transform.
im = cv.imread("RPc9Q.png")
(x,y,w,h) = (406, 14, 29, 29)
warped = cv.warpPolar(
src=im,
dsize=(512, 512),
center=(x + (w-1)/2, y + (h-1)/2),
maxRadius=(w-1)/2,
flags=cv.WARP_POLAR_LINEAR | cv.INTER_LINEAR
)

Here's some more elaboration on the polar warp approach.
polar warp
take a column of pixels, being a circle in the source picture
plot to see what's there
argmax to find the red bits of the arrow
im = cv.imread("RPc9Q.png") * np.float32(1/255)
(x,y,w,h) = (406, 14, 29, 29)
# polar warp...
steps_angle = 360 * 2
steps_radius = 512
warped = cv.warpPolar(
src=im,
dsize=(steps_radius, steps_angle),
center=(x + (w-1)/2, y + (h-1)/2),
maxRadius=(w-1)/2,
flags=cv.WARP_POLAR_LINEAR | cv.INTER_LANCZOS4
)
# goes 360 degrees, starting from 90 degrees (east) clockwise
# sample at 85% of "full radius", picked manually
col = int(0.85 * steps_radius)
# for illustration
imshow(cv.rotate(cv.line(warped.copy(), (col, 0), (col, warped.shape[0]), (0, 0, 255), 1), rotateCode=cv.ROTATE_90_COUNTERCLOCKWISE))
signal = warped[:,col,2] # red channel, that column
# polar warp coordinate system:
# first row of pixels is sampled at exactly 90 degrees (east)
samplepoints = np.arange(steps_angle) / steps_angle * 360 + 90
imax = np.argmax(signal) # peak
def vertex_parabola(y1, y2, y3):
return 0.5 * (y1 - y3) / (y3 - 2*y2 + y1)
# print("samples around maximum:", signal[imax-1:imax+2] * 255)
imax += vertex_parabola(*signal[imax-1:imax+2].astype(np.float32))
# that slice will blow up in your face if the index gets close to the edges
# either use np.roll() or drop the correction entirely
angle = imax / steps_angle * 360 + 90 # ~= samplepoints[imax]
print("angle:", angle) # 176.2
plt.figure(figsize=(16,4))
plt.xlim(90, 360+90)
plt.xticks(np.arange(90, 360+90, 45))
plt.plot(
samplepoints, signal, 'k-',
samplepoints, signal, 'k.')
plt.axvline(x=angle, color='r', linestyle='-')
plt.show()

I have been able to solve my question with the feedback provided.
First I grab the image of the compass:
step_1
After I process the image crop out the middle and edges of the compass as seen here:
step_2
Now I have a cropped compass with only a little bit of red showing where the compass needle points. I masked out the red part of the image.
step_3
From there it is a simple operation to find the center of the blob which roughly outputs where the needle is pointing. Although this is not perfectly accurate I believe it will work for my purposes.
step_4
Now that I know where the needle end is it should be easy to calculate the direction based on that.
Some references:
Finding red color in image using Python & OpenCV
https://www.geeksforgeeks.org/python-opencv-find-center-of-contour/

Is there a way to display a pygame window over OpenGL?

I've been meddling around with PyOpenGL and pygame, and I managed to create an FPS-style camera object. Now I want to add a crosshairs in the middle of the screen, and potentially expand to display statistics on the sides of the window.
I've already looked into this, and it seems like you have to do some weird stuff with OpenGL like disabling depth test and changing the projection matrix, and until now none of that actually renders anything, and reduces performance.
It seems to me that it should be very easy, as all I want is something that is over everything else, and doesn't ever move. Is there really no way to tell pygame to draw over OpenGL so I can just draw two lines in the middle of the screen?

No there is no specified way to do that. Do it in OpenGL it is not that complicate.
According to your previous questions, I assume you want to do it in immediate mode using glBegin - glEnd sequences.
In the following I assume that width is the width of the window and height its height. You have to disable the depth test and back up the current matrices by glPushMatrix/glPopMatrix. Load the Identity matrix for the model view matrix and setup an orthographic projection corresponding to the window size (glOrtho):
cross_size = 100
glMatrixMode(GL_MODELVIEW)
glPushMatrix()
glLoadIdentity()
glMatrixMode(GL_PROJECTION)
glPushMatrix()
glLoadIdentity()
glOrtho(0, width, height, 0, -1, 1)
glDisable(GL_DEPTH_TEST)
glColor3ub(128, 128, 128) # color of the crosshair
glBegin(GL_LINES)
glVertex2f(width/2 - cross_size/2, height/2)
glVertex2f(width/2 + cross_size/2, height/2)
glVertex2f(width/2, height/2 - cross_size/2)
glVertex2f(width/2, height/2 + cross_size/2)
glEnd()
glEnable(GL_DEPTH_TEST)
glMatrixMode(GL_PROJECTION)
glPopMatrix()
glMatrixMode(GL_MODELVIEW)
glPopMatrix()
Ensure that 2 dimensional texturing is disabled (glDisable(GL_TEXTURE_2D))

How to set clipping planes with opengl and pyglet

I am troubleshooting a problem with my code that if the depth value of any primitive is not zero it will not render on the screen. I suspect that it gets clipped away.
Is there an easy pythonic way to set my clipping planes in pyglet ?
This is my code so far:
import pyglet
from pyglet.gl import *
import pywavefront
from camera import FirstPersonCamera
def drawloop(win,camera):
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
#glClearColor(255,255,255,255)
glLoadIdentity()
camera.draw()
pyglet.graphics.draw(2, pyglet.gl.GL_POINTS,
('v3f', (10.0, 15.0, 0.0, 30.0, 35.0, 150.0))
)
glPointSize(20.)
return pyglet.event.EVENT_HANDLED
def main():
win = pyglet.window.Window()
win.set_exclusive_mouse(True)
win.clear()
camera = FirstPersonCamera(win)
#win.event
def on_draw():
drawloop(win,camera)
def on_update(delta_time):
camera.update(delta_time)
pyglet.clock.schedule(on_update)
pyglet.app.run()
if __name__ == '__main__':
main()
I am using the FirstPersonCamera snippet from here:
https://gist.github.com/mr-linch/f6dacd2a069887a47fbc

I am troubleshooting a problem with my code that if the depth value of any primitive is not zero it will not render on the screen. I suspect that it gets clipped away.
You have to set up a projection matrix to solve the issue. Either set up an orthographic projection matrix or a perspective projection matrix.
The projection matrix describes the mapping from 3D points of the view on a scene, to 2D points on the viewport. It transforms from eye space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) by dividing with the w component of the clip coordinates. The NDC are in range (-1,-1,-1) to (1,1,1). Every geometry which is out of the clippspace is clipped.
At Orthographic Projection the coordinates in the view space are linearly mapped to clip space coordinates and the clip space coordinates are equal to the normalized device coordinates, because the w component is 1 (for a cartesian input coordinate).
The values for left, right, bottom, top, near and far define a box. All the geometry which is inside the volume of the box is "visible" on the viewport.
At Perspective Projection the projection matrix describes the mapping from 3D points in the world as they are seen from of a pinhole camera, to 2D points of the viewport. The eye space coordinates in the camera frustum (a truncated pyramid) are mapped to a cube (the normalized device coordinates).
To set a projection matrix the projection matrix stack has to be selected by glMatrixMode.
An orthographic projection can be set by glOrhto:
w, h = 640, 480 # default pyglet window size
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
glOrtho( -w/2, w/2, -h/2, h/2, -1000.0, 1000.0) # [near, far] = [-1000, 1000]
glMatrixMode(GL_MODELVIEW)
....
An perspective projection can be set by gluPerspective:
w, h = 640, 480 # default pyglet window size
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
gluPerspective( 90.0, 640.0/480, 0.1, 1000.0) # fov = 90 degrees; [near, far] = [0.1, 1000]
glMatrixMode(GL_MODELVIEW)
....
I recommend to use the following coordinates, to "see" the points in both of the above cases:
e.g.:
pyglet.graphics.draw(2, pyglet.gl.GL_POINTS,
('v3f', (-50.0, -20.0, -200.0, 40.0, 20.0, -250.0)))
glPointSize(20.0)

coordinates changed when migrating from pygame+rabbyt to pyglet+rabbyt

I'm working on a 2D game and decided to switch from SDL to OpenGL. I took rabbyt as an opengl wrapper for rendering my sprites and using pymunk (chipmunk) for my physics. I used pygame for creating the window and rabbyt for drawing the sprites on the screen.
I discovered that with pygame+rabbyt the (0,0) coordinate is in the middle of the screen. I liked that fact, because the coordinate representation in the physics engine were the same as in my graphics engine (I don't have to recalculate the coordinates when rendering the sprites).
Then I switched to pyglet because I wanted to draw lines with OpenGL - and discovered that suddenly the (0,0) coordinate was at the bottom left of the screen.
I suspected that that has something to do with the glViewport function, but only rabbyt executes that function, pyglet touches it only when the window is resized.
How can I set the (0,0) coordinate at the middle of the Screen?
I'm not very familiar with OpenGL and couldn't find anything after several hours googling and trial&error... I hope someone can help me :)
Edit: Some additional information :)
This is my pyglet screen initialization code:
self.window = Window(width=800, height=600)
rabbyt.set_viewport((800,600))
rabbyt.set_default_attribs()
This is my pygame screen initialization code:
display = pygame.display.set_mode((800,600), \
pygame.OPENGL | pygame.DOUBLEBUF)
rabbyt.set_viewport((800, 600))
rabbyt.set_default_attribs()
Edit 2: I looked at the sources of pyglet and pygame and didn't discover anything in the screen initialization code that has something to do with the OpenGL viewport... Here is the source of the two rabbyt functions:
def set_viewport(viewport, projection=None):
"""
``set_viewport(viewport, [projection])``
Sets how coordinates map to the screen.
``viewport`` gives the screen coordinates that will be drawn to. It
should be in either the form ``(width, height)`` or
``(left, top, right, bottom)``
``projection`` gives the sprite coordinates that will be mapped to the
screen coordinates given by ``viewport``. It too should be in one of the
two forms accepted by ``viewport``. If ``projection`` is not given, it
will default to the width and height of ``viewport``. If only the width
and height are given, ``(0, 0)`` will be the center point.
"""
glMatrixMode(GL_PROJECTION)
glLoadIdentity()
if len(viewport) == 4:
l, t, r, b = viewport
else:
l, t = 0, 0
r, b = viewport
for i in (l,t,r,b):
if i < 0:
raise ValueError("Viewport values cannot be negative")
glViewport(l, t, r-l, b-t)
if projection is not None:
if len(projection) == 4:
l, t, r, b = projection
else:
w,h = projection
l, r, t, b = -w/2, w/2, -h/2, h/2
else:
w,h = r-l, b-t
l, r, b, t = -w/2, w/2, -h/2, h/2
glOrtho(l, r, b, t, -1, 1)
glMatrixMode(GL_MODELVIEW)
glLoadIdentity()
def set_default_attribs():
"""
``set_default_attribs()``
Sets a few of the OpenGL attributes that sprites expect.
Unless you know what you are doing, you should call this at least once
before rendering any sprites. (It is called automatically in
``rabbyt.init_display()``)
"""
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)
glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE)
glEnable(GL_BLEND)
#glEnable(GL_POLYGON_SMOOTH)
Thanks,
Steffen

As l33tnerd suggested the origin can be placed at the center with glTranslatef...
I added the following below my screen initialization code:
pyglet.gl.glTranslatef(width/2, height/2, 0)
Thanks!

Negative coordinates in pygame

What would be the best way to use negative coordinates in pygame?
At the moment I have a surface that is 1.5 times the original surface then everything that needs to be drawn is shifted up by a certain amount (to ensure the negative coordinates become positive) and drawn.
Is there an easier/alternate way of doing this?

A simple solution is to write a linear mapping function from world coordinates to pygame screen coordinates
def coord(x,y):
"Convert world coordinates to pixel coordinates."
return 320+170*x, 400-170*y
and use it when drawing all world objects. Have a look here for a complete example.

There is no way to move the origin of a surface from 0,0.
Implement your own drawing class which transforms all the coordinates passed in into the space of the surface.

If it's similar to an RPG map situation, where you have world coordinates and screen coordinates:
use a function that translates world to local, and vice versa.
But I wasn't sure I'd you were looking for Rect's properties?
rect.bottomright = (width, height) # bottom right at window corner
If you want to use center coordinates to blit, vs top left being (0,0)
ship.rect.center = (20, 30) # don't need to translate by adding w/2 to topleft
See also: Rect.move_ip()