Ok so I've spent an embarrassing amount of time on this problem and still am fairly confused. I'm hoping someone can provide some guidance, resources, similar projects, etc. I am unsure that what I am doing is the right way to go about this problem.
Background:
I am working on a project where I am developing an imitation of rocket league (a video game) so that I can run reinforcement learning algorithms on the game (I can't run the game itself because it would be too slow). The imitation needs to comply with OpenAI Gym's API, which essentially demands a step method (which takes in the actions the AI determines necessary at that timestep and outputs the state of the game after the timestep), and a render method. I have enough of the game implemented that I am currently working on the render method. I want to package this and release it on pypi, so I chose pyglet to implement the graphics because I read it is fairly portable.
Problem:
I don't think I can use an event loop to run my graphics. The way this api is set-up and used is that the user first instantiates the environment (the video game) and then sets up a loop in which they run the step function of the environment and optionally choose to also place the render method in that same loop depending on whether or not they want to see their AI's actions during that run. An example usage is available under environments on this page. So I can't use an event loop because it would stop execution of the user's loop. Instead I need to instantiate a window on the first call of render, and then update it with the state of the game on every subsequent call.
My current solution:
def render():
if self.window is None:
self.window = pyglet.window.Window(width, height)
self.window.clear()
self.window.dispatch_events()
... describe current 3d scence
self.window.flip()
Problem cont:
My current solution feels a bit hacky which I don't love, but more of a problem is that I can't figure out how to implement user input for this solution. I would like to be able to pan and move the camera all around the scene so that I view the 3-dimensional shape of objects, but I don't know how to implement that without the event loop and on_key_press decorator.
Also:
I am struggling to find good resources for 3d programming with OpenGL functions (the game is 3d). I was wondering if anyone knew of a good place to learn that without all complexity I found on https://learnopengl.com/. I don't even know if pyglet/opengl is the right way to go about solving this problem. I know very little about 3d graphics and am open to any suggestions.
So for anyone with a similar problem looking for the solution, here is what I determined:
If you need to render events but not give up control flow to pyglet.app.run() or any other event loop, custom or otherwise, it is possible to still listen for user actions. The following code is an example pseudo-implementation for a class that renders its state each time the render() function is called, with user input optionally changing that state. Be warned that this is far optimal from an efficiency perspective, and you should always use pyglet.app.run() when possible, but this instance demanded an alternative solution.
class Env:
# ...lots of other class stuff
def render(self):
if self.window is None:
self.window = pyglet.window.Window(width, height)
#self.window.event
def on_close():
self.window.close()
#self.window.event
def on_key_press(key, mod):
# ...do stuff on key press
pyglet.clock.tick()
self.window.clear()
self.window.dispatch_events()
# ...transform, update, create all objects that need to be rendered
self.window.flip()
env = Env()
for _ in range(100):
env.doStuff()
env.render()
Related
Note: I am open to different solutions which achieve the desired capability
I am working on a project with many instances of the same game.
Therefore, I am sending keyboard and mouse instructions to each of theses processes, in parallel.
I am currently using win32ui as follows:
After finding the processes hwnd (windows handle) values from Get HWND of each Window?, so a hwnds_list with all the processes with a given name e.g. [788133, 723724, ...]
I am sending instructions to each of the processes, by creating a PyCWnd object:
PyCWnd = win32ui.CreateWindowFromHandle(hwnd)
Then, say I want to press the return key, I used:
def press_return(pycwnd):
pycwnd.SendMessage(win32con.WM_KEYDOWN, win32con.VK_RETURN, 0)
pycwnd.SendMessage(win32con.WM_KEYUP, win32con.VK_RETURN, 0)
Then I run this in parallel with:
def press_return_par(hwnds):
# Get the Window from handle
pycwnd = make_pycwnd(hwnds)
time.sleep(0.1)
press_return(pycwnd)
num_workers = len(hwnds_list)
with Pool(num_workers) as p:
p.map(press_return_par, hwnds_list)
So, I have a good way of sending keyboard commands, and even scrolling with a mouse, but can't work out how to do this with mouse movements.
Ideally, I'd like to say, "Move to (x, y) coordinates over n time". This 'ideal' method needs to not effect the current cursor (or allow a locked cursor for each process/game), as I will want to do this across ~8 instances of the game.
I've looked through the official pywin32 docs http://timgolden.me.uk/pywin32-docs/contents.html, other answers that look bang on https://stackoverflow.com/a/3721198/11181287 but use win32api.mouse_event, so I don't know how to convert this to work with the multiple pycwnd objects.
https://stackoverflow.com/a/3721053/11181287 looks close, but doesn't seem to move the mouse, it just does the right click, although I have made some guesses for the MAKELPARAM function which is not listed.
In addition, https://github.com/oblitum/Interception could be helpful but haven't found good docs for how to apply this here.
As the game is an FPS game, running multiple instances through nucleus-coop, using a VM etc... won't be fast enough (from my current research).
PyAutoGUI is exactly the functionality I want, with the speed, but (as expected) I haven't been able to set it up to work for multiple mice/processes
There could be something in sending DirectX inputs into the game (black ops 2)?
(I'm running windows 10, Python 3.7.11, and only know Python)
I have two possible solutions to your mice issue.
What if you used only one mouse to control all of the windows? With pyautogui you could tab into each window when necessary and control the mouse for that window. I'm not sure how efficient this would be and how fast the mouse control for each window would be, but it's still sort of a solution.
OR
You could control the mouse with the keyboard.
See this article https://www.windowscentral.com/how-control-mouse-using-keyboard-windows-10
I apologize for not just commenting, unfortunately I don't have enough reputation.
I'm using kivy to develop a small project. Currently, I use the animation utility to animate Widgets (change position, cycle through a sprite atlas, etc.). Everything works great so far, but I have no idea how I am supposed to pause animations and continue them at a later point. I know that I could use the Clock schedule mechanism and clock.cancel, but that would make the whole ànimation` class pointless.
The Animation class has start, stop and cancel methods you can use to manipulate it how you like. Perhaps we should have a pause too, but you can achieve the same effect with those.
I'm working with opencv3, python 3 and pyqt5. I want to make a simple GUI in which I want open up a new window to play a video along with some other widgets when a button is clicked on the main window. I've used QPixmap for displaying images in the past so I create a label and try to set the frames in the pixmap in a loop. The loop works fine but I am unable to get a display of the video/new window.
The loop I want to execute in the new window looks something like this:
def setupUi():
vid=cv2.VideoCapture('file')
ret, frame=vid.read()
while ret:
Qimg=convert(frame)
self.label.setpixmap(Qimg)
self.label.update()
ret,frame=vid.read()
convert() is a function I've written myself that converts the cv frame to QImage type to be set into the pixmap.
I'm only a beginner with pyQt so don't know what I am doing wrong. I've read about using signals, threads for the new window and QtApplication.processEvents() but don't know how these work and how they'll fit into my problem.
It would be helpful if someone could set me in the right direction and also point out some resources to create good interfaces for my apps using OpenCV and python.
The reason that this isn't running is that your while loop is blocking Qt's event loop. Basically, you're stuck in the while loop and you never give control back to Qt to redraw the screen.
Your update() call isn't doing what you think it is; it's updating the data stored by the object, but this change does not show up until the program reenters the eventloop.
There are probably multiple ways of handling this, but I see two good options, the first being easier to implement:
1) Call QApplication.processEvents() in every iteration of your while loop. This forces Qt to update the GUI. This will be much more simple to implement than 2).
2) Move the function to a separate class and use QThread combined with moveToThread() to update the data, and communicate with the GUI thread using signals/slots. This will require restructuring your code a bit, but this might be good for your code overall. Right now the code that is generating the data is in your MainWindow class presumably, while the two should be kept separate according to Qt's Model-View design pattern. Not very important for a small one-off app, but will help keep your code base intelligible as your app grows in size.
I'm using Python 3 and I'm trying to rewrite a graphics library that I wrote a while ago. Right now, I have a class called Scene which has three methods in addition to its constructor: logic(self), update(self), and render(self).
logic(self) handles the logic of each Scene. It handles all mouse and keyboard input, and figures out whether a button was clicked, et cetera.
update(self) can be thought of as an abstract method. Its contents are simply raise NotImplementedError. update(self) is called a certain number of times each second, and updates whatever is going on in the program, but does not draw anything on the screen. This is the framerate-independent update. If Pong were to be made with this library, the ball's position would be updated in this method.
render(self) actually updates the screen. This way, the speed of the program is not determined by the framerate.
I like my current implementation of Scene. It's really clean right now, and it seems very intuitive. However, I can't get over how to extend Scene. Suppose I want to make a Scene for a basic GUI like a main menu. I want there to be only one main menu Scene, because I don't think it's a good idea to recreate a whole other Scene every time the main menu will be viewed and used.
I started out by extending Scene with a static class, but quickly realized that it's not really possible to do that. Is the current paradigm just to extend Scene with a normal class, and then recreate and destroy it everytime that the subclass must be used? For example, should I just recreate and destroy a hypothetical MainMenu(Scene) object when viewing and leaving it, respectively?
Implement what the GoF refer to as Strategy Pattern, where you dynamically dispatch to appropriate logic based on current state.
I've created a simple console based Scrabble game using Python. I tried to encapsulate the game model from the I/O as much as possible, meaning I created a few classes to describe the game with its rules and the current state. Basically I came up with these classes:
LetterSet: To describe the tiles in the game (score, total amount, etc.)
Board: A representation of the Board with its tiles and auxiliry functions
Player: A virtual class to subclass real Classes like Human or Bot, got one method play() which should return the players move
Game: Well ...
Everything works fine using a simple linear and synchronous flow with my console app.
But it turns out that its not that easy to port this concept to Qt. I've created all neccessary widgets like a Dragable board, general visual elements describing the game state and simple buttons like 'Pass', 'Continue', 'Exchange'.
The problem is, that I'm not sure how to handle the play() method which may use the Qt interface I created to generate a valid move. That's no problem for the Bot though, which simply searches a move without any interaction.
My current idea is to create a local event loop like described here and to wait for the buttons to get clicked in my play() method defined in Human(Bot). This is kinda ugly so I'm wondering if there is a better way to do it.
I would like the main logic to be the same e.g. the Player class serves a play() method which generates a move and returns it. This way it should be possible to create any type of Player's, like network players or bots. This synchronous way of doing it doesn't work very well with Qt's based signal/slot way of doing things. Hope someone got a neat idea to solve my problem.
Summarized: How to generate the Player's move inside its play() method and returning it like a simple move = player.play(game) call?
Edit: A snapshot to get an idea of what I'm talking about:
(source: b52 at reaktor42.de)
Edit2: This is rather old and I completed the assignment about two years ago successfully. Nevertheless I thought it might be useful for others if I publish the results through github.
Thanks in advance, Oli
What you can do in the Player play function is:
Enabled the buttons and connect them to slots (one per action)
Wait until the player move is validate (or any other reason to quit)
Disconnect signals to slot when the player move has been received (or is validated)
This is one way but you should modify it to fit your game model
My current idea is to create a local event loop like described here and to wait for the buttons to get clicked in my play() method defined in Human(Bot). This is kinda ugly so I'm wondering if there is a better way to do it.
I don't see why you think this is ugly. This is how practically any GUI program works: initialise a set of controls (buttons, etc), wait for the user to interact with the controls, respond to the interaction, and repeat.
I'd recommend you have a button for the player to submit their move. They click that, an event fires, the event handler tells your Player object that a move has been made and it passes it to the game logic event handler. The game logic checks whether the move is legal (you should put the code here rather than in the GUI classes), and passes control to the next Player object if the move was legal.
I think you're right that there's a bit of a disconnect between the event based model in a GUI environment and your original application design. With a console app you write the event loop, but in a GUI app in any of the frameworks I know, the framework has one of it's own.
I'd have the Player object send it's move via a signal to the game object. You would need to structure your base Player class around this design. It shouldn't all that hard to do as you've already got the actual logic worked out, you're just re-wiring it a bit.
Note that your Player objects are actually just interfaces between the game and the actual player which might be someone clicking buttons in the UI, or a remote player over a network connection. Only in the case of a bot can the player object actually be the player, and even in that case you might be best off having separate strategy engine objects. Thinking about it that way might help get a grip on the design.
You don't have to do it that way. You can work around it e.g. by the method you describe, but I wouldn't want to mess around with my own event loops inside an application that has it's own GUI event loop. That's fighting against the GUI framework rather than working with it. It's not just Qt that's like this, you'd have an issue similar to this in any GUI framework. It's a different way to think about structuring an application and I'd recommend embracing it.