I am beginner level in programming(less than a year of experience). I am trying to use https://github.com/Sentdex/pygta5/blob/master/directkeys.py to imitate keyboard inputs in one of RPGs. I successfully managed to implement simple movements(WASD) but when I try to use game specific command as example "choose a closest enemy(Using TAB)" game doesn't react to the input. By doing some research I realized that the game most likely have internal driver level keyboard interaction which cannot be activated using Direct Inputs. I am asking for possible ways to solve the issue.
I tried to use virtual keys to try to imitate the keyboard inputs. Also, I tried to reassign one of the movement keys to "chose closest enemy button" but it produces no inputs as well. I am might start extensive research towards even lower-level keyboard interactions and see what I can do from there.
Here is small code sample
from directkeys import KeyPress, PressKey, ReleaseKey, W, ESC, C, TAB
import time
import win32gui
# open a window
handle = win32gui.FindWindow(0, "Game window")
win32gui.SetForegroundWindow(handle)
# Press and Release key
PressKey(W)
time.sleep(3)
ReleaseKey(W)
print("Script finished")
Related
Note: I am open to different solutions which achieve the desired capability
I am working on a project with many instances of the same game.
Therefore, I am sending keyboard and mouse instructions to each of theses processes, in parallel.
I am currently using win32ui as follows:
After finding the processes hwnd (windows handle) values from Get HWND of each Window?, so a hwnds_list with all the processes with a given name e.g. [788133, 723724, ...]
I am sending instructions to each of the processes, by creating a PyCWnd object:
PyCWnd = win32ui.CreateWindowFromHandle(hwnd)
Then, say I want to press the return key, I used:
def press_return(pycwnd):
pycwnd.SendMessage(win32con.WM_KEYDOWN, win32con.VK_RETURN, 0)
pycwnd.SendMessage(win32con.WM_KEYUP, win32con.VK_RETURN, 0)
Then I run this in parallel with:
def press_return_par(hwnds):
# Get the Window from handle
pycwnd = make_pycwnd(hwnds)
time.sleep(0.1)
press_return(pycwnd)
num_workers = len(hwnds_list)
with Pool(num_workers) as p:
p.map(press_return_par, hwnds_list)
So, I have a good way of sending keyboard commands, and even scrolling with a mouse, but can't work out how to do this with mouse movements.
Ideally, I'd like to say, "Move to (x, y) coordinates over n time". This 'ideal' method needs to not effect the current cursor (or allow a locked cursor for each process/game), as I will want to do this across ~8 instances of the game.
I've looked through the official pywin32 docs http://timgolden.me.uk/pywin32-docs/contents.html, other answers that look bang on https://stackoverflow.com/a/3721198/11181287 but use win32api.mouse_event, so I don't know how to convert this to work with the multiple pycwnd objects.
https://stackoverflow.com/a/3721053/11181287 looks close, but doesn't seem to move the mouse, it just does the right click, although I have made some guesses for the MAKELPARAM function which is not listed.
In addition, https://github.com/oblitum/Interception could be helpful but haven't found good docs for how to apply this here.
As the game is an FPS game, running multiple instances through nucleus-coop, using a VM etc... won't be fast enough (from my current research).
PyAutoGUI is exactly the functionality I want, with the speed, but (as expected) I haven't been able to set it up to work for multiple mice/processes
There could be something in sending DirectX inputs into the game (black ops 2)?
(I'm running windows 10, Python 3.7.11, and only know Python)
I have two possible solutions to your mice issue.
What if you used only one mouse to control all of the windows? With pyautogui you could tab into each window when necessary and control the mouse for that window. I'm not sure how efficient this would be and how fast the mouse control for each window would be, but it's still sort of a solution.
OR
You could control the mouse with the keyboard.
See this article https://www.windowscentral.com/how-control-mouse-using-keyboard-windows-10
I apologize for not just commenting, unfortunately I don't have enough reputation.
I tried to simulate keypresses in python, but they dont get recognised the programm. All work in simple editors, some even in an open admin cmd window, but nothing here.
What I tried so far:
-librarys: pyautogui, keyboard, pydirectinput, pynput, pyKey
Solutions with ctypes and SendInput:
Python simulate keydown
Use Python to send keystrokes to games in Windows?
Simulating a keypress in python
All other solutions i found were all using win32api or SendInput and didn't work either.
The usecase is the programm Parsec, if that helps. Also mouse movement and clicks work fine with all of the solutions above.
Could anyone help me with that, or point me in a direction? Because i don't know how to continue.
Some programs, especially video games, are specifically made to be resistant to what you are trying to do. This is to prevent bots and cheating within the game or program. You could find a way to make a driver that simulates key presses, but considering that the program has taken specific measures to defend against this, it is likely bannable or violates the Terms and Service agreement.
I am trying to make a macro in Minecraft using pynput, but it seems that pynput does not press the correct buttons on the keyboard. I know this because when I ran (code snippet 1) while setting the controls of Hotbar slot 3 it showed as a weird character instead of a number. (see picture) Is there a way to make pynput press the correct buttons?
# Code snippet 1
import time
from pynput.keyboard import Controller
time.sleep(7) # this is just so I have time to switch to minecraft before it activates
board = Controller()
board.press('2')
time.sleep(.1)
board.release('2')
In place of '2' I have also tried KeyCode(49) and gotten the same result. It is properly recognised by the computer, as shown by keycode.info and notepad.
When I replaced '2' with 'e' Minecraft recognised it as the same as '2', and it even pressed the button, so I am extra confused now.
Extra notes:
For now I will just set the controls to what pynput gives minecraft to make my macro work, but I would prefer to be able to use the button(s) outside of the macro as well.
Pynput's click and scroll seem to work fine.
I am not using the mouse for my macro, but if you have knowledge about the mouse there is a stackOverflow question about it here: How to control the mouse in Minecraft using Python?
For anyone wondering, I am making this macro to empty water buckets quickly after I have used them to remove an area of water in the ocean
While I could be wrong about this, based on this stack overflow post I believe that pynput outputs the characters as win32 keycodes, which while they work for programs like notepad or keycode.info, don't seem to work with setting minecraft controls.
If you are ok with using a different library than pynput, I have found that pywinauto correctly inputs keys into minecraft.
from pywinauto import keyboard
import time
time.sleep(3) #A short wait to allow me to switch to minecraft
keyboard.send_keys("{2 down}" "{2 up}") #replace 2 with whatever key you want to press
Above is the code that I used while testing and I have found it works correctly when trying to set controls.
The easiest way i think is using pydirectinput.
pydirectinput is exactly like pyautogui. Its just that its name changed and that it can work with games using DirectX (minecraft, roblox)
To use it you can just do
import pydirectinput
import pyautogui
import time
time.sleep(2)#so u can switch to your game
pydirectinput.keyDown('v')
time.sleep(0.5)
pydirectinput.keyUp('v)
I'm creating a script for a game because I want to automate a certain part of it. So far I have:
import win32api, win32con, time
def click(x,y):
win32api.SetCursorPos((x,y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)
click(100,655)
time.sleep(3)
click(740,580)
time.sleep(1)
raw_input(100)
So far, I click on the correct page I need to go to, then I click on the textbox where I can enter a number, but after selecting the textbox I cannot quite figure out how to enter a number. I thought to use raw_input, but it has acted like a print statement instead.
The raw_input function isn't going to simulate keystrokes to another program. What it will do is print the prompt to its console, wait for you to type a response to that console, and return what you typed to your script. Completely useless here.
What you actually want is a way to send keyboard events to the app, the same way you're sending mouse events.
If you can depend on Windows Scripting Host being present (which I think is always there in Vista and XPSP3 and later, and can be installed for earlier XP), you can just use it instead of doing things at the low level:
wshell = win32com.client.Dispatch("WScript.Shell")
wshell.SendKeys("foo")
Otherwise, you'll need to get a handle to the window (that's explained in the win32api docs, so I assume you already know it) then something like this:
def sendkey(hwnd, keycode):
win32api.PostMessage(hwnd, win32con.WM_CHAR, keycode, 0)
This won't handle special keys like tab, escape, or return properly. For that, you need to instead send WM_KEYDOWN and WM_KEYUP. But for your use, WM_CHAR is what you want.
You also need a function to look up the keycode for each character in your string. For '100' it's actually just ord('1'), ord('0'), ord('0'), but that's not true for everything.
You may want to look at SendKeys and similar modules that wrap all of this up for you.
Or you may want to use a higher-level automation library like AutoPy (there are many of these, and if you search SO you'll find details about all of them).
Or you may want to forget about trying to automate the browser in terms of mouse clicks and key events and instead deal with it at the appropriate (web) level by using selenium.
Or you may want to forget about automating the browser and instead just simulate a browser in your own script by using mechanize.
I am using pygame to write a program and I need some GUI configuration text field and button for control. I've already made the button using pygame, but I just can write a text field out of pygame. Maybe I need to use tkinter together with pygame.
I think if there is no way to made to pygame part and tkinter part together in 1 window, then I could put them into 2 separate windows.
I hope the tkinter part can update the global variable in my pygame part, would there if any problem? I might create a child process of tkinter from the pygame part so that the tkinter part can probably "see" the global variable in pygame part and modify them.
Can I do this? Are there any pitfalls?
Both Tkinter and Pygame have their own event loops, so doing what you want is far from simple. The problem is that Pygame wants to control both the screen and the events the user feeds into the computer. This doesn't work well with GUI libraries, which also want to be "in control".
I would suggest sticking with Pygame, it has some nice GUI toolkits that will help you create buttons and other controls. Go over this page - it should clear things out. You may also find this discussion useful.
Apart from the practical aspects, a GUI created with Pygame is also IMHO more suitable for a game than something done with Tkinter, since games usually have original, thematical user interfaces and not the bland "text box + button" windows we're used to in other applications.
Take a look at some of the sample games on the Pygame wiki, many have GUIs and you can borrow ideas and code from them.
from tkinter import *
import pygame
import random
import os
global playing
playing=False
def playpause():
global playing
if playing==True:
playing=False
else:
playing=True
root = Tk()
embed = Frame(root, width=640, height=480)
embed.grid(row=0,column=2)
playpausebutton=Button(root, command=playpause, text="Play/Pause")
playpausebutton.grid(row=1,column=2)
root.update()
os.environ['SDL_WINDOWID'] = str(embed.winfo_id())
os.environ['SDL_VIDEODRIVER'] = 'windib'
pygame.display.init()
screen = pygame.display.set_mode((640,480))
pygame.display.flip()
while True:
#your code here
if playing:
screen.fill((random.randint(0,255),random.randint(0,255),random.randint(0,255)))
pygame.display.flip()
root.update()
This works just great, I have used this method successfully in multiple cases.
I have also found pgu is awful. However, what you say about the tkinter event loop taking control is wrong. You just call root.update instead of mainloop, and this can go inside of a while loop to replace the mainloop. Answering your main question however, there is something you should be aware of. It seems that whenever I run the two programs alongside each other, this traceback occurs:
TclError: expected boolean value but got "-1"
Fatal Python error: (pygame parachute) Segmentation Fault
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
Because of this I would avoid the combination, although I suspect this particular issue may pertain to my use of threads.