Python; "simulate" mouse and keyboard - python

I recently started learning Python, trying to make short programs with simple functions but that I can use in one way or another while learning.
I have studied a bit several libraries that they functions are what the title say, with some I have had better experiences than with others, but at the moment I have a problem when trying to make a "Shift+Click" in concrete positions of the screen.
Im actually using win32api and win32con since they are the ones that to my way of seeing have better results for what Im trying to do.
Its nothing very complicated, its just do a Shift + Left Click in specific positions, like same way that you can recreate it in a normally with your mouse and keyboard.
Here is a simplified version of the code:
def mouse_click(x, y):
win32api.SetCursorPos((x, y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN, x, y, 0, 0)
time.sleep(0.065)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP, x, y, 0, 0)
def shift_down():
win32api.keybd_event(win32con.VK_LSHIFT, 0, 0, 0)
time.sleep(0.005)
def shift_up():
win32api.keybd_event(win32con.VK_LSHIFT, 0, win32con.KEYEVENTF_KEYUP, 0)
time.sleep(0.005)
def clicks():
for offs in coord:
x, y = offs
x += random.randint(-15, 15)
y += random.randint(-15, 15)
if random.random() > 0.0:
shift_down()
time.sleep(0.005)
mouse_click(x, y)
time.sleep(0.005)
shift_up()
print('click on', offs)
def main():
while True:
if win32api.GetAsyncKeyState(win32con.VK_INSERT):
break
elif win32api.GetAsyncKeyState(win32con.VK_F2):
clicks()
print('clicking end')
if __name__ == '__main__':
main()
Yes, Im aware that some sleep are unnecessary, but here is my current problem. The way you see above is the best way I have had the code respond correctly and in time according to my needs.if I low the time on mouse_click below 0.55 some clicks start to be ignored, from what I have read and seen it should not be necessary make pauses, but at the moment this is the solution I found.So I want to fix the basic formatting of it first.
Am I missing something or is it some kind of "optimization" problem?
Any hints or improvements are welcome even if its about using another library.

I would recommend using the PyAutoGUI module available here: https://pyautogui.readthedocs.io/
It is very beginner friendly and very easy to use!
You would perform a shift-click like this:
import pyautogui
pyautogui.keyDown('shift')
pyautogui.click()
pyautogui.keyUp('shift')

Somehow these randoms sleep are no longer necessary, and the sleep in mouse_click() I could lower it to a maximum of 0.06 without problems.
The press and release Shift seems to work fine in other applications, so I deduce that it is some kind of incompatibility with the application/window itself (Im not trying to force or use it in a particular window or screen). So maybe it can work with some other library than the ones currently in use.
The problem with the pyautogui and pynput library, is that the same thing happens but they also add some kind of delay by default.
In any case, setting the Shift press outside of if in clicks() seems to be the solution as well as using win32con.KEYEVENTF_EXTENDEDKEY inside win32api.keybd_event().
I don't see any point in going on with this and it will probably end up being better if I hold down Shift by myself.

Related

Simulate horizontal scroll in Windows with python

as the title says, I'm looking for a way to simulate horizontal scrolling (specifically in OneNote). I know it is possible to do it in AutoHotKey with a script, but I'm trying to keep the program as localized as possible. I also know it is possible with PyAutoGui on mac and linux, but I've come up empty handed with anything related to windows. If you have any leads, I would greatly appreciate it:)
For anyone running into a similar problem in the future, here's my solution:
import win32api, time, pyautogui as pag, keyboard
from win32con import *
running = True
lastX, lastY = pag.position()
while running:
while keyboard.is_pressed("shift"):
x, y = pag.position()
if lastX!=x:
win32api.mouse_event(MOUSEEVENTF_HWHEEL, 0, 0, x-lastX, 0) # Horizontal scrolling
lastX=x
if lastY!=y:
win32api.mouse_event(MOUSEEVENTF_WHEEL, 0, 0, lastY-y, 0) # Vertical scrolling
lastY=y
Hope this can help anyone in the future:)
Windows 10 already has an out-of-the-box shortcut for horizontal scroll:
Hold Shift down and use the mouse wheel for side scroll. (This shortcut seems to also work on Ubuntu und MacOS)
Considering this shortcut exists it is possible to emulate it with PyAutoGui like so
import pyautogui
offset = 100
pyautogui.keyDown('shift')
pyautogui.scroll(offset)
pyautogui.keyUp('shift')

PyAutoGui While loop keep going when if statement remains true

I've just started learning python and started working on a project.
I'm moving my mouse cursor from one area to another. I just want it to stop when it reach a certain point/area/zone. I'm using the mouse position to tell me where it is, currently.
For some reason, when this loops starts, it keeps looping even when the IF statement is true.
But if I started when the IF statement is true, the loops kinda works as intended, so far it's only reading the 'X' values.
I couldn't find an answer to this, or any questions like it. If anyone has an idea or can point me to a similar post, I'll appreciate it.
import pyautogui, sys, time, autoit
#Search for a position on screen manually
try:
while True:
x, y = pyautogui.position()
print(pyautogui.position())
print('Stopping for 1 seconds, keep searching or CTRL + C to end')
time.sleep(1)
#Confirmed location on screen.
if pyautogui.position(x,y) >= pyautogui.position(710, 15):
pyautogui.leftClick()
print('The Eagle has landed')
print(pyautogui.position())
break
Update: I got it! Following mkrieger1 advice, I manage to get the 'x, y' values to update. Code was rewritten.

Clearing the screen for text based rpg in python

I am trying to make a text based rpg game in python. I just started and this is what my code looks so far:
class Game:
def __init__(self):
self.map = [[" " for i in range(50)]for i in range(30)]
def draw_map(self):
for i in range(len(self.map)):
print(self.map[i])
def add_stuff(self, thing, x, y):
self.map[y][x] = thing
game = Game()
class Player:
def __init__(self):
self.x = 1
self.y = 1
self.player = "ME"
def draw(self):
game.add_stuff(self.player, self.x, self.y)
def move(self):
pass
player = Player()
player.draw()
game.draw_map()
I was trying to find a way to implement a game loop in some way. To do this i was thinking of running the draw_map() and clearing the screen right away and printing the map again, a bit like real game loops. However i am having problems doing this. Based on other answers to other similar questions, i managed to produce the following code(it just shows the main loop and subprocess is imported as sp).
while True:
game.draw_map()
sp.call("cls", shell = True)
However this is not working for me. It simply dosent do anything. I also tried using clear function from clear_screen` module in a similar way and i cant figure out why this wouldnt work. Thanks for any help
so based on your previous comments you want to clear the screen in a Python interpreter. There is no "command" to clear the screen just like this. But you can use a loop to print some new lines until your screen is blank again.
def clear(lines):
for l in range(lines):
print()
while True:
game.draw_map()
clear(35)
time.sleep(0.05)
You may need to increase or decrease the amount of lines cleared. I hope this works for you. P.S.: I would use a normal console.
https://en.wikipedia.org/wiki/Interpreter_(computing)#Efficiency:
Efficiency:
The main disadvantage of interpreters is that an interpreted program typically runs slower than if it had been compiled. The difference in speeds could be tiny or great; often an order of magnitude and sometimes more. It generally takes longer to run a program under an interpreter than to run the compiled code but it can take less time to interpret it than the total time required to compile and run it. This is especially important when prototyping and testing code when an edit-interpret-debug cycle can often be much shorter than an edit-compile-run-debug cycle.
If your problem is clearing the console window, you can just use
import os
os.system('cls')
in place of
sp.call("cls", shell = True)
(Assuming you are working on windows)
EDIT for clarity:
This is the new loop after the change:
while True:
game.draw_map()
os.system('cls')

Using pyautogui with multiple monitors

I'm trying to use the pyautogui module for python to automate mouse clicks and movements. However, it doesn't seem to be able to recognise any monitor other than my main one, which means i'm not able to input any actions on any of my other screens, and that is a huge problem for the project i am working on.
I've searched google for 2 hours but i can't find any straight answers on whether or not it's actually possible to work around. If anyone could either tell me that it is or isn't possible, tell me how to do it if it is, or suggest an equally effective alternative (for python) i would be extremely grateful.
Yes it is possible! Use this code to determine where your mouse is registering on the screen:
import pyautogui
print('Press Ctrl-C to quit.')
try:
while True:
x, y = pyautogui.position()
positionStr = 'X: ' + str(x).rjust(4) + ' Y: ' + str(y).rjust(4)
print(positionStr, end='')
print('\b' * len(positionStr), end='', flush=True)
except KeyboardInterrupt:
print('\nDone.')
This should be run from a command prompt. The output will be nonsensical if run through IDLE.
not sure if this is clear but I subtracted an extended monitor's horizontal resolution from 0 because my 2nd monitor is on the left of my primary display. That allowed me to avoid the out of bounds warning. my answer probably isn't the clearest but I figured I would chime in to let folks know it actually can work.

How to get the coordinates after a mouse click?

I was using the following code to get the coordinates of a point after a mouse click (keep in mind I was clicking on a random point on the screen, not on a figure):
import win32api
posvals = [[],[]]
x = 0
state_left = win32api.GetKeyState(0x01)
while x<2:
a = win32api.GetKeyState(0x01)
if a != state_left:
state_left = a
print(a)
if a >= 0:
print('button down')
z,y = win32api.GetCursorPos()
posvals[x] = [z,y]
print(z,y)
x += 1
time.sleep(.001)
print(posvals)
Here I saved the coordinates in posvals, and the while loop is there because I only wanted to record 2 clicks. I got and tweaked this code from another question on stackoverflow, but I'm not sure which one.
My current problem is that I'm using a Linux computer and the win32api module (its official name is pywin32) won't work since it is only for windows.
How can I adjust (or completely restart) my code?
So there is no easy way to port the code to linux, unless you run in wrapped with WineLib or equivalent wrapper software. One such explanation of this practice is here.
You could try other mouse position packages like PyMouse. This might be a better option. This question also has some good examples of other more agnostic package options for python mouse coordinates.

Categories