Using pyautogui with multiple monitors - python

I'm trying to use the pyautogui module for python to automate mouse clicks and movements. However, it doesn't seem to be able to recognise any monitor other than my main one, which means i'm not able to input any actions on any of my other screens, and that is a huge problem for the project i am working on.
I've searched google for 2 hours but i can't find any straight answers on whether or not it's actually possible to work around. If anyone could either tell me that it is or isn't possible, tell me how to do it if it is, or suggest an equally effective alternative (for python) i would be extremely grateful.

Yes it is possible! Use this code to determine where your mouse is registering on the screen:
import pyautogui
print('Press Ctrl-C to quit.')
try:
while True:
x, y = pyautogui.position()
positionStr = 'X: ' + str(x).rjust(4) + ' Y: ' + str(y).rjust(4)
print(positionStr, end='')
print('\b' * len(positionStr), end='', flush=True)
except KeyboardInterrupt:
print('\nDone.')
This should be run from a command prompt. The output will be nonsensical if run through IDLE.

not sure if this is clear but I subtracted an extended monitor's horizontal resolution from 0 because my 2nd monitor is on the left of my primary display. That allowed me to avoid the out of bounds warning. my answer probably isn't the clearest but I figured I would chime in to let folks know it actually can work.

Related

Python; "simulate" mouse and keyboard

I recently started learning Python, trying to make short programs with simple functions but that I can use in one way or another while learning.
I have studied a bit several libraries that they functions are what the title say, with some I have had better experiences than with others, but at the moment I have a problem when trying to make a "Shift+Click" in concrete positions of the screen.
Im actually using win32api and win32con since they are the ones that to my way of seeing have better results for what Im trying to do.
Its nothing very complicated, its just do a Shift + Left Click in specific positions, like same way that you can recreate it in a normally with your mouse and keyboard.
Here is a simplified version of the code:
def mouse_click(x, y):
win32api.SetCursorPos((x, y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN, x, y, 0, 0)
time.sleep(0.065)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP, x, y, 0, 0)
def shift_down():
win32api.keybd_event(win32con.VK_LSHIFT, 0, 0, 0)
time.sleep(0.005)
def shift_up():
win32api.keybd_event(win32con.VK_LSHIFT, 0, win32con.KEYEVENTF_KEYUP, 0)
time.sleep(0.005)
def clicks():
for offs in coord:
x, y = offs
x += random.randint(-15, 15)
y += random.randint(-15, 15)
if random.random() > 0.0:
shift_down()
time.sleep(0.005)
mouse_click(x, y)
time.sleep(0.005)
shift_up()
print('click on', offs)
def main():
while True:
if win32api.GetAsyncKeyState(win32con.VK_INSERT):
break
elif win32api.GetAsyncKeyState(win32con.VK_F2):
clicks()
print('clicking end')
if __name__ == '__main__':
main()
Yes, Im aware that some sleep are unnecessary, but here is my current problem. The way you see above is the best way I have had the code respond correctly and in time according to my needs.if I low the time on mouse_click below 0.55 some clicks start to be ignored, from what I have read and seen it should not be necessary make pauses, but at the moment this is the solution I found.So I want to fix the basic formatting of it first.
Am I missing something or is it some kind of "optimization" problem?
Any hints or improvements are welcome even if its about using another library.
I would recommend using the PyAutoGUI module available here: https://pyautogui.readthedocs.io/
It is very beginner friendly and very easy to use!
You would perform a shift-click like this:
import pyautogui
pyautogui.keyDown('shift')
pyautogui.click()
pyautogui.keyUp('shift')
Somehow these randoms sleep are no longer necessary, and the sleep in mouse_click() I could lower it to a maximum of 0.06 without problems.
The press and release Shift seems to work fine in other applications, so I deduce that it is some kind of incompatibility with the application/window itself (Im not trying to force or use it in a particular window or screen). So maybe it can work with some other library than the ones currently in use.
The problem with the pyautogui and pynput library, is that the same thing happens but they also add some kind of delay by default.
In any case, setting the Shift press outside of if in clicks() seems to be the solution as well as using win32con.KEYEVENTF_EXTENDEDKEY inside win32api.keybd_event().
I don't see any point in going on with this and it will probably end up being better if I hold down Shift by myself.

PyAutoGui While loop keep going when if statement remains true

I've just started learning python and started working on a project.
I'm moving my mouse cursor from one area to another. I just want it to stop when it reach a certain point/area/zone. I'm using the mouse position to tell me where it is, currently.
For some reason, when this loops starts, it keeps looping even when the IF statement is true.
But if I started when the IF statement is true, the loops kinda works as intended, so far it's only reading the 'X' values.
I couldn't find an answer to this, or any questions like it. If anyone has an idea or can point me to a similar post, I'll appreciate it.
import pyautogui, sys, time, autoit
#Search for a position on screen manually
try:
while True:
x, y = pyautogui.position()
print(pyautogui.position())
print('Stopping for 1 seconds, keep searching or CTRL + C to end')
time.sleep(1)
#Confirmed location on screen.
if pyautogui.position(x,y) >= pyautogui.position(710, 15):
pyautogui.leftClick()
print('The Eagle has landed')
print(pyautogui.position())
break
Update: I got it! Following mkrieger1 advice, I manage to get the 'x, y' values to update. Code was rewritten.

Capturing key presses from the terminal

I am trying to write a program that captures at any time which key is being pressed. The ultimate goal is to control a robot from the keyboard, e.g. using wasd keys to control movement. It's rather easy using pygame, but as I want to be able to access my robot over SSH, I am looking for a purely bash-based solution. The curses library seems to be the way to go (but please let me know if there is a better solution). I now have something like this:
import curses
def main(screen):
screen.timeout(50)
key = ''
while key != 'q':
try:
key = screen.getkey()
screen.addstr(0, 0, 'key: {:<10}'.format(key))
except:
screen.addstr(0, 0, 'key: {:<10}'.format('N/A'))
if __name__ == '__main__':
curses.wrapper(main)
which largely works fine, except for some unexpected behaviour: when I press and hold a key, it very briefly (and only once) alternates between the key in question and the 'N/A' fallback, as if I press and release and then press and hold.
I think my issue is caused by the character repeat delay setting on my machine. Indeed, if I increase the curses timeout setting to half a second, the problem is 'solved', but I want my program to be more responsive than half a second. Any way I can overrule the character repeat delay within my program? Or alternative solutions?
Note: Not sure if this is relevant, but I am using bash on Windows 10 (WSL) for testing purposes, and would ultimately like to run it on Raspbian.

How to get the coordinates after a mouse click?

I was using the following code to get the coordinates of a point after a mouse click (keep in mind I was clicking on a random point on the screen, not on a figure):
import win32api
posvals = [[],[]]
x = 0
state_left = win32api.GetKeyState(0x01)
while x<2:
a = win32api.GetKeyState(0x01)
if a != state_left:
state_left = a
print(a)
if a >= 0:
print('button down')
z,y = win32api.GetCursorPos()
posvals[x] = [z,y]
print(z,y)
x += 1
time.sleep(.001)
print(posvals)
Here I saved the coordinates in posvals, and the while loop is there because I only wanted to record 2 clicks. I got and tweaked this code from another question on stackoverflow, but I'm not sure which one.
My current problem is that I'm using a Linux computer and the win32api module (its official name is pywin32) won't work since it is only for windows.
How can I adjust (or completely restart) my code?
So there is no easy way to port the code to linux, unless you run in wrapped with WineLib or equivalent wrapper software. One such explanation of this practice is here.
You could try other mouse position packages like PyMouse. This might be a better option. This question also has some good examples of other more agnostic package options for python mouse coordinates.

not able to hide message box in wxpython

hi guys tried everything and now i am asking your suggestions.
I am not able to hide my messagebox.
code is:
ans = wx.MessageBox("length = %s , height = %s " % (str(len), str(ht)))
if ans = wx.Yes:
subprocess.call("pword | sudo -S ./Install.sh %s %s" % (str(len),str(ht)))
3rd line take around 6-7 min to complete .till that time message box will be on screen ,i want to make it hide till installation process get finish.
I tried hide ,future call,close ,Disable..but everytime i get 'int object has no attribute " ".
The message box must be hidden and destroyed by the time wxMessageBox() returns, there must be something else going on here.
In general, if you want to be sure that all pending events have been processed (and so all windows that should be repainted/resized/closed were indeed handled), use wx.CallAfter() to launch your long-running process at a slightly later time.
Of course, running a blocking operation taking several minutes from a GUI program is a bad idea anyhow, but this is another story...
If you need more control over the message box use a message dialog instead. The message box returns the answer directly, so you can't control the window. Creating a message dialog lets you control the window.
Here is a simple example:
import wx
app = wx.App(False)
box = wx.MessageDialog(None, 'Yes or No?', caption='Dialog Title', style=wx.YES_NO | wx.YES_DEFAULT)
result = box.ShowModal()
box.Destroy()
if result == wx.ID_YES:
print 'yes'
else:
print 'no'

Categories