How can I detect mouse clicks regardless of the window the mouse is in?
Perferabliy in python, but if someone can explain it in any langauge I might be able to figure it out.
I found this on microsoft's site:
http://msdn.microsoft.com/en-us/library/ms645533(VS.85).aspx
But I don't see how I can detect or pick up the notifications listed.
Tried using pygame's pygame.mouse.get_pos() function as follows:
import pygame
pygame.init()
while True:
print pygame.mouse.get_pos()
This just returns 0,0.
I'm not familiar with pygame, is something missing?
In anycase I'd prefer a method without the need to install a 3rd party module.
(other than pywin32 http://sourceforge.net/projects/pywin32/ )
The only way to detect mouse events outside your program is to install a Windows hook using SetWindowsHookEx. The pyHook module encapsulates the nitty-gritty details. Here's a sample that will print the location of every mouse click:
import pyHook
import pythoncom
def onclick(event):
print event.Position
return True
hm = pyHook.HookManager()
hm.SubscribeMouseAllButtonsDown(onclick)
hm.HookMouse()
pythoncom.PumpMessages()
hm.UnhookMouse()
You can check the example.py script that is installed with the module for more info about the event parameter.
pyHook might be tricky to use in a pure Python script, because it requires an active message pump. From the tutorial:
Any application that wishes to receive
notifications of global input events
must have a Windows message pump. The
easiest way to get one of these is to
use the PumpMessages method in the
Win32 Extensions package for Python.
[...] When run, this program just sits
idle and waits for Windows events. If
you are using a GUI toolkit (e.g.
wxPython), this loop is unnecessary
since the toolkit provides its own.
I use win32api. It works when clicking on any windows.
# Code to check if left or right mouse buttons were pressed
import win32api
import time
state_left = win32api.GetKeyState(0x01) # Left button down = 0 or 1. Button up = -127 or -128
state_right = win32api.GetKeyState(0x02) # Right button down = 0 or 1. Button up = -127 or -128
while True:
a = win32api.GetKeyState(0x01)
b = win32api.GetKeyState(0x02)
if a != state_left: # Button state changed
state_left = a
print(a)
if a < 0:
print('Left Button Pressed')
else:
print('Left Button Released')
if b != state_right: # Button state changed
state_right = b
print(b)
if b < 0:
print('Right Button Pressed')
else:
print('Right Button Released')
time.sleep(0.001)
It's been a hot minute since this question was asked, but I thought I'd share my solution: I just used the built-in module ctypes. (I'm using Python 3.3 btw)
import ctypes
import time
def DetectClick(button, watchtime = 5):
'''Waits watchtime seconds. Returns True on click, False otherwise'''
if button in (1, '1', 'l', 'L', 'left', 'Left', 'LEFT'):
bnum = 0x01
elif button in (2, '2', 'r', 'R', 'right', 'Right', 'RIGHT'):
bnum = 0x02
start = time.time()
while 1:
if ctypes.windll.user32.GetKeyState(bnum) not in [0, 1]:
# ^ this returns either 0 or 1 when button is not being held down
return True
elif time.time() - start >= watchtime:
break
time.sleep(0.001)
return False
Windows MFC, including GUI programming, is accessible with python using the Python for Windows extensions by Mark Hammond. An O'Reilly Book Excerpt from Hammond's and Robinson's book shows how to hook mouse messages, .e.g:
self.HookMessage(self.OnMouseMove,win32con.WM_MOUSEMOVE)
Raw MFC is not easy or obvious, but searching the web for python examples may yield some usable examples.
The windows way of doing it is to handle the WM_LBUTTONDBLCLK message.
For this to be sent, your window class needs to be created with the CS_DBLCLKS class style.
I'm afraid I don't know how to apply this in Python, but hopefully it might give you some hints.
Related
I have been learning python recently and decided to learn more about pyautogui, what you see down below is a macro of mine. My question is simple; is there a way to let this macro run in a specific window. For example: I want this macro to run in google chrome while I am in discord chatting with my friends (text channel so I'm not in the google chrome window). (Ignore my sloppy method of writing code)
import pyautogui
import random
import time
import mouse
#############################
tijd = 0
actief = 0
float (actief)
#############################
while not mouse.is_pressed('right'):
time.sleep(0.01)
bank_x1, bank_y1 = pyautogui.position()
time.sleep (0.5)
while not mouse.is_pressed('right'):
time.sleep(0.01)
bank_x2, bank_y2 = pyautogui.position()
print ("{} {} {} {}".format(bank_x1,bank_x2,bank_y1,bank_y2))
#############################
lijst = [[bank_x1,bank_x2,bank_y1,bank_y2,200,243],[1203,1236,721,749,23,49],[390,422,112,140,22,46]]
while not mouse.is_pressed('middle') or actief > tijd:
for i in range(0, 4):
x = random.randint(lijst[i][0], lijst[i][1])
y = random.randint(lijst[i][2], lijst[i][3])
pyautogui.moveTo(x, y)
wacht = random.randint(lijst[i][4], lijst[i][5]) / 100
time.sleep(wacht)
str (actief_str)
pyautogui.click()
pyautogui.press('esc')
No. Not with pyautogui.
Pyautogui simulates actual keyboard and mouse input to the system, not to a specific window. Thus, it will always act exactly the same as if you were pressing the keys and clicking the mouse yourself. So, no. You cannot use it to send keystrokes to background applications. The keystrokes will always effect the window that is focused, just as they do when you physically input them with a keyboard or mouse.
This question shows how you could achieve it with winapi, but it is much more complicated and less user-friendly than pyautogui.
I was using the following code to get the coordinates of a point after a mouse click (keep in mind I was clicking on a random point on the screen, not on a figure):
import win32api
posvals = [[],[]]
x = 0
state_left = win32api.GetKeyState(0x01)
while x<2:
a = win32api.GetKeyState(0x01)
if a != state_left:
state_left = a
print(a)
if a >= 0:
print('button down')
z,y = win32api.GetCursorPos()
posvals[x] = [z,y]
print(z,y)
x += 1
time.sleep(.001)
print(posvals)
Here I saved the coordinates in posvals, and the while loop is there because I only wanted to record 2 clicks. I got and tweaked this code from another question on stackoverflow, but I'm not sure which one.
My current problem is that I'm using a Linux computer and the win32api module (its official name is pywin32) won't work since it is only for windows.
How can I adjust (or completely restart) my code?
So there is no easy way to port the code to linux, unless you run in wrapped with WineLib or equivalent wrapper software. One such explanation of this practice is here.
You could try other mouse position packages like PyMouse. This might be a better option. This question also has some good examples of other more agnostic package options for python mouse coordinates.
I need to do some macros and I wanna know what is the most recommended way to do it.
So, I need to write somethings and click some places with it and I need to emulate the TAB key to.
I do automated testing stuff in Python. I tend to use the following:
http://www.tizmoi.net/watsup/intro.html
Edit: Link is dead, archived version: https://web.archive.org/web/20100224025508/http://www.tizmoi.net/watsup/intro.html
http://www.mayukhbose.com/python/IEC/index.php
I do not always (almost never) simulate key presses and mouse movement. I usually use COM to set values of windows objects and call their .click() methods.
You can send keypress signals with this:
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shell.SendKeys("^a") # CTRL+A may "select all" depending on which window's focused
shell.SendKeys("{DELETE}") # Delete selected text? Depends on context. :P
shell.SendKeys("{TAB}") #Press tab... to change focus or whatever
This is all in Windows. If you're in another environment, I have no clue.
Maybe you are looking for Sendkeys?
SendKeys is a Python module for
Windows that can send one or more
keystrokes or keystroke combinations
to the active window.
it seems it is windows only
Also you have pywinauto (copied from my SO answer)
pywinauto is a set of open-source
(LGPL) modules for using Python as a
GUI automation 'driver' for Windows NT
based Operating Systems (NT/W2K/XP).
and example from the web page
> from pywinauto import application
> app = application.Application.start("notepad.exe")
> app.notepad.TypeKeys("%FX")
> app.Notepad.MenuSelect("File->SaveAs")
> app.SaveAs.ComboBox5.Select("UTF-8")
> app.SaveAs.edit1.SetText("Example-utf8.txt")
> app.SaveAs.Save.Click()
pyautogui is a great package to send keys and automate several keyboard / mouse related tasks. Check out Controlling the Keyboard and Mouse with GUI Automation and PyAutoGUI’s documentation.
You can use PyAutoGUI library for Python which works on Windows, macOS, and Linux.
Mouse
Here is a simple code to move the mouse to the middle of the screen:
import pyautogui
screenWidth, screenHeight = pyautogui.size()
pyautogui.moveTo(screenWidth / 2, screenHeight / 2)
Docs page: Mouse Control Functions.
Related question: Controlling mouse with Python.
Keyboard
Example:
pyautogui.typewrite('Hello world!') # prints out "Hello world!" instantly
pyautogui.typewrite('Hello world!', interval=0.25) # prints out "Hello world!" with a quarter second delay after each character
Docs page: Keyboard Control Functions.
More reading: Controlling the Keyboard and Mouse with GUI Automation (Chapter 18 of e-book).
Related questions:
Python GUI automation library for simulating user interaction in apps.
Python simulate keydown.
Two other options are:
pynput - https://pypi.org/project/pynput/ - which is for Windows (tested), Linux and MacOS- docs are at https://pynput.readthedocs.io/en/latest/
PyDirectInput - https://pypi.org/project/PyDirectInput/ - which is for Windows only and can be used with (or without) PyAutoGUI
Warning - if you are wanting to use keyboard control for games, then pynput doesn't always work - e.g. it works for Valheim, but not for the Witcher 3 - which is where PyDirectInput will work instead. I also tested PyDirectInput and it works for Half life 2 (as a test of an older game).
Tip - You will likely need to reduce (don't remove for games) the delay between character typing - use pydirectinput.PAUSE = 0.05
As an example, here is a function that allows virtual keyboard typing - currently only tested on Windows:
from pynput import keyboard
try:
import pydirectinput
pydirectinput.PAUSE = 0.05
except ImportError as err:
pydirectinput = False
print("pydirectinput not found:")
def write_char(ch):
upper = ch.isupper()
if pydirectinput and pydirectinput.KEYBOARD_MAPPING.get(ch.lower(), False):
if upper:
pydirectinput.keyDown('shift')
print('^')
pydirectinput.write(ch.lower(), interval=0.0)
print(ch)
if upper:
pydirectinput.keyUp('shift')
else:
keyboard.Controller().type(ch)
This allows a string to be sent in, with upper case alphabetic characters handled through pydirectinput. When characters don't simply map, the function falls back to using pynput. Note that PyAutoGUI also can't handled some shifted characters - such as the £ symbol, etc.
I need to do some macros and I wanna know what is the most recommended way to do it.
So, I need to write somethings and click some places with it and I need to emulate the TAB key to.
I do automated testing stuff in Python. I tend to use the following:
http://www.tizmoi.net/watsup/intro.html
Edit: Link is dead, archived version: https://web.archive.org/web/20100224025508/http://www.tizmoi.net/watsup/intro.html
http://www.mayukhbose.com/python/IEC/index.php
I do not always (almost never) simulate key presses and mouse movement. I usually use COM to set values of windows objects and call their .click() methods.
You can send keypress signals with this:
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shell.SendKeys("^a") # CTRL+A may "select all" depending on which window's focused
shell.SendKeys("{DELETE}") # Delete selected text? Depends on context. :P
shell.SendKeys("{TAB}") #Press tab... to change focus or whatever
This is all in Windows. If you're in another environment, I have no clue.
Maybe you are looking for Sendkeys?
SendKeys is a Python module for
Windows that can send one or more
keystrokes or keystroke combinations
to the active window.
it seems it is windows only
Also you have pywinauto (copied from my SO answer)
pywinauto is a set of open-source
(LGPL) modules for using Python as a
GUI automation 'driver' for Windows NT
based Operating Systems (NT/W2K/XP).
and example from the web page
> from pywinauto import application
> app = application.Application.start("notepad.exe")
> app.notepad.TypeKeys("%FX")
> app.Notepad.MenuSelect("File->SaveAs")
> app.SaveAs.ComboBox5.Select("UTF-8")
> app.SaveAs.edit1.SetText("Example-utf8.txt")
> app.SaveAs.Save.Click()
pyautogui is a great package to send keys and automate several keyboard / mouse related tasks. Check out Controlling the Keyboard and Mouse with GUI Automation and PyAutoGUI’s documentation.
You can use PyAutoGUI library for Python which works on Windows, macOS, and Linux.
Mouse
Here is a simple code to move the mouse to the middle of the screen:
import pyautogui
screenWidth, screenHeight = pyautogui.size()
pyautogui.moveTo(screenWidth / 2, screenHeight / 2)
Docs page: Mouse Control Functions.
Related question: Controlling mouse with Python.
Keyboard
Example:
pyautogui.typewrite('Hello world!') # prints out "Hello world!" instantly
pyautogui.typewrite('Hello world!', interval=0.25) # prints out "Hello world!" with a quarter second delay after each character
Docs page: Keyboard Control Functions.
More reading: Controlling the Keyboard and Mouse with GUI Automation (Chapter 18 of e-book).
Related questions:
Python GUI automation library for simulating user interaction in apps.
Python simulate keydown.
Two other options are:
pynput - https://pypi.org/project/pynput/ - which is for Windows (tested), Linux and MacOS- docs are at https://pynput.readthedocs.io/en/latest/
PyDirectInput - https://pypi.org/project/PyDirectInput/ - which is for Windows only and can be used with (or without) PyAutoGUI
Warning - if you are wanting to use keyboard control for games, then pynput doesn't always work - e.g. it works for Valheim, but not for the Witcher 3 - which is where PyDirectInput will work instead. I also tested PyDirectInput and it works for Half life 2 (as a test of an older game).
Tip - You will likely need to reduce (don't remove for games) the delay between character typing - use pydirectinput.PAUSE = 0.05
As an example, here is a function that allows virtual keyboard typing - currently only tested on Windows:
from pynput import keyboard
try:
import pydirectinput
pydirectinput.PAUSE = 0.05
except ImportError as err:
pydirectinput = False
print("pydirectinput not found:")
def write_char(ch):
upper = ch.isupper()
if pydirectinput and pydirectinput.KEYBOARD_MAPPING.get(ch.lower(), False):
if upper:
pydirectinput.keyDown('shift')
print('^')
pydirectinput.write(ch.lower(), interval=0.0)
print(ch)
if upper:
pydirectinput.keyUp('shift')
else:
keyboard.Controller().type(ch)
This allows a string to be sent in, with upper case alphabetic characters handled through pydirectinput. When characters don't simply map, the function falls back to using pynput. Note that PyAutoGUI also can't handled some shifted characters - such as the £ symbol, etc.
I created a small Python script using win32api to use on the popular game Cookie Clicker (a game where you have to click on a Big Cookie to gain points) just for fun. It has a function called "auto_clicker" that do just that: keeps clicking on the screen on the point the user defined. This is the script:
# -*- coding: utf-8 -*-
import win32con
import win32api
def clicker(x,y):
"""Clicks on given position x,y
Input:
x -- Horizontal position in pixels, starts from top-left position
y -- Vertical position in pixels, start from top-left position
"""
win32api.SetCursorPos((x,y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)
def auto_clicker(x = -1,y = -1):
"""Keep clicking on position x,y. If no input is given, gets from actual
mouse position.
"""
if x == -1 | y == -1:
x,y = win32api.GetCursorPos()
while True:
clicker(x,y)
It works nicely, but I want to make some improvements:
How can I get the cursor position only when the user clicks instead when the function is called? I would prefer to not add another module
since win32api seems to contain everything I needed. Tried this
method without success.
How can I detect a keypress like "Escape", so I can exit from my program without the ugly hack I am using now (Ctrl+Alt+Del seems to give SetCursorPos denied access, so Python throws a error and exit the program).
Can I make this program portable? Seems like I can do using Tkinter and generating a invisible Tk window, but I tried to write something without success.
I don't think with win32api you can listen to clicks you can just generate them (not sure though). However, try using pyHook, it's a simple api easy to use and can be found here http://sourceforge.net/apps/mediawiki/pyhook/index.php?title=Main_Page. With pyhook you can create a listener to listen to a mouse event and upon a mouse click you can do whatever you want, the example in the link shows you how. As for key press, you can use the same api for that too, also an example is provided, good luck!
use pynput . It can control mouse, keyboard, etc.
examples:
from pynput.mouse import Button, Controller
mouse = Controller()
# Read pointer position
print('The current pointer position is {0}'.format(
mouse.position))
# Set pointer position
mouse.position = (10, 20)
print('Now we have moved it to {0}'.format(
mouse.position))
# Move pointer relative to current position
mouse.move(5, -5)
# Press and release
mouse.press(Button.left)
mouse.release(Button.left)
# Double click; this is different from pressing and releasing
# twice on Mac OSX
mouse.click(Button.left, 2)
# Scroll two steps down
mouse.scroll(0, 2)