We Keep Coding

Taking Single Click Event and Propagating it to Multiple Areas - python, pynput, pyautogui

Having trouble turning 1 mouse click into multiple mouse clicks. Basically what I want to do is to control multiple windows at once. I want to click on one master window and have the clicks propagate to the subsequent windows. In this snippet there are 4 windows and I track them via determining the offset between it and the master window.
I'm using python3 with pynput for the mouse listener and pyautogui for mouse control.
What I'm having a problem with is setting up the mouse listener such that it listens to my actual clicks but ignores the programmatic clicks. Right now, I think it's getting stuck in an infinite loop where my initial click triggers the on_click event, propagates the clicks, each triggering an additional on_click event, propagates the clicks, etc. When I run the below code it starts fine, and then when I first click it just heavily lags my mouse for a minute before return back to normal with no mouse listener active anymore. My guess is that a failsafe kicks in to return it to normal.
Things I have tried:
using pynput for listener and control - this does not change the outcome
stopping the listener and creating a new one after propagated clicks have finished - bad hacky solution that still did not change the outcome
semaphore locking with _value peeking to ignore events if semaphore has already been acquired - also hacky and did not work
calling propagateActions via threading and waiting for completion before returning from on_click event - did not work
commenting out pyautogui.click() - this allows for expected behavior to move the mouse to the subsequent locations and return it back to its initial position after. Without the click, it works perfect. With the click, it lags and the listener dies.
searching stackoverflow - this question bears a resemblance in terms of outcome, but is unanswered and is trying to achieve something different.
My snippet is below:
from pynput import mouse, keyboard
import pyautogui
pyautogui.PAUSE = 0.01
mouseListener = None
killSwitch = False
# this is just a keyboard listener for a kill switch
def on_release(key):
if key == keyboard.Key.f1:
global killSwitch
print('### Kill switch activated ###')
killSwitch = True
# on mouse release I want to propogate a click to 4 other areas
def on_click(x, y, button, pressed):
print('{0} at {1}'.format('Pressed' if pressed else 'Released', (x, y)))
if not pressed:
propogateActions(x, y, button)
# propogates clicks
def propogateActions(x, y, button):
print('propogating actions to {0} windows'.format(len(offsets)+1))
for offset in offsets:
pyautogui.moveTo(x+offset.x, y+offset.y)
print('mouse moved')
if button == mouse.Button.left:
print('left clicking at ({0}, {1})'.format(x+offset.x, y+offset.y))
pyautogui.click()
pyautogui.moveTo(x, y)
# point class for ease of use
class Point():
def __init__(self, x, y):
self.x = x
self.y = y
def __repr__(self):
return 'Point(x={0}, y={1})'.format(self.x, self.y)
# main method
def doTheThing():
print('started')
while not killSwitch:
pass
# initializations and starting listeners
# offsets tracks how far the subsequent clicks are from the initial click point
offsets = [Point(50, 0), Point(50, 50), Point(0, 50)]
keyboardListener = keyboard.Listener(on_release=on_release)
mouseListener = mouse.Listener(on_click=on_click)
keyboardListener.start()
mouseListener.start()
doTheThing()
My Question:
Is there some way to listen only for "real" clicks and not programmatic clicks?
If not, can I pause the mouse listener and then restart it some way after the propagated clicks have occurred?
This is the small section of code that's relevant to the issue at hand. offsets has an initialization that sets it more appropriately and there's other bells and whistles, but this is the section relevant to the problem. I appreciate your help.

Found the answer! Had to go a layer deeper.
Pynput has a method of suppressing events that exposes the win32 data behind the click event. Ran a test of one of my clicks vs a pyautogui.click() and lo-and-behold there is a difference. The data.flags was set to value 0 on a user click event and set to value 1 on a programmatic click.
That's good enough for me to filter on. This is the pertinent filter:
def win32_event_filter(msg, data):
if data.flags:
print('suppressing event')
return False
added that to my above code and changed the
mouseListener = mouse.Listener(on_click=on_click)
to
mouseListener = mouse.Listener(on_click=on_click, win32_event_filter=win32_event_filter)
and it works!
My real clicks prevail, programmatic clicks are propagated, and I am not stuck in an infinite loop. Hope this helps if others run into this issue.

Trying to create a mouse recorder, but it keeps looping endlessly?

I'm trying my hand at Pynput, and I'm starting off with creating a simple program to record the movements of a mouse, and then replay those movements once a button is clicked.
However, every time I click the mouse, it just starts to freak out and endlessly loop. I think it's going through the movements at a super high speed, but I eventually have to Alt-F4 the shell to stop it.
Any help would be appreciated.
import pynput
arr = []
from pynput import mouse
mou = pynput.mouse.Controller()
def on_move(x,y):
Pos = mou.position
arr.append(Pos)
def on_click(x, y, button, pressed):
listener.stop()
for i in arr:
mou.position = i
print("Done")
listener = mouse.Listener(on_move = on_move, on_click=on_click)
listener.start()

You have to be careful when using multiple threads (which is the case here, since mouse.Listener runs in its own thread). Apparently, as long as you are in the callback function, all events are still processed, even after you have called listener.stop(). So when replaying, for each mouse position you set, the on_move callback function is called, so that mouse position is added to your list again, which causes the endless loop.
In general, it's bad practice to implement too much functionality (in this case the "replaying") in a callback function. A better solution would be to use an event to signal another thread that the mouse button has been clicked. See the following example code. A few remarks:
I've added a few print statements to see what's happening.
I've added a small delay between the mouse positions to really see the playback. (NB: This also might make breaking out of the application a bit easier in case it hangs!)
I've changed a few variable names to make more sense. Calling an array "arr" is not a good idea. Try to use names that really describe the variable. In this case it is a list of positions, so I choose to call it positions.
I'm using return False to stop the mouse controller. The documentation states "Call pynput.mouse.Listener.stop from anywhere, raise StopException or return False from a callback to stop the listener.", but personally, I think returning False is the cleanest and safest solution.
import threading
import time
import pynput
positions = []
clicked = threading.Event()
controller = pynput.mouse.Controller()
def on_move(x, y):
print(f'on_move({x}, {y})')
positions.append((x, y))
def on_click(x, y, button, pressed):
print(f'on_move({x}, {y}, {button}, {pressed})')
# Tell the main thread that the mouse is clicked
clicked.set()
return False
listener = pynput.mouse.Listener(on_move=on_move, on_click=on_click)
listener.start()
try:
listener.wait()
# Wait for the signal from the listener thread
clicked.wait()
finally:
listener.stop()
print('*REPLAYING*')
for position in positions:
controller.position = position
time.sleep(0.01)
Note that when you run this in a Windows command prompt, the application might hang because you have pressed the mouse button and are then starting to send mouse positions. This causes a "drag" movement, which pauses the terminal. If this happens, you can just press Escape and the program will continue to run.

You got yourself an infinite loop. I think the listener you referred to in the on_click method might be null or undefined. Also according to some documentation I found you need to return false for the on_click method to stop listening
This is what I was looking at:
https://pythonhosted.org/pynput/mouse.html

Pygame: Why do both buttons activate if one hasn't been checked yet?

I've set up a basic Pygame graphical interface, but I'm having trouble with my buttons. I created a Button class, and the function to be executed by the button is determined in the __init__() method. In other words, I input the function when I create an instance of Button via lambda expression. The relevant code of the buttons basically looks like this:
class Button():
def __init__(self, action):
self.command = action
def check(self): # To be called while iterating through pygame.event.get()
if event.type == MOUSEBUTTONUP and self.rect.collidepoint(pygame.mouse.get_pos()):
self.command()
I also created a Window class where each instance is list of the buttons to be seen at a time:
class Window():
def __init__(self, buttons):
self.is_visible = False # This determines whether the buttons in this
# window should be updated, checked, and drawn
self.buttons = list(buttons)
def open(self):
self.is_visible = True
def close(self):
self.is_visible = False
def trans(self, new_window):
self.close()
new_window.open()
Next, I set up two instances of Window, each with a Button to toggle back to the other:
WINDOW_1 = Window([Button(lambda: WINDOW_1.trans(WINDOW_2))])
WINDOW_2 = Window([Button(lambda: WINDOW_2.trans(WINDOW_1))])
And finally:
WINDOW_1.is_visible = True
Here comes the problem.
Each button works exactly how it is supposed to: it closes the open window and opens the closed window. Unfortunately, if I click the mouse in a spot where both buttons overlap (or where they would overlap if they were both visible), the function for WINDOW_2's button is called immediately after the function for WINDOW_1's button is called. Basically, WINDOW_1 -> WINDOW_2 -> WINDOW_1, and it all happens in the same loop.
However, if we start from WINDOW_2, then this happens: WINDOW_2 -> WINDOW_1. It appears that the glitch is only one-way. I just can't figure out what's wrong here, I would really appreciate some help. (Just in case, here's a link to the full code so you can maybe reproduce the problem; I've set the buttons' location so that the bottom half of the first button overlaps the top half of the second: http://pastebin.com/m1zCQLRF). Thank you for reading and thank you in advance for answering!

Consider these lines (from the pastebin):
all_windows=[WINDOW_1,WINDOW_2]
....
for event in pygame.event.get():
for window in all_windows:
if window.is_visible:
for button in window.buttons:
button.update()
button.check()
If WINDOW_1's button is clicked, WINDOW_2.is_visible becomes True.
In the next iteration of the for window in all_windows loop, the check method of WINDOW_2 will be called because its is_visible attribute is now True. Because this is still the same iteration of for event in pygame.event.get(), WINDOW_2.check() sees the same MOUSEBUTTONUP event. The Button objects overlap, so the event causes the windows' visibility to toggle a second time, back to the state where WINDOW_1 is visible, and this is what is drawn.
Incidentally, using event.pos would be more accurate than pygame.mouse.get_pos() in the check() method. The former is the mouse's position at the time the event was posted, while the latter is the current position of the mouse.
Edit
I got the pastebin running with some tweaking and verified that what I described above is the problem by applying a quick and dirty fix.
First, I edited Button.check() so it returns a boolean that shows whether or not its check validated:
def check(self):
if event.type==MOUSEBUTTONUP and self.rect.collidepoint(event.pos):
self.command()
return True
return False
Then altered the code shown above to break out of the all_windows loop if check returns True (i.e. if a window is closed).
for window in all_windows:
if window.is_visible:
for button in window.buttons:
button.update()
_break = button.check()
if _break:
break
Now the windows close and open as expected when a button is clicked. Again, that's not a "real" fix just a confirmation of what caused the problem.

Python script to control mouse clicks

I created a small Python script using win32api to use on the popular game Cookie Clicker (a game where you have to click on a Big Cookie to gain points) just for fun. It has a function called "auto_clicker" that do just that: keeps clicking on the screen on the point the user defined. This is the script:
# -*- coding: utf-8 -*-
import win32con
import win32api
def clicker(x,y):
"""Clicks on given position x,y
Input:
x -- Horizontal position in pixels, starts from top-left position
y -- Vertical position in pixels, start from top-left position
"""
win32api.SetCursorPos((x,y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)
def auto_clicker(x = -1,y = -1):
"""Keep clicking on position x,y. If no input is given, gets from actual
mouse position.
"""
if x == -1 | y == -1:
x,y = win32api.GetCursorPos()
while True:
clicker(x,y)
It works nicely, but I want to make some improvements:
How can I get the cursor position only when the user clicks instead when the function is called? I would prefer to not add another module
since win32api seems to contain everything I needed. Tried this
method without success.
How can I detect a keypress like "Escape", so I can exit from my program without the ugly hack I am using now (Ctrl+Alt+Del seems to give SetCursorPos denied access, so Python throws a error and exit the program).
Can I make this program portable? Seems like I can do using Tkinter and generating a invisible Tk window, but I tried to write something without success.

I don't think with win32api you can listen to clicks you can just generate them (not sure though). However, try using pyHook, it's a simple api easy to use and can be found here http://sourceforge.net/apps/mediawiki/pyhook/index.php?title=Main_Page. With pyhook you can create a listener to listen to a mouse event and upon a mouse click you can do whatever you want, the example in the link shows you how. As for key press, you can use the same api for that too, also an example is provided, good luck!

use pynput . It can control mouse, keyboard, etc.
examples:
from pynput.mouse import Button, Controller
mouse = Controller()
# Read pointer position
print('The current pointer position is {0}'.format(
mouse.position))
# Set pointer position
mouse.position = (10, 20)
print('Now we have moved it to {0}'.format(
mouse.position))
# Move pointer relative to current position
mouse.move(5, -5)
# Press and release
mouse.press(Button.left)
mouse.release(Button.left)
# Double click; this is different from pressing and releasing
# twice on Mac OSX
mouse.click(Button.left, 2)
# Scroll two steps down
mouse.scroll(0, 2)

Blender Mouse Over not returning correct HitObject

So I have a Python controller which has two sensors hooked up to it, a Mouse Left Button sensor and a Mouse Over sensor, both have TRUE level triggering enabled with a frequency of 0. The Python controller is linked to a script which is shown below:
# Gather information.
scene = GameLogic.getCurrentScene();
camera = scene.active_camera;
controller = GameLogic.getCurrentController();
# Change to Earth camera.
clicked = controller.sensors['MouseClick'].positive;
if clicked:
hitObject = controller.sensors['MouseOver'].hitObject;
if hitObject is not None:
print(hitObject.name);
if(hitObject.name == 'Earth'):
scene.active_camera = 'Earth Camera';
else:
print('Nothing hit!');
Basically, it is supposed to check if the mouse's left button was clicked, and if so grab the hitObject of the mouse over sensor and print out the name of it, if the mouse over object is None, it prints out "Nothing hit!". However, in the game when I click on an object (such as the Earth, which is hard to miss clicking on) all it does is print out, "Nothing hit!".
Is there something wrong with the way I am using the sensors? Is my Python incorrect here? Why is controller.sensors['MouseOver'].hitObject always of type None?

the object, you want to click should not have the Physics Type 'No Collision'. I know that 'Static' works.
Good luck.