I am trying to write a python script that will type out lines from a text document as if they were coming from a keyboard.
I already have a code snippet working in some apps (see below), and this types every line from the file I open correctly, I tested the output into notepad++ for example and it types it all out.
import keyboard
import time
time.sleep(3) """ this gives me enough time to alt-tab into the game (Witcher 3)
that I am trying to have the keypresses inserted into, I also tried some code with
win32gui that brough the Witcher 3 app to the front, but this is simpler. """
with open('w3recipes.txt', 'r', encoding='utf-8') as recipes:
for line in recipes:
keyboard.write(line)
time.sleep(0.05)
The issue is that these keystrokes are not registered by Witcher 3, the game I am trying to write all these keystrokes to. I tried changing the game from fullscreen to windowed with no luck, and I tried compiling the script to a .exe and running it as an admin as well, no dice. I also tried the pynput library as opposed to the keyboard library used here and that yielded the same result.
Any help would be appreciated, I am trying to write a few hundred console commands to this game and there is no newline character in the game's console; it only supports 1 command at a time before hitting enter. My only other option is sitting here copy-pasting all the lines in which would be tiresome.
Thanks in advance.
Use another library, pydirectinput. It's an updated version of pyautogui, and I've found it works with most (if not all) games.
Quoting from docs:
This library aims to replicate the functionality of the PyAutoGUI mouse and keyboard inputs, but by utilizing DirectInput scan codes and the more modern SendInput() win32 function. PyAutoGUI uses Virtual Key Codes (VKs) and the deprecated mouse_event() and keybd_event() win32 functions. You may find that PyAutoGUI does not work in some applications, particularly in video games and other software that rely on DirectX. If you find yourself in that situation, give this library a try!
Write functions:
>>> import pyautogui
>>> import pydirectinput
>>> pydirectinput.moveTo(100, 150) # Move the mouse to the x, y coordinates 100, 150.
>>> pydirectinput.click() # Click the mouse at its current location.
>>> pydirectinput.click(200, 220) # Click the mouse at the x, y coordinates 200, 220.
>>> pydirectinput.move(None, 10) # Move mouse 10 pixels down, that is, move the mouse relative to its current position.
>>> pydirectinput.doubleClick() # Double click the mouse at the
>>> pydirectinput.press('esc') # Simulate pressing the Escape key.
>>> pydirectinput.keyDown('shift')
>>> pydirectinput.keyUp('shift')
# And this is the one you want,
>>> pydirectinput.write('string') # Write string
>>> pydirectinput.typewrite("string")
Related
I want to write an app in Python on Windows to do some jobs repeatedly.
For example, I need to convert some files into other type. I have a software installed in Windows to do that. However, that program was designed to do it file by file. Now I want to do it automatically.
Therefore, I need to write a software to simulate the key press on active windows. There are a lot of code on autokeyboard but it only works in terminal which run Python script. Specially, after I run Python script, I minimize the terminal, then open some program then the Python script will simulate the key press and/or mouse click in this program.
I found a lot of program can do something like hotkey and after press hotkey, it will simulate some key press and mouse. So I think it is possible.
Could anyone give me a solution for that?
Thanks.
This will help you to automate:
for mouse clicks:
import pyautogui
pyautogui.click(1319, 45)
pyautogui.scroll(200)
pyautogui.hotkey("ctrlleft", "a")
For keyboard
import keyboard
# It writes the keys r, k and endofline
keyboard.press_and_release('shift + r, shift + k, \n')
keyboard.press_and_release('R, K')
# it blocks until esc is pressed
keyboard.wait('esc')
# It records all the keys until escape is pressed
rk = keyboard.record(until='Esc')
# It replay back the all keys
keyboard.play(rk, speed_factor=1)
I started to learn pyautogui for my personal project and almost instantly ran into the problems when trying to open OSX dock icons.
I want to open local Spotify which is under Mac Launchpad.
My code to do this.
import pyautogui
launchpad = pyautogui.locateOnScreen('img/Launchpad.png')
This return None so the image was not found.
image example attached
However, if I open Mac OSX Notes window and paste the same image into it and ran the program again the image is found every time. Similarly, if I just leave image open in my Editor.
Is dock actually part of the OSX screen pyautogui can search from? If not how to interact with it?
Figured that using application hotkeys vs find on the screen is a much less brittle approach. Below how I finally build Spotify bot.
import time
import pyautogui
# use pyauutogui key shortcut to open OSX spotlight search
pyautogui.hotkey('command', 'space')
# type spotify and press enter to open application
pyautogui.typewrite('Spotify')
pyautogui.hotkey('enter')
# use Spotify keyboard shortcuts to select search.
# key docs here: https://support.spotify.com/ie/article/Keyboard-shortcuts/
time.sleep(5)
pyautogui.hotkey('command', 'l')
# typewrite allows passing string arguments using keyboard
pyautogui.typewrite('concentration music')
# move to select the song with tab and press enter to play
pyautogui.hotkey('tab', 'tab', 'tab', 'tab')
time.sleep(2)
pyautogui.hotkey('enter')
pyautogui.hotkey('space')
# sleeps 30 seconds while music is playing
time.sleep(30)
pyautogui.hotkey('command', 'q')
I made a program to pertaining your problem. It is a spotify bot aswell.
import pyautogui as p
from time import sleep as t
p.keyDown("command")
p.press("space")
t(2)
p.keyUp("command")
p.typewrite("Spotify")
p.press("enter")
t(3)
p.moveTo(153,132)
p.click()
t(1)
p.typewrite("Never Gonna Give You Up")
p.press("enter")
p.moveTo(707,324)
t(4)
p.click()
t(10)
p.keyDown("command")
p.press("q")
t(1)
p.keyUp("command")
The coordinates will be different pertaining to the position of your spotify window, so you can do p.mouseInfo/pyautogui.mouseInfo to find the exact coordinates and substitute them in the code I gave above.
I need to do some macros and I wanna know what is the most recommended way to do it.
So, I need to write somethings and click some places with it and I need to emulate the TAB key to.
I do automated testing stuff in Python. I tend to use the following:
http://www.tizmoi.net/watsup/intro.html
Edit: Link is dead, archived version: https://web.archive.org/web/20100224025508/http://www.tizmoi.net/watsup/intro.html
http://www.mayukhbose.com/python/IEC/index.php
I do not always (almost never) simulate key presses and mouse movement. I usually use COM to set values of windows objects and call their .click() methods.
You can send keypress signals with this:
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shell.SendKeys("^a") # CTRL+A may "select all" depending on which window's focused
shell.SendKeys("{DELETE}") # Delete selected text? Depends on context. :P
shell.SendKeys("{TAB}") #Press tab... to change focus or whatever
This is all in Windows. If you're in another environment, I have no clue.
Maybe you are looking for Sendkeys?
SendKeys is a Python module for
Windows that can send one or more
keystrokes or keystroke combinations
to the active window.
it seems it is windows only
Also you have pywinauto (copied from my SO answer)
pywinauto is a set of open-source
(LGPL) modules for using Python as a
GUI automation 'driver' for Windows NT
based Operating Systems (NT/W2K/XP).
and example from the web page
> from pywinauto import application
> app = application.Application.start("notepad.exe")
> app.notepad.TypeKeys("%FX")
> app.Notepad.MenuSelect("File->SaveAs")
> app.SaveAs.ComboBox5.Select("UTF-8")
> app.SaveAs.edit1.SetText("Example-utf8.txt")
> app.SaveAs.Save.Click()
pyautogui is a great package to send keys and automate several keyboard / mouse related tasks. Check out Controlling the Keyboard and Mouse with GUI Automation and PyAutoGUI’s documentation.
You can use PyAutoGUI library for Python which works on Windows, macOS, and Linux.
Mouse
Here is a simple code to move the mouse to the middle of the screen:
import pyautogui
screenWidth, screenHeight = pyautogui.size()
pyautogui.moveTo(screenWidth / 2, screenHeight / 2)
Docs page: Mouse Control Functions.
Related question: Controlling mouse with Python.
Keyboard
Example:
pyautogui.typewrite('Hello world!') # prints out "Hello world!" instantly
pyautogui.typewrite('Hello world!', interval=0.25) # prints out "Hello world!" with a quarter second delay after each character
Docs page: Keyboard Control Functions.
More reading: Controlling the Keyboard and Mouse with GUI Automation (Chapter 18 of e-book).
Related questions:
Python GUI automation library for simulating user interaction in apps.
Python simulate keydown.
Two other options are:
pynput - https://pypi.org/project/pynput/ - which is for Windows (tested), Linux and MacOS- docs are at https://pynput.readthedocs.io/en/latest/
PyDirectInput - https://pypi.org/project/PyDirectInput/ - which is for Windows only and can be used with (or without) PyAutoGUI
Warning - if you are wanting to use keyboard control for games, then pynput doesn't always work - e.g. it works for Valheim, but not for the Witcher 3 - which is where PyDirectInput will work instead. I also tested PyDirectInput and it works for Half life 2 (as a test of an older game).
Tip - You will likely need to reduce (don't remove for games) the delay between character typing - use pydirectinput.PAUSE = 0.05
As an example, here is a function that allows virtual keyboard typing - currently only tested on Windows:
from pynput import keyboard
try:
import pydirectinput
pydirectinput.PAUSE = 0.05
except ImportError as err:
pydirectinput = False
print("pydirectinput not found:")
def write_char(ch):
upper = ch.isupper()
if pydirectinput and pydirectinput.KEYBOARD_MAPPING.get(ch.lower(), False):
if upper:
pydirectinput.keyDown('shift')
print('^')
pydirectinput.write(ch.lower(), interval=0.0)
print(ch)
if upper:
pydirectinput.keyUp('shift')
else:
keyboard.Controller().type(ch)
This allows a string to be sent in, with upper case alphabetic characters handled through pydirectinput. When characters don't simply map, the function falls back to using pynput. Note that PyAutoGUI also can't handled some shifted characters - such as the £ symbol, etc.
I need to do some macros and I wanna know what is the most recommended way to do it.
So, I need to write somethings and click some places with it and I need to emulate the TAB key to.
I do automated testing stuff in Python. I tend to use the following:
http://www.tizmoi.net/watsup/intro.html
Edit: Link is dead, archived version: https://web.archive.org/web/20100224025508/http://www.tizmoi.net/watsup/intro.html
http://www.mayukhbose.com/python/IEC/index.php
I do not always (almost never) simulate key presses and mouse movement. I usually use COM to set values of windows objects and call their .click() methods.
You can send keypress signals with this:
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shell.SendKeys("^a") # CTRL+A may "select all" depending on which window's focused
shell.SendKeys("{DELETE}") # Delete selected text? Depends on context. :P
shell.SendKeys("{TAB}") #Press tab... to change focus or whatever
This is all in Windows. If you're in another environment, I have no clue.
Maybe you are looking for Sendkeys?
SendKeys is a Python module for
Windows that can send one or more
keystrokes or keystroke combinations
to the active window.
it seems it is windows only
Also you have pywinauto (copied from my SO answer)
pywinauto is a set of open-source
(LGPL) modules for using Python as a
GUI automation 'driver' for Windows NT
based Operating Systems (NT/W2K/XP).
and example from the web page
> from pywinauto import application
> app = application.Application.start("notepad.exe")
> app.notepad.TypeKeys("%FX")
> app.Notepad.MenuSelect("File->SaveAs")
> app.SaveAs.ComboBox5.Select("UTF-8")
> app.SaveAs.edit1.SetText("Example-utf8.txt")
> app.SaveAs.Save.Click()
pyautogui is a great package to send keys and automate several keyboard / mouse related tasks. Check out Controlling the Keyboard and Mouse with GUI Automation and PyAutoGUI’s documentation.
You can use PyAutoGUI library for Python which works on Windows, macOS, and Linux.
Mouse
Here is a simple code to move the mouse to the middle of the screen:
import pyautogui
screenWidth, screenHeight = pyautogui.size()
pyautogui.moveTo(screenWidth / 2, screenHeight / 2)
Docs page: Mouse Control Functions.
Related question: Controlling mouse with Python.
Keyboard
Example:
pyautogui.typewrite('Hello world!') # prints out "Hello world!" instantly
pyautogui.typewrite('Hello world!', interval=0.25) # prints out "Hello world!" with a quarter second delay after each character
Docs page: Keyboard Control Functions.
More reading: Controlling the Keyboard and Mouse with GUI Automation (Chapter 18 of e-book).
Related questions:
Python GUI automation library for simulating user interaction in apps.
Python simulate keydown.
Two other options are:
pynput - https://pypi.org/project/pynput/ - which is for Windows (tested), Linux and MacOS- docs are at https://pynput.readthedocs.io/en/latest/
PyDirectInput - https://pypi.org/project/PyDirectInput/ - which is for Windows only and can be used with (or without) PyAutoGUI
Warning - if you are wanting to use keyboard control for games, then pynput doesn't always work - e.g. it works for Valheim, but not for the Witcher 3 - which is where PyDirectInput will work instead. I also tested PyDirectInput and it works for Half life 2 (as a test of an older game).
Tip - You will likely need to reduce (don't remove for games) the delay between character typing - use pydirectinput.PAUSE = 0.05
As an example, here is a function that allows virtual keyboard typing - currently only tested on Windows:
from pynput import keyboard
try:
import pydirectinput
pydirectinput.PAUSE = 0.05
except ImportError as err:
pydirectinput = False
print("pydirectinput not found:")
def write_char(ch):
upper = ch.isupper()
if pydirectinput and pydirectinput.KEYBOARD_MAPPING.get(ch.lower(), False):
if upper:
pydirectinput.keyDown('shift')
print('^')
pydirectinput.write(ch.lower(), interval=0.0)
print(ch)
if upper:
pydirectinput.keyUp('shift')
else:
keyboard.Controller().type(ch)
This allows a string to be sent in, with upper case alphabetic characters handled through pydirectinput. When characters don't simply map, the function falls back to using pynput. Note that PyAutoGUI also can't handled some shifted characters - such as the £ symbol, etc.
I created a small Python script using win32api to use on the popular game Cookie Clicker (a game where you have to click on a Big Cookie to gain points) just for fun. It has a function called "auto_clicker" that do just that: keeps clicking on the screen on the point the user defined. This is the script:
# -*- coding: utf-8 -*-
import win32con
import win32api
def clicker(x,y):
"""Clicks on given position x,y
Input:
x -- Horizontal position in pixels, starts from top-left position
y -- Vertical position in pixels, start from top-left position
"""
win32api.SetCursorPos((x,y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)
def auto_clicker(x = -1,y = -1):
"""Keep clicking on position x,y. If no input is given, gets from actual
mouse position.
"""
if x == -1 | y == -1:
x,y = win32api.GetCursorPos()
while True:
clicker(x,y)
It works nicely, but I want to make some improvements:
How can I get the cursor position only when the user clicks instead when the function is called? I would prefer to not add another module
since win32api seems to contain everything I needed. Tried this
method without success.
How can I detect a keypress like "Escape", so I can exit from my program without the ugly hack I am using now (Ctrl+Alt+Del seems to give SetCursorPos denied access, so Python throws a error and exit the program).
Can I make this program portable? Seems like I can do using Tkinter and generating a invisible Tk window, but I tried to write something without success.
I don't think with win32api you can listen to clicks you can just generate them (not sure though). However, try using pyHook, it's a simple api easy to use and can be found here http://sourceforge.net/apps/mediawiki/pyhook/index.php?title=Main_Page. With pyhook you can create a listener to listen to a mouse event and upon a mouse click you can do whatever you want, the example in the link shows you how. As for key press, you can use the same api for that too, also an example is provided, good luck!
use pynput . It can control mouse, keyboard, etc.
examples:
from pynput.mouse import Button, Controller
mouse = Controller()
# Read pointer position
print('The current pointer position is {0}'.format(
mouse.position))
# Set pointer position
mouse.position = (10, 20)
print('Now we have moved it to {0}'.format(
mouse.position))
# Move pointer relative to current position
mouse.move(5, -5)
# Press and release
mouse.press(Button.left)
mouse.release(Button.left)
# Double click; this is different from pressing and releasing
# twice on Mac OSX
mouse.click(Button.left, 2)
# Scroll two steps down
mouse.scroll(0, 2)