This question already has answers here:
Python code to automate desktop activities in windows
(6 answers)
Closed 5 years ago.
I love automating daily tasks using AutoHotKey and Python. I like using my keyboard instead of my mouse, so I tend to make hotkeys with AutoHotKey that do various actions that you'd otherwise need to use the mouse for, or that you'd need to use too many keyboard actions to do.
But there are some actions that I don't succeed in automating. For example, the dropdown menu for changing the number of monitors that a VM uses in VMWare Workstation. I think that one thing that would really help me with this automation task, and possibly with more in the future, is to be able to simulate menu item clicks.
Is this possible? I want to fool a program into thinking that one of its menu items were clicked. I know to program in Python and AutoHotKey. Is it possible at all, and specifically in these two languages?
Presuming that this is Windows, if the app is using standard HMENU items you can do one of two things with (relatively) minimal effort
If there is a hotkey, like Ctrl+S for Save, you could just send Ctrl+S to the window using the SendKeys API (https://msdn.microsoft.com/en-us/library/windows/desktop/ms646310(v=vs.85).aspx)
If there isn't, you can try to find which WM_COMMAND is sent to the window after the item is selected. You can use Spy++ from Microsoft (https://msdn.microsoft.com/en-us/library/dd460756.aspx) to get the code and then do a PostMessage to that window with the code.
Mind you, you'll need to be sure that you do a FindWindow to get the right HWND handle to send it to.
Best of luck :)
Edit Followup: Quick tutorial on how to use Spy++ with Notepad.exe
Start Spy++
NOTE: There are two of them, spyxx.exe and spyxx_amd64.exe. If one doesn't log messages - use the other
NOTE: if you started Spy++ first, hit F5 to refresh the window list AFTER Notepad.exe starts
Spy++ main menu > Search > Find Window
on the "Window search" dialog, click and hold on the "target" icon next to "finder tool"
drag your cursor over the Notepad title bar
Spy++ will show Caption: "Untitled - Notepad" and Class: "Notepad"
Click "OK"
You will now see Notepad in the window list
Right-click on that entry and select "Messages"
To control the flow, Main Menu > Messages > Logging Options, Messages Tab
Clear All, only select WM_COMMAND, click ok
in Notepad, go to the Main Menu > Edit > Go to
in Spy++, you will see a line reading "WM_COMMAND wNotifyCode: 0 (sent from a menu) wID:24"
Result: Now we know that if you were to PostMessage(WM_COMMAND, MAKE_WPARAM(0,24), HWND of any Notepad.exe on the system), the "Go to" window would appear.
Related
I am writing a python script that automates running a program and performing different tasks within the program. My main problem is figuring out how to click buttons and interact with the GUI of the program to be controlled.
I am currently using the pyautogui library and using pyautogui.click(X,Y) to advance through prompts and click on different menus and menu items. The problem with this approach is that I am relying on a separate script to inform me of the coordinates of interest in my environment by telling me the coordinates of where my cursor is hovering. This probably will not work on other machines and just seems like a one case solution.
My question is how can I automate using a program in windows (clicking around) without having to hard code the exact position of the items I need to click?
For example, If I need to click a "ok" box to accept some setting, how can I make Windows grab the program window, read through the options and click what I need without any prior knowledge of the position of the dialog box and where the "Ok" button is located?
Code:
import pyautogui as gui
gui.click(x,y)
The way you can do this using pyautogui is with their locating methods. You will need a picture (for example of the OK box) and then you can have pyautogui find it on the screen and give you its coordinates. Check out the official documentation on this.
Ctrl+Escape is a global Windows shortcut for opening main system menu. But I would like my Qt application to use this shortcut without triggering Windows main menu. I know it is probably a bad idea to override system shortcuts in general, but I would like to use this shortcut is a very limited use case.
This usecase is as follows. I have a popup window containing several rows or items. This window is opened by Ctrl+Tab and while the user holds Ctrl and keep pressing Tab, the current rows are cycled through. When the user releases Ctrl, the current row is used for some operation... But sometimes it happens that user presses Ctrl+Tab and then realizes he does not want to continue. He usually presses Escape while still holding Ctrl. And then it triggers Windows system menu and normal user gets confused, choleric user get angry... which is a bad thing. In other words I would like to be able to close the popup window when user presses Ctrl+Escape. How to do that? It is even possible?
If I write the code using this shortcut like any other short, it does not work and it always triggers Windows main menu.
As I understand it, Qt will typically not receive the key event if the underlying window system has intercepted it. For example even QtCreator cannot override system-wide shortcuts.
This question is almost a duplicate of: C++/Qt Global Hotkeys
While that question is asking specifically to capture shortcuts in a hidden/background application, I think the basic concept is the same -- capture shortcuts before the window system processes them.
From that answer, UGlobalHotkey seems pretty good, and the How to use System-Wide Hotkeys in your Qt application blog post could be useful for your limited-use case (but read the comments on that blog post about fixing the example).
Also found:
https://github.com/mitei/qglobalshortcut
https://github.com/Skycoder42/QHotkey (looks like a more detailed version of above)
There is command In the AutoHotKey that shows tooltip with some message on the top of the screen. I want to do it on python.
In more details I need to make an application, that will work in background, always tracking a keyboard. On the specific hotkey it should show on the top of the screen a message without grabbing focus from other application. And it should remove this message after some conditions (e.g. mouse moving). It should work with full screen apps too.
I couldn't find how to do it on python, or what libraries should I use. Need your help.
Global tooltips are created via WINAPI CreateWindowEx with TOOLTIPS_CLASS for window class. There are some examples which you can adapt. See also the Autohotkey implementation.
Shell tray tooltips are created via Shell_NotifyIcon.
A working example: wontoncc/balloontip.py and the Autohotkey implementation.
I am using Ubuntu 13.04 with Unity.
I created an Application Indicator using the AppIndicator class in Python, with a gtk.Menu attached to it. The menu is showing some status information about GIT repositories that I don't want to be updated as long as the menu is not visible. Instead, when the user clicks on the AppIndicator and the menu becomes visible, I would like to get notified and update the information shown.
So far, I could not find a signal emitted by the Menu whenver it pops up. Nor does the AppIndicator have a signal emitted when it gets clicked. In fact, there is already this question but the answers only discuss if AppIndicator has (or should have) the appropriate signal. But what about the gtk.Menu itself?
So, my question is, how can I get notified that my menu becomes visible?
This question is to old to go into details but if you still interested you can monitor the DBus system in ubuntu so you can catch when the call is done to you menu, you can even track submenu if you are into that.
I am writing a script to automate repeatedly registering new users for a website (not boosting metrics, not what you think!). I can boil down the process of registering to a series of mouse clicks and typing. I know there are some macro recorders that will let me record how I use the GUI and repeat it, but I need to type something a little different every time - however the mouse clicks are always the same.
Imagining script to look kindof like:
username = "something"
for i in range(0,100):
playback recorded series of mouse clicks A
type username + str(i)
type some other stuff
playback recorded series of mouse clicks B
But I can't find a good tool to record a series of mouse clicks so that I can play them back from either an applescript or a python script.
I found a useful script (http://www.bluem.net/en/mac/cliclick/) I can call that takes in 2 coordinates and clicks. So even if I could find a tool to record the coordinates of mouseclicks that would work!
You can also run JavaScripts in browsers with something like tell application "Safari" to tell document 1 to do JavaScript or tell application "Google Chrome" to tell tab 1 of window 1 to execute javascript.
tell application "Safari" to tell document 1
do JavaScript "document.querySelectorAll('.mainnavs ul li:nth-child(2) a')[0].click()"
delay 1
repeat until do JavaScript "document.readyState" is "complete"
delay 0.1
end repeat
do JavaScript "document.getElementById('tagfilter').value = 'aa'"
end tell
System Events has commands for simulating clicks and keystrokes:
delay 1
tell application "System Events" to tell process "Safari"
click at {76, 117}
repeat with i from 1 to 3
keystroke "aa" & i & return
end repeat
keystroke "a" using command down
end tell
You can see the positions of UI elements from Accessibility Inspector.app. It is part of Xcode, but it can also be downloaded from developer.apple.com/downloads.
If you want to get the screen coordinates of where you want to perform mouse clicks, you can use a free tool I made called MouseTools found here. I made it for just this purpose. You'll want to do something like I show in the example #1 applescript near the bottom of the page. Just move your mouse to a position on the screen and run the applescript to get the coordinates. Note: you'll want to move your mouse to a location and press command-r (while the applescript is frontmost) to run the script in AppleScript Editor.
In addition, I found that clicking at screen coordinates, as Lauri Ranta showed in her applescript using System Events, often does not reliably work. System Events is good at clicking in the Finder however it often has trouble clicking in the window of an application like Safari. If you find that problem too then you can also use the MouseTools to perform the mouse clicks. Again, this is why I created the program and there are examples of how to do this on the web page.
I hope it helps. Good luck.
How about Carsten Blum's Clicclick?
His link
It is a shell app which will emulate mouse clicks at arbitrary screen coordinates. This will work with applescript.
Hopefully you are doing test automation!
Have you looked into using something like Selenium WebDriver instead?