Firstly, I am aware of this post and this post here StackOverflow. However, most of the information in these posts is either severely outdated or not applicable to my use case.
I would like to know if it is possible to simulate n mouse pointers with n different instances of Python. That is, I would like to be able to run as many mouse pointers on my screen as possible. All of these mouse pointers would be controlled by the same script, but would still be doing all their work independently of each other.
Is it possible to create such an application using Python?
Related
Note: I am open to different solutions which achieve the desired capability
I am working on a project with many instances of the same game.
Therefore, I am sending keyboard and mouse instructions to each of theses processes, in parallel.
I am currently using win32ui as follows:
After finding the processes hwnd (windows handle) values from Get HWND of each Window?, so a hwnds_list with all the processes with a given name e.g. [788133, 723724, ...]
I am sending instructions to each of the processes, by creating a PyCWnd object:
PyCWnd = win32ui.CreateWindowFromHandle(hwnd)
Then, say I want to press the return key, I used:
def press_return(pycwnd):
pycwnd.SendMessage(win32con.WM_KEYDOWN, win32con.VK_RETURN, 0)
pycwnd.SendMessage(win32con.WM_KEYUP, win32con.VK_RETURN, 0)
Then I run this in parallel with:
def press_return_par(hwnds):
# Get the Window from handle
pycwnd = make_pycwnd(hwnds)
time.sleep(0.1)
press_return(pycwnd)
num_workers = len(hwnds_list)
with Pool(num_workers) as p:
p.map(press_return_par, hwnds_list)
So, I have a good way of sending keyboard commands, and even scrolling with a mouse, but can't work out how to do this with mouse movements.
Ideally, I'd like to say, "Move to (x, y) coordinates over n time". This 'ideal' method needs to not effect the current cursor (or allow a locked cursor for each process/game), as I will want to do this across ~8 instances of the game.
I've looked through the official pywin32 docs http://timgolden.me.uk/pywin32-docs/contents.html, other answers that look bang on https://stackoverflow.com/a/3721198/11181287 but use win32api.mouse_event, so I don't know how to convert this to work with the multiple pycwnd objects.
https://stackoverflow.com/a/3721053/11181287 looks close, but doesn't seem to move the mouse, it just does the right click, although I have made some guesses for the MAKELPARAM function which is not listed.
In addition, https://github.com/oblitum/Interception could be helpful but haven't found good docs for how to apply this here.
As the game is an FPS game, running multiple instances through nucleus-coop, using a VM etc... won't be fast enough (from my current research).
PyAutoGUI is exactly the functionality I want, with the speed, but (as expected) I haven't been able to set it up to work for multiple mice/processes
There could be something in sending DirectX inputs into the game (black ops 2)?
(I'm running windows 10, Python 3.7.11, and only know Python)
I have two possible solutions to your mice issue.
What if you used only one mouse to control all of the windows? With pyautogui you could tab into each window when necessary and control the mouse for that window. I'm not sure how efficient this would be and how fast the mouse control for each window would be, but it's still sort of a solution.
OR
You could control the mouse with the keyboard.
See this article https://www.windowscentral.com/how-control-mouse-using-keyboard-windows-10
I apologize for not just commenting, unfortunately I don't have enough reputation.
I am executing automation testing with Squish for Windows using Image-Based Testing. But some of the events are not working!
For example, i have a control looking like this:
My sample control
I would like to hold and drag it to a specific position (when cropping images) but i can't figure how to do it with Squish yet.
I want something like this:
mouseDrag(waitForObject(":_CropTopButton"), 5, 30, 0, 280)
But with Image-Based Testing, it won't work. Only mouseClick(), doubleClick() and tapObject() are supported! (As far as i know)
Is there any way to treat (or drag) images like a control-object with Squish for Windows using Image-Based Testing?
I'm completely new to Squish, StackOverFlow and writing in English so if anything is inappropriate or annoyed, please be kind enough to let me know, thank you so so much!)
You should be able to use the functions mouseMove(), mousePress() [Windows], mouseRelease() [Windows]. Whether that triggers a drag may depend. You may have to "wiggle" the mouse back and forth a few pixels in a long/short enough amount of time to trigger it.
As for supporting mouseDrag() for the image objects, please contact the vendor regarding that.
I'm using:
ctypes.windll.user32.SystemParametersInfoA(SPI_SETDESKWALLPAPER,
0, "picturefile", 0)
To change the wallpaper.
But I'm wondering if there's any simple way to put different wallpapers on each screen.
This feature isn't standard in windows though, but there are external applications like ultramon that do this. Anyone know how that works?
The way I thought it might work if I join the two images together into one and then make that the wallpaper, but then I still need a way to span one image accross both screens.
Also, how could I grab some info about the monitor setup, the resolution of each screen and their placement? Like what you see in the gui display settings in windows, but in numbers.
After joining the images together into a big image, you have to set the wallpaper mode to tiled to make it so the image spans the desktop (otherwise it will restart on each monitor).
Couple of ways to do this:
a) Using IActiveDesktop (which does not require Active Desktop to be used, don't worry). This is nicest as on Win7 the new wallpaper will fade in.
You create an IActiveDesktop / CLSID_ActiveDesktop COM object and then call SetWallpaper, SetWallpaperOptions and finally ApplyChanges. (As I'm not a Python dev, I'm not sure exactly how you access the COM object, sorry.)
OR:
b) Via the registry. This isn't as nice, but works well enough.
Under HKEY_CURRENT_USER\Control Panel\Desktop set:
TileWallpaper to (REG_SZ) 1 (i.e. the string "1" not the number 1)
WallpaperStyle to (REG_SZ) 0 (i.e. the string "0" not the number 0)
Then call SystemParameterInfo(SPI_SETDESKTOPWALLPAPER...) as you do already.
.
By the way, the code I'm looking at, which uses IActiveDesktop and falls back on the registry if that fails, passes SPIF_UPDATEINIFILE | SPIF_SENDCHANGE as the last argument to SystemParameterInfo; you're currently passing 0 which could be wrong.
EnumDisplayMonitors is the Win32 API for getting details on the monitors, including their screen sizes and positions relative to each other.
That API returns its results via a callback function that you have to provide. (It calls it once for each monitor.) I am not a Python developer so I'm not sure how you can call such a function from Python.
A quick Google for "Python EnumWindows" (EnumWindows being a commonly-used API which returns results in the same way) finds people talking about that, and using a Lambda function for the callback, so it looks like it's possible but I'll leave it to someone who knows more about Python.
Note: Remember to cope with monitors that aren't right next to each other or aren't aligned with each other. Your compiled image may need to have blank areas to make things line up right on all the monitors. If you move one of the monitors around and do a PrtScn screenshot of the whole desktop you'll see what I mean in the result.
Generally I want to write a program to run in the background on Mac and when I push a keyboard shortcut, the current active window would be resized and positioned to the way I have set.
Something similar to the tool called SizeUp on Mac. I think this shouldn't be difficult to implement and would be fun to take a try.
I would appreciate any resources you could point me to. Thanks.
I think you're going to have an easier time attacking this in applescript. Upon casual googling, this link seems to have more or less what you want. If you were more looking for a programming challenge in python, and less for a solution, then disregard this.
I'm not familiar with PowerBuilder but I have a task to create Automatic UI Test Application for PB. We've decided to do it in Python with pywinauto and iaccesible libraries. The problem is that some UI elements like newly added lists record can not be accesed from it (even inspect32 can't get it).
Any ideas how to reach this elements and make them testable?
I'm experimenting with code for a tool for automating PowerBuilder-based GUIs as well. From what I can see, your best bet would be to use the PowerBuilder Native Interface (PBNI), and call PowerScript code from within your NVO.
If you like, feel free to send me an email (see my profile for my email address), I'd be interested in exchanging ideas about how to do this.
I didn't use PowerBuilder for a while but I guess that the problem that you are trying to solve is similar to the one I am trying to address for people making projects with SCADA systems like Wonderware Intouch.
The problem with such an application is that there is no API to get or set the value of a control. So a pywinauto approach can't work.
I've made a small tool to simulate the user events and to get the results from a screencapture. I am usig PIL and pytesser ORM for the analysis of the screen captures. It is not the easiest way but it works OK.
The tool is open-source and free of charge and can be downloaded from my website (Sorry in french). You just need an account but it's free as well. Just ask.
If you can read french, here is one article about testing Intouch-based applications
Sorry for the self promotion, but I was facing a similar problem with no solution so I've written my own. Anyway, that's free and open-source...
I've seen in AutomatedQa support that they a recipe recommending using msaa and setting some properties on the controls. I do not know if it works.
If you are testing DataWindows (the class is pbdwxxx, e.g. pbdw110) you will have to use a combination of clicking at specific coordinates and sending Tab keys to get to the control you want. Of course you can also send up and down arrow keys to move among rows. The easiest thing to do is to start with a normal control like an SLE and tab into the DataWindow. The problem is that the DataWindow is essentially just an image. There is no control for a given field until you move the focus there by clicking or tabbing. I've also found that the DataWindow's iAccessible interface is a bit strange. If you ask the DataWindow for the object with focus, you don't get the right answer. If you enumerate through all of the children you can find the one that has focus. If you can modify the source I also advise that you set AccessibleName for your DataWindow controls, otherwise you probably won't be able to identify the controls except by position (by DataWindow controls I mean the ones inside the DataWindow, not the DataWindow itself). If it's an MDI application, you may also find it useful to locate the MicroHelp window (class fnhelpxxx, e.g. fnhelp110, find from the main application window) to help determine your current context.
Edited to add:
Sikuli looks very promising for testing PowerBuilder. It works by recognizing objects on the screen from a saved fragment of screenshot. That is, you take a screenshot of the part of the screen you want it to find.