I'm writing a simple OSX app using Python and PyObjC. I designed the settings dialog using Interface Builder and I use ibtool to compile it, then load it from Python. The problem is how to access the controls I have in this window from the Python code? I played around with iPhone development a bit before and I remember I need to have an IBOutlet in the controller class which will be connected to the UI control in the interface builder. It should look something like this in Python:
class MyClass(NSObject):
my_outlet = objc.IBOutlet('my_outlet')
But since I'm not working in XCode (all I have is a .py file and a .xib file), Interface Builder doesn't know about my outlets. How can I do the binding in this case? Or how else can I access the UI elements from the code?
First, the use of Xcode or not has nothing to do with NIB loading (beyond making it more convenient).
As Ole said, you can use IB to manually add the outlet's you need to file's owner or to the custom object instances that you have in the NIB file. By doing so, it will all "just work".
However, this statement is what prompted my relatively similar answer:
all I have is a .py file and a .xib
file
Are you trying to write a bit of UI code outside of a .app wrapper? If so, that is a wholly unsupported pattern, very difficult to get correct, and quite likely to break across software updates or major releases (as it has many times in the past).
The best way to solve your problem is to use an Xcode project and build a standard application. The templates are no longer shipped with the dev tools. Just download them separately.
If you need to run it from the command line, you can still do so.
I haven't tried this, but you can also define outlets directly in IB. Open the Library panel, select Classes in the segmented control at the top and select your custom class you want to define an outlet for. Let's say you have a NSWindow subclass called MyWindow. Select the NSWindow class in the list, click on the action button at the bottom left, select New Subclass... and name it MyWindow. Now switch to the Outlets tab and create a NSButton outlet for your window. Now you connect a button to the outlet.
I don't know how this will transfer to PyObjC but I'd love to see your results when you try it out.
Related
I want to retrieve information from a tooltip in the system tray programmatically.
The image shows the tooltip.
Now, I found that by using Microsoft's inspect.exe, which is "a Windows-based tool that enables you select any UI element and view the element's accessibility data", that it is in theory possible to retrieve this value programmatically.
Hovering over the the pandora icon in the toolbar shows me the following properties
It shows one property "name" that contains the exact data I need. I'm unsure how to retrieve this value programmatically using the win32api. I have a hwnd to the pandora icon already.
Additionally, a different ui spy tool, UiSpy.exe calls this same property "helpText" (different song name :p)
I tried using getWindowText(pandoraSystrayIconHwnd) but that returns a different text. Does anyone know what this "name" value is, and how I can retrieve it using the win32api? It should be possible because inspect.exe is an external program that can access the data somehow
I'm doing this in Python, as the target application is written in Python already.
These spy apps are probably using Active Accessibility and/or UI Automation.
You can try calling AccessibleObjectFromWindow on the toolbar HWND or AccessibleObjectFromPoint if you care about the mouse position and then call IAccessible::get_accName.
Keep in mind that the classname and window hierarchy of the tray icon toolbar is undocumented.
If you only care about Pandora and not other applications then I would strongly suggest that you look for other alternatives first. Perhaps they have a hidden window with the title etc.
If you don't mind hacks then you could take a look at TraySaver, it is open source and knows the internal format of the data stored for each icon in the tray toolbar. Keep in mind that it is pretty old and might not work on newer versions of Windows. If you go down this path (and I don't recommend it) then you have to remember that you need to support both 32-bit and 64-bit Explorer.
Maybe GUI automation library pywinauto could help you. It uses Win32 API or UI Automation under the hood (by your choice). Core concept is described in the Getting Started Guide.
Method .window_text() returns exactly the same as Name property shows in Inspect.exe.
To interact with tray area icons you can use this example on StackOverflow.
This is complex to explain, I hope this will not end up being a vague question getting vague answers.
If this is not the right place to ask this, you may help me to find the proper one.
I have a plugin for Photoshop based on the Listener, so it captures any input from the user.
The plugin creates a python module (called here "ps") containing basically the hInstance and the hwnd of the photoshop window.
Then this plugin, using plain python commands in the plugin for the module like those
PyRun_SimpleString("import Photoshop");
PyRun_SimpleString("Photoshop.showTools()");
will load a special module (here called "Photoshop") that will initialize pyqt and using the QtWinMigrate and the ps module to get the hInstance like this: QMfcApp.pluginInstance(ps.GetPluginInstance()), will start pyqt in photoshop. Here an example code of the Photoshop module using the ps module:
from PyQt4.QtWinMigrate import QMfcApp
from PyQt4.QtGui import QPushButton
import ps #this is implemented in the photoshop plugin (based on the Listener plugin)
#create the plugin instance here
app=QMfcApp.pluginInstance(ps.GetPluginInstance())
def showTools():
box = QPushButton()
box.show()
app.exec_()
Again then, the sequence is like this:
When the plugin starts in photoshop "ps" module is created, then it will load the "Photoshop" module that will load and bind properly pyqt. In the "Photoshop" module I can load any python module, widgets are properly working and everything works really well inside Photoshop.
But now the problem is: using Wacom tablets in Photoshop loose stroke sensitivity, the driver works and everything else works but the pressure sensitivity.
Apparently QMfcApp.pluginInstance will install an event filter to drive the Qt event loop while photoshop still owns the event loop. ( http://doc.qt.digia.com/solutions/4/qtwinmigrate/qmfcapp.html )
and on the paper looks fine to me.. but I could not manage to solve this by myself and I tried, more or less carefully, different approaches:
the listener plugin is not the problem. If Listener plugin runs but python is not initialized sensitivity works fine.
python itself is not a problem. If the listener starts python without gui nor pyqt, then works fine.
as soon as I call pluginInstance which should create the QApplication the issue starts and pressure is lost from the tablet. Even with the small code I wrote before.
Someone may have put pyqt as a plugin somewhere else, since the only purpose of QMfcApp is apparently this one. There is something I can configure to make it work? Is a known issue?
I would rather keep the approach (instead of connecting to photoshop externally like with COM)
I am not able to post the entire code here but let me know if you need something.. I probably can show more.
Thanks a lot for your help
I'm trying to write a program than will detect when my mouse pointer will change icon and automatically send out a mouse click. Is there a better way to do this than to take screenshots and parse the image for the mouse icon?
EDIT:
I'm running my program on windows 7.
I'm trying to learn some image processing and make a simple flash game i made automated.
Rules: when the curses changes shape, click to get a point.
Also what imaging modules for python will allow you to take a specific size screenshot not just the whole screen? This question has moved to a new thread: "Taking Screen shots of specific size"
The way to do this in Windows is to install either a global message hook with SetWindowsHookEx or SetWinEventHook. (Alternatively, you could build a DLL that embeds Python and hooks into the browser or its Flash wrapper app and do it less intrusively from within the app, but that's much more work.)
The message you want is WM_SETCURSOR. Note that this is the message sent by Windows to the app to ask whether it wants to change the cursor, not a message sent when the cursor changes. So, IIRC, you will want to put a WH_CALLWNDPROC and a WH_CALLWNDPROCRET and check GetCursorInfo before and after to see if the app has done so.
So, how do you do this from Python? Honestly, if you don't already know both win32api and friends from the pywin32 package, and how to write Windows message procs in some language, you probably don't want to. If you do want to, I'd start off with the (abandoned) pyHook project from UNC Assist. Even if you can't get it working, it's full of useful source code.
You should also search SO for [python] SetWinEventHook and [python] SetWindowsHookEx, and google around a bit; there are some examples out there (I even wrote one here somewhere…)
You can look at higher-level wrapper frameworks like pywinauto and winGuiAuto, but as far as I know, none of them has much help for capturing events.
I believe there are other tools, maybe AutoIt, that have all the functionality you need, but not in Python module. (AutoIt, for example, has its own VB-like scripting language instead.)
How can I programmatically open the 'Proxies' tab in 'Network' dialog box?
System Preferences > Network > Advanced > Proxies
For those using Chrome, if you go to Menu > Settings > Show Advanced Settings > Change proxy settings... , the 'Network' box shows up, and its already on the 'Proxies' tab.
I want to achieve this using python.
The way to do this is through Apple Events. If you open AppleScript Editor, you can Open Dictionary on System Preferences and see the commands:
tell application "System Preferences"
reveal pane "com.apple.preference.network"
end tell
So, how do you do this from Python? There are three options:
Create some AppleScript and run it via PyObjC, or via a wrapper like py-applescript.
Use ScriptingBridge, Apple's AppleEvents-to-Python (and -Ruby and -ObjC) bridge.
Use appscript, a third-party AppleEvents-to-Python (and …) bridge.
Appscript is a lot better, but it's effectively an abandoned project, and ScriptingBridge comes with Apple's version of Python. So, I'll show that first:
import ScriptingBridge
sp = ScriptingBridge.SBApplication.applicationWithBundleIdentifier_('com.apple.SystemPreferences')
panes = sp.panes()
pane = panes.objectWithName_('com.apple.preference.network')
anchors = pane.anchors()
dummy_anchor = anchors.objectAtIndex_(0)
dummy_anchor.reveal()
You may notice that the ScriptingBridge version is a lot more verbose and annoying than the AppleScript. There are a few reasons for this.
ScriptingBridge isn't really an AppleEvent-Python bridge, it's an AppleEvent-ObjC bridge wrapped up in PyObjC, so you have to use horribleObjectiveCSyntax_withUnderscores_forEachParameterNamed_.
It's inherently horribly verbose.
The "obsolete" method of looking applications up by name isn't exposed in ScriptingBridge, so you have to find the bundle ID (or file:// URL) of the app and open that.
Most importantly, ScriptingBridge doesn't expose the actual object model; it forces it into a CocoaScripting OO-style model and exposes that. So, while System Preferences knows how to reveal anything, the ScriptingBridge wrapper only knows how to call the reveal method on an anchor object.
While the last two are the most troublesome, the first two can be annoying as well. For example, even using bundle IDs and following the CocoaScripting model, here's what the equivalent looks like in AppleScript:
tell application "com.apple.SystemPreferences"
reveal first anchor of pane "com.apple.preference.network"
end tell
… and in Python with appscript:
import appscript
sp = appscript.app('com.apple.SystemPreferences')
sp.panes['com.apple.preference.network'].anchors[1].reveal()
Meanwhile, in general, I wouldn't recommend any Python programmer move any of their logic into AppleScript, or try to write logic that crosses the boundaries (because I subscribe to the Geneva conventions against torture). So, I immediately start with ScriptingBridge or appscript in any case where we might need so much as an if statement. But in this case, as it turns out, we don't need that. So, using an AppleScript solution might be the best answer. Here's the code with py-applescript, or with nothing but what Apple gives you out of the box:
import applescript
scpt = 'tell app "System Preferences" to reveal pane "com.apple.preference.network"'
applescript.AppleScript(scpt).run()
import Foundation
scpt = 'tell app "System Preferences" to reveal pane "com.apple.preference.network"'
ascpt = Foundation.NSAppleScript.alloc()
ascpt.initWithSource_(scpt)
ascpt.executeAndReturnError_(None)
I'm not familiar with PowerBuilder but I have a task to create Automatic UI Test Application for PB. We've decided to do it in Python with pywinauto and iaccesible libraries. The problem is that some UI elements like newly added lists record can not be accesed from it (even inspect32 can't get it).
Any ideas how to reach this elements and make them testable?
I'm experimenting with code for a tool for automating PowerBuilder-based GUIs as well. From what I can see, your best bet would be to use the PowerBuilder Native Interface (PBNI), and call PowerScript code from within your NVO.
If you like, feel free to send me an email (see my profile for my email address), I'd be interested in exchanging ideas about how to do this.
I didn't use PowerBuilder for a while but I guess that the problem that you are trying to solve is similar to the one I am trying to address for people making projects with SCADA systems like Wonderware Intouch.
The problem with such an application is that there is no API to get or set the value of a control. So a pywinauto approach can't work.
I've made a small tool to simulate the user events and to get the results from a screencapture. I am usig PIL and pytesser ORM for the analysis of the screen captures. It is not the easiest way but it works OK.
The tool is open-source and free of charge and can be downloaded from my website (Sorry in french). You just need an account but it's free as well. Just ask.
If you can read french, here is one article about testing Intouch-based applications
Sorry for the self promotion, but I was facing a similar problem with no solution so I've written my own. Anyway, that's free and open-source...
I've seen in AutomatedQa support that they a recipe recommending using msaa and setting some properties on the controls. I do not know if it works.
If you are testing DataWindows (the class is pbdwxxx, e.g. pbdw110) you will have to use a combination of clicking at specific coordinates and sending Tab keys to get to the control you want. Of course you can also send up and down arrow keys to move among rows. The easiest thing to do is to start with a normal control like an SLE and tab into the DataWindow. The problem is that the DataWindow is essentially just an image. There is no control for a given field until you move the focus there by clicking or tabbing. I've also found that the DataWindow's iAccessible interface is a bit strange. If you ask the DataWindow for the object with focus, you don't get the right answer. If you enumerate through all of the children you can find the one that has focus. If you can modify the source I also advise that you set AccessibleName for your DataWindow controls, otherwise you probably won't be able to identify the controls except by position (by DataWindow controls I mean the ones inside the DataWindow, not the DataWindow itself). If it's an MDI application, you may also find it useful to locate the MicroHelp window (class fnhelpxxx, e.g. fnhelp110, find from the main application window) to help determine your current context.
Edited to add:
Sikuli looks very promising for testing PowerBuilder. It works by recognizing objects on the screen from a saved fragment of screenshot. That is, you take a screenshot of the part of the screen you want it to find.