We Keep Coding

Difference between pyOpenCL and opencl4py

Today I stumbled over a post in stackoverflow (see also here):
We are developing opencl4py, higher level bindings. This project uses CFFI, so it works on Pypy.
The major issue we encountered with pyopencl is that 'import pyopencl' does OpenCL initialization and takes the whole virtual memory in case of NVIDIA driver, preventing from correct forking and effectively disabling multiprocessing (yes, we claim that using pyopencl disables multiprocessing at least with NVIDIA). opencl4py uses lazy OpenCL initialization, resolving this "import hell".
Later, it gained some nice features like super easy binary program caching, etc. Unfortunately, the documentation is somewhat brief. The best way to learn how it works is go through the tests.
As there is also pyOpenCL, I was woundering what the difference between these two packages is. Does anybody know where I can find an overview on the pro's and con's for these both packages?
Edit: To include benshope's comment as I would also be interested: what does "disable[s] multiprocessing" mean? Like, it can't run kernels on several devices at one time?

As far as I know, there is no such overview. I'll try to list some key points:
pyOpenCL is a mature project with a relatively large user base. There are tutorials, FAQ, etc. opencl4py appeared on 03/2014; no tutorials, FAQ and so on - only unit tests and docstrings.
pyOpenCL is a native cPython extension, whereas opencl4py uses cffi, so that it works on PyPy (pyOpenCL does NOT) and it does not require to be recompiled each time cPython changes version.
PyOpenCL has extras, such as random number generator and OpenGL interoperability.
opencl4py is extensively tested in Samsung production real world scenarios and is being actively developed.
what does "disable[s] multiprocessing" mean? Like, it can't run kernels on several devices at one time?
Of course, it can, I was trying to say that after importing pyopencl, os.fork() or multiprocessing.Process() lead to crashes inside NVIDIA OpenCL userspace library. It is always a bad idea of doing work during import.

Launch times of compiled Python vs IronPython

I'm currently writing some Windows GUI applications in Python 2.7 using pyQT4 and pyInstaller. These compiled executables are taking about 3-6 seconds to open. I've eliminated UPX already and seen some time shaved off the load, and some more by not packaging it as a single exe but I'd rather have a single distributable file.
I was wondering if IronPython would be a better at providing a quick loading application. Or any of the other version of Python, or better yet if there is anything else i can do to minimize the wait for the GUI to be drawn on the screen.

IronPython does add a performance hit to a number of specific areas compared to Python. However, I would not draw the conclusion that it is always slower. There has been a post already on that topic Why is IronPython faster than the Official Python Interpreter
The most important aspect to consider is have you fully optimized everything in your Python project? As in, have you performed any code profiling?
If the answer is yes then you could attempt to migrate to a different toolkit or language. If you chose IronPython (link) it would be wise to migrate your GUI to .NET (winforms, XAML, etc). Another option (albeit unpopular to most) would be to use tkinter. Tkinter is very powerful in that one can fully-customize the code to be nothing more and nothing less than the project requires. Of course that will add considerable typing time. But I have made Python apps that load and draw a window almost immediately and quite honestly almost looked like it came out of VS2012.

App created with PyInstaller has a slow startup

I have an application written in Python and 'compiled' with PyInstaller. It also uses PyQt for the GUI framework.
Running this application has a delay of about 10 seconds before the main window loads and is shown. As far as I can tell, this is not due to slowness in my code. Instead, I suspect this is due to the Python runtime initializing.
The problem is that this application is started with a custom laucncher / taskbar application. The user will click the button to launch the app, see nothing appear to happen, and click elsewhere on another application. When my application shows it's window, it cannot come to the foreground due to the rules for SetForegroundWindow.
I have access to the source for the PyInstaller win32 loader, the Python code, and even the launcher code.
My questions are:
How can I make this application start faster?
How can I measure the time spend i the first few seconds of the process's lifetime?
What is the generally accepted technique for reducing time until the first window is shown?
I'd like to avoid adding a splash screen for two reasons - one, I expect it won't help (the overhead is before Python code runs) and two, I just don't like splash screens :)
If I need to, I could probably edit the PyInstaller loader stub to create a window, but that's another route i'd rather not take.

I suspect that you're using pyinstaller's "one file" mode -- this mode means that it has to unpack all of the libraries to a temporary directory before the app can start. In the case of Qt, these libraries are quite large and take a few seconds to decompress. Try using the "one directory" mode and see if that helps?

Tell PyInstaller to create a console-mode executable. This gives you a working console you can use for debugging.
At the top of your main script, even before the first import is run, add a print "Python Code starting". Then run your packaged executable from the command line. This way you can get a clear picture whether the time is spent in PyInstaller's bootloader or in your application.
PyInstaller's bootloader is usually quite fast in one-dir mode, but it can be much slower in one-file mode, because it depacks everything into a temporary directory. On Windows, I/O is very slow, and then you have antiviruses that will want to double check all those DLL files.
PyQt itself is a non-issue. PyQt is generated by SIP which generates very fast lazy bindings; importing the whole PyQt is faster than any other GUI library because it basically does nothing: all bindings to classes/functions are dynamically created when (and if!) you access them, saving lots of memory too.
If your application is slow at coming up, that will be true without PyInstaller as well. In that case, your only solution is either a splash screen (import just PyQt, create QApplication, create a display the splashscreen, then import the rest of your program and run it), or rework your code. I can't help you much without details.

I agree with above answers. My Qt python program needed about 5 seconds to start up on a decent PC when using onefile mode. After I change to --onedir, it only took around one second to start; almost immediately after user double clicks the exe file. But the drawback is that there are many files in that directory which is not so neat.

For my application, the long startup time almost entirely was caused by the antivirus system. Switching it off reduced the startup in my case from 3 minutes to less than 10secs!
To bring these measurements into perspective: My application was bundled with extra data files (about 150 files with a payload of 250MB), besides carrying around Qt and numpy (that may depend on Intel MKL, which alone adds another 200MB to the bundle!) dependencies. It even didn't help much that the tested system was running with a solid state drive...
In conclusion: if you have a large application with lots of dependencies, the startup time may be affected strongly by the antivirus system!

In case anyone is still have this issue, I resolved mine by running the exe locally and not on any sharedrives. This took the startup time from over a minute to under 10 seconds.

I solved this by adding an exception to the anti-virus monitoring software (F-Secure). It removed the wait of several minutes before running.

I have 'compiled' a few wxPython apps using py2exe and cx_Freeze, None of them take more than 4 seconds to start.
Are you sure sure it's not your code ?
maybe some network or some I/O resource call holding your app ?
Have you tried other machine than yours? Even the fastest hardware can be slow sometimes with the wrong software config, apps or OS, try it.
Try timing it with timeit module.
I never used pyQT, but with wxPython the startup speed is OK, and after the first initialize if I close and open again, it's faster than the first time.

PyQt or PySide - which one to use [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I started learning a bit of python and would now like to toy around a bit with gui-building. Qt seems to be a good choice because of its cross-platformishness.
Now there seem to be two bindings available: PyQt by Riverbank Computing and PySide, originally developed by Nokia.
So which one should I choose? All I can find are two year old feature comparisons, but what differences are there nowadays?
Which one is easier to use, has more/better documentation? Are both still in active development?
Licensing isn't of much concern to me since I don't intend to write commercial applications.

Both toolkits are actively maintained, and by now more or less equal in features and quality. There are only few, rather unimportant differences.
Still, I'd recommend PySide for Python 2. It has a more reasonable API, mainly it doesn't expose Qt types, which have a direct equivalent in Python (e.g. QString, QList, etc.) or which are completely superfluous due to Python's dynamic nature, like QVariant. This avoids many tedious conversions to and from Qt types, and thus eases programming and avoids many errors.
PyQt also supports this modern API, and uses it by default for Python 3, but not for Python 2 to maintain backwards compatibility.

There is also the licensing difference. PySide is LGPL while PyQt is GPL. This could make a difference if you don't wish to make your project opensource. Although PyQt always has the propriety version available for a fairly reasonable price.
I tend to find the PySide documentation more intuitive. The API, in my opinion is slightly more Pythonic and the rate of bug fixes is quite impressive at the moment.
PyQt has the advantage of Python 3 support and incumbency. There is a lot more 3rd party documentation/tutorials for it.

I recently ported a significant code base (over 8,000 lines of code) from PyQt to PySide.
Right now I'd say PyQt is a much more mature, performant and stable project. I hit a number of bugs in PySide, and suspect that any big project will hit issues. Having said that, I reported a bug to the project and it was fixed and in a new release within a few weeks. I'm also having a problem where the app takes about 15 seconds to quit. I've not yet spent the time to find out why. However it's only a matter of time before there will be no reasons for choosing PyQt over PySide.
If you do decide to go with PyQt for now, make sure you use API v2 throughout. It is a better API, and will ease any future transition to PySide. Also if you do port, just follow the guidelines on the PySide wiki. Even for an 8+ kloc app consisting of about 20 source files it just took an afternoon.

Although they might have similar interface for Qt/C++ classes, their interface for Qt/C++ macros such as signal/slot/property are very different.
Porting one to another is not an easy job. It would be better to make the right decision at the very beginning.
Beyond the grammar/license differences, I just want to point out some deficiency of PyQt in language binding, which might be essential to write QML project in Python.
These differences finfally push me to PySide from PyQt.
qmlRegisterType
qmlRegisterType is essential to create runtime C++ binding with QML.
In PySide, it is part of PySide.QtDeclarative. And this works pretty well with Python.
In PyQt, qmlRegisterType does not exist. And I could not find an alternative approach.
I know some simple task could be done by setting QML context.
But if you really need runtime binding with qmlRegister and Q_INVOKABLE, I think PySide is the only choice at the moment.
Shiboken VS SIP
Both can wrap Qt/C++ into python plugin.
For Shiboken, I feel it is simpler and requires less coding.
Just create a typesystem xml including the NAME of classes you want to export, and that's all.
Shiboken does not require extra manual descriptions for the structure of the target classes.
For SIP, it would require more extra coding. We will have to create a SIP file that almost reimplements everything the C++ header.
It requires not only the NAME of the class, but also the DETAILS of what methods the target classes have.
If the C++ class is in good design using Pimp and we want to export all methods within it, SIP should provide a way of automatically export all class methods, which it cannot at the moment.
This would also add the burden for maintaining the consistency between the SIP and the C++ headers.
But I have to say that the documentation for Shiboken on Qt wiki is very bad and misleading.
Creating Python plugin with Shiboken on Windows do not necessarily require CMake at all.
generatorrunner is also not required.
I only use a windows cmd script to invoke shiboken, and a qmake pro for compiling the target plugin.

An important fact is that PyQt4 has two versions of its APIs for some things. Version 1 items are such things as using QString instead of unicode, and QVariant (basically just a wrapper, I believe - I've never actually done anything which uses it) instead the wrapped. Version 2, which can be enabled in Python 2 and is enabled in Python 3, is much better (though still unpythonic in many places - PySide is too, but it's getting distinctly better. There are still some remaining incompatibilities with them; PyQt4 has QtCore.pyqt(Signal|Slot|Property), PySide has QtCore.(Signal|Slot|Property).
For a project of my own, I decided that I wanted to support both with no changes to the code. I prefer PySide, but on Windows I distribute with PyQt4 as at present it's quite a bit smaller for distribution at present. My solution is to check for PySide and if it's there insert an import hook to redirect PyQt4 imports to PySide, or if it's not, fix up PyQt4 to work like it should.
The files used:
pyqt4pysideimporter.py
zip_imp.py (for py2exe support)
make_gui.py (my script for building .ui files and .qrc files with pyside or pyqt4 tools, and fixing the imports to be consistent; polls for file changes and rebuilds the changed - nothing high tech like inotify)
Then you just import pyqt4pysideimporter and pyqt4pysideimporter.autoselect() (as in main.py in that repository). And after that you can just import PyQt4.
Aside: it was also stated a few days ago on the PySide mailing list that they are planning on supporting Python 3 fully within the next few months.

I have a 20k line Python app that I unsuccessfully tried to convert to PySide.
Conversion is easy and most of the functionality works.
There are several methods that are not implemented because they are 'deprecated', so I had to fix those. That was OK.
On Windows, using PySide-1.1.2, the '==' operator is not implemented for many Qt objects. One workaround is to say: "if id(item1) == id(item2):".
The other observation is that PySide seemed noticeably slower. I did not isolate PySide as the cause of the slowness, but the problem went away when I reverted back to PyQt.
Lastly, as of now, the Android kit with PySide does not seem ready for prime time.

How do I deploy a Python desktop application?

I have started on a personal python application that runs on the desktop. I am using wxPython as a GUI toolkit. Should there be a demand for this type of application, I would possibly like to commercialize it.
I have no knowledge of deploying "real-life" Python applications, though I have used py2exe in the past with varied success. How would I obfuscate the code? Can I somehow deploy only the bytecode?
An ideal solution would not jeopardize my intellectual property (source code), would not require a direct installation of Python (though I'm sure it will need to have some embedded interpreter), and would be cross-platform (Windows, Mac, and Linux). Does anyone know of any tools or resources in this area?
Thanks.

You can distribute the compiled Python bytecode (.pyc files) instead of the source. You can't prevent decompilation in Python (or any other language, really). You could use an obfuscator like pyobfuscate to make it more annoying for competitors to decipher your decompiled source.
As Alex Martelli says in this thread, if you want to keep your code a secret, you shouldn't run it on other people's machines.
IIRC, the last time I used cx_Freeze it created a DLL for Windows that removed the necessity for a native Python installation. This is at least worth checking out.

Wow, there are a lot of questions in there:
It is possible to run the bytecode (.pyc) file directly from the Python interpreter, but I haven't seen any bytecode obfuscation tools available.
I'm not aware of any "all in one" deployment solution, but:
For Windows you could use NSIS(http://nsis.sourceforge.net/Main_Page). The problem here is that while OSX/*nix comes with python, Windows doesn't. If you're not willing to build a binary with py2exe, I'm not sure what the licensing issues would be surrounding distribution of the Python runtime environment (not to mention the technical ones).
You could package up the OS X distribution using the "bundle" format, and *NIX has it's own conventions for installing software-- typically a "make install" script.
Hope that was helpful.

Maybe IronPython can provide something for you? I bet those .exe/.dll-files can be pretty locked down. Not sure how such features work on mono, thus no idea how this works on Linux/OS X...

I have been using py2exe with good success on Windows. The code needs to be modified a bit so that the code analysis picks up all modules needed, but apart from that, it works.
As for Linux, there are several important distribution formats:
DEB (Debian, Ubuntu and other derivatives)
RPM (RedHat, Fedora, openSuSE)
DEBs aren't particularly difficult to make, especially when you're already using distutils/setuptools. Some hints are given in the policy document, examples for packaging Python applications can be found in the repository.
I don't have any experience with RPM, but I'm sure there are enough examples to be found.

Try to use scraZ obfuscator (http://scraZ.me).
This is obfuscator for bytecode, not for source code.
Free version have good, but not perfect obfuscation methods.
PRO version have very very strong protection for bytecode.
(after bytecode obfuscation a decompilation is impossible)