I am trying to display my loader-dialog in a thread because when I start the upload system of my files I can't see the window until the processing is finished.
I tried something like this:
thread.start_new_thread(self.display_loader(),(self))
but it didn't work.
There is a special way to run a new window in a thread?
Everything is done with Python and PyGtk
When you say self.display_thread(), that will actually call the display_thread function right there, and pass its return value as the first argument to thread.start_new_thread. That's probably not what you intended.
That said, I think you're better off doing it the other way around; let the main thread own the UI, and do the loading off in a thread.
Remember that GTK+ is not very thread-safe, it's absolutely best to do all interaction with GTK+ from a single thread.
UPDATE: Actually, the above is perhaps over-simplifying a bit, it's the "common truth" as I've understood it (I've been using GTK+ for ~15 years but seldom with threads). This article re-states it in a more forgiving way, not sure if it really makes life that much easier in this case, though. I wanted to mention in for completeness' sake.
Related
I have a GUI driven image analysis package in IDL that needs some serious rewriting. Python has been suggested to be as an alternative to IDL (with benefits of cost and some nice libraries among other things). I've poked around now with PyQT4 and it looks like it should work nicely. However, one of the best things about IDL (being interpreted) is that if the code hits a bug, you can correct it on the fly, type 'retall' and then continue with your work. If you are hours into some analysis and have lots of datafiles open, etc., this is a HUGE improvement over having to exit, then change and restart the program. Not only that but we can quickly try some things on the command line, then if it looks good, code up a routine, put it in the menu structure, and then 'retall' and we are back with the new functionality, all without ever having to restart the program.
My question is, is this possible with Python? A little googling makes the answer seem like no but since it is an interpreted language I don't understand why not. If the answer really is no I'd strongly urge someone to think about implementing this -- it is probably the feature that made me most happy about IDL over the past decade.
Thanks in advance,
Eric
I don't think it can be done.
However, you may consider structuring your package to cache intermediate results on disk and to allow resume from thoses cached results. This has benefits outside the bug case you describe.
Fixing a bug "on the fly" seems potentially dangerous to me as it is so easy to overlook possible side-effects of the bug. It breaks the link between execution flow and code and make it extremely difficult to debug subsequent issues, etc ...
Actually working on a project in python with PyQT, we choose to create widgets that were "unpleasant" or that didn't have a good enough behaviour.
So, we finally found that QToolbox, QDate and some others had a behaviour non acceptable for the project, so we had to adapt these.
We had also to create a complete new widget : A scheduler.
As we were creating these, it has been decided that it took too much time. So we were asked to think about other libraries.
I actually found a project of a scheduler in wxPython, that actually looks like what we want ( but we believe that we'll have to adapt it a lot ). Here it is : http://code.google.com/p/wxscheduler/
So, I ask everyone that have some more experience than me in GUI programming in python : Do we need to start again the project in anything other than PyQT? I know the question is weird, but what you need to know is :
The project has now been going on for 2 months
I know only PyQT, and started working in python 2 month ago
We are currently 3 in the project, and we currently know only PyQT
We have currently managed a lot of the PyQT widgets, and were starting to code these new widgets.
Please help us =)
Thanks
Edit : I should have add that the project is opensource and multi-platform
Feel free to look at other libraries if you like. Robin Dunn, the creator of wxPython, recently started working on PySide and he found it somewhat similar to wx, so you might find that wxPython will fit your brain fairly well too. I certainly think wx's class names are more intuitive than PyQt's. The only way to know for certain is to actually experiment a little and see if it works. I will say that the wxPython community is one of the best Python communities I've dealt with over the years.
One possibility would be to use an HTML scheduler control via QtWebKit. If your UI can accommodate a QWebView in the place where you'd otherwise have a custom scheduler widget, there are probably a number of excellent scheduler widgets (implemented as jQuery plugins, etc.) you could choose from.
we choose to create widgets that were "unpleasant" or that didn't have
a good enough behaviour.
Why don't you create "pleasant" widgets ?
so we had to adapt these.
Yes. It is the solution.
it has been decided that it took too much time
Don't you think it would take much more time if you change the whole GUI API ?
As for i know, there is not such native Scheduler in any Python GUI library, especially one you could use with Qt. I don't think it would be so long to recreate one, unless you have very specific needs, that would confirm you wouldn't find a such existing thing in an existing library.
Concerning wxScheduler, i guess you can have a look to the code, even it uses wxWidget and you're working with Qt, to get an idea how to do it.
I would need to run a python script for some random amount of time, pause it, get a stack traceback, and unpause it. I've googled around for a way to do this, but I see no obvious solution.
There's the statprof module
pip install statprof (or easy_install statprof), then to use:
import statprof
statprof.start()
try:
my_questionable_function()
finally:
statprof.stop()
statprof.display()
There's a bit of background on the module from this blog post:
Why would this matter, though? Python already has two built-in profilers: lsprof and the long-deprecated hotshot. The trouble with lsprof is that it only tracks function calls. If you have a few hot loops within a function, lsprof is nearly worthless for figuring out which ones are actually important.
A few days ago, I found myself in exactly the situation in which lsprof fails: it was telling me that I had a hot function, but the function was unfamiliar to me, and long enough that it wasn’t immediately obvious where the problem was.
After a bit of begging on Twitter and Google+, someone pointed me at statprof. But there was a problem: although it was doing statistical sampling (yay!), it was only tracking the first line of a function when sampling (wtf!?). So I fixed that, spiffed up the documentation, and now it’s both usable and not misleading. Here’s an example of its output, locating the offending line in that hot function more accurately:
% cumulative self
time seconds seconds name
68.75 0.14 0.14 scmutil.py:546:revrange
6.25 0.01 0.01 cmdutil.py:1006:walkchangerevs
6.25 0.01 0.01 revlog.py:241:__init__
[...blah blah blah...]
0.00 0.01 0.00 util.py:237:__get__
---
Sample count: 16
Total time: 0.200000 seconds
I have uploaded statprof to the Python package index, so it’s almost trivial to install: "easy_install statprof" and you’re up and running.
Since the code is up on github, please feel welcome to contribute bug reports and improvements. Enjoy!
I can think of a couple of few ways to do this:
Rather than trying to get a stack trace while the program is running, just fire an interrupt at it, and parse the output. You could do this with a shell script or with another python script that invokes your app as a subprocess. The basic idea is explained and rather thoroughly defended in this answer to a C++-specific question.
Actually, rather than having to parse the output, you could register a postmortem routine (using sys.excepthook) that logs the stack trace. Unfortunately, Python doesn't have any way to continue from the point at which an exception occurred, so you can't resume execution after logging.
In order to actually get a stack trace from a running program, you will may have to hack the implementation. So if you really want to do that, it may be worth your time to check out pypy, a Python implementation written mostly in Python. I've no idea how convenient it would be to do this in pypy. I'm guessing that it wouldn't be particularly convenient, since it would involve introducing a hook into basically every instruction, which would I think be prohibitively inefficient. Also, I don't think there will be much advantage over the first option, unless it takes a very long time to reach the state where you want to start doing stack traces.
There exists a set of macros for the gdb debugger intended to facilitate debugging Python itself. gdb can attach to an external process (in this case the instance of python which is executing your application) and do, well, pretty much anything with it. It seems that the macro pystack will get you a backtrace of the Python stack at the current point of execution. I think it would be pretty easy to automate this procedure, since you can (at worst) just feed text into gdb using expect or whatever.
Python already contains everything you need to do what you described, no need to hack the interpreter.
You just have to use the traceback module in conjunction with the sys._current_frames() function. All you need is a way to dump the tracebacks you need at the frequency you want, for example using UNIX signals, or another thread.
To jump-start your code, you can do exactly what is done in this commit:
Copy the threads.py module from that commit, or at least the stack trace dumping function (ZPL license, very liberal):
Hook it up to a signal handler, say, SIGUSR1
Then you just need to run your code and "kill" it with SIGUSR1 as frequently as you need.
For the case where a single function of a single thread is "sampled" from time to time with the same technique, using another thread for timing, I suggest dissecting the code of Products.LongRequestLogger and its tests (developed by yours truly, while under the employ of Nexedi):
Whether or not this is proper "statistical" profiling, the answer by Mike Dunlavey referenced by intuited makes a compelling argument that this is a very powerful "performance debugging" technique, and I have personal experience that it really helps zoom in quickly on the real causes of performance issues.
To implement an external statistical profiler for Python, you're going to need some general debugging tools that let you interrogate another process, as well as some Python specific tools to get a hold of the interpreter state.
That's not an easy problem in general, but you may want to try starting with GDB 7 and the associated CPython analysis tools.
Seven years after the question was asked, there are now several good statistical profilers available for Python. In addition to vmprof, already mentioned by Dmitry Trofimov in this answer, there are also vprof and pyflame. All of them support flame graphs one way or another, giving you a nice overview of where the time was spent.
Austin is a frame stack sampler for CPython that can be used to make statistical profilers for Python that require no instrumentation and introduce minimal overhead. The simplest thing to do is to pipe the output of Austin with FlameGraph. However, you can just grab Austin's output with a custom application to make your very own profiler that is targeted at precisely your needs.
This is a screenshot of Austin TUI, a terminal application that provides a top-like view of everything that is happening inside a running Python application.
This is Web Austin, a web application that shows you a live flame graph of the collected samples. You can configure the address where to serve the application which then allows you to do remote profiling.
There is a cross-platform sampling(statistical) Python profiler written in C called vmprof-python.
Developed by the members of PyPy team, it supports PyPy as well as CPython.
It works on Linux, Mac OSX, and Windows. It is written in C, thus has a very small overhead.
It profiles Python code as well as native calls made from Python code.
Also, it has a very useful option to collect statistics about execution lines inside functions in addition to function names.
It can also profile memory usage (by tracing the heap size).
It can be called from the Python code via API or from the console.
There is a Web UI to view the profile dumps: vmprof.com, which is also open sourced.
Also, some Python IDEs (for example PyCharm) have integration with it, allowing to run the profiler and see the results in the editor.
For Python there is py-spy to dump the stacktraces. The dumps can get analyzed by speedscope
Source: Guidelines
When starting up a new project, as a beginner, which would you use?
For example, in my situation. I'm going to have a program running on an infinite loop, constantly updating values. I need these values to be represented as a bar graph as they're updating. At the same time, the GUI has to be responsive to user feedback as there will be some QObjects that will be used to updated parameters within that infinite loop. So these need to be on separate threads, if I'm not mistaken. Which choice would give the most/least hassle?
If I understood your question correctly, updating the GUI has a little to do with the way you programmed it.
From my experience, it's easier to design a main window (or whatever your top level object is) in Designer, and add some dynamically updated content in a widget(s) created in your code. In most cases, it saves your time spent on digging through QT documentation, and additionally, you are able to visually inspect positioning, aligning etc.
You don't lose anything by using a Designer, every part of the GUI can be modified in your code afterwards, if it needs some custom behavior.
Having said that, without knowing all the details of your project is hard to tell which option (QT or in-code) is faster.
Your right threading is your answer. Use the QT threads they work very well.
Where I work when people start out using QT a lot of them start with designer but eventually end up hand coding it. I think you will end up hand coding it but if you are someone who really likes GUIs you may want to start with Designer. I know that isn't a definitive answer but it really depends.
First of all, the requirements that you've mentioned don't (or shouldn't) have much affect on this decision.
Either way, you're going to have to learn something. You might as well investigate both options, and make the decision yourself. Write a couple of "Hello, World!" apps, then start adding some extra widgets/behavior to see how each approach scales.
Since you asked, I would probably use Qt Designer. But I'm not you, and I'm not working on (nor do I know much of anything about) your project.
The tutorials I've found on WxPython all use examples from Linux, but there seem to be differences in some details.
For example, in Windows a Panel behind the widgets is mandatory to show the background properly. Additionally, some examples that look fine in the tutorials don't work in my computer.
So, do you know what important differences are there, or maybe a good tutorial that is focused on Windows?
EDIT: I just remembered this: Does anybody know why when subclassing wx.App an OnInit() method is required, rather than the more logical __init__()?
I've noticed odd peculiarities in a small GUI I wrote a while back, but it's been a long time since I tried to the specifics are a rather distant memory. Do you have some specific examples which fail? Maybe we can improve them and fix the bugs?
Have you tried the official wxPython tutorials? ...or were you after something more specific?
r.e. your edit - You should use OnInit() because you're subclassing wx.App (i.e. it's a requirement for wxWidgets rather than Python) and the wxPython implementation is wherever possible, just a wrapper for wxWidgets.
[Edit] Zetcode has a fairly lengthy tutorial on wxPython. I've not looked through it all myself, but it might be of some help?
The wxWidgets::wxApp::OnInit() documentation is fairly clear:
This must be provided by the application, and will usually create the application's main window, optionally calling wxApp::SetTopWindow. You may use OnExit to clean up anything initialized here, provided that the function returns true.
If wxWidgets didn't provide a common interface then you'd have to do different things in C++ (using a constructor) compared to Python's __init__(self,...). Using a language-independent on-initialisation allows wxWidgets ports to other languages look more alike which should be a good thing right? :-)
EDIT: I just remembered this: Does anybody know why when subclassing wx.App an OnInit() method is required, rather than the more logical __init__()?
I use OnInit() for symmetry: there's also an OnExit() method.
Edit: I may be wrong, but I don't think using OnInit() is required.
I find a number of small differences, but don't remember all of them. Here are two:
1) The layout can be slightly different, for example, causing things to not completely fit in the window in one OS when the do in the other. I haven't investigated the reasons for this, but it happens most often when I use positions rather than sizers to arrange things.
2) I have to explicitly call Refresh more in Windows. For example, if you place one panel over another, you won't see it the top panel in Windows until you call Refresh.
I general, I write apps in Linux and run them in Windows, and things work similarly enough so this is a reasonable approach, but it's rare for me when something runs perfectly straight out of the gate after an OS switch.