Sometimes, I do live updates to a plot in a loop. Normally, this works fine, but when the processing within the loop takes a long time, the plot 'greys out'/sleeps, for all but the first 10 seconds of this time. This can be quite annoying, as it makes it typically not possible to distinguish the curves (I could use dotted lines of course, but...). I'm using Ubuntu, and about 10 seconds is the threshold where this starts to happen for me.
Below is some toy code to reproduce the problem, and some pictures to demonstrate what happens.
Is there an easy way to prevent this 'greying out' behaviour?
import numpy as np
import pylab as p
import time
def create_data(i):
time.sleep(10) # INCREASE THIS VALUE TO MAKE THE PLOT GREY OUT WHILE IT WAITS
return np.sin(np.arange(i) * 0.1)
def live_plot(y):
p.cla()
p.plot(y)
p.plot(y**2)
p.draw()
p.pause(0.01)
for i in xrange(1000):
y = create_data(i)
live_plot(y)
The issue is that the GUI window becomes non-responsive from the PoV of the window manager so it 'helpfully' grays it out to tell you this. There may be a setting in gnome/unity (not sure which you are using) to disable this. (system dependent, maybe impossible :( )
One solution is to push the computation to another thread/process which will allow you to use a blocking show to leave the GUI main loop responsive. (complex :( )
Another solution is to in your slow loop periodically poke the GUI loop. I think calling fig.canvas.flush_events() should be sufficient. (dirty :( )
In short, GUIs (and asynchronous stuff in general) are hard to get right.
In Ubuntu 18.04, go to "search program" and type in "compiz" in the search box. When the ccsm window comes up, look for and uncheck the "fade window" box.
Related
I would like to have some advice, what would be the best solution to my problem. I want to write a python program, that runs on Raspberry Pi. It runs in an infinite loop, and checks various stuff, like contents of some folders, and GPIO states.
A basic operation looks like this:
pictures arrive in a folder from a different source
the code notices, reads the images, does some processing
it starts to listen to GPIO input: in case of GPIO state change, the code displays the image.
if the GPIO state changes again, the code switches the picture to an another. And the loop resets, files are deleted, and the cycle starts again
I solved most of the problems, but I have difficulties with displaying the images.
I tried using opencv, I structured the code something like this, it's only a dummy:
state = "first_phase"
while True:
state = decide_state() #this part checks everything,
GPIO, folders, etc
if state = "first_phase":
cv2.imread(some_picture)
cv2.imshow(the picture)
cv2.waitKey(1500) #I had freezes with less delay, but this seems to work
state = "next_phase"
continue
elif state = "next_phase":
cv2.imread(some_other_picture)
cv2.imshow(the_picture)
cv2.waitKey(1500)
state = "first_phase"
continue
destroyAllWindows()
This is much much more complex in reality, with way more states and GPIO ports used, it really is a difficult application, but the idea is something like this, and it is working. 99% of the time...
1% of the time I get weird errors, like "GDK_IS_DEVICE" and Segmentation Faults. (I checked, resources are ok) I can't resolve them, and they come erratically, sometimes right away, sometimes after the 994th picture, and always at the point where the code tries to show the images.
I'm now convinced that it has to do with the highgui opencv uses, but - you can probably guess from my code - I have not enough experience to debug this. I cannot completely omit using opencv because I'll also do some operations on the images, but for displaying them I think I should use an another method. PILLOW is not okay, because I don't want to write out the results of the image manipulations, only draw them on the display. (also it is only possible to kill the display with subprocesses, and I don't feel comfortable doing that). I tried using tkinter but I'm lost how threading and communicating between my main loop and the tkinter loop would be possible and crash-free.
I can restart the code, but the last images will be lost, and it is important to show them again, if the code crashed. I could write a workaround, where I write the manipulated pictures on the disk, and after a successfull display it deletes them, so after starting up again, it sees that there are still images left to show, and shows them... but this only solves the symptoms, and also kills the sd card faster.
Is there any other solution to this? TL;DR: I want to draw pictures on the display in case of some events, while my code does it's thing in the background for eternity.
I want to draw a line, but not instantaneously as it usually appears with DrawLine(). I want it to be fast, but just slow enough that you can tell it's being drawn.
I've tried testing a for loop that draws a line a pixel at a time with a sleep of .05 seconds, but it doesn't seem to draw any of it until the for loop ends at which point it draws it all, instantly. I figured this had to do with it not getting drawing actions through the mainloop, so I used CallAfter on the drawline, that didn't work, either.
It seems like there should be a way to simply tell drawline to draw it over the course of x seconds or send it through an animation object in which you can specify # of frames, etc. Not really sure where to look. Any suggestions?
The issue is that time.sleep(...) will also block GUI as the for loop itself. wx.Yield should normally signal "GUI, I have halted the time-consuming process, do whatever you nned to do NOW". But by experience, this basically never works (I have not found out yet why).
So instead of using sleep you could use wx.CallLater (Link to wxPython Phoenix docs) which is like wx.CallAfter, but a delay can be specified. By calling it at the end of the inner for loop drawing the line part draw events should not pile up.
You will probably want to draw a series of lines. That would likely be the easiest. I would probably use a wx.Timer. It runs in it's own main loop. You can set up the timer to fire every so often, say every 0.5 second. When it fires, it can call your draw method, which will draw the next line. You can read more about timers here:
http://wiki.wxpython.org/Timer
http://www.blog.pythonlibrary.org/2009/08/25/wxpython-using-wx-timers/
http://www.wxpython.org/docs/api/wx.Timer-class.html
I've been fighting for three hours now to get this process multithreaded, so that I can display a progress box. I finally got it working, insomuch as the process completes as expected, and all the functions call, including the ones to update the progress indicator on the window.
However, the window never actually displays. This is a PyGObject interface designed in Glade. I am not having fun.
def runCompile(obj):
compileWindow = builder.get_object("compilingWindow")
compileWindow.show_all()
pool = ThreadPool(processes=1)
async_result = pool.apply_async(compileStrings, ())
output = async_result.get()
#output = compileStrings() #THIS IS OLD
compileWindow.hide()
return output
As I mentioned, everything works well, except for the fact that the window doesn't appear. Even if I eliminate the compileWindow.hide() command, the window never shows until the process is done. In fact, the whole stupid program freezes until the process is done.
I'm at the end of my rope. Help?
(By the way, the "recommended" processes of using generators doesn't work, as I HAVE to have a return from the "long process".)
I'm not a pyGobject expert and i don't really understand your code. I think that you should post more. Why are you calling the builder in a function? you can call it at the init of the GUI?
Anyways.. It seems that you are having the common multithread problems..
are you using at the startup GObject.threads_init() and Gdk.threads_init() ?
Then, if you want to show a window from a thread you need to use Gdk.threads_enter() and Gdk.threads_leave().
here is an useful doc
I changed the overall flow of my project, so that may affect it. However, it is imperative that Gtk be given a chance to go through its own main loop, by way of...
if Gtk.events_pending():
Gtk.main_iteration()
In this instance, I only want to call it once, to ensure the program doesn't hang.
(The entire program source code can be found on SourceForge. The function in question is on line 372 as of this posting, in function compileModel().
I would like to perform a measurement and plot a graph while the measurement is
running. This measurements takes quite some time in python (it has to retrieve data over a slow connection). The problem is that the graph freezes when measuring. The measurement
consists of setting a center wavelength, and then measuring some signal.
My program looks something like this:
# this is just some arbitrary library that has the functions set_wavelength and
# perform_measurement
from measurement_module import set_wavelength, perform_measurement
from pylab import *
xdata = np.linspace(600,1000,30) # this will be the x axis
ydata = np.zeros(len(xdata)) # this will be the y data. It will
for i in range(len(xdata)):
# this call takes approx 1 s
set_wavelength(xdata[i])
# this takes approx 10 s
ydata[i] = perform_measurement(xdata)
# now I would like to plot the measured data
plot(xdata,ydata)
draw()
This will work when it is run in IPython with the -pylab module switched on,
but while the measurement is running the figure will freeze. How can modify
the behaviour to have an interactive plot while measuring?
You cannot simply use pylab.ion(), because python is busy while performing the measurements.
regards,
Dirk
You can, though maybe a bit awkward, run the data-gathering as a serparate process. I find Popen in the subprocess module quite handy. Then let that data-gathering script save what it does to disk somewhere and you use
Popen.poll()
To check if it has completed.
It ought to work.
I recommend buffering the data in large chunks and render/re-render when the buffer fills up. If you want it to be nonblocking look at greenlets.
from gevent.greenlet import Greenlet
import copy
def render(buffer):
'''
do rendering stuff
'''
pass
buff = ''
while not_finished:
buff = connection.read()
g = Greenlet(render, copy.deepcopy(buff))
g.start()
Slow input and output is the perfect time to use threads and queues in Python. Threads have there limitations, but this is the case where they work easily and effectively.
Outline of how to do this:
Generally the GUI (e.g., the matplotlib window) needs to be in the main thread, so do the data collection in a second thread. In the data thread, check for new data coming in (and if you do this in some type of infinite polling loop, put in a short time.sleep to release the thread occasionally). Then, whenever needed, let the main thread know that there's some new data to be processed/displayed. Exactly how to do this depends on details of your program and your GUI, etc. You could just use a flag in the data thread that you check for from the main thread, or a theading.Event, or, e.g., if you have a wx backend for matplotlib wx.CallAfter is easy. I recommend looking through one of the many Python threading tutorials to get a sense of it, and also threading with a GUI usually has a few issues too so just do a quick google on threading with your particular backend. This sounds cumbersome as I explain it so briefly, but it's really pretty easy and powerful, and will be smoother than, e.g., reading and writing to the same file from different processes.
Take a look at Traits and Chaco, Enthought's type system and plotting library. They provide a nice abstraction to solve the problem you're running into. A Chaco plot will update itself whenever any of its dependencies change.
I have run into this trouble with show() over and over again, and I'm sure I'm doing something wrong but not sure of the 'correct' way to do what I want.
And [I think] what I want is some way to block in the main thread until an event happens in the GUI thread, something like this works the first time:
from matplotlib import pyplot as p
from scipy import rand
im = (255*rand(480,640)).astype('uint8')
fig = p.figure()
ax = fig.add_subplot(111)
ax.imshow(im)
# just any mutable container for storing a click
s = [-1, -1]
def onclick(event):
if event.xdata is not None and event.ydata is not None:
s[0] = event.xdata
s[1] = event.ydata
p.close()
cid = fig.canvas.mpl_connect('button_press_event', onclick)
p.show()
print s
the p.show() blocks until p.close() is called in the event handler. But when run the same code the second time, it races past the p.show() and prints that original s, [-1, -1].
I have read conflicting information on whether or not p.show() can or should be called more than once from the same program. It seems it was designed to be used once, and only once at the end of a script. Other use cases seem to break pyplot somehow (state machine?).
I've tried to use combinations of p.draw() and p.ion() and p.ioff(), but could not get the behaviour I wanted (either things weren't blocking properly or the plots weren't appearing at the right times).
I'm also confused about how the event handler is able to see s at all here, and whether this is a poor way of passing in/out information. If I don't use a mutable container like an array or list, the information I want set by the event handler just gets lost as a local variable. is there some other method I'm missing out on , where the GUI thread can pass signals back to the main thread? is there a way to block in main, without periodic polling or busy waiting , for a signal from the event handler before continuing?
So I guess ultimately my main question is:
Is there a neat replacement for p.show(), that does what I want (the same behaviour as p.show() has the first time), or does this kind of code require a complete rethink/rewrite ?
Couple ideas of varying quality:
If you don't like s being a global variable, you could make onclick() a callable object attach it to that.
Your callback could acquire/release a lock to control program flow (little dirty).
You could actively poll s to control program flow (very dirty).
You can manually control the drawing of your figures via fig.canvas.draw()
I was able to resolve my issue today. if anyone else is interested in changing the behaviour of show(), read on for how you can do it:
I noticed this paragraph titled multiple calls to show supported on the what's new part of the matplotlib webpage:
A long standing request is to support multiple calls to show(). This has been difficult because it is hard to get consistent behavior across operating systems, user interface toolkits and versions. Eric Firing has done a lot of work on rationalizing show across backends, with the desired behavior to make show raise all newly created figures and block execution until they are closed. Repeated calls to show should raise newly created figures since the last call. Eric has done a lot of testing on the user interface toolkits and versions and platforms he has access to, but it is not possible to test them all, so please report problems to the mailing list and bug tracker.
This was in 'what's new' for version 1.0.1, at time of writing the version in synaptic is still back on 0.99.3. I was able to download and build from source v1.0.1. Additional packages I also required to satisfy dependencies were libfreetype6-dev tk-dev tk8.5-dev tcl8.5-dev python-gtk2-dev.
Now with matplotlib.__version__ == 1.0.1 , the following code blocks how I would expect:
import matplotlib.pyplot as p
from scipy import eye
p.imshow(eye(3))
p.show()
print 'a'
p.imshow(eye(6))
p.show()
print 'b'
p.imshow(eye(9))
p.show()
print 'c'
I noticed a difference between running the code
Directly within the Python interpretor (command line)
Putting it in a Python script and running it from the command line ("python script.py")
Both give a blocking behavior, which is ok.
From the interpretor both images appear, from the command line only the first appears.