freezing gui while socket - python

i am building a small gui apllication that allowd the user to
download file from the server. it got a use in socket and tkinter mostly.
but when i download a file (a movie for example) its take a time, lets say 5 minuites for example. and in that time i want a progress bar that will tart looping till the file fully downloaded. but when the client getting the file data line by line using sock.recv,
the all gui program is freezing!
so because of that the progress bar cannot move,
and i cannot push any buttons.
so my question is - how can i fix it? mean that the gui application wont be stack while getting data from the server, and then i can make the progress bar work.
thanks lot you guys.

Here I tried to describe (mostly pseudo-code) how you can implement a non-blocking download function with progress bar.
Hope it helps.
def update_progress(percentage):
# update your progress bar in GUI
def download(callback):
# implement your download function
# check the full size of the file to be downloaded.
# try to download a reasonable amount at once
# call callback with percentage that you have downloaded to update GUI
total = 1000000 # get total size of the file to be downloaded.
current = 0
block_size = 1000 # i.e., 1 KB
while True:
# do required operations to download data of block_size amount
# example: sock.recv(block_size)
current += block_size
# calculate downloaded percentage
percentage = (block_size * 100) / total # you may add precision if you prefer
# call the callback function to update GUI based on the downloaded percentage
callback(percentage)
# check if download completed
if current >= total:
break
def start_download(): # bind this function to your button's click on GUI.
# import threading
# create a thread to execute download
# see how 'update_progress' is passed as an argument
thread = threading.Thread(target=download, args=[update_progress])
thread.start()
# execution will not be blocked here as the thread runs in the background.
# so, any code here will run without waiting for download to be completed.

thanks you guys for the help and especially to you ohannes, i used in background
thread class, this is the code: (you need change the 'root' to the name of your tkinter window)
class ThreadedClient(threading.Thread):
def __init__(self, queue, fcn):
threading.Thread.__init__(self)
self.queue = queue
self.fcn = fcn
def run(self):
time.sleep(1)
self.queue.put(self.fcn())
def spawnthread(fcn):
thread = ThreadedClient(queue, fcn)
thread.start()
periodiccall(thread)
def periodiccall(thread):
if(thread.is_alive()):
root.After(100, lambda: periodiccall(thread))
#END

Related

PyQt5 UI "freezes" when updating it too quickly from another thread

I'm making a GUI to play videos using the python bindings for VLC, and I've run into a snag with the progress slider. Due to VLC's extreme inaccuracy and inconsistency when it comes to reporting a video's progress, I've been faking the QSlider by simply incrementing it + updating the rest of the UI (which includes multiple QLabels, QLineEdits, and QSpinBoxes) once per frame in a separate thread (not a QThread).
However, something strange has been happening: eventually, the entire UI "freezes"... except the video continues playing, the console continues outputting, and the UI instantly jumps back to life after manually interacting with it (such as by pressing the pause button). Qt is running and keeping track of the current state of the UI, just without painting anything.
I've tried manually updating the UI, manually repainting the UI (which always crashes...?), auto-resizing the UI, running processEvents(), running QTest.qWait(1), but nothing has worked. Using a QTimer to increment the progress bar, however, DOES prevent the UI from freezing, which confirms I'm not crazy. Of course, I can't actually use a QTimer though, since even a Qt.PreciseTimer results in... a very imprecise timer.
I've read that this is likely caused by Qt mistakenly trying to optimize my code by clumping the rapid updates into one, which would explain why the UI seemingly stops being painted despite clearly being active under the hood.
Though I have a feeling that the solution is probably something obvious (maybe even a different implementation of something I mentioned trying above) that doesn't require a code sample, here's a significantly stripped down version of what I'm currently working with:
def setup(self):
self.progress_thread = Thread(target=self.update_slider_thread, daemon=True)
self.progress_thread.start()
def update_slider_thread(self):
current_frame = self.progress_slider.value
is_playing = self.vlc.player.is_playing
update_progress = self.update_progress
while True:
start = time.time()
while not is_playing():
sleep(0.01)
start = time.time()
while is_playing() and not self.lock_progress_updates:
frame_multiplier = self.frame_rate
next_frame = current_frame() + 1
if next_frame <= self.frame_count: # don't update if we're at the end
update_progress(next_frame)
delay = self.delay # frame rate / total frames
try:
sleep(0.0001) # forces time.time() to update (otherwise we get the same "time" for several loops)
sleep(delay - (time.time() - start) - 0.00075) # 0.00075 to account for executing this line of code
except: pass
finally:
start = time.time()
continue
def update_progress(self, frame):
if self.get_player_state() == vlc.State.Ended:
if self.ready_to_restart and self.vlc.is_paused:
self.restart_video(frame=frame, pause=True)
self.current_time = round(self.duration * (frame / self.frame_count), 2)
h, m, s, ms = get_hms(self.current_time)
current_time_string = f'{m:02}:{s:02}.{ms:02}'
# these are just aliases for things like setValue() for each widget (for performance)
self.set_progress_slider(frame)
if not self.current_time_text_has_focus(): self.set_current_time_text(current_time_string)
self.set_hour_spin(h)
self.set_minute_spin(m)
self.set_second_spin(s)
self.set_frame_spin(frame)
Uhhhhhhhhhhhhhhhhh Ok so some basics. 1st of all, any interaction with widgets have to happen in main thread. So if you are changing slider value/etc. from daemon/worker thread. Then you are messing up Qt insides.
I would suggest that you use signals/slots. Here is a small example of it
class sliderUpdate(QObject):
handleUpdate = Signal(int)
def __init__(self):
print "Current thread (Should be main thread ) : " QThread.currentThread()
# Lets conect our signal to function/slot.
self.handleUpdate.connect(self.doUpdate,Qt.QueuedConnection) # We force it in to QueuedConnection so that Qt pass the data from worker thread to Main thread.
self.slider = QSlider()
def doUpdate(self,val):
print "Current thread (Should be main thread ) : " QThread.currentThread()
self.slider.setValue(val)
def processInThread(self):
print "Current thread (Should be worker thread ) : " QThread.currentThread()
self.handleUpdate.emit(10)
2nd. If VLC is sending updates A LOT per second. Then you may be bombarding Qt with refresh requests & lagging app. I would suggest implementing some sort of... delayed report... A sudo example :
timer = QElapsedTimer()
timer.start()
...
...
if timer.elapsed()>500:
emit new Value:
timer.restart()
else:
skip, emit on next call if timeout over 500 ms.

changing label color 2 times within a funtion in tkinter python

I have a funtion, that parses the log. I am updating the label color and text at different lines within the same funtion. I see that only the last changed color and text only reflecting in the UI. i do not see the intermediate color and text changes.
Below is my code:
def OnclickLogParser(self):
if LogFileName == '':
messagebox.showinfo("Error", "Please select a valid Log")
if LogPathName == '':
messagebox.showinfo("Error", "Please select a valid Log Path")
self.lb_log_status.configure(bg="#08DFE8", fg="#010101", text='Parsing inProgress...')
m_logParser = CAdpBrrLogParser()
m_logReader = CAdpBrrLogReader('mrr', m_logParser)
status = m_logReader.readFile(LogPathName)
if status == True:
self.lb_log_status.configure(bg="#F6F50B", fg="#010101", text='Log Ready')
self.btn_log_start["state"] = "normal"
global m_injector
m_injector = CAdpUdpDataInjector()
you can see that i am changing the color and text of lb_log_status at two different places. The time gap between those two lines would be around 3 -5 secs. But i can not witness the first color change. '
Tkinter must process events regularly to perform any UI operations. If you want to see the color changes here you have to give Tk a chance to process the configuration and paint events that get generated when you re-configure a widget. Your code just immediately starts processing the log and will not process any events until this function exits and you return to the Tk mainloop() which is the event processing method. You would see the same problem if you used a progress bar widget and tried to update the progress during processing.
One way to resolve this is to use the Tk after() method and schedule your processing in chunks that take a limited amount of time per chunk. After reading and processing a chunk call after() again to schedule the next chunk.
Another method is to put the processing on a worker thread and use event_generate() to post events back to the Tk thread to announce progress and completion.

How can I launch pyqt GUI multiple times consequtively in a process?

How can I architect code to run a pyqt GUI multiple times consecutively in a process?
(pyqtgraph specifically, if that is relevant)
The context
A python script that performs long running data capture on measurement equipment (a big for loop). During each capture iteration a new GUI appear and displays live data from the measurement equipment to the user, while the main capture code is running.
I'd like to do something like this:
for setting in settings:
measurement_equipment.start(setting)
gui = LiveDataStreamGUI(measurement_equipment)
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
The main issue
I'd like the data capture code to be the main thread. However pyqt doesn't seems to allow this architecture, as its app.exec_() is a blocking call, allowing a GUI to be created only once per process (e.g., in gui.display() above).
An application is an executable process that runs on one or more foreground threads each of which can also start background threads to perform parallel operations or operations without blocking the calling thread. An application will terminate after all foreground threads have ended, therefore, you need at least one foreground thread which in your case is created when you call the app.exec_() statement. In a GUI application, this is the UI thread where you should create and display the main window and any other UI widget. Qt will automatically terminate your application process when all widgets are closed.
IMHO, you should try to follow the normal flow described above as much as possible, the workflow could be as follows:
Start Application > Create main window > Start a background thread for each calculation > Send progress to UI thread > Show results in a window after each calculation is finished > Close all windows > End application
Also, you should use ThreadPool to make sure you don't run out of resources.
Here is a complete example:
import sys
import time
import PyQt5
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtCore import QRunnable, pyqtSignal, QObject
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QDialog
class CaptureDataTaskStatus(QObject):
progress = pyqtSignal(int, int) # This signal is used to report progress to the UI thread.
captureDataFinished = pyqtSignal(dict) # Assuming your result is a dict, this can be a class, a number, etc..
class CaptureDataTask(QRunnable):
def __init__(self, num_measurements):
super().__init__()
self.num_measurements = num_measurements
self.status = CaptureDataTaskStatus()
def run(self):
for i in range(0, self.num_measurements):
# Report progress
self.status.progress.emit(i + 1, self.num_measurements)
# Make your equipment measurement here
time.sleep(0.1) # Wait for some time to mimic a long action
# At the end you will have a result, for example
result = {'a': 1, 'b': 2, 'c': 3}
# Send it to the UI thread
self.status.captureDataFinished.emit(result)
class ResultWindow(QWidget):
def __init__(self, result):
super().__init__()
# Display your result using widgets...
self.result = result
# For this example I will just print the dict values to the console
print('a: {}'.format(result['a']))
print('b: {}'.format(result['b']))
print('c: {}'.format(result['c']))
class MainWindow(QMainWindow):
def __init__(self, *args, **kwargs):
super(MainWindow, self).__init__(*args, **kwargs)
self.result_windows = []
self.thread_pool = QtCore.QThreadPool().globalInstance()
# Change the following to suit your needs (I just put 1 here so you can see each task opening a window while the others are still running)
self.thread_pool.setMaxThreadCount(1)
# You could also start by clicking a button, menu, etc..
self.start_capturing_data()
def start_capturing_data(self):
# Here you start data capture tasks as needed (I just start 3 as an example)
for setting in range(0, 3):
capture_data_task = CaptureDataTask(300)
capture_data_task.status.progress.connect(self.capture_data_progress)
capture_data_task.status.captureDataFinished.connect(self.capture_data_finished)
self.thread_pool.globalInstance().start(capture_data_task)
def capture_data_progress(self, current, total):
# Update progress bar, label etc... for this example I will just print them to the console
print('Current: {}'.format(current))
print('Total: {}'.format(total))
def capture_data_finished(self, result):
result_window = ResultWindow(result)
self.result_windows.append(result_window)
result_window.show()
class App(QApplication):
"""Main application wrapper, loads and shows the main window"""
def __init__(self, sys_argv):
super().__init__(sys_argv)
self.main_window = MainWindow()
self.main_window.show()
if __name__ == '__main__':
app = App(sys.argv)
sys.exit(app.exec_())
If you want your GUI to keep updating in realtime and to not be freezed, you have two main ways to do it:
Refresh the GUI from time to time calling QApplication.processEvents() inside your time consuming function.
Create a separate thread (I mean, QThread) where you run your time consuming function
My personal preference is to go for the latter way. Here is a good tutorial for getting started on how to do multi-threading in Qt.
Having a look at your code:
...
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
...
it seems you are calling app.exec_ inside gui.display. Its very likely you will have to decouple both functions and call app.exec_ outside of gui.display and after calling capture_data. You will also have to connect the finished signal of the new thread to gui.close. It will be something like this:
...
gui.display() # dont call app.exec_ here
thread = QThread.create(measurement_equipment.capture_data, 300)
thread.finished.connect(gui.close)
app.exec_()
...
I hope this can help you and to not be late!!
You can have only One graphic GUI thread. This would imply to have some Threads to capture data and sync data with the graphic application when needed.
We need to know if the GUI data display is displaying realtime data or only oneshot.

Tkinter progress bar with Excel

We have a Pyxll app (Excel app written in python) that makes a bunch of requests to get data when the workbook is opened. We would like to display a loading bar to the user while the requests are being made and update the loading bar after each request returns.
I'm trying to use Tkinter to do this, but have run into issues. I can get a progress bar to pop up, but it blocks Excel from running until you close the window for the progress bar. I can't put it in a different thread because I want to be able to update the progress based on when the HTTP requests return. Is there an easy way to do this?
Here is my code so far. I have made a basic loading bar class:
import Tkinter as tk
import ttk
class OrderingProgressBar(tk.Tk):
def __init__(self, *args, **kwargs):
tk.Tk.__init__(self, *args, **kwargs)
# progress goes from 0-100 (for percentage)
self.progress = 0
self.max_progress = 100
self.progress_bar = ttk.Progressbar(self, orient="horizontal", length=200, mode="determinate", maximum=self.max_progress)
self.progress_bar.pack()
And then I have a macro that gets called to launch the app and start making requests.
def launch_ordering_terminal(ribbon):
"""Launch all of the apps required for the Ordering Terminal"""
ordering_progress_bar = OrderingProgressBar()
ordering_progress_bar.mainloop()
excel_utils.turn_off_excel_updates(xl_app())
launch_allocation_app(manager_id)
ordering_progress_bar.progress_bar.step(25)
launch_si_app(manager_id)
ordering_progress_bar.progress_bar.step(25)
launch_accounting_app(manager_id)
ordering_progress_bar.progress_bar.step(25)
launch_reports_builder(terminal_mode)
ordering_progress_bar.progress_bar.step(25)
excel_utils.turn_on_excel_updates(xl_app())
With this code, when I call the macro a loading bar pops up, but block Excel. If I close the window it continues. If I move the mainloop call to the end of launch_ordering_terminal, then the entire terminal loads, and then it pops up the loading bar. I can't get them to work at the same time.
Thanks in advance for the help!
Maintaining a GUI is a full time job that requires an endless loop, called the "mainloop". If you want to do something else in the meantime, you need to add it to the mainloop, or do it in a different thread. In your case I think a new thread would be easiest. Tkinter runs best in the main thread, so put your code in a child thread.
As a totally untested guess:
from threading import Thread
def do_stuff(ordering_progress_bar):
excel_utils.turn_off_excel_updates(xl_app())
launch_allocation_app(manager_id)
ordering_progress_bar.progress_bar.step(25)
launch_si_app(manager_id)
ordering_progress_bar.progress_bar.step(25)
launch_accounting_app(manager_id)
ordering_progress_bar.progress_bar.step(25)
launch_reports_builder(terminal_mode)
ordering_progress_bar.progress_bar.step(25)
excel_utils.turn_on_excel_updates(xl_app())
ordering_progress_bar.quit() # kill the GUI
def launch_ordering_terminal(ribbon):
"""Launch all of the apps required for the Ordering Terminal"""
ordering_progress_bar = OrderingProgressBar()
t = Thread(target=do_stuff, args=(ordering_progress_bar,))
t.start()
ordering_progress_bar.mainloop()

Memory leak when embedding and updating a matplotlib graph in a PyQt GUI

I am trying to embed a matplotlib graph that updates every second into a PyQt GUI main window.
In my program I call an update function every second using threading.Timer via the timer function shown below. I have a problem: my program grows bigger every second - at a rate of about 1k every 4 seconds. My initial thoughts are that the append function (that returns a new array in update_figure) does not delete the old array? Is it possible this is the cause of my problem?
def update_figure(self):
self.yAxis = np.append(self.yAxis, (getCO22()))
self.xAxis = np.append(self.xAxis, self.i)
# print(self.xAxis)
if len(self.yAxis) > 10:
self.yAxis = np.delete(self.yAxis, 0)
if len(self.xAxis) > 10:
self.xAxis = np.delete(self.xAxis, 0)
self.axes.plot(self.xAxis, self.yAxis, scaley=False)
self.axes.grid(True)
self.i = self.i + 1
self.draw()
This is my timer function - this is triggered by the click of a button in my PyQt GUI and then calls itself as you can see:
def timer(self):
getCH4()
getCO2()
getConnectedDevices()
self.dc.update_figure()
t = threading.Timer(1.0, self.timer)
t.start()
EDIT: I cant post my entire code because it requires a lot of .dll includes. So i'll try to explain what this program does.
In my GUI I want to show the my CO2 value over time. My get_co22 function just returns a float value and I'm 100% sure this works fine. With my timer, shown above, I want to keep append a value to a matplotlib graph - the Axes object is available to me as self.axes. I try to plot the last 10 values of the data.
EDIT 2: After some discussion in chat, I tried putting the call to update_figure() in a while loop and using just one thread to call it and was able to make this minimal example http://pastebin.com/RXya6Zah. This changed the structure of the code to call update_figure() to the following:
def task(self):
while True:
ui.dc.update_figure()
time.sleep(1.0)
def timer(self):
t = Timer(1.0, self.task())
t.start()
but now the program crashes after 5 iterations or so.
The problem is definitely not with how you are appending to your numpy array, or truncating it.
The problem here is with your threading model. Integrating calculation loops with a GUI control loop is difficult.
Fundamentally, you need your GUI threading to have control of when your update code is called (spawning a new thread to handle it if necessary) - so that
your code does not block the GUI updating,
the GUI updating does not block your code executing and
you don't spawn loads of threads holding multiple copies of objects (which might be where your memory leak comes from).
In this case, as your main window is controlled by PyQt4, you want to use a QTimer (see a simple example here)
So - alter your timer code to
def task(self):
getCH4()
getCO2()
getConnectedDevices()
self.dc.update_figure()
def timer(self):
self.t = QtCore.QTimer()
self.t.timeout.connect(self.task)
self.t.start(1000)
and this should work. Keeping the reference to the QTimer is essential - hence self.t = QtCore.QTimer() rather than t = QtCore.QTimer(), otherwise the QTimer object will be garbage collected.
Note:
This is a summary of a long thread in chat clarifying the issue and working through several possible solutions. In particular - the OP managed to mock up a simpler runnable example here: http://pastebin.com/RXya6Zah
and the fixed version of the full runnable example is here: http://pastebin.com/gv7Cmapr
The relevant code and explanation is above, but the links might help anyone who wants to replicate / solve the issue. Note that they require PyQt4 to be installed
if you are creating a new figure for every time this is quite common.
matplotlib do not free the figures you create, unless you ask it, somethink like:
pylab.close()
see How can I release memory after creating matplotlib figures

Categories