We Keep Coding

GUI freezing in python PySide6

I am developing a software as part of my work, as soon as I make a call to the API to fetch data the GUI freezes, at first I understood the problem and transferred the functions to threads, the problem is that once I used the join() function the app froze again.
What I would like is to wait at the same point in the function until the threads end and continue from the same point in the function, is there any way to do this in Python?
threads = []
def call_api(self, query, index, return_dict):
thread = threading.Thread( target=self.get_data, args=(query, index, return_dict))
self.threads.append(thread)
thread.start()
def get_all_tickets(self, platform):
if platform == 'All':
self.call_api(query1, 0, return_dict)
self.call_api(query2, 1, return_dict)
for thread in self.threads:
thread.join()
# The app freezes here
# Is there a way to wait at this point asynchronously until the processes are complete
and continue from that point without the GUI freezing?

One possible option would be to use QThreads finished signal it emits that you can connect to with a slot that contains the remaining logic from your get_all_tickets method.
threads = []
def call_api(self, query, index, return_dict):
thread = QThread()
worker = Worker(query, index, return_dict)
worker.moveToThread(thread)
thread.started.connect(worker.run)
worker.finished.connect(thread.terminate)
thread.finished.connect(self.continue_getting_all_tickets)
self.threads.append(thread)
thread.start()
def get_all_tickets(self, platform):
if platform == 'All':
self.call_api(query1, 0, return_dict)
self.call_api(query2, 1, return_dict)
def continue_getting_all_tickets(self):
# this will be called once for each and every thread created
# if you only want it to run once all the threads have completed
# you could do something like this:
if all([thread.isFinished() for thread in self.threads]):
# ... do something
The worker class could look something like this.
class Worker(QObject):
finished = Signal()
def __init__(self, query, index, return_dict):
super().__init__()
self.query = query
self.index = index
self.return_dict = return_dict
def run(self):
# this is where you would put `get_data` code
# once it is finished it should emit the finished signal
self.finished.emit()
Hopefully this will help you find the right direction.

Stopping eval code dinamically on event fired [duplicate]

What's the proper way to tell a looping thread to stop looping?
I have a fairly simple program that pings a specified host in a separate threading.Thread class. In this class it sleeps 60 seconds, the runs again until the application quits.
I'd like to implement a 'Stop' button in my wx.Frame to ask the looping thread to stop. It doesn't need to end the thread right away, it can just stop looping once it wakes up.
Here is my threading class (note: I haven't implemented looping yet, but it would likely fall under the run method in PingAssets)
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
def run(self):
config = controller.getConfig()
fmt = config['timefmt']
start_time = datetime.now().strftime(fmt)
try:
if onlinecheck.check_status(self.asset):
status = "online"
else:
status = "offline"
except socket.gaierror:
status = "an invalid asset tag."
msg =("{}: {} is {}. \n".format(start_time, self.asset, status))
wx.CallAfter(self.window.Logger, msg)
And in my wxPyhton Frame I have this function called from a Start button:
def CheckAsset(self, asset):
self.count += 1
thread = PingAssets(self.count, asset, self)
self.threads.append(thread)
thread.start()

Threaded stoppable function
Instead of subclassing threading.Thread, one can modify the function to allow
stopping by a flag.
We need an object, accessible to running function, to which we set the flag to stop running.
We can use threading.currentThread() object.
import threading
import time
def doit(arg):
t = threading.currentThread()
while getattr(t, "do_run", True):
print ("working on %s" % arg)
time.sleep(1)
print("Stopping as you wish.")
def main():
t = threading.Thread(target=doit, args=("task",))
t.start()
time.sleep(5)
t.do_run = False
if __name__ == "__main__":
main()
The trick is, that the running thread can have attached additional properties. The solution builds
on assumptions:
the thread has a property "do_run" with default value True
driving parent process can assign to started thread the property "do_run" to False.
Running the code, we get following output:
$ python stopthread.py
working on task
working on task
working on task
working on task
working on task
Stopping as you wish.
Pill to kill - using Event
Other alternative is to use threading.Event as function argument. It is by
default False, but external process can "set it" (to True) and function can
learn about it using wait(timeout) function.
We can wait with zero timeout, but we can also use it as the sleeping timer (used below).
def doit(stop_event, arg):
while not stop_event.wait(1):
print ("working on %s" % arg)
print("Stopping as you wish.")
def main():
pill2kill = threading.Event()
t = threading.Thread(target=doit, args=(pill2kill, "task"))
t.start()
time.sleep(5)
pill2kill.set()
t.join()
Edit: I tried this in Python 3.6. stop_event.wait() blocks the event (and so the while loop) until release. It does not return a boolean value. Using stop_event.is_set() works instead.
Stopping multiple threads with one pill
Advantage of pill to kill is better seen, if we have to stop multiple threads
at once, as one pill will work for all.
The doit will not change at all, only the main handles the threads a bit differently.
def main():
pill2kill = threading.Event()
tasks = ["task ONE", "task TWO", "task THREE"]
def thread_gen(pill2kill, tasks):
for task in tasks:
t = threading.Thread(target=doit, args=(pill2kill, task))
yield t
threads = list(thread_gen(pill2kill, tasks))
for thread in threads:
thread.start()
time.sleep(5)
pill2kill.set()
for thread in threads:
thread.join()

This has been asked before on Stack. See the following links:
Is there any way to kill a Thread in Python?
Stopping a thread after a certain amount of time
Basically you just need to set up the thread with a stop function that sets a sentinel value that the thread will check. In your case, you'll have the something in your loop check the sentinel value to see if it's changed and if it has, the loop can break and the thread can die.

I read the other questions on Stack but I was still a little confused on communicating across classes. Here is how I approached it:
I use a list to hold all my threads in the __init__ method of my wxFrame class: self.threads = []
As recommended in How to stop a looping thread in Python? I use a signal in my thread class which is set to True when initializing the threading class.
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
self.signal = True
def run(self):
while self.signal:
do_stuff()
sleep()
and I can stop these threads by iterating over my threads:
def OnStop(self, e):
for t in self.threads:
t.signal = False

I had a different approach. I've sub-classed a Thread class and in the constructor I've created an Event object. Then I've written custom join() method, which first sets this event and then calls a parent's version of itself.
Here is my class, I'm using for serial port communication in wxPython app:
import wx, threading, serial, Events, Queue
class PumpThread(threading.Thread):
def __init__ (self, port, queue, parent):
super(PumpThread, self).__init__()
self.port = port
self.queue = queue
self.parent = parent
self.serial = serial.Serial()
self.serial.port = self.port
self.serial.timeout = 0.5
self.serial.baudrate = 9600
self.serial.parity = 'N'
self.stopRequest = threading.Event()
def run (self):
try:
self.serial.open()
except Exception, ex:
print ("[ERROR]\tUnable to open port {}".format(self.port))
print ("[ERROR]\t{}\n\n{}".format(ex.message, ex.traceback))
self.stopRequest.set()
else:
print ("[INFO]\tListening port {}".format(self.port))
self.serial.write("FLOW?\r")
while not self.stopRequest.isSet():
msg = ''
if not self.queue.empty():
try:
command = self.queue.get()
self.serial.write(command)
except Queue.Empty:
continue
while self.serial.inWaiting():
char = self.serial.read(1)
if '\r' in char and len(msg) > 1:
char = ''
#~ print('[DATA]\t{}'.format(msg))
event = Events.PumpDataEvent(Events.SERIALRX, wx.ID_ANY, msg)
wx.PostEvent(self.parent, event)
msg = ''
break
msg += char
self.serial.close()
def join (self, timeout=None):
self.stopRequest.set()
super(PumpThread, self).join(timeout)
def SetPort (self, serial):
self.serial = serial
def Write (self, msg):
if self.serial.is_open:
self.queue.put(msg)
else:
print("[ERROR]\tPort {} is not open!".format(self.port))
def Stop(self):
if self.isAlive():
self.join()
The Queue is used for sending messages to the port and main loop takes responses back. I've used no serial.readline() method, because of different end-line char, and I have found the usage of io classes to be too much fuss.

Depends on what you run in that thread.
If that's your code, then you can implement a stop condition (see other answers).
However, if what you want is to run someone else's code, then you should fork and start a process. Like this:
import multiprocessing
proc = multiprocessing.Process(target=your_proc_function, args=())
proc.start()
now, whenever you want to stop that process, send it a SIGTERM like this:
proc.terminate()
proc.join()
And it's not slow: fractions of a second.
Enjoy :)

My solution is:
import threading, time
def a():
t = threading.currentThread()
while getattr(t, "do_run", True):
print('Do something')
time.sleep(1)
def getThreadByName(name):
threads = threading.enumerate() #Threads list
for thread in threads:
if thread.name == name:
return thread
threading.Thread(target=a, name='228').start() #Init thread
t = getThreadByName('228') #Get thread by name
time.sleep(5)
t.do_run = False #Signal to stop thread
t.join()

I find it useful to have a class, derived from threading.Thread, to encapsulate my thread functionality. You simply provide your own main loop in an overridden version of run() in this class. Calling start() arranges for the object’s run() method to be invoked in a separate thread.
Inside the main loop, periodically check whether a threading.Event has been set. Such an event is thread-safe.
Inside this class, you have your own join() method that sets the stop event object before calling the join() method of the base class. It can optionally take a time value to pass to the base class's join() method to ensure your thread is terminated in a short amount of time.
import threading
import time
class MyThread(threading.Thread):
def __init__(self, sleep_time=0.1):
self._stop_event = threading.Event()
self._sleep_time = sleep_time
"""call base class constructor"""
super().__init__()
def run(self):
"""main control loop"""
while not self._stop_event.isSet():
#do work
print("hi")
self._stop_event.wait(self._sleep_time)
def join(self, timeout=None):
"""set stop event and join within a given time period"""
self._stop_event.set()
super().join(timeout)
if __name__ == "__main__":
t = MyThread()
t.start()
time.sleep(5)
t.join(1) #wait 1s max
Having a small sleep inside the main loop before checking the threading.Event is less CPU intensive than looping continuously. You can have a default sleep time (e.g. 0.1s), but you can also pass the value in the constructor.

Sometimes you don't have control over the running target. In those cases you can use signal.pthread_kill to send a stop signal.
from signal import pthread_kill, SIGTSTP
from threading import Thread
from itertools import count
from time import sleep
def target():
for num in count():
print(num)
sleep(1)
thread = Thread(target=target)
thread.start()
sleep(5)
pthread_kill(thread.ident, SIGTSTP)
result
0
1
2
3
4
[14]+ Stopped

Pause worker thread and wait for event from main thread

We have an application that executes different queries. It starts up to four threads, and runs the extractions on them.
That part looks like this:
if len(self.threads) == 4:
self.__maxThreadsMsg(base)
return False
else:
self.threads.append(Extractor(self.ui, base))
self.threads[-1].start()
self.__extractionMsg(base)
return True
Our Extractor class inherits QThread:
class Extractor(QThread):
def init(self, ui, base):
QThread.__init__(self)
self.ui = ui
self.base = base
def run(self):
self.run_base(base)
and self.ui is set to Ui_MainWindow():
class Cont(QMainWindow):
def __init__(self, parent=None):
QWidget.__init__(self,parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)
There is a specific base that sends data to the user (back to the main window) before proceeding (in this case, a pop-up with two buttons):
#This code is in the main file inside a method, not in the Extractor class
msg_box = QMessagebox()
msg_box.setText('Quantity in base: '.format(n))
msg_box.setInformativeText('Would you like to continue?')
msg_box.setStandardButtons(QMessageBox.Ok | QMessageBox.Cancel)
signal = msg_box.exec_()
How can I pause the thread at a certain point, display the window (which I believe would be returning to the main thread) and return to the worker thread, passing the button clicked event?
I read a bit about signals but it seems confusing as it is my first time dealing with threads.
Edit: After reading this question: Similar question, I altered the code to this:
On a method inside of the Cont class
thread = QThread(self)
worker = Worker()
worker.moveToThread(thread)
worker.bv.connect(self.bv_test)
thread.started.connect(worker.process()) # This, unlike in the linked question..
#doesn't work if I remove the parentheses of the process function.
#If I remove it, nothing happens and I get QThread: "Destroyed while thread is still running"
thread.start()
#pyqtSlot(int)
def bv_test(self, n):
k = QMessageBox()
k.setText('Quantity: {}'.format(n))
k.setStandardButtons(QMessageBox.Yes | QMessageBox.No)
ret = k.exec_()
return ret
and this is the Worker class:
class Worker(QObject):
#Signals
bv = pyqtSignal(int)
def process(self):
self.bv.emit(99)
Now I just need to figure out how to send the ret value back to the worker thread so it starts the second process. I also keep getting this error:
TypeError: connect() slot argument should be a callable or a signal, not 'NoneType'

Below is a simple demo based on the code in your question which does what you want. There is not much to say about it, really, other than that you need to communicate between the worker and the main thread via signals (in both directions). The finished signal is used to quit the thread, which will stop the warning message QThread: "Destroyed while thread is still running" being shown.
The reason why you are seeing the error:
TypeError: connect() slot argument should be a callable or a signal, not `NoneType'
is because you are trying to connect a signal with the return value of a function (which is None), rather than the function object itself. You must always pass a python callable object to the connect method - anything else will raise a TypeError.
Please run the script below and confirm that it works as expected. Hopefully it should be easy to see how to adapt it to work with your real code.
from PyQt4.QtCore import *
from PyQt4.QtGui import *
class Cont(QWidget):
confirmed = pyqtSignal()
def __init__(self):
super(Cont, self).__init__()
self.thread = QThread()
self.worker = Worker()
self.worker.moveToThread(self.thread)
self.worker.bv.connect(self.bv_test)
self.worker.finished.connect(self.thread.quit)
self.confirmed.connect(self.worker.process_two)
self.thread.started.connect(self.worker.process_one)
self.thread.start()
def bv_test(self, n):
k = QMessageBox(self)
k.setAttribute(Qt.WA_DeleteOnClose)
k.setText('Quantity: {}'.format(n))
k.setStandardButtons(QMessageBox.Yes | QMessageBox.No)
if k.exec_() == QMessageBox.Yes:
self.confirmed.emit()
else:
self.thread.quit()
class Worker(QObject):
bv = pyqtSignal(int)
finished = pyqtSignal()
def process_two(self):
print('process: two: started')
QThread.sleep(1)
print('process: two: finished')
self.finished.emit()
def process_one(self):
print('process: one: started')
QThread.sleep(1)
self.bv.emit(99)
print('process: one: finished')
app = QApplication([''])
win = Cont()
win.setGeometry(100, 100, 100, 100)
win.show()
app.exec_()

If you want the thread to wait for the action, connect to a signal from the thread using
PyQt4.QtCore.Qt.BlockingQueuedConnection
as flag.
Now I do not understand why you need threading if you let them wait, which brings in a lot of complexity. For me the better solution would be to cut the task you want to perform in the threads in smaller pieces. Each time a piece is ready, you can ask if the user wants the next too.

GUI not responding after child processes finishes due to redirection of Stdout

I've spent the past few days reading various threads about making tkinter thread-safe and running children without blocking the main thread. I thought I had arrived at a solution that allowed my code to run as I wanted it to, but now my main thread becomes non-responsive when my child process finishes. I can move the window around but the GUI part shows a loading cursor, whites out, and says "Not Responding" in the title of the window. I can let it sit like that forever and nothing will happen. I know what part of the code is causing the problem but I am not sure why it's causing the GUI to freeze. I'm using Windows.
I want my GUI to run another process using multiprocess. I have sys.stdout and sys.stderr routed to a queue and I use threading to create a thread that holds an automatic queue checker that updates the GUI every 100 ms so my GUI updates in "real time". I have tried every way of sending the child's stdout/stderr to the GUI and this is the only way that works the way I want it to (except for the freezing bit), so I would like to find out why it's freezing. Or I would like help setting up a proper way of sending the child's output to the GUI. I have tried every method I could find and I could not get them to work.
My main thread:
#### _______________IMPORT MODULES_________________###
import Tkinter
import multiprocessing
import sys
from threading import Thread
import qBMPchugger
###____________Widgets__________________###
class InputBox(Tkinter.Tk):
def __init__(self,parent):
Tkinter.Tk.__init__(self, parent)
self.parent = parent
self.initialize()
def initialize(self):
# Styles
self.grid()
# Approval
self.OKbutton = Tkinter.Button(self, text=u"OK", command=self.OKgo, anchor="e")
self.OKbutton.pack(side="right")
self.view = Tkinter.Text(self)
self.view.pack(side="left")
self.scroll = Tkinter.Scrollbar(self, orient=Tkinter.VERTICAL)
self.scroll.config(command=self.view.yview)
self.view.config(yscrollcommand=self.scroll.set)
self.scroll.pack(side="left")
def write(self, text):
self.view.insert("end", text)
def OKgo(self):
sys.stdout = self
sys.stderr = self
checker = Thread(target=self._update)
checker.daemon = True
checker.start()
self.view.delete(1.0, "end")
self.update_idletasks()
print("Loading user-specified inputs...")
path = "C:/"
inarg = (q, path)
print("Creating the program environment and importing modules...")
# Starts the text monitor to read output from the child process, BMPchugger
p = multiprocessing.Process(target=qBMPchugger.BMPcode, args=inarg)
p.daemon = 1
p.start()
def _update(self):
msg = q.get()
self.write(msg)
self.update_idletasks()
self.after(100, self._update)
if __name__ == "__main__":
app = InputBox(None)
app.title("File Inputs and Program Settings")
q = multiprocessing.Queue()
app.mainloop()
My child process (qBMPchugger):
#### _______________INITIALIZE_________________###
import os
import sys
import tkMessageBox
import Tkinter
class BadInput(Exception):
pass
def BMPcode(q, path):
# Create root for message boxes
boxRoot = Tkinter.Tk()
boxRoot.withdraw()
# Send outputs to the queue
class output:
def __init__(self, name, queue):
self.name = name
self.queue = queue
def write(self, msg):
self.queue.put(msg)
def flush(self):
sys.__stdout__.flush()
class error:
def __init__(self, name, queue):
self.name = name
self.queue = queue
def write(self, msg):
self.queue.put(msg)
def flush(self):
sys.__stderr__.flush()
sys.stdout = output(sys.stdout, q)
sys.stderr = error(sys.stderr, q)
print("Checking out the Spatial Analyst extension from GIS...")
# Check out extension and overwrite outputs
### _________________VERIFY INPUTS________________###
print("Checking validity of specified inputs...")
# Check that the provided file paths are valid
inputs = path
for i in inputs:
if os.path.exists(i):
pass
else:
message = "\nInvalid file path: {}\nCorrect the path name and try again.\n"
tkMessageBox.showerror("Invalid Path", message.format(i))
print message.format(i)
raise BadInput
print("Success!")
It's the part under # Send outputs to the queue (starting with the output class and ending with sys.stderr = error(sys.stderr, q)) that is causing my program to freeze. Why is that holding up my main thread when the child process finishes executing? EDIT: I think the freezing is being caused by the queue remaining open when the child process closes... or something. It's not the particular snippet of code like I thought it was. It happens even when I change the print statements to q.put("text") in either the parent or the child.
What is a better way to send the output to the queue? If you link me to a topic that answers my question, PLEASE show me how to implement it within my code. I have not been successful with anything I've found so far and chances are that I've already tried that particular solution and failed.

Use a manager list or dictionary to communicate between processes https://docs.python.org/2/library/multiprocessing.html#sharing-state-between-processes . You can have a process update the dictionary and send it to the GUI/some code outside the processes, and vice versa. The following is a simple, and a little sloppy, example of doing it both ways.
import time
from multiprocessing import Process, Manager
def test_f(test_d):
""" frist process to run
exit this process when dictionary's 'QUIT' == True
"""
test_d['2'] = 2 ## add as a test
while not test_d["QUIT"]:
print "P1 test_f", test_d["QUIT"]
test_d["ctr"] += 1
time.sleep(1.0)
def test_f2(test_d):
""" second process to run. Runs until the for loop exits
"""
for j in range(0, 10):
## print to show that changes made anywhere
## to the dictionary are seen by this process
print " P2", j, test_d
time.sleep(0.5)
print "second process finished"
if __name__ == '__main__':
##--- create a dictionary via Manager
manager = Manager()
test_d = manager.dict()
test_d["ctr"] = 0
test_d["QUIT"] = False
##--- start first process and send dictionary
p = Process(target=test_f, args=(test_d,))
p.start()
##--- start second process
p2 = Process(target=test_f2, args=(test_d,))
p2.start()
##--- sleep 2 seconds and then change dictionary
## to exit first process
time.sleep(2.0)
print "\nterminate first process"
test_d["QUIT"] = True
print "test_d changed"
print "dictionary updated by processes", test_d
##--- may not be necessary, but I always terminate to be sure
time.sleep(5.0)
p.terminate()
p2.terminate()

For my particular problem, the main thread was trying to read from the queue when the queue was empty and not having anything else put into it. I don't know the exact details as to why the main loop got hung up on that thread (self._update in my code) but changing _update to the following stopped making the GUI non-responsive when the child finished:
def _update(self):
if q.empty():
pass
else:
msg = q.get()
self.write(msg)
self.update_idletasks()

Python: Non-modal window with subprocesses

I need to implement the following UI:
- There is a window with a label "running experiment 1/X" and a button
- When the window is loaded, some experiments are started. The experiments are run by os.system of subprocess.Popen, they are just pre-compiled C++ programs
- The experiments should run strictly one after another and not simultaneously (hence I can't use subprocess.Popen)
- The window should be active while the experiments are running and the user can press the button
- When the button is pressed, the experiments stop (we can just wait until the current experiment ends and stop) and the window closes
- When all experiments are over, the window should close itself
First I tried running experiments in threading.Thread, but it still blocked the window. So I switched to multiprocessing.Process:
class StoppableProcess(Process):
def __init__(self, name, alg, proj, parent):
Process.__init__(self)
self.stop = False
self.name = name
self.alg = alg
self.proj = proj
self.parent = parent
def stop(self):
self.stop = True
def stopped(self):
return self.stop
def run(self):
count = len([k for k in self.proj.values()])
i = 1
for p in self.proj.values():
self.parent.label.setText("Running experiment " + str(i) + " / " + str(count))
os.system("some command here")
i += 1
if self.stopped():
break
self.parent.hide()
class Runner(QDialog):
def __init__(self, parent):
QDialog.__init__(self, parent)
self.layout = QVBoxLayout()
self.label = QLabel("Running experiment 0 / 0")
self.setWindowTitle("Running experiments")
button = QPushButton("Break experiments")
self.layout.addWidget(self.label)
self.layout.addWidget(button)
self.setLayout(self.layout)
QObject.connect(button, SIGNAL("clicked()"), self.Break)
def Run(self, name, alg, proj):
self.thread = StoppableProcess(name, alg, proj, self)
self.thread.start()
self.show()
self.thread.join()
def Break(self):
self.thread.stop()
self.hide()
However, this doesn't work at all, apparently because the Runner object should be pickled to be passed to a subprocess, but pickling fails. I was thinking about avoiding passing the parent argument and using Qt signals instead, but maybe there's a better solution?

First of all, you can indeed use subprocess.Popen to start background processes and wait for their completion. See the documentation, specifically, the poll() method. Run the UI event loop until the process has exited.
Second, it is usually a good idea to avoid threads in Python. The multiprocessing module is mostly useful when you want to parallelize tasks written in Python. IMO, I think it is easier to use the subprocess module if you are just launching external child processes.
The following pseudocode illustrates the idea:
experiments = [...]
process = None
def start_next_experiment():
if not experiments:
print "Done!"
else:
experiment = experiments.pop()
process = subprocess.Popen(experiment)
def on_start_clicked():
start_next_experiment()
def on_stop_clicked():
# Clear the queue
experiments = []
# optional: Kill the process
if process:
process.terminate()
def on_idle():
if process:
# use e.g. a PyQT timer to run this method periodically
process.poll()
if process.returncode is not None:
process = None
start_next_experiment()