I'm writing a simple time tracking application in Python3 and PyQt5. Time is tracked in separate thread. Function that this thread is running doesn't access GUI code. On Windows10 application freezes after trying to close it. It's caused by calling thread.join(). I need to end the process in task manager to close it. On Linux Mint it works fine. I'm using threads from threading library. It doesn't work also with QThread's. If I comment out the thread.join() line it closes without a problem, but the code that's running by this thread doesn't finish.
Thread is initialized in __init__() function of Window class.
self.trackingThread = Thread(target = self.track)
Function that is responsible for tracking time:
def track(self):
startTime = time()
lastWindowChangeTime = startTime
while self.running:
# check if active window has changed
if self.active_window_name != get_active_window_name():
if self.active_window_name in self.applications_time:
self.applications_time[self.active_window_name] += int(time() - lastWindowChangeTime) // 60 # time in minutes)
else:
self.applications_time[self.active_window_name] = int(time() - lastWindowChangeTime) // 60 # time in minutes
lastWindowChangeTime = time()
self.active_window_name = get_active_window_name()
totalTime = int(time() - startTime) // 60 # time in minutes
if date.today() in self.daily_time:
self.daily_time[date.today()] += totalTime
else:
self.daily_time[date.today()] = totalTime
Joining the thread:
def saveAndQuit(self):
self.running = False
self.trackingThread.join() # the line that's causing application freeze
self.save()
QApplication.instance().quit()
EDIT:
Example:
https://pastebin.com/vt3BfKJL
relevant code:
def get_active_window_name():
active_window_name = ''
if system() == 'Linux':
active_window_name = check_output(['xdotool', 'getactivewindow', 'getwindowname']).decode('utf-8')
elif system() == 'Windows':
window = GetForegroundWindow()
active_window_name = GetWindowText(window)
return active_window_name
EDIT2:
After removing those 2 lines app closes without any problem. Is there any other way of getting active window name on Windows except win32gui?:
window = GetForegroundWindow()
active_window_name = GetWindowText(window)
The issue occurs because GetWindowText() is blocking, and so your thread can never join. To understand why, we have to delve into the win32 documentation
If the target window is owned by the current process, GetWindowText causes a WM_GETTEXT message to be sent to the specified window or control. If the target window is owned by another process and has a caption, GetWindowText retrieves the window caption text. If the window does not have a caption, the return value is a null string. This behavior is by design. It allows applications to call GetWindowText without becoming unresponsive if the process that owns the target window is not responding. However, if the target window is not responding and it belongs to the calling application, GetWindowText will cause the calling application to become unresponsive.
You are attempting to join the thread from within a function (saveAndQuit) that has been called by the Qt event loop. As such, until this function returns, the Qt event loop will not process any messages. This means the call to GetWindowText in the other thread has sent a message to the Qt event loop which won't be processed until saveAndQuit finishes. However, saveAndQuit is waiting for the thread to finish, and so you have a deadlock!
There are several ways to solve the deadlock, probably the easiest to implement is to recursively call join, with a timeout, from the Qt event loop. It's somewhat "hacky", but other alternatives mean things like changing the way your thread behaves or using QThreads.
As such, I would modify your saveAndQuit as follows:
def saveAndQuit(self):
self.running = False
self.trackingThread.join(timeout=0.05)
# if thread is still alive, return control to the Qt event loop
# and rerun this function in 50 milliseconds
if self.trackingThread.is_alive():
QTimer.singleShot(50, self.saveAndQuit)
return
# if the thread has ended, then save and quit!
else:
self.save()
QApplication.instance().quit()
I had a similar problem and someone here on SO advised me to use something like this:
class MyThread(QThread):
def __init__(self):
super().__init__()
# initialize your thread, use arguments in the constructor if needed
def __del__(self):
self.wait()
def run(self):
pass # Do whatever you need here
def run_qt_app():
my_thread = MyThread()
my_thread.start()
qt_app = QApplication(sys.argv)
qt_app.aboutToQuit.connect(my_thread.terminate)
# Setup your window here
return qt_app.exec_()
Works fine for me, my_thread runs as long as qt_app is up, and finishes it's work on quit.
edit: typos
Related
I'm using threading to run a long task, but I ran into an issue. The thread just hung while trying to set an IntVar after I clicked the close button. It doesn't even error. I don't want to use a daemon thread because the function is a critical part of the program, which might have consequences if it stops midway through (it deals with a bunch of files).
Here's an oversimplified version of my program, meant to demonstrate my issue.
import tkinter as tk
import tkinter.ttk as ttk
import threading
class Window(tk.Tk):
def __init__(this, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
this.threads = []
this.var = tk.IntVar(value=0)
this.label = ttk.Label(textvariable=this.var)
this.button = ttk.Button(text='Start counter', command=this.startCounter)
this.label.pack()
this.button.pack()
this.stop = False
this.protocol("WM_DELETE_WINDOW", this.close)
def startCounter(this):
thread = threading.Thread(target=this.counter)
this.threads.append(thread)
thread.start()
def counter(this):
while True:
if this.stop:
print(f'self.stop = ')
break
this.var.set(this.var.get() + 1)
def close(this):
print('Stopping threads')
this.stop = True
this.waitThreads()
print('Stopped threads')
this.destroy()
def waitThreads(this):
for thread in this.threads:
thread.join()
Window().mainloop()
My program is using an InVar for a progress bar, not a counter, this was just the best way I could demonstrate the issue.
I tried a bunch of different methods to stop all threads, but none of them worked (that was before I knew what the issue was). For some reason in my actual program, if I log the var and the value of the var before the stop check, it's able to stop. I could not reproduce that with my test script.
I'm expecting the set var line to move on, or error instead of just hang.
Why is it hanging, and how can I fix this? I want to be able to safely stop the thread(s), and I don't want to use a daemon.
you have a race condition, a deadlock, and an undefined behavior in your application ... that's how simple it is to mess up a small code snippet when multithreading.
the tk interpreter isn't threadsafe, and shouldn't be called from different threads, notably the event_generate function is threadsafe and should be used for instructing GUI changes, incrementing the variable from another thread is likely going to crash the interpreter, it's also a race condition, and the results will be wrong, increments should only happen in the main thread, by generating an event from the other thread.
lastly you need to make your threads drop the GIL momentarily, this can be done by a small sleep time.sleep(0.0000001).
import tkinter as tk
import tkinter.ttk as ttk
import threading
import time
class Window(tk.Tk):
def __init__(this, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
this.threads = []
this.var = tk.IntVar(value=0)
this.label = ttk.Label(textvariable=this.var)
this.button = ttk.Button(text='Start counter', command=this.startCounter)
this.bind("<<increment_counter>>",this.increment_var)
this.label.pack()
this.button.pack()
this.stop = False
this.protocol("WM_DELETE_WINDOW", this.close)
def startCounter(this):
thread = threading.Thread(target=this.counter)
this.threads.append(thread)
thread.start()
def increment_var(this, event):
this.var.set(this.var.get() + 1)
def counter(this):
while True:
time.sleep(0.0000001) # drop the GIL momentarily
if this.stop:
print(f'self.stop = ')
break
this.event_generate("<<increment_counter>>") # all increments happen in main thread
def close(this):
print('Stopping threads')
this.stop = True
this.waitThreads()
print('Stopped threads')
this.destroy()
def waitThreads(this):
for thread in this.threads:
thread.join()
Window().mainloop()
note that the first argument of a method is by convention called self in python, calling it this will confuse a lot of linters, parsers, other coders, autocorrect IDEs and documentation generators. please don't do that to everyone and use self instead of this.
I have a few suggestions to make your code work safer.
Use a costume defined event. This will place an event in the event queue of tkinters event-loop.
Have a thread limit, too many threads might mess things up an become unreachable.
Use a threading primitive to signal the termination of your threads, like threading.Event()
Instead of joining the threads within the event-loop of tkinter, make sure they are done after your app has been terminated. This will lead to a better user-experience.
In addition I would propose:
Using self instead of this because of convention
Using a set() instead of a list()
import tkinter as tk
import tkinter.ttk as ttk
import threading
import time
class Window(tk.Tk):
def __init__(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
self.threads = set()
self.var = tk.IntVar(value=0)
self.label = ttk.Label(self, textvariable=self.var)
self.button = ttk.Button(
self, text='Start counter', command=self.startCounter)
self.label.pack()
self.button.pack()
self.stop = threading.Event()
self.event_generate('<<UpdateLabel>>')
self.bind('<<UpdateLabel>>', self.update_counter)
self.protocol("WM_DELETE_WINDOW", self.close)
def update_counter(self, event):
self.var.set(self.var.get() + 1)
def startCounter(self):
if threading.active_count()+1 <= THREAD_LIMIT:
t = threading.Thread(target=self.counter)
self.threads.add(t)
t.start()
else:
print('too many threads might messing things up')
def counter(self):
current = threading.current_thread()
while not self.stop.is_set():
time.sleep(1) #simulate task
if not self.stop.is_set():
#if application still exists
self.event_generate('<<UpdateLabel>>')
else:
self.threads.discard(current)
break
self.threads.discard(current)
def close(self):
print('Stopping threads')
self.stop.set()
print('Stop event is set')
self.destroy()
if __name__ == '__main__':
COUNT = threading.active_count()
THREAD_LIMIT = COUNT + 7
window = Window()
window.mainloop()
print('mainloop ended')
while threading.active_count() > COUNT: #simulate join
time.sleep(0.1)
print(threading.active_count())
print('all threads ended, mainthread ends now')
I am pretty new to Python and have a question about threading.
I have one function that is called pretty often. This function starts another function in a new Thread.
def calledOften(id):
t = threading.Thread(target=doit, args=(id))
t.start()
def doit(arg):
while true:
#Long running function that is using arg
When calledOften is called everytime a new Thread is created. My goal is to always terminate the last running thread --> At all times there should be only one running doit() Function.
What I tried:
How to stop a looping thread in Python?
def calledOften(id):
t = threading.Thread(target=doit, args=(id,))
t.start()
time.sleep(5)
t.do_run = False
This code (with a modified doit Function) worked for me to stop the thread after 5 seconds.
but i can not call t.do_run = False before I start the new thread... Thats pretty obvious because it is not defined...
Does somebody know how to stop the last running thread and start a new one?
Thank you ;)
I think you can decide when to terminate the execution of a thread from inside the thread by yourself. That should not be creating any problems for you. You can think of a Threading manager approach - something like below
import threading
class DoIt(threading.Thread):
def __init__(self, id, stop_flag):
super().__init__()
self.id = id
self.stop_flag = stop_flag
def run(self):
while not self.stop_flag():
pass # do something
class CalledOftenManager:
__stop_run = False
__instance = None
def _stop_flag(self):
return CalledOftenManager.__stop_run
def calledOften(self, id):
if CalledOftenManager.__instance is not None:
CalledOftenManager.__stop_run = True
while CalledOftenManager.__instance.isAlive():
pass # wait for the thread to terminate
CalledOftenManager.__stop_run = False
CalledOftenManager.__instance = DoIt(id, CalledOftenManager._stop_flag)
CalledOftenManager.__instance.start()
# Call Manager always
CalledOftenManager.calledOften(1)
CalledOftenManager.calledOften(2)
CalledOftenManager.calledOften(3)
Now, what I tried here is to make a controller for calling the thread DoIt. Its one approach to achieve what you need.
I have been making a GUI for a genetic algorithm I am working on and I made the mistake of leaving the threading so late simply because I did not (and still don't) know how to do it. So essentially when the start button is clicked the function 'run' starts the whole infinite loop process which actually happens in generation_loop. Each generation the loop checks to see if it should still be running. The idea is that if the stop or pause button has been clicked it will stop looping (with the stop button all the data is cleared with the pause button it remains and the unpause button just sets running to True and calls generation_loop)
So I need to work out a way to make my GUI responsive while generation_loop is running. Here is my code, I tried to minimise it but I am unsure what is important information for threading:
class Window(main_window, QMainWindow):
def __init__(self):
QMainWindow.__init__(self)
main_window.__init__(self)
self.setupUi(self)
self.scene = QGraphicsScene()
self.im_view.setScene(self.scene)
self.setWindowTitle('Fantasy Generator')
self.running = False
self.first_run = True
self.im = Image.new('RGBA', (400, 400), (0, 0, 0, 255))
self.saved_gens = deque([('A', self.im, self.im, self.im)])
self.set_save_amount(self.sb_saveamt.value())
self.population = []
self.btn_exit.clicked.connect(self.close)
self.actionQuit.triggered.connect(self.close)
self.btn_pauser.clicked.connect(self.pause_button)
self.sb_saveamt.valueChanged[int].connect(self.set_save_amount)
self.btn_restart.clicked.connect(self.start_button)
self.btn_loadimage.clicked.connect(self.get_image)
self.actionLoad_Image.triggered.connect(self.get_image)
self.gen_sldr.valueChanged[int].connect(self.display_gen)
self.cb_display.currentIndexChanged.connect(self.change_quality)
self.has_image = True
self.display_gen(0)
def get_image(self):
pass
# To save you time I removed the code here. It just sets self.im using a file dialog basically
def set_save_amount(self, amt):
if amt == -1:
self.saved_gens = deque(self.saved_gens)
else:
self.saved_gens = deque(self.saved_gens, amt + 1)
def pause_button(self):
if self.first_run:
self.run()
elif self.running:
self.running = False
self.btn_pauser.setText('Resume Execution')
# pause stuff goes here
else:
self.running = True
self.btn_pauser.setText('Pause Execution')
self.generation_loop()
# resume from pause stuff goes here
def start_button(self):
if self.first_run:
self.run()
else:
self.end()
# The run function should start the actual process
def run(self):
self.btn_restart.setText('End')
self.btn_pauser.setText('Pause Execution')
self.first_run = False
self.running = True
settings = dict(ind_per_gen=self.sb_ipg.value(), shapes_per_im=self.sb_spi.value(),
complexity=self.sb_complexity.value(), mut_rate=self.sb_mutation.value(),
cross_chance=self.sb_cross.value(), seed=self.sb_seed.value())
self.population = Population(self.im, **settings)
self.generation_loop()
# This is the loop I want to be able to exit out of using buttons
def generation_loop(self):
while self.running:
if self.first_run:
break
self.add_generation_data(self.population.next_gen())
def end(self):
self.btn_restart.setText('Start')
self.btn_pauser.setText('Start Execution')
self.first_run = True
self.running = False
self.saved_gens = deque([('A', self.im, self.im, self.im)])
self.set_save_amount()
self.display_gen(0)
def add_generation_data(self, data):
self.saved_gens.append(data)
self.gen_sldr.setMaximum(len(self.saved_gens) - 1)
self.gen_sldr.setValue(len(self.saved_gens) - 1)
self.display_gen(data[0] + 1)
def change_quality(self):
self.display_gen(self.gen_sldr.value())
def resizeEvent(self, e):
if self.has_image:
self.im_view.fitInView(QRectF(0, 0, self.width, self.height), Qt.KeepAspectRatio)
self.scene.update()
def display_image(self, image):
self.scene.clear()
if image.mode != 'RGBA':
image = image.convert('RGBA')
self.width, self.height = image.size
qim = ImageQt.ImageQt(image)
pixmap = QPixmap.fromImage(qim)
self.scene.addPixmap(pixmap)
self.im_view.fitInView(QRectF(0, 0, self.width, self.height), Qt.KeepAspectRatio)
self.scene.update()
def display_gen(self, index):
self.lcd_cur_gen.display(self.saved_gens[index][0])
if self.cb_display.currentIndex() == 0:
self.display_image(self.saved_gens[index][1])
elif self.cb_display.currentIndex() == 1:
self.display_image(self.saved_gens[index][2])
else:
self.display_image(self.saved_gens[index][3])
if __name__ == '__main__':
app = QApplication(sys.argv)
w = Window()
w.show()
sys.exit(app.exec_())
EDIT: I also just found at that I can't even change the graphics view from within the generation_loop but it works and changes if I limit the loop
In order to move your long running code to a thread, you need to first identify which parts of the long running code interact with the GUI and which parts don't. The key reason for this is that interacting with the GUI from a secondary thread is forbidden, and will lead to segfaults.
It looks like self.population.next_gen() is the long running bit of the code and doesn't interact with the GUI (although what this does is not provided so I can't be sure) while self.add_generation_data(...) updates the GUI which should be reasonably fast.
As such, this makes it reasonably simple to separate, which I'll show below.
Now, about threads. Python provides threads through the threading module (as the other answers show), however these not are recommended for use with a PyQt application if you want your thread to have any relation to the GUI (see here). PyQt also provides threading via the QThread object, which integrates support for sending and receiving Qt signals (which are thread safe). In short, the QThread has a separate event loop, and processes signals received asynchronously to the main thread, thus leaving the event loop in the main thread to process GUI events (like button clicks).
Typically you create a new class that inherits from QObject, instantiate it and move it to a QThread. Slots (aka methods) in the object that are triggered by a signal emission, then run in the thread.
So you'll want to do something like this
class MyWorker(QObject):
done = pyqtSignal(object) # you may need to update "object" to the type returned by Population.next_gen()
def __init__(self, settings):
# create the population object with whatever settings you need
# Note that this method runs in the parent thread as you have
# yet to move the object to a new thread. It shouldn't cause any
# problems, but may depend on what the Population class is/does.
# TODO: I've removed the reference to an image here...
#it may or may not be thread safe. I can't tell from your code.
self.population = Population(..., settings)
#pyqtSlot()
def next_gen(self):
new_gen = self.population.next_gen()
self.done.emit(new_gen)
class Window(....):
make_next_generation = pyqtSignal()
....
def run(self):
self.btn_restart.setText('End')
self.btn_pauser.setText('Pause Execution')
self.first_run = False
self.running = True
settings = dict(ind_per_gen=self.sb_ipg.value(), shapes_per_im=self.sb_spi.value(),
complexity=self.sb_complexity.value(), mut_rate=self.sb_mutation.value(),
cross_chance=self.sb_cross.value(), seed=self.sb_seed.value())
self.setupThread(settings)
def setupThread(self, settings):
self.thread = QThread()
self.worker = MyWorker(settings)
self.worker.moveToThread(self.thread)
# connect a signal in the main thread, to a slot in the worker.
# whenever you emit the signal, a new generation will be generated
# in the worker thread
self.make_next_generation.connect(self.worker.next_gen)
# connect the signal from the worker, to a slot in the main thread.
# This allows you to update the GUI when a generation has been made
self.worker.done.connect(self.process_generation)
# Start thread
self.thread.start()
# emit the signal to start the process!
self.make_next_generation.emit()
def process_generation(new_gen):
# run the GUI side of the code
# ignore the new generation if the "end" button was clicked
if not self.first_run:
self.add_generation_data(new_gen)
if self.running:
# make another generation in the thread!
self.make_next_generation.emit()
def pause_button(self):
if self.first_run:
self.run()
elif self.running:
self.running = False
self.btn_pauser.setText('Resume Execution')
# pause stuff goes here
else:
self.running = True
self.btn_pauser.setText('Pause Execution')
# make another generation in the thread!
self.make_next_generation.emit()
Things to note:
I haven't included all of your code in my answer. Merge as appropriate.
I'm unsure what self.im is. It's passed to Population so there might be some thread unsafe behaviour in your code that I can't see. I've left it to you to fix
I'm familiar with PyQt4, not PyQt5, so there is a possibility some things I've done don't work quite right. It should be easy for you to work out what to change from any error messages that are raised.
It's a bit messy recreating the thread and worker each time it is started from scratch. You might want to consider moving the instantiation of Population to a method in the worker (one that isn't __init__ and invoking it each time you want to start from scratch (in the same way we trigger a new generation). This would allow you to move pretty much all of setupThread to the Window.__init__ method and then when the start button was clicked, you'd just emit a signal to recreate Population followed by one to make the first generation.
You can use Threading events here.
from threading import Thread, Event
Once you detect the button click,
class MyThread(Thread):
def __init__(self, the_function, <any input param you want to provide>):
Thread.__init__(self)
self.stop_event = Event()
self.exec_func = the_function
def set_stop_flag(self, value):
if value:
self.stop_event.set()
else:
self.stop_event.clear()
def run(self):
while True:
try:
if not self.stop_event.is_set()
self.exec_func()
else:
break # once the event is set, you can break which will kill this thread.
# To stop busy waiting you can make this thread sleep for some seconds after each cycle.
import time
time.sleep(10) # 10 seconds wait before the next cycle.
except Exception, excep:
print "Got exception > ", str(excep)
Now in your code you embed this code piece and keep a reference for this thread.
Let's say
self.my_thread = MyThread(self.function_to_perform, <blabla>)
self.my_thread.setDaemon(True) # So that you don't have to worry about it when the Main process dies!
self.my_thread.start()
Now once you get a STOP button click event you call
self.my_thread.set_stop_flag(True) # Bingo! Your thread shall quit.
the following code:
import time
import threading
tasks = dict()
class newTask(object):
def __init__(self, **kw):
[setattr(self, x, kw[x]) for x in kw]
self.object_ret()
def object_ret(self): return self
def task_create(name, timeout, function):
task = newTask(**{
'timeout': int(timeout),
'function': function,
'start': time.time()
})
def set_timeout(v):
while True:
if (time.time() - v.start) > v.timeout:
v.function()
v.start = time.time()
tasks[name] = threading.Thread(target=set_timeout, args=(task,))
tasks[name].start()
def stop(x):
#stops the thread in tasks[x]
is a simple task system that i am using for minor tasks such as pings and timeouts. This works for my needs but if i ever wanted to stop a ping or task that was running, there is no way for me to do so. Is there a way for me to delete or stop that thread that i created using any means possible? I do not care if it is bad or messy to do so, i just want it stopped.
I suggest the following:
In your newTask.init function, add a line "self.alive = True"
In the set_timeout function, replace "while True:" with "while v.alive:"
Store newTask objects in your "tasks" dictionary, not thread objects.
The stop(x) function has one line: "tasks[x].alive = False"
This will cause the thread to die when you call stop(x), where x is the thread's name. It provides a mechanism that allows a thread to die without killing it in some bogus way. I know you said you don't care, but you really should care if you want your multithreaded programs to work.
Second suggestion: read Ulrich Eckhardt's comment carefully and take it seriously; all of his points are well taken.
Signal handler:::
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
tasks[name].stop()
sys.exit(0)
in the main script, register the handler:::
signal.signal(signal.SIGINT, signal_handler)
signal.pause()
I have simple PyGTK app. Since I have to run multiple periodic tasks to fetch some data and refresh GUI, I extended Thread like this:
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.setDaemon(True)
self.event = threading.Event()
self.event.set()
def run(self):
while self.event.is_set():
timer = threading.Timer(60, self._run)
timer.start()
timer.join()
def cancel(self):
self.event.clear()
def _run(self):
gtk.threads_enter()
# do what need to be done, fetch data, update GUI
gtk.threads_leave()
I start threads on app bootstrap, save them in some list and cancel them before exit. This works just perfect.
But now I want to add refresh button which will force one of the threads to run immediately and not wait period of time to be run, if not currently running.
I tried to do that by adding bool var to MyThread to indicate whether a thread is running or not (set before _run, reset on complete), and then just call MyThread._run() if not running, but that causes my app to become unresponsive and _run task to never finish execution.
I'm not sure why this happens. What is the best way to solve this problem? It would be also fine if I can make refresh running in background so it does not block GUI.
Maybe to call run and pass in number of seconds to 1 so timer can trigger it sooner?
Instead of using a Timer, use another Event object in combination with a timeout. You can then set that event from within your button callback. The following code illustrates this (I've stripped your cancelling code to keep it short):
import threading
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.sleep_event = threading.Event()
self.damon = True
def run(self):
while True:
self.sleep_event.clear()
self.sleep_event.wait(60)
threading.Thread(target=self._run).start()
def _run(self):
print "run"
my_thread = MyThread()
my_thread.start()
while True:
raw_input("Hit ENTER to force execution\n")
my_thread.sleep_event.set()
By default "run" will be printed every 60 seconds. If you hit ENTER it will be printed immediately, and then again after 60 seconds, etc.