I'm trying to wrap the blocking calls in pyaudio with a thread to give me non-blocking access through queues. However, the problem I have is not with pyaudio, or queues, but with the issue of trying to test a thread. In keeping with "strip the example down to the minimum possible", all the pyaudio stuff has vanished, to leave only the thread class, and its instantiation in a main.
What I was hoping for was an object that I could create, and leave to get on with its stuff in the background, while I do control things with the console or tk. I figure the following max-stripped down example should have the thread doing stuff, while main runs and asks me if it is working. The raw_input prompt never appears. I would not be surprised at this if I was running it from IDLE, which is not thread safe, but I get the same behaviour if I run the script directly from the OS. I was prepared to see the raw input prompt disappear up the screen pushed by 'running' prints, but not even that happens. The prompt never appears. What's going on? It does respond to ctrl-C and to closing the window, but I'd still like to be able to see main running.
import threading
import time
class TestThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.running=True
self.run()
def run(self):
while self.running:
time.sleep(0.5)
print 'running'
def stop(self):
self.running=False
if __name__=='__main__':
tt=TestThread()
a=raw_input('simple stuff working ? -- ')
tt.stop()
You should start the thread with self.start() instead of self.run(). In this case you are just running the thread function like any other normal function.
Normally you do not inherit from Thread. Instead, you use Thread(target=func2run).start()
Related
Simply put, I want to properly implement threading in a Python GTK application. This is in order to prevent UI freezing due to functions/code taking a long time to finish running. Hence, my approach was to move all code which took a long time to run into separate functions, and run them in their separate threads as needed. This however posed a problem when trying to run the functions in sequence.
For example, take a look at the following code:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
def threaded_function(self):
time.sleep(20)
print('this is a threaded function')
def first_normal_function(self):
print('this is a normal function')
def second_normal_function(self):
print('this is a normal function')
def main_function(self, widget):
self.first_normal_function()
self.threaded_function()
self.second_normal_function()
Pressing the button starts main_function which then starts 3 functions in sequence. threaded_function represents a function which would take a long time to complete. Running this as is will freeze the UI. Hence it should be threaded as such:
...
...
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
self.second_normal_function()
What should happen is that the following first_normal_function should run, then threaded_function in a background thread - the UI should remain responsive as the background thread is working. Finally, second_normal_function should run, but only when threaded_function is finished.
The issue with this is that the functions will not run in sequence. The behaviour I am looking for could be achieved by using thread.join() however this freezes the UI.
So I ask, what's the proper way of doing this? This is a general case, however it concerns the general issue of having code which takes a long time to complete in a graphical application, while needing code to run sequentially. Qt deals with this by using signals, and having a QThread emit a finished signal. Does GTK have an equivalent?
I'm aware that this could be partially solved using Queue , with a put() and get() in relevant functions, however I don't understand how to get this to work if the main thread is calling anything other than functions.
EDIT: Given that it's possible to have threaded_function call second_normal_function using GLib.idle_add, let's take an example where in main_function, the second_normal_function call is replaced with a print statement, such that:
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
print('this comes after the thread is finished')
...
...
...
#some more code here
With GLib.idle_add, the print statement and all the code afterwards would need to be moved into a separate function. Is it possible to avoid moving the print statement into its own function while maintaining sequentiality, such that the print statement remains where it is and still gets called after threaded_function is finished?
Your suggestion on how to do this was very close to the actual solution, but it's indeed not going to work.
In essence, what you'll indeed want to do, is to run the long-running function in a different thread. That'll mean you get 2 threads: one which is running the main event loop that (amongs other things) updates your UI, and another thread which does the long-running logic.
Of course, that bears the question: how do I notify the main thread that some work is done and I want it to react to that? For example, you might want to update the UI while (or after) some complex calculation is going on. For this, you can use GLib.idle_add() from within the other thread. That function takes a single callback as an argument, which it will run as soon as it can ("on idle").
So a possibility to use here, would be something like this:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
def threaded_function(self):
# Really intensive stuff going on here
sleep(20)
# We're done, schedule "on_idle" to be called in the main thread
GLib.idle_add(self.on_idle)
# Note, this function will be run in the main loop thread, *not* in this one
def on_idle(self):
second_normal_function()
return GLib.SOURCE_REMOVE # we only want to run once
# ...
For more context, you might want to read the pygobject documentation on threading and concurrency
I was attempting to create a thread class that could be terminated by an exception (since I am trying to have the thread wait on an event) when I created the following:
import sys
class testThread(threading.Thread):
def __init__(self):
super(testThread,self).__init__()
self.daemon = True
def run(self):
try:
print('Running')
while 1:
pass
except:
print('Being forced to exit')
test1 = testThread()
test2 = testThread()
print(test1.daemon)
test1.run()
test2.run()
sys.exit()
However, running the program will only print out one Running message, until the other is terminated. Why is that?
The problem is that you're calling the run method.
This is just a plain old method that you implement, which does whatever you put in its body. In this case, the body is an infinite loop, so calling run just loops forever.
The way to start a thread is the start method. This method is part of the Thread class, and what it does is:
Start the thread’s activity.
It must be called at most once per thread object. It arranges for the object’s run() method to be invoked in a separate thread of control.
So, if you call this, it will start a new thread, make that new thread run your run() method, and return immediately, so the main thread can keep doing other stuff.1 That's what you want here.
1. As pointed out by Jean-François Fabre, you're still not going to get any real parallelism here. Busy loops are never a great idea in multithreaded code, and if you're running this in CPython or PyPy, almost all of that busy looping is executing Python bytecode while holding the GIL, and only one thread can hold the GIL at a time. So, from a coarse view, things look concurrent—three threads are running, and all making progress. But if you zoom in, there's almost no overlap where two threads progress at once, usually not even enough to make up for the small scheduler overhead.
I've wrote a class that inherits from object and has instances of sub-objects that uses some threads for tasks. There are two socket listeners that creates other threads for each accepted connection. They do what they have to do. To finish them, they are looking on a Threading.Event object to know that they have to finish.
I've noticed that, when exit the python console they are not notified (or don't catch the notification) and the exit don't return control to the bash console, unless a Close() is called before.
First idea to fix it has been to implement the '__del__' method to use the garbage collector to clean it when exit.
class ServiceProvider(object):
def __init__(self):
super(ServiceProvider,self).__init__()
#...
self.Open()
def Open(self):
#... Some threads are created.
def Close(self):
#.... Threading.Event to report the threads to finish
def __del__(self):
self.Close()
But the behaviour is the same. If I place a print in those methods, non in '__del__', neither in 'Close' they are written. Unless it is closed before, then the print in the del is wrote.
Then I've implemented the '__enter__' and '__exit__' methods to manage the with statement. And the exit behaves as expected and when the with ends, things are release. But what I really want is to have something like the file descriptors that event if file.close() is not called, it is executed when exits the program.
class ServiceProvider(object):
#...
def __enter__(self):
return self
def __exit__(self):
self.Close()
Searching for more solutions I've tried with atexit but not. I have similar results that doesn't fix the issue. Even I collect all the objects created of this class, the doOnExit only writes its print if the objects in the list are already Close.
import atexit
global objects2Close
objects2Close = []
#atexit.register
def doOnExit():
for obj in objects2Close:
obj.Close()
class ServiceProvider(object):
def __init__(self):
super(ServiceProvider,self).__init__()
objects2Close.append(self)
It's usually a good idea to use with when you have resources that you don't want to leak (files, connections, whatever else you care about).
Somewhere, just outside your main loop you should have something like:
with ServiceProvider(some_params) as service_provider:
rest_of_the_code()
What this does is that regardless of how you exit rest_of_the_code() (except for kill -9) it will call service_provider.Close() at the end. This works for exceptions and interrupts as well. Kill -9 doesn't work because the process is kill at os level and doesn't have a chance to attempt to recover.
I've got a solution for this issue. The posted information in this question was not related with the real issue.
This is as simple as daemon threading.
A the implementation uses some threads for listening remote connections they have to finish their execution when the program goes to exit. But the program ends when all the no daemon thread has finished.
Mistakenly those listeners and talkers where not set to be daemons and that's why the execution waits for them.
I have to launch a thread in the background but the output seems to be following the thread rather than sticking within main. I have something like this:
import threading
def work()
while True:
do stuff
def recieve()
while True:
await instruction
#main
recieve()
if instruction == "do work"
threading.Thread(target=work()).start()
I have many other instructions that get recieved and dealt with sequentially but as work() takes a long time to complete I need the thread, now, I would expect a thread to be launched that does work in the background whilst we continue to await instruction except this doesn't happen. What happens is focus is kept on the newly created thread so further instructions can't be received.
Why is this? What is wrong?
Many thanks
receive() never end because of endless loop; thread does not start.
Start thread first.
if instruction == "do work":
threading.Thread(target=work).start()
recieve()
and drop () from threading.Thread(target=work()).start(). work() make work function call run in main thread.
I am writing a program by a framework using pygtk. The main program doing the following things:
Create a watchdog thread to monitor some resource
Create a client to receive data from socket
call gobject.Mainloop()
but it seems after my program enter the Mainloop, the watchdog thread also won't run.
My workaround is to use gobject.timeout_add to run the monitor thing.
But why does creating another thread not work?
Here is my code:
import gobject
import time
from threading import Thread
class MonitorThread(Thread):
def __init__(self):
Thread.__init__(self)
def run(self):
print "Watchdog running..."
time.sleep(10)
def main():
mainloop = gobject.MainLoop(is_running=True)
def quit():
mainloop.quit()
def sigterm_cb():
gobject.idle_add(quit)
t = MonitorThread()
t.start()
print "Enter mainloop..."
while mainloop.is_running():
try:
mainloop.run()
except KeyboardInterrupt:
quit()
if __name__ == '__main__':
main()
The program output only "Watchdog running...Enter mainloop..", then nothing.
Seems thread never run after entering mainloop.
Can you post some code? It could be that you have problems with the Global Interpreter Lock.
Your problem solved by someone else :). I could copy-paste the article here, but in short gtk's c-threads clash with Python threads. You need to disable c-threads by calling gobject.threads_init() and all should be fine.
You have failed to initialise the threading-based code-paths in gtk.
You must remember two things when
using threads with PyGTK:
GTK Threads must be initialised with gtk.gdk.threads_init:
From http://unpythonic.blogspot.com/2007/08/using-threads-in-pygtk.html, copyright entirely retained by author. This copyright notice must not be removed.
You can think glib/gobject instead of pygtk, it's the same thing.