I am using the python threading module for the first time and this is what I am trying to find out -- can we save and load state of python thread object in some database or file? Let's say I have the below piece of code in a program (this program runs like a server using python twisted) to do something asynchronously (like copying files in a non-blocking fashion).
def foo():
print('Hello world!')
import threading
thr = threading.Thread(target=foo, args=(), kwargs={})
thr.start()
Now, I don't do thr.join() here as I don't want to wait for the thread to complete. Now, is it even possible to save the state of thread or object and later retrieve it and find out if the thread is still alive or not? I can get the thread id and status by doing this but only through the thread object.
# thr.ident
# check status of thread by doing thr.is_alive()
I may be completely wrong here - but is it possible to save/load thread objects? Also, looking for suggestions. Thanks a lot!
is it possible to save/load thread objects?
No, it is not.
Related
I am trying to call a thread I define in a function from another function. Here is the first function, its purpose is to create and start a thread:
def startThread(func):
listen = threading.Thread(target = func)
listen.start()
I am trying to implement a function that will close the thread created in that first function, how should I go about it? I don't know how to successfully pass the thread.
def endThread(thread):
thread.exit()
Thank you!
This problem is almost FAQ material.
To summarise, there is no way to kill a thread from the outside. You can of course pass the thread object to any function you want, but threading library is missing kill and exit calls.
There are more or less two distinct ways around this, depending on what your thread does.
The first method is to make it so that your thread co-operates. This approach is discussed here: Is there any way to kill a Thread in Python? This method adds a check to your thread loop and a way to raise a "stop signal", which will then cause the thread to exit from the inside when detected.
This method works fine if your thread is a relatively busy loop. If it is something that is blocking in IO wait, not so much, as your thread could be blocking in a read call for days or weeks before receiving something and executing the signal check part. Many IO calls accept a timeout value, and if it is acceptable to wait a couple of seconds before your thread exits, you can use this to force the exit check every N seconds without making your thread a busy loop.
The other approach is to replace threads with processes. You can force kill a subprocess. If you can communicate with your main program with queues instead of shared variables, this is not too complicated, either. If your program relies heavily on sharing global variables, this would require a major redesign.
If your program is waiting in IO loops, you need instantaneous termination and you are using shared global variables, then you are somewhat out of luck, as you either need to accept your threads not behaving nicely or you need to redesign some parts of your code to untangle either the IO wait or shared variables.
I'm making a virtual assistant using Python.
For that I need one main thread running continuously which is required for Speech recognition and want to run other threads like actions after detecting speech to run in background.
For the tasks like timer, I want to make it running in background while the main thread is running, so that I can perform other tasks even when the timer is running... and after reaching the time it should return it as tts to main thread
current structure I'm using is
main.py
-> class Main()
->Running logger in background // which is meant to exit with mainLoop
-> and Command() loop for speech recognition continuously
->`which links to Brain.py to timer.py
A few words about multithreading vs multiprocessing:
In multithreading you start a thread in the current process. Python runs (through the global interpreter lock) threads in short sequential order, never really in parallel. The upside is that threads can access the same variables (i.e. share memory).
On the other side in multiprocessing you run a new process (in the OS it appears as a separate program). They can really run in parallel but sharing variables is a lot more tricky (and much slower as well).
For your use case it seems that not two things are "CPU bound" i.e. it will not be the case that two things will need the CPU for 100% at the same time. In this situation, multithreading is probably the better solution, that is you should go for James Lim's solution.
If you still want to go for multiprocessing, then the following code could be your basic setup for timer. For the speech recognition function it would be accordingly (espeically the part about returning the list should be sufficient for returning tts from the speech recognition):
import multiprocessing
import time
def timer_with_return(seconds, return_list):
time.sleep(seconds)
return_list.append('foo')
if __name__ == "__main__":
# variables by manager are shared among process
# e.g. for return values
manager = multiprocessing.Manager()
timer_return = manager.list()
timer = multiprocessing.Process(target=timer_with_return, args=(3, timer_return))
timer.start()
while True:
time.sleep(1)
if not timer.is_alive():
break
print("timer is still running")
timer.join() # make sure the process is really finished
print("timer finished, return value is {}".format(timer_return))
Running this produces:
timer is still running
timer is still running
timer is still running
timer finished, return value is ['foo']
here's some example code
while True: #main-loop
if command_received:
thread = Thread(target = doItNOW)
thread.start()
......
def doItNOW():
some_blocking_operations()
my problem is I need "some_blocking_operations" to start IMMEDIATELY (as soon as command_received is True).
but since they're blocking i can't execute them on my main loop
and i can't change "some_blocking_operations" to be non-blocking either
for "IMMEDIATELY" i mean as soon as possible, not more than 10ms delay.
(i once got a whole second of delay).
if it's not possible, a constant delay would also be acceptable. (but it MUST be constant. with very few milliseconds of error)
i'm currently working on a linux system (Ubuntu, but it may be another one in the future. always linux)
a python solution would be amazing.. but a different one would be better than nothing
any ideas?
thanks in advance
from threading import Thread
class worker(Thread):
def __init__(self, someParameter=True):
Thread.__init__(self)
# This is how you "send" parameters/variables
# into the thread on start-up that the thread can use.
# This is just an example in case you need it.
self.someVariable = someParameter
self.start() # Note: This makes the thread self-starting,
# you could also call .start() in your main loop.
def run():
# And this is how you use the variable/parameter
# that you passed on when you created the thread.
if self.someVariable is True:
some_blocking_operations()
while True: #main-loop
if command_received:
worker()
This is a non-blocking execution of some_blocking_operations() in a threaded manner. I'm not sure if what you're looking for is to actually wait for the thread to finish or not, or if you even care?
If all you want to do is wait for a "command" to be received and then to execute the blocking operations without waiting for it, verifying that it completes, then this should work for you.
Python mechanics of threading
Python will only run in one CPU core, what you're doing here is simply running multiple executions on overlapping clock invervals in the CPU. Meaning every other cycle in the CPU an execution will be made in the main thread, and the other your blocking call will get a chance to run a execution. They won't actually run in parallel.
There are are some "You can, but..." threads.. Like this one:
is python capable of running on multiple cores?
Yet the thread module works for me. How to check if thread made by module thread (in Python 3 _thread) is running? When the function the thread is doing ends, the thread ends too, or doesn't?
def __init__(self):
self.thread =None
......
if self.thread==None or not self.thread.isAlive() :
self.thread = thread.start_new_thread(self.dosomething,())
else:
tkMessageBox.showwarning("XXXX","There's no need to have more than two threads")
I know there is no function called isAlive() in "thread" module, is there any alternative?
But there isn't any reason why to use "threading" module, is there?
Unless you really need the low-level capabilities of the internal thread (_thread module, you really should use the threading module instead. It makes everything easier to use and does come with helpers such as is_alive.
Btw. the alternative of restarting a thread like you do in your example code would be to keep it running but have it wait for additional jobs. E.g. you could have a queue somewhere which keeps track of all jobs you want the thread to do, and the thread keeps working on them until the queue is empty—and then it will not terminate but wait for new jobs to appear. And only at the end of the application, you signalize the thread to stop waiting and terminate it.
I had some annoyances with spawning subprocesses, like getting correct output and so on. A wrapper library, envoy, solved all of my problems with an easy-to-use interface that gets rid of most problems.
Using thread, I sometimes struggle with hanging processes that does not end, external programs launched within threads that I can't reach anymore and so on.
Is there any "threading for dummies" python library out there? Thanks
Is there any "threading for dummies" python library out there?
No, there is not. threading is pretty simple to use in simple cases. You want to use it to introduce concurrency in your program. This means you want to use it whenever you want two or more actions to happen simultaneously, i.e. at the same time.
This is how you can let Peter build a house and let Igor drive to Moskow at the same time:
from threading import Thread
import time
def drive_bus():
time.sleep(1)
print "Igor: I'm Igor and I'm driving to... Moskow!"
time.sleep(9)
print "Igor: Yei, Moskow!"
def build_house():
print "Peter: Let's start building a large house..."
time.sleep(10.1)
print "Peter: Urks, we have no tools :-("
threads = [Thread(target=drive_bus), Thread(target=build_house)]
for t in threads:
t.start()
for t in threads:
t.join()
Isn't that simple? Define your function to be run in another thread. Create a threading.Thread instance with that function as target. Nothing happend so far, until you invoke start. It fires off the thread and immediately returns.
Before letting your main thread exit, you should wait for all the threads you have spawned to finish. This is what t.join() does: it blocks and waits for the thread t to finish. Only then it returns.
I would recommend reading more about the actual Python library - it is simple enough. Your problem with hanging threads, provided it prevents your application from exiting, may be solved by using daemon threads.
What kind of task are you trying to achieve? If you are trying to run a task in parallel without actual use of the custom threading, you may find the package multiprocessing useful. Furthermore, there is an interesting piece of information on the python wiki about parallel processing.
Could you elaborate a bit more on the task please?