this code works in idle3 but in a console(MAC, Windows Linux) thread2 is instantly closing if set to daemon. Is there any explanation for that ? Maybe also a workaround to properly have a daemon thread asking for user input ?
import queue
import threading
import sys
def worker(q):
_text = ''
while _text == '':
_text = q.get()
print('[worker]input was ',_text)
sys.exit()
def dialog(q):
while True:
try:
_select = input('[dialog]enter text:')
if _select != '':
q.put(_select)
except EOFError:
pass
except KeyboardInterrupt:
print("bye")
sys.exit(0)
except Exception as e:
print(e)
sys.exit(1)
if 'esc'.lower() in _select.lower():
sys.exit()
q = queue.Queue()
thread1 = threading.Thread(target=worker,args=(q,))
thread2 = threading.Thread(target=dialog,args=(q,))
thread1.setDaemon(True)
thread2.setDaemon(True)
print('start asking')
thread1.start()
thread2.start()
thanks for any hint on the issue
Normally the child threads die when the main thread exits. The code you've given as example exits directly after starting two child threads. To solve this, you should 'join' the threads back to the main thread. This will make it so the main thread waits for the child threads to die.
thread1.join()
thread2.join()
at the end of your file should solve this problem.
https://docs.python.org/3.5/library/threading.html#threading.Thread.join
Also, why do you want to run this application as daemon?
Related
I am testing Python threading with the following script:
import threading
class FirstThread (threading.Thread):
def run (self):
while True:
print 'first'
class SecondThread (threading.Thread):
def run (self):
while True:
print 'second'
FirstThread().start()
SecondThread().start()
This is running in Python 2.7 on Kubuntu 11.10. Ctrl+C will not kill it. I also tried adding a handler for system signals, but that did not help:
import signal
import sys
def signal_handler(signal, frame):
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
To kill the process I am killing it by PID after sending the program to the background with Ctrl+Z, which isn't being ignored. Why is Ctrl+C being ignored so persistently? How can I resolve this?
Ctrl+C terminates the main thread, but because your threads aren't in daemon mode, they keep running, and that keeps the process alive. We can make them daemons:
f = FirstThread()
f.daemon = True
f.start()
s = SecondThread()
s.daemon = True
s.start()
But then there's another problem - once the main thread has started your threads, there's nothing else for it to do. So it exits, and the threads are destroyed instantly. So let's keep the main thread alive:
import time
while True:
time.sleep(1)
Now it will keep print 'first' and 'second' until you hit Ctrl+C.
Edit: as commenters have pointed out, the daemon threads may not get a chance to clean up things like temporary files. If you need that, then catch the KeyboardInterrupt on the main thread and have it co-ordinate cleanup and shutdown. But in many cases, letting daemon threads die suddenly is probably good enough.
KeyboardInterrupt and signals are only seen by the process (ie the main thread)... Have a look at Ctrl-c i.e. KeyboardInterrupt to kill threads in python
I think it's best to call join() on your threads when you expect them to die. I've taken the liberty to make the change your loops to end (you can add whatever cleanup needs are required to there as well). The variable die is checked on each pass and when it's True, the program exits.
import threading
import time
class MyThread (threading.Thread):
die = False
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run (self):
while not self.die:
time.sleep(1)
print (self.name)
def join(self):
self.die = True
super().join()
if __name__ == '__main__':
f = MyThread('first')
f.start()
s = MyThread('second')
s.start()
try:
while True:
time.sleep(2)
except KeyboardInterrupt:
f.join()
s.join()
An improved version of #Thomas K's answer:
Defining an assistant function is_any_thread_alive() according to this gist, which can terminates the main() automatically.
Example codes:
import threading
def job1():
...
def job2():
...
def is_any_thread_alive(threads):
return True in [t.is_alive() for t in threads]
if __name__ == "__main__":
...
t1 = threading.Thread(target=job1,daemon=True)
t2 = threading.Thread(target=job2,daemon=True)
t1.start()
t2.start()
while is_any_thread_alive([t1,t2]):
time.sleep(0)
One simple 'gotcha' to beware of, are you sure CAPS LOCK isn't on?
I was running a Python script in the Thonny IDE on a Pi4. With CAPS LOCK on, Ctrl+Shift+C is passed to the keyboard buffer, not Ctrl+C.
(Python 3.8.3)
I am using two python threads right now, one that has a while True loop
import threading
def threadOne():
while True:
do(thing)
print('loop ended!')
t1=threading.Thread(threadOne)
t1.start()
And another that checks for a ctrl+r input. When recieved, I need the second thread to tell the first thread to break from the while loop. Is there a way to do this?
Note that I cannot change the loop to 'while Break == False' as do(thing) waits for user input, but i need this to be interrupted.
The recommended way is to use threading.event (You can combine this with event.wait if you want to sleep in that thread too however as you are waiting for a user event, probably dont need that).
import threading
e = threading.Event()
def thread_one():
while True:
if e.is_set():
break
print("do something")
print('loop ended!')
t1=threading.Thread(target=thread_one)
t1.start()
# and in other thread:
import time
time.sleep(0.0001) # just to show thread_one keeps printing
# do something for little bit and then it break
e.set()
EDIT: To interrupt the thread while it's waiting for user input you can send SIGINT to that thread and and it will raise KeyboardInterrupt which you can then handle. Unfortunate limitation of python, including python3, is that signals to all threads are handled in the main thread so you need to wait for the user input in the main thread:
import threading
import sys
import os
import signal
import time
def thread_one():
time.sleep(10)
os.kill(os.getpid(), signal.SIGINT)
t1=threading.Thread(target=thread_one)
t1.start()
while True:
try:
print("waiting: ")
sys.stdin.readline()
except KeyboardInterrupt:
break
print("loop ended")
One library that I need to use with python3 (iperf3) needs that the library 'run' function is executed in the main thread.
I'm performing some tests to verify if a new process with the multiprocessing library will let me use the main thread of the process but it seems that with the snippet above I cannot have a new 'main thread' for the process.
What would be the recommended way to run a forked process as the main thread of the new process? Is that possible? Will a system like Celery help with this? I'm planning to run this from a Flask app.
Thanks!
#! /usr/bin/python3
import threading
import multiprocessing as mp
def mp_call():
try:
print("mp_call is mainthread? {}".format(isinstance(threading.current_thread(), threading._MainThread)))
except Exception as e:
print('create iperf e:{}'.format(e))
def thread_call():
try:
print("thread_call is mainthread? {}".format(isinstance(threading.current_thread(), threading._MainThread)))
p = mp.Process(target=mp_call, args=[])
p.daemon = False
p.start()
p.join()
print('Process ended')
except Exception as e:
print('thread e:{}'.format(e))
t = threading.Thread(target=thread_call)
t.daemon = False
t.start()
t.join()
print('Thread ended')
In fact, all threads are dead after fork, you will get a new "main" thread which is your current thread, your checking method is wrong. threading._MainThread is not a public api, use threading.main_thread() instead:
assert threading.current_thread() == threading.main_thread()
because main thread got replaced after subprocess fork, no longer a _MainThread subclass.
When you import and use package, this package can run non daemon threads. Until these threads are finished, python cannot exit properly (like with sys.exit(0)). For example imagine that thread t is from some package. When unhandled exception occurs in the main thread, you want to terminate. But this won't exit immediately, it will wait 60s till the thread terminates.
import time, threading
def main():
t = threading.Thread(target=time.sleep, args=(60,))
t.start()
a = 5 / 0
if __name__ == '__main__':
try:
main()
except:
sys.exit(1)
So I came up with 2 things. Replace sys.exit(1) with os._exit(1) or enumerate all threads and make them daemon. Both of them seems to work, but what do you thing is better? os._exit won't flush stdio buffers but setting daemon attribute to threads seems like a hack and maybe it's not guaranteed to work all the time.
import time, threading
def main():
t = thread.Thread(target=time.sleep, args=(60,))
t.start()
a = 5 / 0
if __name__ == '__main__':
try:
main()
except:
for t in threading.enumerate():
if not t.daemon and t.name != "MainThread":
t._daemonic = True
sys.exit(1)
import sys
import time
import threading
class exThread(threading.Thread):
def __init__(self, threadId):
threading.Thread.__init__(self)
self.threadId = threadId
def run(self):
try:
while 1:
pass
except KeyboardInterrupt:
print "Ctrl-C caught by thread"
finally:
print "Thread's finally clause being executed"
sys.exit() # Same as thread.exit()
cond = True
def func():
pass
try:
th = exThread(1)
th.start()
while True:
if cond:
func()
except KeyboardInterrupt:
print "Ctrl-C caught by main thread"
sys.exit(0)
finally:
print "Executing the finally clause from main thread"
On executing above code, when I press Ctrl-C, the main thread exits after printing from its finally clause. Now, since the child thread is a non-daemon, it is still running in the try: except KeyboardInterrupt block. However, this child thread doesn't seem to respond to Ctrl-C, even though it is supposed to catch KeyboardInterrupt exception. Why?
As far as I know, KeyboardInterrup is only raised in the main thread. To be able to stop the other threads, make them check for an event from time to time, and then exit voluntarily.
Another option is to mark the child thread as daemon=True so that it will auto-terminate when the main thread terminates:
th.daemon = True
Additional reading:
How daemon threads work Daemon Threads Explanation
But also, daemon threads can be harmful: http://joeshaw.org/2009/02/24/605/