Is there a clear way to create a timeout function like the signal module but is compatible with Windows? Thanks
Yes, it can be done in windows without signal and it will also work in other os as well. The logic is to create a new thread and wait for a given time and raise an exception using _thread(in python3 and thread in python2). This exception will be thrown in the main thread and the with block will get exit if any exception occurs.
import threading
import _thread # import thread in python2
class timeout():
def __init__(self, time):
self.time= time
self.exit=False
def __enter__(self):
threading.Thread(target=self.callme).start()
def callme(self):
time.sleep(self.time)
if self.exit==False:
_thread.interrupt_main() # use thread instead of _thread in python2
def __exit__(self, a, b, c):
self.exit=True
Usuage Example :-
with timeout(2):
print("abc")
#other stuff here
The program in the with block should exit within 2 seconds otherise it will be exited after 2 seconds.
Related
I want to create a module that uses a long-running thread. But if a client program calls exit() while the thread is running, the process hangs. How can I make it safe for the client to exit()?
Here's the code for my module:
from threading import Lock, Thread
import signal
import time
import atexit
class Test:
def __init__(self):
self.__update_thread = Thread(target=self.__update)
self.__signals_mutex = Lock()
self.__update_thread_alive = True
signal.signal(signal.SIGINT, self.at_exit)
signal.signal(signal.SIGTERM, self.at_exit)
self.__update_thread.start()
def at_exit(self, a=None, b=None):
print('at_exit called')
self.__update_thread_alive = False
self.__update_thread.join()
def __del__(self):
self.at_exit()
def __update(self):
while self.__update_thread_alive:
#do_stuff
time.sleep(0.01)
__test = Test()
atexit.register(__test.at_exit)
Why after importing this module and then calling exit() function the interpreter freezes both in Windows and Linux? And when I'm just executing it.
This is caused by a thread, obviously. I guess exit() just joins the __update_thread.
So the main question is: how should I get a signal to terminate it without removing a global variable and caring about it outside of the module?
python-running-autobahnpython-asyncio-websocket-server-in-a-separate-subproce
can-an-asyncio-event-loop-run-in-the-background-without-suspending-the-python-in
Was trying to solve my issue with this two links above but i have not.
I have the following error : RuntimeError: There is no current event loop in thread 'Thread-1'.
Here the code sample (python 3):
from autobahn.asyncio.wamp import ApplicationSession
from autobahn.asyncio.wamp import ApplicationRunner
from asyncio import coroutine
import time
import threading
class PoloniexWebsocket(ApplicationSession):
def onConnect(self):
self.join(self.config.realm)
#coroutine
def onJoin(self, details):
def on_ticker(*args):
print(args)
try:
yield from self.subscribe(on_ticker, 'ticker')
except Exception as e:
print("Could not subscribe to topic:", e)
def poloniex_worker():
runner = ApplicationRunner("wss://api.poloniex.com:443", "realm1")
runner.run(PoloniexWebsocket)
def other_worker():
while True:
print('Thank you')
time.sleep(2)
if __name__ == "__main__":
polo_worker = threading.Thread(None, poloniex_worker, None, (), {})
thank_worker = threading.Thread(None, other_worker, None, (), {})
polo_worker.start()
thank_worker.start()
polo_worker.join()
thank_worker.join()
So, my final goal is to have 2 threads launched at the start. Only one need to use ApplicationSession and ApplicationRunner. Thank you.
A separate thread must have it's own event loop. So if poloniex_worker needs to listen to a websocket, it needs its own event loop:
def poloniex_worker():
asyncio.set_event_loop(asyncio.new_event_loop())
runner = ApplicationRunner("wss://api.poloniex.com:443", "realm1")
runner.run(PoloniexWebsocket)
But if you're on a Unix machine, you will face another error if you try to do this. Autobahn asyncio uses Unix signals, but those Unix signals only work in the main thread. You can simply turn off Unix signals if you don't plan on using them. To do that, you have to go to the file where ApplicationRunner is defined. That is wamp.py in python3.5 > site-packages > autobahn > asyncio on my machine. You can comment out the signal handling section of the code like so:
# try:
# loop.add_signal_handler(signal.SIGTERM, loop.stop)
# except NotImplementedError:
# # signals are not available on Windows
# pass
All this is a lot of work. If you don't absolutely need to run your ApplicationSession in a separate thread from the main thread, it's better to just run the ApplicationSession in the main thread.
This is part of a complex project, I will try and simplify it.
I have a class that gets a callable and executes it, the callable can run for any duration of time. If I get a signal (can be using Signal or any other flag I watch) to terminate I want to terminate the callable's execution on the spot (without exiting the process of course)
class FooRunner(object):
def goo(self, foo):
try:
foo()
except:
pass
def on_stop_signal(self):
pass
On a single-threaded signal not running on Windows, (i.e., any Unix flavor) you can use signal.alarm for that.
Check the first example on the documentation - it is more or less what you are asking for:
https://docs.python.org/2/library/signal.html
If anyone ever needs this here is a code sample of it working (One thing to note signal.signal can be called only from the main thread):
#!/usr/bin/python
import time
import signal
import threading
class MyException(Exception):
pass
class FooRunner(object):
def goo(self, foo):
try:
signal.signal(signal.SIGALRM, self.on_stop_signal)
foo()
except MyException:
print('caugt alarm exception')
def on_stop_signal(self, *args):
print('alarm triggered')
raise MyException()
def sample_foo():
time.sleep(30)
def stop_it():
signal.alarm(3)
print('alarm was set for 3 seconds')
if __name__ == "__main__":
print('starting')
fr = FooRunner()
t = threading.Thread(target=stop_it)
t.start()
fr.goo(sample_foo)
Thanks #jsbueno
This is part of a complex project, I will try and simplify it.
I have a class that gets a callable and executes it, the callable can run for any duration of time. If I get a signal (can be using Signal or any other flag I watch) to terminate I want to terminate the callable's execution on the spot (without exiting the process of course)
class FooRunner(object):
def goo(self, foo):
try:
foo()
except:
pass
def on_stop_signal(self):
pass
On a single-threaded signal not running on Windows, (i.e., any Unix flavor) you can use signal.alarm for that.
Check the first example on the documentation - it is more or less what you are asking for:
https://docs.python.org/2/library/signal.html
If anyone ever needs this here is a code sample of it working (One thing to note signal.signal can be called only from the main thread):
#!/usr/bin/python
import time
import signal
import threading
class MyException(Exception):
pass
class FooRunner(object):
def goo(self, foo):
try:
signal.signal(signal.SIGALRM, self.on_stop_signal)
foo()
except MyException:
print('caugt alarm exception')
def on_stop_signal(self, *args):
print('alarm triggered')
raise MyException()
def sample_foo():
time.sleep(30)
def stop_it():
signal.alarm(3)
print('alarm was set for 3 seconds')
if __name__ == "__main__":
print('starting')
fr = FooRunner()
t = threading.Thread(target=stop_it)
t.start()
fr.goo(sample_foo)
Thanks #jsbueno
This should be very simple and I'm very surprised that I haven't been able to find this questions answered already on stackoverflow.
I have a daemon like program that needs to respond to the SIGTERM and SIGINT signals in order to work well with upstart. I read that the best way to do this is to run the main loop of the program in a separate thread from the main thread and let the main thread handle the signals. Then when a signal is received the signal handler should tell the main loop to exit by setting a sentinel flag that is routinely being checked in the main loop.
I've tried doing this but it is not working the way I expected. See the code below:
from threading import Thread
import signal
import time
import sys
stop_requested = False
def sig_handler(signum, frame):
sys.stdout.write("handling signal: %s\n" % signum)
sys.stdout.flush()
global stop_requested
stop_requested = True
def run():
sys.stdout.write("run started\n")
sys.stdout.flush()
while not stop_requested:
time.sleep(2)
sys.stdout.write("run exited\n")
sys.stdout.flush()
signal.signal(signal.SIGTERM, sig_handler)
signal.signal(signal.SIGINT, sig_handler)
t = Thread(target=run)
t.start()
t.join()
sys.stdout.write("join completed\n")
sys.stdout.flush()
I tested this in the following two ways:
1)
$ python main.py > output.txt&
[2] 3204
$ kill -15 3204
2)
$ python main.py
ctrl+c
In both cases I expect this written to the output:
run started
handling signal: 15
run exited
join completed
In the first case the program exits but all I see is:
run started
In the second case the SIGTERM signal is seemingly ignored when ctrl+c is pressed and the program doesn't exit.
What am I missing here?
The problem is that, as explained in Execution of Python signal handlers:
A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction)
…
A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
Your main thread is blocked on threading.Thread.join, which ultimately means it's blocked in C on a pthread_join call. Of course that's not a "long-running calculation", it's a block on a syscall… but nevertheless, until that call finishes, your signal handler can't run.
And, while on some platforms pthread_join will fail with EINTR on a signal, on others it won't. On linux, I believe it depends on whether you select BSD-style or default siginterrupt behavior, but the default is no.
So, what can you do about it?
Well, I'm pretty sure the changes to signal handling in Python 3.3 actually changed the default behavior on Linux so you won't need to do anything if you upgrade; just run under 3.3+ and your code will work as you're expecting. At least it does for me with CPython 3.4 on OS X and 3.3 on Linux. (If I'm wrong about this, I'm not sure whether it's a bug in CPython or not, so you may want to raise it on python-list rather than opening an issue…)
On the other hand, pre-3.3, the signal module definitely doesn't expose the tools you'd need to fix this problem yourself. So, if you can't upgrade to 3.3, the solution is to wait on something interruptible, like a Condition or an Event. The child thread notifies the event right before it quits, and the main thread waits on the event before it joins the child thread. This is definitely hacky. And I can't find anything that guarantees it will make a difference; it just happens to work for me in various builds of CPython 2.7 and 3.2 on OS X and 2.6 and 2.7 on Linux…
abarnert's answer was spot on. I'm still using Python 2.7 however. In order to solve this problem for myself I wrote an InterruptableThread class.
Right now it doesn't allow passing additional arguments to the thread target. Join doesn't accept a timeout parameter either. This is just because I don't need to do that. You can add it if you want. You will probably want to remove the output statements if you use this yourself. They are just there as a way of commenting and testing.
import threading
import signal
import sys
class InvalidOperationException(Exception):
pass
# noinspection PyClassHasNoInit
class GlobalInterruptableThreadHandler:
threads = []
initialized = False
#staticmethod
def initialize():
signal.signal(signal.SIGTERM, GlobalInterruptableThreadHandler.sig_handler)
signal.signal(signal.SIGINT, GlobalInterruptableThreadHandler.sig_handler)
GlobalInterruptableThreadHandler.initialized = True
#staticmethod
def add_thread(thread):
if threading.current_thread().name != 'MainThread':
raise InvalidOperationException("InterruptableThread objects may only be started from the Main thread.")
if not GlobalInterruptableThreadHandler.initialized:
GlobalInterruptableThreadHandler.initialize()
GlobalInterruptableThreadHandler.threads.append(thread)
#staticmethod
def sig_handler(signum, frame):
sys.stdout.write("handling signal: %s\n" % signum)
sys.stdout.flush()
for thread in GlobalInterruptableThreadHandler.threads:
thread.stop()
GlobalInterruptableThreadHandler.threads = []
class InterruptableThread:
def __init__(self, target=None):
self.stop_requested = threading.Event()
self.t = threading.Thread(target=target, args=[self]) if target else threading.Thread(target=self.run)
def run(self):
pass
def start(self):
GlobalInterruptableThreadHandler.add_thread(self)
self.t.start()
def stop(self):
self.stop_requested.set()
def is_stop_requested(self):
return self.stop_requested.is_set()
def join(self):
try:
while self.t.is_alive():
self.t.join(timeout=1)
except (KeyboardInterrupt, SystemExit):
self.stop_requested.set()
self.t.join()
sys.stdout.write("join completed\n")
sys.stdout.flush()
The class can be used two different ways. You can sub-class InterruptableThread:
import time
import sys
from interruptable_thread import InterruptableThread
class Foo(InterruptableThread):
def __init__(self):
InterruptableThread.__init__(self)
def run(self):
sys.stdout.write("run started\n")
sys.stdout.flush()
while not self.is_stop_requested():
time.sleep(2)
sys.stdout.write("run exited\n")
sys.stdout.flush()
sys.stdout.write("all exited\n")
sys.stdout.flush()
foo = Foo()
foo2 = Foo()
foo.start()
foo2.start()
foo.join()
foo2.join()
Or you can use it more like the way threading.thread works. The run method has to take the InterruptableThread object as a parameter though.
import time
import sys
from interruptable_thread import InterruptableThread
def run(t):
sys.stdout.write("run started\n")
sys.stdout.flush()
while not t.is_stop_requested():
time.sleep(2)
sys.stdout.write("run exited\n")
sys.stdout.flush()
t1 = InterruptableThread(run)
t2 = InterruptableThread(run)
t1.start()
t2.start()
t1.join()
t2.join()
sys.stdout.write("all exited\n")
sys.stdout.flush()
Do with it what you will.
I faced the same problem here signal not handled when multiple threads join. After reading abarnert's answer, I changed to Python 3 and solved the problem. But I do like to change all my program to python 3. So, I solved my program by avoiding calling thread join() before signal sent. Below is my code.
It is not very good, but solved my program in python 2.7. My question was marked as duplicated, so I put my solution here.
import threading, signal, time, os
RUNNING = True
threads = []
def monitoring(tid, itemId=None, threshold=None):
global RUNNING
while(RUNNING):
print "PID=", os.getpid(), ";id=", tid
time.sleep(2)
print "Thread stopped:", tid
def handler(signum, frame):
print "Signal is received:" + str(signum)
global RUNNING
RUNNING=False
#global threads
if __name__ == '__main__':
signal.signal(signal.SIGUSR1, handler)
signal.signal(signal.SIGUSR2, handler)
signal.signal(signal.SIGALRM, handler)
signal.signal(signal.SIGINT, handler)
signal.signal(signal.SIGQUIT, handler)
print "Starting all threads..."
thread1 = threading.Thread(target=monitoring, args=(1,), kwargs={'itemId':'1', 'threshold':60})
thread1.start()
threads.append(thread1)
thread2 = threading.Thread(target=monitoring, args=(2,), kwargs={'itemId':'2', 'threshold':60})
thread2.start()
threads.append(thread2)
while(RUNNING):
print "Main program is sleeping."
time.sleep(30)
for thread in threads:
thread.join()
print "All threads stopped."