I have a very simple class with one attribute and one method. i would like the method to run based on the value of the attribute. below is the class code:
class TestWhile()
def __init__(self):
self.status = "OFF"
def StreamV(self):
while self.Status == "ON"
print "nonstop"
basically what I want to do is set Status to ON, then run StreamV until Status is set to off. This is what i want as follows:
P = TestWhile()
T.Status = "ON"
T.StreamV()
T.Status = "OFF"
When I run the above test, it never makes it to the Status = "OFF" line and infinitely runs the loop. How can I fix this?
your call to T.StreamV() does what is commonly referred to as Busy looping and blocks the main thread. What you have effectively described you want is a background thread to update some value until a signal comes in (from your T.Status attribute)
Threading is relatively approachable for simple problems, and is an incredibly useful tool to get to know. From a super wide viewpoint a thread in python is a function that can run at the same time the main script is doing something else. In this regard python threads are indeed created by passing them a function to execute.
Example:
import threading
import time
class TestWhile(object):
def __init__(self):
self.status = "OFF"
self.streamVthread = threading.Thread(target=self.StreamV) #create a thread for our streamV function
def StreamV(self):
print "starting thread"
while self.Status == "ON":
print "nonstop"
time.sleep(1)
print "stopping thread"
T = TestWhile() #our thread is actually created here, but is hasn't started yet
print 'T.Status -> "ON"'
T.Status = "ON" #T.StreamV could be modified to run at creation and wait for this to change to "ON"
T.streamVthread.start() #this calls the T.StreamV function in a separate thread
time.sleep(6)
T.Status = "OFF"
print 'T.Status -> "OFF"'
I've added a bunch of print statements to help you understand the flow of the program. Please comment with any questions.
Edit: passing arguments to a thread
Threads exist in the same namespace as the main thread, so if they share a variable with the main script they will both have access to it. This can cause problems with multiple threads accessing the same thing at the same time (read more on locks and other mutex constructs). Passing arguments at creation of the thread however can be done as such:
import threading
import time
def sayHello(after_secs, name):
time.sleep(after_secs)
print("hello from {}".format(name))
thread1 = threading.Thread(target=sayHello, args=(3,"thread1"))
thread2 = threading.Thread(target=sayHello, args=(1,"thread2"))
thread3 = threading.Thread(target=sayHello, args=(5,"thread3"))
print "starting thread1"
thread1.start()
print "starting thread2"
thread2.start()
print "starting thread3"
thread3.start()
thread1.join() #wait for thread1 to finish
print "thread1 is done"
thread2.join() #wait for thread2 to finish
print "thread2 is done"
thread3.join() #wait for thread3 to finish
print "thread3 is done"
See if you can determine why these print statements happen in the order they do..
Related
I've been looking all over google and can't seem to get this working.
I'm trying to thread 2 functions, both of which are infinite loops.
Looking at the extract below, it only starts the 1st thread and does not proceed to the next one in line.
PS: When I swap the 2 threads around, then I have the same problem with the 2nd thread.
def syslog_service():
syslog_server = socketserver.UDPServer((syslog_host,syslog_port), Syslog_Server)
syslog_server.serve_forever()
def cleanup_old_logs_service():
# lock = threading.Lock()
# threading.Thread.__init__(self)
global syslog_retention_hours
global RUNNING
while RUNNING:
# cleanup_old_logs_service.lock.acquire()
cleanup.old_logs(syslog_retention_hours)
# cleanup_old_logs_service.lock.release()
time.sleep(10)
if __name__ == "__main__":
try:
logger.info("Starting main thread")
config()
logger.info("Starting system testing")
test()
logger.info("Config loaded")
thread1 = cleanup_old_logs_service()
thread2 = syslog_service()
thread1.start()
logger.info("Syslog cleanup service running")
thread2.start()
logger.info("Syslog server running")
The reason why only the first thread is executed is that you actually have ONLY one thread in your program. When you write thread1 = cleanup_old_logs_service() and thread2 = syslog_service()you are not creating new threads, but just assigning the return values of your functions to 2 different variables. For this reason, as soon as the program encounters thread1, it executes cleanup_old_logs_service() and gets stuck in an infinite loop.
To create a new thread, I would import the threading module, create a new threadObj object and start the thread as follows:
import threading
threadObj = threading.Thread(target=cleanup_old_logs_service)
threadObj.start()
This way, the function cleanup_old_logs_service() will be executed in a new thread.
By saying thread1 = cleanup_old_logs_service() you are actually executing the function cleanup_old_logs_service not saving a reference to a thread. You would have to say
import threading # If you have not already
thread1 = threading.Thread(target=cleanup_old_logs_service)
thread2 = threading.Thread(target=syslog_service)
# Now you can start the thread
thread1.start()
logger.info("Syslog cleanup service running")
thread2.start()
logger.info("Syslog server running")
You can look at https://docs.python.org/3.5/library/threading.html for documentation and https://pymotw.com/2/threading/ for examples, because I believe you would need to use locks to manage access to your resources
This should be very simple and I'm very surprised that I haven't been able to find this questions answered already on stackoverflow.
I have a daemon like program that needs to respond to the SIGTERM and SIGINT signals in order to work well with upstart. I read that the best way to do this is to run the main loop of the program in a separate thread from the main thread and let the main thread handle the signals. Then when a signal is received the signal handler should tell the main loop to exit by setting a sentinel flag that is routinely being checked in the main loop.
I've tried doing this but it is not working the way I expected. See the code below:
from threading import Thread
import signal
import time
import sys
stop_requested = False
def sig_handler(signum, frame):
sys.stdout.write("handling signal: %s\n" % signum)
sys.stdout.flush()
global stop_requested
stop_requested = True
def run():
sys.stdout.write("run started\n")
sys.stdout.flush()
while not stop_requested:
time.sleep(2)
sys.stdout.write("run exited\n")
sys.stdout.flush()
signal.signal(signal.SIGTERM, sig_handler)
signal.signal(signal.SIGINT, sig_handler)
t = Thread(target=run)
t.start()
t.join()
sys.stdout.write("join completed\n")
sys.stdout.flush()
I tested this in the following two ways:
1)
$ python main.py > output.txt&
[2] 3204
$ kill -15 3204
2)
$ python main.py
ctrl+c
In both cases I expect this written to the output:
run started
handling signal: 15
run exited
join completed
In the first case the program exits but all I see is:
run started
In the second case the SIGTERM signal is seemingly ignored when ctrl+c is pressed and the program doesn't exit.
What am I missing here?
The problem is that, as explained in Execution of Python signal handlers:
A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction)
…
A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
Your main thread is blocked on threading.Thread.join, which ultimately means it's blocked in C on a pthread_join call. Of course that's not a "long-running calculation", it's a block on a syscall… but nevertheless, until that call finishes, your signal handler can't run.
And, while on some platforms pthread_join will fail with EINTR on a signal, on others it won't. On linux, I believe it depends on whether you select BSD-style or default siginterrupt behavior, but the default is no.
So, what can you do about it?
Well, I'm pretty sure the changes to signal handling in Python 3.3 actually changed the default behavior on Linux so you won't need to do anything if you upgrade; just run under 3.3+ and your code will work as you're expecting. At least it does for me with CPython 3.4 on OS X and 3.3 on Linux. (If I'm wrong about this, I'm not sure whether it's a bug in CPython or not, so you may want to raise it on python-list rather than opening an issue…)
On the other hand, pre-3.3, the signal module definitely doesn't expose the tools you'd need to fix this problem yourself. So, if you can't upgrade to 3.3, the solution is to wait on something interruptible, like a Condition or an Event. The child thread notifies the event right before it quits, and the main thread waits on the event before it joins the child thread. This is definitely hacky. And I can't find anything that guarantees it will make a difference; it just happens to work for me in various builds of CPython 2.7 and 3.2 on OS X and 2.6 and 2.7 on Linux…
abarnert's answer was spot on. I'm still using Python 2.7 however. In order to solve this problem for myself I wrote an InterruptableThread class.
Right now it doesn't allow passing additional arguments to the thread target. Join doesn't accept a timeout parameter either. This is just because I don't need to do that. You can add it if you want. You will probably want to remove the output statements if you use this yourself. They are just there as a way of commenting and testing.
import threading
import signal
import sys
class InvalidOperationException(Exception):
pass
# noinspection PyClassHasNoInit
class GlobalInterruptableThreadHandler:
threads = []
initialized = False
#staticmethod
def initialize():
signal.signal(signal.SIGTERM, GlobalInterruptableThreadHandler.sig_handler)
signal.signal(signal.SIGINT, GlobalInterruptableThreadHandler.sig_handler)
GlobalInterruptableThreadHandler.initialized = True
#staticmethod
def add_thread(thread):
if threading.current_thread().name != 'MainThread':
raise InvalidOperationException("InterruptableThread objects may only be started from the Main thread.")
if not GlobalInterruptableThreadHandler.initialized:
GlobalInterruptableThreadHandler.initialize()
GlobalInterruptableThreadHandler.threads.append(thread)
#staticmethod
def sig_handler(signum, frame):
sys.stdout.write("handling signal: %s\n" % signum)
sys.stdout.flush()
for thread in GlobalInterruptableThreadHandler.threads:
thread.stop()
GlobalInterruptableThreadHandler.threads = []
class InterruptableThread:
def __init__(self, target=None):
self.stop_requested = threading.Event()
self.t = threading.Thread(target=target, args=[self]) if target else threading.Thread(target=self.run)
def run(self):
pass
def start(self):
GlobalInterruptableThreadHandler.add_thread(self)
self.t.start()
def stop(self):
self.stop_requested.set()
def is_stop_requested(self):
return self.stop_requested.is_set()
def join(self):
try:
while self.t.is_alive():
self.t.join(timeout=1)
except (KeyboardInterrupt, SystemExit):
self.stop_requested.set()
self.t.join()
sys.stdout.write("join completed\n")
sys.stdout.flush()
The class can be used two different ways. You can sub-class InterruptableThread:
import time
import sys
from interruptable_thread import InterruptableThread
class Foo(InterruptableThread):
def __init__(self):
InterruptableThread.__init__(self)
def run(self):
sys.stdout.write("run started\n")
sys.stdout.flush()
while not self.is_stop_requested():
time.sleep(2)
sys.stdout.write("run exited\n")
sys.stdout.flush()
sys.stdout.write("all exited\n")
sys.stdout.flush()
foo = Foo()
foo2 = Foo()
foo.start()
foo2.start()
foo.join()
foo2.join()
Or you can use it more like the way threading.thread works. The run method has to take the InterruptableThread object as a parameter though.
import time
import sys
from interruptable_thread import InterruptableThread
def run(t):
sys.stdout.write("run started\n")
sys.stdout.flush()
while not t.is_stop_requested():
time.sleep(2)
sys.stdout.write("run exited\n")
sys.stdout.flush()
t1 = InterruptableThread(run)
t2 = InterruptableThread(run)
t1.start()
t2.start()
t1.join()
t2.join()
sys.stdout.write("all exited\n")
sys.stdout.flush()
Do with it what you will.
I faced the same problem here signal not handled when multiple threads join. After reading abarnert's answer, I changed to Python 3 and solved the problem. But I do like to change all my program to python 3. So, I solved my program by avoiding calling thread join() before signal sent. Below is my code.
It is not very good, but solved my program in python 2.7. My question was marked as duplicated, so I put my solution here.
import threading, signal, time, os
RUNNING = True
threads = []
def monitoring(tid, itemId=None, threshold=None):
global RUNNING
while(RUNNING):
print "PID=", os.getpid(), ";id=", tid
time.sleep(2)
print "Thread stopped:", tid
def handler(signum, frame):
print "Signal is received:" + str(signum)
global RUNNING
RUNNING=False
#global threads
if __name__ == '__main__':
signal.signal(signal.SIGUSR1, handler)
signal.signal(signal.SIGUSR2, handler)
signal.signal(signal.SIGALRM, handler)
signal.signal(signal.SIGINT, handler)
signal.signal(signal.SIGQUIT, handler)
print "Starting all threads..."
thread1 = threading.Thread(target=monitoring, args=(1,), kwargs={'itemId':'1', 'threshold':60})
thread1.start()
threads.append(thread1)
thread2 = threading.Thread(target=monitoring, args=(2,), kwargs={'itemId':'2', 'threshold':60})
thread2.start()
threads.append(thread2)
while(RUNNING):
print "Main program is sleeping."
time.sleep(30)
for thread in threads:
thread.join()
print "All threads stopped."
the following code:
import time
import threading
tasks = dict()
class newTask(object):
def __init__(self, **kw):
[setattr(self, x, kw[x]) for x in kw]
self.object_ret()
def object_ret(self): return self
def task_create(name, timeout, function):
task = newTask(**{
'timeout': int(timeout),
'function': function,
'start': time.time()
})
def set_timeout(v):
while True:
if (time.time() - v.start) > v.timeout:
v.function()
v.start = time.time()
tasks[name] = threading.Thread(target=set_timeout, args=(task,))
tasks[name].start()
def stop(x):
#stops the thread in tasks[x]
is a simple task system that i am using for minor tasks such as pings and timeouts. This works for my needs but if i ever wanted to stop a ping or task that was running, there is no way for me to do so. Is there a way for me to delete or stop that thread that i created using any means possible? I do not care if it is bad or messy to do so, i just want it stopped.
I suggest the following:
In your newTask.init function, add a line "self.alive = True"
In the set_timeout function, replace "while True:" with "while v.alive:"
Store newTask objects in your "tasks" dictionary, not thread objects.
The stop(x) function has one line: "tasks[x].alive = False"
This will cause the thread to die when you call stop(x), where x is the thread's name. It provides a mechanism that allows a thread to die without killing it in some bogus way. I know you said you don't care, but you really should care if you want your multithreaded programs to work.
Second suggestion: read Ulrich Eckhardt's comment carefully and take it seriously; all of his points are well taken.
Signal handler:::
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
tasks[name].stop()
sys.exit(0)
in the main script, register the handler:::
signal.signal(signal.SIGINT, signal_handler)
signal.pause()
I create a python thread.One it's kick to run by calling it's start() method , I monitor a falg inside the thread , if that flag==True , I know User no longer wants the thread to keep running , so I liek to do some house cleaning and terminate the thread.
I couldn't terminate the thread however. I tried thread.join() , thread.exit() ,thread.quit() , all throw exception.
Here is how my thread looks like .
EDIT 1 : Please notice the core() function is called within standard run() function , which I haven't show it here.
EDIT 2 : I just tried sys.exit() when the StopFlag is true , and it looks thread terminates ! is that safe to go with ?
class workingThread(Thread):
def __init__(self, gui, testCase):
Thread.__init__(self)
self.myName = Thread.getName(self)
self.start() # start the thread
def core(self,arg,f) : # Where I check the flag and run the actual code
# STOP
if (self.StopFlag == True):
if self.isAlive():
self.doHouseCleaning()
# none of following works all throw exceptions
self.exit()
self.join()
self._Thread__stop()
self._Thread_delete()
self.quit()
# Check if it's terminated or not
if not(self.isAlive()):
print self.myName + " terminated "
# PAUSE
elif (self.StopFlag == False) and not(self.isSet()):
print self.myName + " paused"
while not(self.isSet()):
pass
# RUN
elif (self.StopFlag == False) and self.isSet():
r = f(arg)
Several problems here, could be others too but if you're not showing the entire program or the specific exceptions this is the best I can do:
The task the thread should be performing should be called "run" or passed to the Thread constructor.
A thread doesn't call join() on itself, the parent process that started the thread calls join(), which makes the parent process block until the thread returns.
Usually the parent process should be calling run().
The thread is complete once it finishes (returns from) the run() function.
Simple example:
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
super(MyThread,self).__init__()
self.count = 5
def run(self):
while self.count:
print("I'm running for %i more seconds" % self.count)
time.sleep(1)
self.count -= 1
t = MyThread()
print("Starting %s" % t)
t.start()
# do whatever you need to do while the other thread is running
t.join()
print("%s finished" % t)
Output:
Starting <MyThread(Thread-1, initial)>
I'm running for 5 more seconds
I'm running for 4 more seconds
I'm running for 3 more seconds
I'm running for 2 more seconds
I'm running for 1 more seconds
<MyThread(Thread-1, stopped 6712)> finished
There's no explicit way to kill a thread, either from a reference to thread instance or from the threading module.
That being said, common use cases for running multiple threads do allow opportunities to prevent them from running indefinitely. If, say, you're making connections to an external resource via urllib2, you could always specify a timeout:
import urllib2
urllib2.urlopen(url[, data][, timeout])
The same is true for sockets:
import socket
socket.setdefaulttimeout(timeout)
Note that calling the join([timeout]) method of a thread with a timeout specified will only block for hte timeout (or until the thread terminates. It doesn't kill the thread.
If you want to ensure that the thread will terminate when your program finishes, just make sure to set the daemon attribute of the thread object to True before invoking it's start() method.
I'm sorry if it is a stupid question. I am trying to use a number of classes of multi-threading to finish different jobs, which involves invoking these multi-threadings at different times for many times. But I am not sure which method to use. The code looks like this:
class workers1(Thread):
def __init__(self):
Thread.__init__(self)
def run(self):
do some stuff
class workers2(Thread):
def __init__(self):
Thread.__init__(self)
def run(self):
do some stuff
class workers3(Thread):
def __init__(self):
Thread.__init__(self)
def run(self):
do some stuff
WorkerList1=[workers1(i) for i in range(X)]
WorkerList2=[workers2(i) for i in range(XX)]
WorkerList2=[workers3(i) for i in range(XXX)]
while True:
for thread in WorkerList1:
thread.run (start? join? or?)
for thread in WorkerList2:
thread.run (start? join? or?)
for thread in WorkerList3:
thread.run (start? join? or?)
do sth .
I am trying to have all the threads in all the WorkerList to start functioning at the same time, or at least start around the same time. After sometime once they were all terminated, I would like to invoke all the threads again.
If there were no loop, I can just use .start; but since I can only start a thread once, start apparently does not fit here. If I use run, it seems that all the threads start sequentially, not only the threads in the same list, but also threads from different lists.
Can anyone please help?
there are a lot of misconceptions here:
you can only start a specific instance of a thread once. but in your case, the for loop is looping over different instances of a thread, each instance being assigned to the variable thread in the loop, so there is no problem at all in calling the start() method over each thread. (you can think of it as if the variable thread is an alias of the Thread() object instantiated in your list)
run() is not the same as join(): calling run() performs as if you were programming sequentially. the run() method does not start a new thread, it simply execute the statements in in the method, as for any other function call.
join() does not start executing anything: it only waits for a thread to finish. in order for join() to work properly for a thread, you have to call start() on this thread first.
additionally, you should note that you cannot restart a thread once it has finished execution: you have to recreate the thread object for it to be started again. one workaround to get this working is to call Thread.__init__() at the end of the run() method. however, i would not recommend doing this since this will disallow the use of the join() method to detect the end of execution of the thread.
If you would call thread.start() in the loops, you would actually start every thread only once, because all the entries in your list are distinct thread objects (it does not matter they belong to the same class). You should never call the run() method of a thread directly -- it is meant to be called by the start() method. Calling it directly would not call it in a separate thread.
The code below creates a class that is just a thread but the start and calls the initialization of the Thread class again so that the thread doesn't know it has been called.
from threading import Thread
class MTThread(Thread):
def __init__(self, name = "", target = None):
self.mt_name = name
self.mt_target = target
Thread.__init__(self, name = name, target = target)
def start(self):
super().start()
Thread.__init__(self, name = self.mt_name, target = self.mt_target)
def run(self):
super().run()
Thread.__init__(self, name = self.mt_name, target = self.mt_target)
def code():
#Some code
thread = MTThread(name = "SomeThread", target = code)
thread.start()
thread.start()
I had this same dilemma and came up with this solution which has worked perfectly for me. It also allows a thread-killing decorator to be used efficiently.
The key feature is the use of a thread refresher which is instantiated and .started in main. This thread-refreshing thread will run a function that instantiates and starts all other (real, task-performing) threads. Decorating the thread-refreshing function with a thread-killer allows you to kill all threads when a certain condition is met, such as main terminating.
#ThreadKiller(arg) #qu'est-ce que c'est
def RefreshThreads():
threadTask1 = threading.Thread(name = "Task1", target = Task1, args = (anyArguments))
threadTask2 = threading.Thread(name = "Task2", target = Task2, args = (anyArguments))
threadTask1.start()
threadTask2.start()
#Main
while True:
#do stuff
threadRefreshThreads = threading.Thread(name = "RefreshThreads", target = RefreshThreads, args = ())
threadRefreshThreads.start()
from threading import Thread
from time import sleep
def runA():
while a==1:
print('A\n')
sleep(0.5)
if __name__ == "__main__":
a=1
t1 = Thread(target = runA)
t1.setDaemon(True)
t1.start()
sleep(2)
a=0
print(" now def runA stops")
sleep(3)
print("and now def runA continue")
a=1
t1 = Thread(target = runA)
t1.start()
sleep(2)