I'm running the following python code:
import threading
import multiprocessing
def forever_print():
while True:
print("")
def main():
t = threading.Thread(target=forever_print)
t.start()
return
if __name__=='__main__':
p = multiprocessing.Process(target=main)
p.start()
p.join()
print("main process on control")
It terminates.
When I unwrapped main from the new process, and just ran it directly, like this:
if name == '__main__':
main()
The script went on forever, as I thought it should. Am I wrong to assume that, given that t is a non-daemon process, p shouldn't halt in the first case?
I basically set up this little test because i've been developing an app in which threads are spawned inside subprocesses, and it's been showing some weird behaviour (sometimes it terminates properly, sometimes it doesn't). I guess what I wanted to know, in a broader sense, is if there is some sort of "gotcha" when mixing these two python libs.
My running environment: python 2.7 # Ubuntu 14.04 LTS
For now, threads created by multiprocessing worker processes act like daemon threads with respect to process termination: the worker process exits without waiting for the threads it created to terminate. This is due to worker processes using os._exit() to shut down, which skips most normal shutdown processing (and in particular skips the normal exit processing code (sys.exit()) that .join()'s non-daemon threading.Threads).
The easiest workaround is for worker processes to explicitly .join() the non-daemon threads they create.
There's an open bug report about this behavior, but it hasn't made much progress: http://bugs.python.org/issue18966
You need to call t.join() in your main function.
As your main function returns, the process gets terminated with both its threads.
p.join() blocks the main thread waiting for the spawned process to end. Your spawned process then, creates a thread but does not wait for it to end. It returns immediately thus trashing the thread itself.
If Threads share memory, Processes don't. Therefore, the Thread you create in the newly spawned process remains relegated to that process. The parent process is not aware of it.
The gotcha is that the multiprocessing machinery calls os._exit() after your target function exits, which violently kills the child process, even if it has background threads running.
The code for Process.start() looks like this:
def start(self):
'''
Start child process
'''
assert self._popen is None, 'cannot start a process twice'
assert self._parent_pid == os.getpid(), \
'can only start a process object created by current process'
assert not _current_process._daemonic, \
'daemonic processes are not allowed to have children'
_cleanup()
if self._Popen is not None:
Popen = self._Popen
else:
from .forking import Popen
self._popen = Popen(self)
_current_process._children.add(self)
Popen.__init__ looks like this:
def __init__(self, process_obj):
sys.stdout.flush()
sys.stderr.flush()
self.returncode = None
self.pid = os.fork() # This forks a new process
if self.pid == 0: # This if block runs in the new process
if 'random' in sys.modules:
import random
random.seed()
code = process_obj._bootstrap() # This calls your target function
sys.stdout.flush()
sys.stderr.flush()
os._exit(code) # Violent death of the child process happens here
The _bootstrap method is the one that actually executes the target function you passed passed to the Process object. In your case, that's main. main returns right after you start your background thread, even though the process doesn't exit, because there's still a non-daemon thread running.
However, as soon execution hits os._exit(code), the child process is killed, regardless of any non-daemon threads still executing.
Related
I'm fiddling with multiprocessing and signal.
I'm creating a pool, and have the workers catch SIGTERMs.
With no apparent reasons, I observe that the subprocesses randomly receive SIGTERMs.
Here is a MWE:
import multiprocessing as mp
import signal
import os
import time
def start_process():
print("Starting process #{}".format(os.getpid()))
def sigterm_handler(signo, _frame):
print("Process #{} received a SIGTERM".format(os.getpid()))
def worker(i):
time.sleep(1)
signal.signal(signal.SIGTERM, sigterm_handler)
while True:
with mp.Pool(initializer=start_process) as pool:
pool.map(worker, range(10))
time.sleep(2)
Output:
Starting process #7735
Starting process #7736
Starting process #7737
Starting process #7738
Starting process #7739
Starting process #7740
Starting process #7741
Starting process #7742
Job done.
Starting process #7746
Starting process #7747
Starting process #7748
Starting process #7749
Starting process #7750
Starting process #7751
Starting process #7752
Starting process #7753
Process #7748 received a SIGTERM
Process #7746 received a SIGTERM
Job done.
Starting process #7757
Starting process #7758
Starting process #7759
Starting process #7760
Starting process #7761
Starting process #7762
Starting process #7763
Starting process #7764
As you can see, that looks unpredictable.
So, where do these SIGTERMs come from?
Is this normal?
Am I guaranteed that the workers will finish their job?
And in the end, is it OK to have the subprocesses capture SIGTERMs?
It's normal and can happen while your pool is executing __exit__ upon leaving the context-manager.
Since the workers have finished their jobs at that point, there's nothing to worry about.
The pool itself causes the SIGTERM for workers which don't have an exitcode available when the
pool checks for it. This gets triggered in the Pool._terminate_pool-method (Python 3.7.1):
# Terminate workers which haven't already finished.
if pool and hasattr(pool[0], 'terminate'):
util.debug('terminating workers')
for p in pool:
if p.exitcode is None:
p.terminate()
The pool-workers will get joined a few lines later:
if pool and hasattr(pool[0], 'terminate'):
util.debug('joining pool workers')
for p in pool:
if p.is_alive():
# worker has not yet exited
util.debug('cleaning up worker %d' % p.pid)
p.join()
In a scenario where you would call pool.terminate() explicitly while your workers
are still running (for example you are using pool.map_async and then use pool.terminate()),
your application would deadlock waiting on the p.join() (unless you let your sigterm_handler eventually call sys.exit()).
Better don't mess with signal handlers if you don't have to.
I think it normal, but can't say anything about the random message printing. You can get more info, insert this in the main:
mp.log_to_stderr(logging.DEBUG)
and change the start_process():
def start_process():
proc= mp.current_process()
print("Starting process #{}, its name is {}".format(os.getpid(),proc.name))
The answer might be right in front of me on the link below but I still don't understand. I'm sure after someone explains this to me, Darwin will be making a call to me.
The example is at this link here, although I've made some changes to try to experiment and help my understanding.
Here's the code:
import multiprocessing
import time
import sys
def daemon():
p = multiprocessing.current_process()
print 'Starting: ', p.name, p.pid
sys.stdout.flush()
time.sleep(2)
print 'Exiting: ', p.name, p.pid
sys.stdout.flush()
def non_daemon():
p = multiprocessing.current_process()
print 'Starting: ', p.name, p.pid
sys.stdout.flush()
time.sleep(6)
print 'Exiting: ', p.name, p.pid
sys.stdout.flush()
if __name__ == '__main__':
d = multiprocessing.Process(name='daemon', target=daemon)
d.daemon = True
n = multiprocessing.Process(name='non-daemon', target=non_daemon)
n.daemon = False
d.start()
time.sleep(1)
n.start()
# d.join()
And the output of the code is:
Starting: daemon 6173
Starting: non-daemon 6174
Exiting: non-daemon 6174
If the join() at the end is uncommented, then the output is:
Starting: daemon 6247
Starting: non-daemon 6248
Exiting: daemon 6247
Exiting: non-daemon 6248
I'm confused b/c the sleep of the daemon is 2 sec, whereas the non-daemon is 6 sec. Why doesn't it print out the "Exiting" message in the first case? The daemon should have woken up before the non-daemon and printed the message.
The explanation from the site is as such:
The output does not include the “Exiting” message from the daemon
process, since all of the non-daemon processes (including the main
program) exit before the daemon process wakes up from its 2 second
sleep.
but I changed it such that the daemon should have woken up before the non-daemon does. What am I missing here? Thanks in advance for your help.
EDIT: Forgot to mention I'm using python 2.7 but apparently this problem is also in python 3.x
This was a fun one to track down. The docs are somewhat misleading, in that they describe the non-daemon processes as if they are all equivalent; the existence of any non-daemon process means the process "family" is alive. But that's not how it's implemented. The parent process is "more equal" than others; multiprocessing registers an atexit handler that does the following:
for p in active_children():
if p.daemon:
info('calling terminate() for daemon %s', p.name)
p._popen.terminate()
for p in active_children():
info('calling join() for process %s', p.name)
p.join()
So when the main process finishes, it first terminates all daemon child processes, then joins all child processes to wait on non-daemon children and clean up resources from daemon children.
Because it performs cleanup in this order, a moment after your non-daemon Process starts, the main process begins cleanup and forcibly terminates the daemon Process.
Note that fixing this can be as simple as joining the non-daemon process manually, not just joining the daemon process (which defeats the whole point of a daemon completely); that prevents the atexit handler from being called, delaying the cleanup that would terminate the daemon child.
It's arguably a bug (one that seems to exist up through 3.5.1; I reproed myself), but whether it's a behavior bug or a docs bug is arguable.
I am using multiprocessing module's Process class to spawn multiple processes, those processes execute some script and then dies.What I wanted, a timeout to be applied on each process, so that a process would die if cant execute in time timeout. I am using join(timeout) on Process objects.
Since the join() function doesn;t kill the process, it just blocks the process until it finishes
Now my question : Is there any side-effects of using join() with timeout ..like, would the processes be cleaned automatically, after the main process dies ?? or I have to kill those processes manually ??
I am a newbie to python and its multiprocessing module, please be patient.
My Code, which is creating Processes in a for loop ::
q = Queue()
jobs = [
Process(
target=get_current_value,
args=(q,),
kwargs=
{
'device': device,
'service_list': service_list,
'data_source_list': data_source_list
}
) for device in device_list
]
for j in jobs:
j.start()
for k in jobs:
k.join()
The timeout argument just tells join how long to wait for the Process to exit before giving up. If timeout expires, the Process does not exit; the join call simply unblocks. If you want to end your workers when the timeout expires, you need to do so manually. You can either use terminate, as suggested by wRAR, to uncleanly shut things down, or use some other signaling mechanism to tell the children to shutdown cleanly:
p = Process(target=worker, args=(queue,))
p.start()
p.join(50)
if p.isalive(): # join timed out without the process actually finishing
#p.terminate() # unclean shutdown
If you don't want to use terminate, the alternative approach is really dependent on what the workers are doing. If they're consuming from a queue, you can use a sentinel:
def worker(queue):
for item in iter(queue.get, None): # None will break the loop
# Do normal work
if __name__ == "__main__":
queue = multiprocessing.Queue()
p = multiprocessing.Process(target=worker, args=(queue,))
p.start()
# Do normal work here
# Time to shut down
queue.put(None)
Or you could use an Event, if you're doing some other operation in a loop:
def worker(event):
while not event.is_set():
# Do work here
if __name__ == "__main__":
event= multiprocessing.Event()
p = multiprocessing.Process(target=worker, args=(event,))
p.start()
# Do normal work here
# Time to shut down
event.set()
Using terminate could be just fine, though, unless your child processes are using resources that could be corrupted if the process is unexpectedly shut down (like writing to a file or db, or holding a lock). If you're just doing some calculations in the worker, using terminate won't hurt anything.
join() does nothing with the child process. If you really want to terminate worker process in a non-clean manner you should use terminate() (you should understand the consequences).
If you want children to be terminated when the main process exits you should set daemon attribute on them.
So I essentially have a case like this where in my main script I have
command = 'blender -b ' + settings.BLENDER_ROOT + 'uploadedFileCheck.blend -P ' + settings.BLENDER_ROOT + 'uploadedFileCheck.py -noaudio'
process = Popen(command.split(' ') ,stdout=PIPE, stderr=PIPE)
out, err = process.communicate()
And in the subprocess script uploadedFileCheck.py I have the line
exportFile(fileIn, fileOut)
Thread(target=myfunction).start()
So I expect the subprocess to be finished, or at least to return to out, err after the exportFile() call, but it seems it's waiting for the Thread to finish as well. Does anyone understand this behavior?
Also, in case you're wondering, I'm calling that other python file as a subprocess because the main script is in python2 and that script (blender) is in python3, but that's irrelevant (and can't change)
A process won't exit until all its non-daemon threads have exited. By default, Thread objects in Python are created as non-daemon threads. If you want your script to exit as soon as the main thread is done, rather than waiting for the spawned thread to finish, set the daemon flag on the Thread object to True prior to starting it:
t = Thread(target=myfunction)
t.daemon = True
t.start()
Note that this will kill the daemon thread in a non-graceful way, without any cleanup occuring. If you're doing any kind of work in that thread that needs to be cleaned up, you should consider an approach where you signal the thread to shut itself down, instead.
I am reading The Python Standard Library by Example and get confused when I arrived page 509.
Up to this point, the example programs have implicitly waited to exit until all threads have completed their work. Programs sometimes spawn a thread as a daemon that runs without blocking the main program from exiting.
but after I run some codes, I get result that is opposite. The code is like this:
#!/usr/bin/env python
# encoding: utf-8
#
# Copyright (c) 2008 Doug Hellmann All rights reserved.
#
"""Creating and waiting for a thread.
"""
#end_pymotw_header
import threading
import time
def worker():
"""thread worker function"""
print 'Worker'
# time.sleep(10000)
return
threads = []
for i in range(5):
t = threading.Thread(target=worker)
threads.append(t)
t.start()
print "main Exit"
and sometime the result is this:
Worker
Worker
WorkerWorker
main Exit
Worker
So I want to ask when will main thread exit in python after it starts several thread?
The main thread will exit whenever it is finished executing all the code in your script that is not started in a separate thread.
Since the t.start() will start the thread and then return to the main thread, the main thread will simply continue to execute until it reaches the bottom of your script and then exit.
Since you started the other threads in a non-daemon mode, they will continue running until they are finished.
If you wanted the main thread to not exit until all the threads have finished, you should explicitly join them to the main thread. The join will cause the thread calling join to wait until the the thread being joined is finished.
for i in range(5):
threads[i].join()
print "main Exit"
As #codesparkle pointed out, the more Pythonic way to write this would be to skip the index variable entirely.
for thread in threads:
thread.join()
print "main Exit"
According to the threading docs:
The entire Python program exits when only daemon threads are left
This agrees with the quote you give, but the slight difference in wording shows the result you get. The 'main thread' exits when you would expect it to. Note that the worker threads keep running at this point - you can see this in the test output you give. So, the main thread has finished, but the whole process is still running because there are other threads still running.
The difference is that if some of those worker threads were daemonised, they would be forcibly killed when the last non-daemon thread finished. If all of the workers were daemon threads, then the entire process would finish - and you would be back at your systems shell prompt - very soon after you print 'main exit', and it would be very rare (though not impossible, owing to race conditions) for any worker to print after that.
Your main thread will exit as soos as for loop completes its execution. Your main thread is starting new asynchronous threads. Which means that it will not wait untill new thread finishes its execution. So in your case main thread will start 5 threads in parallel and exit itself.
Note that Main does not exit when you print main Exit, but after it. Consider this program:
import threading
import time
def worker():
"""thread worker function"""
print 'Worker'
time.sleep(1)
print 'Done'
return
class Exiter(object):
def __init__(self):
self.a = 5.0
print 'I am alive'
def __del__(self):
print 'I am dying'
exiter = Exiter()
threads = []
for i in range(5):
t = threading.Thread(target=worker)
threads.append(t)
t.start()
print "main Exit"
I have created an object whose sole purpose is to print "I am dying" when it is being finalised. I am not deleting it explicitly anywhere, so it will only die when the main thread finishes, that is, when Python starts to kill everything to return memory to the OS.
If you run this you will see that workers are working when the main thread is done, but the objects are still alive. I am dying always comes after all the workers have finished their job.
For example as follow:
class ThreadA(Thread):
def __init__(self, mt):
Thread.__init__(self)
self.mt = mt
def run(self):
print 'T1: sleeping...'
time.sleep(4)
print 'current thread is ', self.isAlive()
print 'main thread is ', self.mt.isAlive()
print 'T1: raising...'
if __name__ == '__main__':
mt = threading.currentThread()
ta = ThreadA(mt)
ta.start()
logging.debug('main end')
>
T1: sleeping...
(MainThread) main end
current thread is True
main thread is False
T1: raising...
you can see the main thread active state is false?