I have a program using a thread. When my program is closed, my thread is still running and that's normal. I would like to know how my thread can detect that the main program is terminated; by itself ONLY. How would I do that?
My thread is in an infinite loop and process many object in a Queue. I can't define my thread as a daemon, else I can lose some data at the end of the main program. I don't want that my main program set a boolean value when it closed.
If you can get a handle to the main thread, you can call is_alive() on it.
Alternatively, you can call threading.enumerate() to get a list of all currently living threads, and check to see if the main thread is in there.
Or if even that is impossible, then you might be able to check to see if the child thread is the only remaining non-daemon thread.
Would it work if your manager tracked how many open threads there were, then the children killed themselves when starved of input? So the parent would start pushing data on to the queue, and the workers would consume data from the queue. If a worker found nothing on the queue for a certain timeout period, it would kill itself. The main thread would then track how many workers were operating and periodically start new workers if the number of active workers were under a given threshold.
Related
I am currently building a python app which should trigger functions at given timestamps entered by the user (not entered in chronological order).
I ran into a problem because I don't want my program to be busy-waiting checking if a new timestamp has been entered that must be added to the timer queue but also not creating a whole bunch of threads for every time a new timestamp is creating with its single purpose waiting for that timestamp.
What I thought of is putting it all together in one thread and doing something like an interruptable sleep, but I can't think of another way besides this:
while timer_not_depleted:
sleep(1)
if something_happened:
break
which is essentially busy-waiting.
So any suggestions on realizing an interruptable sleep?
Your intuition of using threads is correct. The following master-worker construction can work:
The master thread spawns a worker thread that waits for "jobs";
The two threads share a Queue - whenever a new job needs to be scheduled, the master thread puts a job specification into the queue;
Meanwhile, the worker thread does the following:
Maintain a separate list of future jobs to run and keep track of how long to keep sleeping until the next job runs;
Continue listening to new jobs by calling Queue.get(block=True, timeout=<time-to-next-job>);
In this case, if no new jobs are scheduled until the timeout, Queue.get will raise Empty exception - and at this point the worker thread should run the scheduled function and get back to polling. If a new job is scheduled in the meantime, Queue.get returns the new job, such that you can update the timeout value and then get back to waiting.
I'd like to suggest select.
Call it with a timeout equal to the delay to the nearest event (heap queue is a good data structure to maintain a queue of future timestamps) and provide a socket (as an item in the rlist arg), where your program listens on for updates from the user.
The select call returns when the socket has incoming data or when the timeout has occurred.
I'm working on a python application, in which the main thread creates an object, say x, of a particular class.
Then it starts one thread which starts the execution in one of the methods of this object x. The method has a while True: loop, so its infinite.
Then it starts another thread which starts the execution in another method of the same object x. This method also has a while True: infinite loop.
I have made both the threads as daemon by calling t1.setDaemon(True), but it seems both stop execution once the main thread exits.
How do I keep the children alive after the parent thread is finished?
Or should I change my design to use a cron job or process fork?
The documentation says
A thread can be flagged as a "daemon thread". The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property.
If you want to keep your program running, you must have at least one non daemon thread.
When running my code I start a thread that runs for around 50 seconds and does a lot of background stuff. If I run this program and then close it soon after, the stuff still goes on in the background for a while because the thread never dies. How can I kill the thread gracefully in my closeEvent method in my MianWindow class? I've tried setting up a method called exit(), creating a signal 'quitOperation' in the thread in question, and then tried to use
myThread.quitOperation.emit()
I expected that this would call my exit() function in my thread because I have this line in my constructor:
self.quitOperation.connect(self.exit)
However, when I use the first line it breaks, saying that 'myThread' has no attribute 'quitOperation'. Why is this? Is there a better way?
I'm not sure for python, but I assume this myThread.quitOperation.emit() emits a signal for the thread to exit. The point is that while your worker is using the thread and does not return, nor runs QCoreApplication::processEvents(), myThread will never have the chance to actually process your request (this is called thread starvation).
Correct answer may depend on the situation, and the nature of the "stuff" your thread is doing. The most common practice is that the main thread sends a signal to the worker thread where a slot sets a flag. In the blocking process you regularly check this flag. It it is set you stop whatever "stuff" you are doing, tell your worker thread that it can quit (with a signal preferably with queued connection), call a deleteLater() on the worker object itself, and return from any functions you are currently in, so that the thread's event handler can run, and clear your worker object and itself up, the finally quit.
In case your "stuff" is a huge cycle of very fast operation like simple mathematics or directory navigation one-by-one that takes only a few milliseconds each, this will be enough.
In case your "stuff" contain huge blocking parts that you have no control of (an thus you can't place this flag checking call in it), you may need to wait in the main thread until the worker thread quits.
In case you use direct connect to set the flag, or you set it directly, be sure to protect the read/write access of the flag with a QMutex to prevent inconsistent reads, or user a queued connection to ensure single thread access of the flag.
While highly discouraged, optionally you can use QThread's terminate() method to instantaneously kill the thread. You should never do this as it may cause memory leak, heap corruption, resource leaking and any nasty stuff as destructors and clean-up codes will not run, and the execution can be halted at an undesired state.
I have an application (actually a plugin for another application) that manages threads to communicate with external sensor devices. The external sensors send events to the application, but the application may also send actions to the sensors. There are several types of devices and each has unique qualities (temperature, Pressure, etc.) that require special coding. All communications with the sensor devices is over IP.
In the applications, I create a thread for each instance of a sensor. This is an example of the code
self.phThreadDict[phDevId] = tempsensor(self, phDevId, phIpAddr, phIpPort, phSerial, self.triggerDict)
self.phThreadDict[phDevId].start()
In each thread I setup callback handlers for events sent by the sensor and then go into a loop at the end.
while not self.shutdown:
self.plugin.sleep(0.5)
The threads then handle incoming events and make calls into the main thread, or the actual program that spawned the main thread. All of this works quite well.
But, at times I also need to send requests to a specific sensor. Methods are defined in each thread for that purpose and I call those methods from the main thread. For example:
self.phThreadDict[dev.id].writeDigitalOutput(textLine, lcdMessage)
This also works, but I believe the code is actually executed in the main thread rather than in the thread specific to the sensor.
My question is: What options do I have for passing work to the specific target thread and having the thread execute the work and then return success or fail?
Expanding a bit on Thomas Orozco's spot-on comments,
self.phThreadDict[dev.id].writeDigitalOutput(textLine, lcdMessage)
is executed in whichever thread runs it. If you run it from the main thread, then the main thread will do all of it. If from some other thread, then that thread will run it.
In addition to a Queue per thread, for the threads to receive descriptions of work items to process, you also want a single Queue for threads to put results on (you can also use another Queue per thread for this, but that's overkill).
The main thread will pull results off the latter Queue. Note that you can - and it's very common to do so - put tuples on Queues. So, for example, on the talk-back-to-the-main-thread Queue threads will likely put tuples of the form:
(result, my_thread_id, original_work_description)
That's enough to figure out which thread returned what result in response to which work item. Maybe you don't need all of that. Maybe you need more than that. Can't guess ;-)
Indeed, this is executing code in the main thread.
Use queues, that's what they're meant for (task synchronization and message passing between threads).
Use one queue per sensor manager thread.
Your sensor manager threads should be getting items from the queue instead of sleeping (this is a blocking call).
Your "main" thread should be putting items in the queue instead of running functions (this is generally a non-blocking call).
All you need to do is define a message format that lets the main thread tell the manager threads what functions to execute and what arguments to use.
I have a multiprocessing application where the parent process creates a queue and passes it to worker processes. All processes use this queue for creating a queuehandler for the purpose of logging. There is a worker process reading from this queue and doing logging.
The worker processes continuously check if parent is alive or not. The problem is that when I kill the parent process from command line, all workers are killed except for one. The logger process also terminates. I don't know why one process keeps executing. Is it because of any locks etc in queue? How to properly exit in this scenario? I am using
sys.exit(0)
for exiting.
I would use sys.exit(0) only if there is no other chance. It's always better to cleanly finish each thread / process. You will have some while loop in your Process. So just do break there, so that it can come to an end.
Tidy up before you leave, i.e., release all handles of external resources, e.g., files, sockets, pipes.
Somewhere in these handles might be the reason for the behavior you see.