I am trying to learn how mutex works in multithread application, but I have some doubts. In my case I have thread which read from device, thread which write to device and synchronization thread. In sync thread I have a timer in this form:
def CheckConnection(self):
. . .
threading.Timer(1, self.CheckConnection).start()
This timer runs periodically CheckConnection function.
Could you tell me:
When thread is locked (acquired) it means that thread stops execution and waits? this is sth like pause?
What will happen when I start synchronization thread, my checkConnection function executes and timer starts, after it synchronization-thread is locked... This will stop the timer and execution of CheckConnection function?
The mymutex.acquire() call will block the thread calling it until the mutex is available, then lock the mutex (so blocking any other thread which calls mymutex.acquire()), then return so the thread can continue execution. The call to mymutex.release() releases/unlocks the mutex and the oldest thread blocked on mymutex.acquire() gets to lock the mutex and return, i.e. unblocks this other thread.
Is self.CheckConnection the same function as you are defining in def CheckConnection()? So you're trying to get the CheckConnection function called every 1 (in your case) second?. There is a nice description of how to do this in the top answer here Python threading.timer - repeat function every 'n' seconds
Each thread using the mutex should be doing something like:
themutex.acquire()
try:
print('Do some stuff')
finally:
themutex.release()
In general, the code between acquire and release, i.e. 'do some stuff', should be as quick as possible because the maximum duration of any of these sections of code where mutex has been acquired is also the maximum time another thread will be blocked (unless several threads are blocked, when obviously the maximum delay can get even longer).
Related
I have a query. I have seen examples where developers write something like the code as follows:
import threading
def do_something():
return true
t = threading.Thread(target=do_something)
t.start()
t.join()
I know that join() signals the interpreter to wait till the thread is completely executed. But what if I do not write t.join()? Will the thread get closed automatically and will it be reused later?
Please let me know the answer. It's my first attempt at creating a multi-threaded application in Python 3.5.0.
A Python thread is just a regular OS thread. If you don't join it, it still keeps running concurrently with the current thread. It will eventually die, when the target function completes or raises an exception. No such thing as "thread reuse" exists, once it's dead it rests in peace.
Unless the thread is a "daemon thread" (via a constructor argument daemon or assigning the daemon property) it will be implicitly joined for before the program exits, otherwise, it is killed abruptly.
One thing to remember when writing multithreading programs in Python, is that they only have limited use due to infamous Global interpreter lock. In short, using threads won't make your CPU-intensive program any faster. They can be useful only when you perform something involving waiting (e.g. you wait for certain file system event to happen in a thread).
The join part means the main program will wait for the thread to end before continuing. Without join, the main program will end and the thread will continue.
Now if you set the daemon parameter to "True", it means the thread will depends on the main program, and it will ends if the main program ends before.
Here is an example to understand better :
import threading
import time
def do_something():
time.sleep(2)
print("do_something")
return True
t = threading.Thread(target=do_something)
t.daemon = True # without the daemon parameter, the function in parallel will continue even your main program ends
t.start()
t.join() # with this, the main program will wait until the thread ends
print("end of main program")
no daemon, no join:
end of main program
do_something
daemon only:
end of main program
join only:
do_something
end of main program
daemon and join:
do_something
end of main program
# Note : in this case the daemon parameter is useless
Without join(), non-daemon threads are running and are completed with the main thread concurrently.
Without join(), daemon threads are running with the main thread concurrently and when the main thread is completed, the daemon threads are exited without completed if the daemon threads are still running.
You can see my answer in this post explaining about it in detail.
I am trying to call a thread I define in a function from another function. Here is the first function, its purpose is to create and start a thread:
def startThread(func):
listen = threading.Thread(target = func)
listen.start()
I am trying to implement a function that will close the thread created in that first function, how should I go about it? I don't know how to successfully pass the thread.
def endThread(thread):
thread.exit()
Thank you!
This problem is almost FAQ material.
To summarise, there is no way to kill a thread from the outside. You can of course pass the thread object to any function you want, but threading library is missing kill and exit calls.
There are more or less two distinct ways around this, depending on what your thread does.
The first method is to make it so that your thread co-operates. This approach is discussed here: Is there any way to kill a Thread in Python? This method adds a check to your thread loop and a way to raise a "stop signal", which will then cause the thread to exit from the inside when detected.
This method works fine if your thread is a relatively busy loop. If it is something that is blocking in IO wait, not so much, as your thread could be blocking in a read call for days or weeks before receiving something and executing the signal check part. Many IO calls accept a timeout value, and if it is acceptable to wait a couple of seconds before your thread exits, you can use this to force the exit check every N seconds without making your thread a busy loop.
The other approach is to replace threads with processes. You can force kill a subprocess. If you can communicate with your main program with queues instead of shared variables, this is not too complicated, either. If your program relies heavily on sharing global variables, this would require a major redesign.
If your program is waiting in IO loops, you need instantaneous termination and you are using shared global variables, then you are somewhat out of luck, as you either need to accept your threads not behaving nicely or you need to redesign some parts of your code to untangle either the IO wait or shared variables.
Please consider this code:
import threading
def printer():
for i in range(2):
with lock:
print ['foo', 'bar', 'baz']
def main():
global lock
lock = threading.Lock()
threads = [threading.Thread(target=printer) for x in xrange(2)]
for t in threads:
t.start()
t.join()
main()
I can understand this code and it is clear: We create two threads and we run them sequentially - we run second thread only when first thread is finished. Ok, now consider another variant:
import threading
def printer():
for i in range(2):
with lock:
print ['foo', 'bar', 'baz']
def main():
global lock
lock = threading.Lock()
threads = [threading.Thread(target=printer) for x in xrange(2)]
for t in threads:
t.start()
for t in threads:
t.join()
main()
What happens here? Ok, we run them in parallel, but what is the purpose of make main thread waiting for child threads in second variant? How it can influence on the output?
In the second variant, the ordering of execution is much less defined.
The lock is released each time through the loop in printer. In both variants, you have two threads and two loops within a thread.
In the first variant, since only one thread runs at a time, you know the total ordering.
In the second variant, each time the lock is released, the thread running may change.
So you might get
thread 1 loop 1
thread 1 loop 2
thread 2 loop 1
thread 2 loop 2
or perhaps
* thread 2 loop 1
* thread 1 loop 1
* thread 1 loop 2
* thread 2 loop 2
The only constraint is that loop1 within a given thread runs before loop 2 within that thread and that the two print statements come together since the lock is held for both of them.
In this particular case I'm not sure the call to t.join() in the second variant has an observable effect. It guarantees that the main thread will be the last thread to end, but I'm not sure that in this code you can observe that in any way. In more complex code, joining the threads can be important so that cleanup actions are only performed after all threads terminate. This can also be very important if you have daemon threads, because the entire program will terminate when all non-daemon threads terminate.
To better understand the multithreading in python, you need to first understand the relationship between the main thread and the children threads.
The main thread is the entry of the program, it is created by your system when you run your script. For example, in your script, the main function is run in the main thread.
While the children thread is created by your main thread when you instanate the Thread class.
The most important thing is how the main thread controls the children thread. Basically, the instance of the Thread is everything that the main thread know about and control over this child thread. At the time when a child thread is created, this child thread does not run immediately, until the main thread call start function on this thread instance. After the start the child thread, you can assume that the main thread and the child thread is running parallelly now.
But one more important thing is how the main thread knows that the task of child thread is done. Though the main thread knows nothing about how the task is done by the child thread, it does be aware of the running status of the child thread. Thread.is_alive can check the status of a thread by the main thread. In pratice, the Thread.join function is always used to tell the main thread wait until the child thread is done. This function will block the main thread.
Okay, let's examine the two script you are confused with. For the first script:
for t in threads:
t.start()
t.join()
The children threads in the loop are started and then joined one by one. Note that start does not block main thread, while join will block the main thread wait until this child thread is done. Thus they are running sequentially.
While for the second script:
for t in threads:
t.start()
for t in threads:
t.join()
All children threads are started in the first loop. As the Thread.start function will not block the main thread, all children threadings are running parallelly after the first loop. In the second loop, the main thread will wait for the task done of each child thread one by one.
Now I think you should notice the difference between these two script: in the first one, children threads running one by one, while in the second script, they are running simultaneously.
There are other useful topics for the python threading:
(1) How to handle the Keyboard Interrupt Exception, e.g., when I want to terminate the program by Ctrl-C? Only the main thread will receive the exception, you have to handle the termination of children threads.
(2) Multithreading vs Multiprocessing. Although we are saying that threading is parallel, it is not the real parallel in CPU level. So if your application is CPU intensive, try multiprocessing, and if your application is I/O intensive, multithreading maybe sufficient.
By the way, read through the documentation of python threading section and try some code may help you understand it.
Hope this would be helpful. Thanks.
I have a main thread and another thread which starts after threading.Timer(1,success).start() calls it.
In the defined function success I need to kill the whole python script, I tried sys.exit() but that only ends the thread. I can't signal the main thread as the reason the timer went off was because the main thread took too long to respond, so there's no guarantee the signal would be read by the main thread.
I considered using os.exit() which works, but it's messy as the script is reloaded after a second by another program and memory fills up.
You can join the created thread with a timeout:
join(timeout=None)
Thus, it will not wait for completion.
here's some example code
while True: #main-loop
if command_received:
thread = Thread(target = doItNOW)
thread.start()
......
def doItNOW():
some_blocking_operations()
my problem is I need "some_blocking_operations" to start IMMEDIATELY (as soon as command_received is True).
but since they're blocking i can't execute them on my main loop
and i can't change "some_blocking_operations" to be non-blocking either
for "IMMEDIATELY" i mean as soon as possible, not more than 10ms delay.
(i once got a whole second of delay).
if it's not possible, a constant delay would also be acceptable. (but it MUST be constant. with very few milliseconds of error)
i'm currently working on a linux system (Ubuntu, but it may be another one in the future. always linux)
a python solution would be amazing.. but a different one would be better than nothing
any ideas?
thanks in advance
from threading import Thread
class worker(Thread):
def __init__(self, someParameter=True):
Thread.__init__(self)
# This is how you "send" parameters/variables
# into the thread on start-up that the thread can use.
# This is just an example in case you need it.
self.someVariable = someParameter
self.start() # Note: This makes the thread self-starting,
# you could also call .start() in your main loop.
def run():
# And this is how you use the variable/parameter
# that you passed on when you created the thread.
if self.someVariable is True:
some_blocking_operations()
while True: #main-loop
if command_received:
worker()
This is a non-blocking execution of some_blocking_operations() in a threaded manner. I'm not sure if what you're looking for is to actually wait for the thread to finish or not, or if you even care?
If all you want to do is wait for a "command" to be received and then to execute the blocking operations without waiting for it, verifying that it completes, then this should work for you.
Python mechanics of threading
Python will only run in one CPU core, what you're doing here is simply running multiple executions on overlapping clock invervals in the CPU. Meaning every other cycle in the CPU an execution will be made in the main thread, and the other your blocking call will get a chance to run a execution. They won't actually run in parallel.
There are are some "You can, but..." threads.. Like this one:
is python capable of running on multiple cores?