I have the following code:
s.settimeout( 300 )
while notend:
data = s.recv()
datacheck()
...
The code above is running on a thread so if the user set the notend variable to False the thread will end. But in this case it won't exit immediately, it takes 300 second to exit, because of the recv function. How can exit from this thread?
# Wait all the threads
for thread in threading.enumerate():
if thread is not threading.currentThread():
thread.join()
The problem is that, if you have a blocking socket, recv() will block for the entire duration of the timeout. You could specify a shorter timeout, but that may not be what you want to do (what if the ping really is many thousands of milliseconds?).
Instead of simply trying to recv() all day, you could use a select.poll object to poll the socket of interest at a more frequent rate (smaller timeout), and call recv() only when the polling reports that there is some data to read. In between polls, you could break out of the polling loop if you discover at any point that notend is now false.
If your platform doesn't support poll(), then see if it supports select(). Then, you can use the select.select() function to accomplish more-or-less the same task.
Here's a reference for the described functionality:
http://docs.python.org/2/library/select.html#poll-objects
http://docs.python.org/2/library/select.html#select.select
Simple answer - close s from another thread. The recv() call will then return with an error, so allowing the thread that called it to clean up and terminate.
There is no need for timeouts, select() etc.
Make use of socket.settimeout because recv is a blocking method
Related
I am trying to call a thread I define in a function from another function. Here is the first function, its purpose is to create and start a thread:
def startThread(func):
listen = threading.Thread(target = func)
listen.start()
I am trying to implement a function that will close the thread created in that first function, how should I go about it? I don't know how to successfully pass the thread.
def endThread(thread):
thread.exit()
Thank you!
This problem is almost FAQ material.
To summarise, there is no way to kill a thread from the outside. You can of course pass the thread object to any function you want, but threading library is missing kill and exit calls.
There are more or less two distinct ways around this, depending on what your thread does.
The first method is to make it so that your thread co-operates. This approach is discussed here: Is there any way to kill a Thread in Python? This method adds a check to your thread loop and a way to raise a "stop signal", which will then cause the thread to exit from the inside when detected.
This method works fine if your thread is a relatively busy loop. If it is something that is blocking in IO wait, not so much, as your thread could be blocking in a read call for days or weeks before receiving something and executing the signal check part. Many IO calls accept a timeout value, and if it is acceptable to wait a couple of seconds before your thread exits, you can use this to force the exit check every N seconds without making your thread a busy loop.
The other approach is to replace threads with processes. You can force kill a subprocess. If you can communicate with your main program with queues instead of shared variables, this is not too complicated, either. If your program relies heavily on sharing global variables, this would require a major redesign.
If your program is waiting in IO loops, you need instantaneous termination and you are using shared global variables, then you are somewhat out of luck, as you either need to accept your threads not behaving nicely or you need to redesign some parts of your code to untangle either the IO wait or shared variables.
I am writing a simple multithreaded chat server in Python3. Sometimes i want to interrupt another thread that is waiting in socket.recv() to allow the theread to perform cleanup tasks.
Is calling socket.shutdown(socket.SHUT_RDWR) from another thread safe?
Calling socket.shutdown from another thread is reasonably safe, and, as #kinokikuf mentioned, is an appropriate way to terminate a blocking recv() in another thread, as long as you can guarantee that the socket is not yet closed when the call to shutdown() happens.
If you cannot guarantee that the socket isn't yet closed, you may get an exception when calling shutdown or you might even terminate another unrelated connection if a race-condition happens and the socket handle is reused by the OS!
The Linux man page for the shutdown() system call doesn't explicitly state that blocking recv() calls will be terminated, only that "further receptions and transmissions will be disallowed". But in practice this seems to include blocking recv() calls on the same socket on both Unixoid systems and Windows.
Another way would be to use socket.settimeout() to circumvent the indefinite blocking problem and regularly check (e.g. with global variables) inside the thread if the connection should be closed.
Yet another reasonable way to interrupt the recv() would be to send a non-lethal signal to the thread, which will interrupt the recv() system call, but there seems to be no wrapper for the low-level pthread_kill function in Python and since Python 3.5 system calls will be automatically restarted if interrupted, which makes this option unusable.
When running my code I start a thread that runs for around 50 seconds and does a lot of background stuff. If I run this program and then close it soon after, the stuff still goes on in the background for a while because the thread never dies. How can I kill the thread gracefully in my closeEvent method in my MianWindow class? I've tried setting up a method called exit(), creating a signal 'quitOperation' in the thread in question, and then tried to use
myThread.quitOperation.emit()
I expected that this would call my exit() function in my thread because I have this line in my constructor:
self.quitOperation.connect(self.exit)
However, when I use the first line it breaks, saying that 'myThread' has no attribute 'quitOperation'. Why is this? Is there a better way?
I'm not sure for python, but I assume this myThread.quitOperation.emit() emits a signal for the thread to exit. The point is that while your worker is using the thread and does not return, nor runs QCoreApplication::processEvents(), myThread will never have the chance to actually process your request (this is called thread starvation).
Correct answer may depend on the situation, and the nature of the "stuff" your thread is doing. The most common practice is that the main thread sends a signal to the worker thread where a slot sets a flag. In the blocking process you regularly check this flag. It it is set you stop whatever "stuff" you are doing, tell your worker thread that it can quit (with a signal preferably with queued connection), call a deleteLater() on the worker object itself, and return from any functions you are currently in, so that the thread's event handler can run, and clear your worker object and itself up, the finally quit.
In case your "stuff" is a huge cycle of very fast operation like simple mathematics or directory navigation one-by-one that takes only a few milliseconds each, this will be enough.
In case your "stuff" contain huge blocking parts that you have no control of (an thus you can't place this flag checking call in it), you may need to wait in the main thread until the worker thread quits.
In case you use direct connect to set the flag, or you set it directly, be sure to protect the read/write access of the flag with a QMutex to prevent inconsistent reads, or user a queued connection to ensure single thread access of the flag.
While highly discouraged, optionally you can use QThread's terminate() method to instantaneously kill the thread. You should never do this as it may cause memory leak, heap corruption, resource leaking and any nasty stuff as destructors and clean-up codes will not run, and the execution can be halted at an undesired state.
I have client and server module, each one can be started by a function. I just need to find a way to run booth in parallel which:
in case of an exception in the client/server would stop the other so the test runner would not stay stuck
in case of an exception in client/server would print the exception or propagate it to the runner so I could see it and debug the client/server using the test suite
would preferably use threads for performance reasons
The first tentative with simple threads ended with an ugly os._exit(1) when catching a exception in the run method of the thread (which kills the test runner...) Edit: with the threading package
The second tentative (to try to avoid os._exit()) was with concurrent.futures.ThreadPoolExecutor. It allows to get the exception out of the thread but I still can't find a way to abort the other thread.
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
server_future = executor.submit(server)
client_future = executor.submit(client)
concurrent.futures.wait([server_future, client_future],
return_when=concurrent.futures.FIRST_EXCEPTION)
if client_future.done() && client_future.exception():
# we can handle the client exception here
# but how to stop the server from waiting the client?
# also, raise is blocking
if server_future.done() && server_future.exception():
# same here
Is there a way to achieve this with threads?
If not with threads, is there a simple way to test a client server app at all? (I think the two first requirements are enough to have a usable solution)
Edit: The client or the server would be blocked on an accept() or a receive() call so I can't periodically pool a flag a decide to exit.(one of classic method to stop a thread)
You can use the threading package. Be aware though that force killing thread is not a good idea, as discussed here. It seems there is no official way to kill Thread in Python, but you can follow one of the example given on the linked post.
Now you need to wait for one thread to exit before stopping the other one, avoiding your test runner to be stuck. You can use Threads wrapping your server/client launch, and have your main Thread waiting for either client/server Thread to exit before killing the other one.
You can define your client/server Thread like this:
# Server thread (replace
class testServerThread (threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
# Do stuff if required
def run(self):
try:
startServer() # Or startClient() for your client thread
except: Exception
# Print your exception here, so you can debug
Then, start both client and server thread, and wait for one of them to exit. Once one of them is not alive anymore, you can kill the other and continue on testing.
# Create and start client/server
serverThread = testServerThread ()
clientThread = testClientThread ()
serverThread.start()
clientThread.start()
# Wait at most 5 seconds for them to exit, and loop if they're still both alive
while(serverThread.is_alive() and clientThread.is_alive()):
serverThread.join(5)
clientThread.join(5)
# Either client or server exited. Kill the other one.
# Note: the kill function you'll have to define yourself, as said above
if(serverThread.is_alive()):
serverThread.kill()
if(clientThread.islive()):
clientThread.kill()
# Done! Your Test runner can continue its work
The central piece of code is the join() function:
Wait until the thread terminates. This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception –, or until the optional timeout occurs.
So in our case, it will wait 5 seconds for the client and 5 seconds for the server, and if both of them are still alive afterward it will loop again. Whenever one of them exit, the loop will stop, and the remaining thread will be killed.
I am trying wait for any of multiple multiprocessing events at the same time, so I came up with code like this:
if e1.wait(timeout) or e2.wait(timeout):
# this part will be reached if either of both
# events is set or the wait timed out
It works like the comment says. But how does this work? Is the if polling bot methods all the time? Or is it called as soon as one event gets set?
Bonus question: Is there some clever way to adjust the code to wait for any number of events, i.e. a list of events? if True in [e1.wait(timeout),e2.wait(timeout)] does not work as expected.
It only waits for the first one. This is due to python's support of short circuiting.
Wait on a thread or process is blocking, so it will block the current thread for going future until the timeout or the thread has finished. The semantics of if in Python is short circuit, which means that if the first one returns true, then the second one will not be called - simonzack said.
Waiting on a number of threads would be kinda hard to implement and maintain for a variety of threads. I would suggest you to use Message passing, and get each process to send a message to a Queue when it is finished. This way you could just check if the queue was of ´len(n)´, where ´n´ is the number of threads/processes. see more here Queues in multiprocessing