I am writing a simple multithreaded chat server in Python3. Sometimes i want to interrupt another thread that is waiting in socket.recv() to allow the theread to perform cleanup tasks.
Is calling socket.shutdown(socket.SHUT_RDWR) from another thread safe?
Calling socket.shutdown from another thread is reasonably safe, and, as #kinokikuf mentioned, is an appropriate way to terminate a blocking recv() in another thread, as long as you can guarantee that the socket is not yet closed when the call to shutdown() happens.
If you cannot guarantee that the socket isn't yet closed, you may get an exception when calling shutdown or you might even terminate another unrelated connection if a race-condition happens and the socket handle is reused by the OS!
The Linux man page for the shutdown() system call doesn't explicitly state that blocking recv() calls will be terminated, only that "further receptions and transmissions will be disallowed". But in practice this seems to include blocking recv() calls on the same socket on both Unixoid systems and Windows.
Another way would be to use socket.settimeout() to circumvent the indefinite blocking problem and regularly check (e.g. with global variables) inside the thread if the connection should be closed.
Yet another reasonable way to interrupt the recv() would be to send a non-lethal signal to the thread, which will interrupt the recv() system call, but there seems to be no wrapper for the low-level pthread_kill function in Python and since Python 3.5 system calls will be automatically restarted if interrupted, which makes this option unusable.
Related
When running my code I start a thread that runs for around 50 seconds and does a lot of background stuff. If I run this program and then close it soon after, the stuff still goes on in the background for a while because the thread never dies. How can I kill the thread gracefully in my closeEvent method in my MianWindow class? I've tried setting up a method called exit(), creating a signal 'quitOperation' in the thread in question, and then tried to use
myThread.quitOperation.emit()
I expected that this would call my exit() function in my thread because I have this line in my constructor:
self.quitOperation.connect(self.exit)
However, when I use the first line it breaks, saying that 'myThread' has no attribute 'quitOperation'. Why is this? Is there a better way?
I'm not sure for python, but I assume this myThread.quitOperation.emit() emits a signal for the thread to exit. The point is that while your worker is using the thread and does not return, nor runs QCoreApplication::processEvents(), myThread will never have the chance to actually process your request (this is called thread starvation).
Correct answer may depend on the situation, and the nature of the "stuff" your thread is doing. The most common practice is that the main thread sends a signal to the worker thread where a slot sets a flag. In the blocking process you regularly check this flag. It it is set you stop whatever "stuff" you are doing, tell your worker thread that it can quit (with a signal preferably with queued connection), call a deleteLater() on the worker object itself, and return from any functions you are currently in, so that the thread's event handler can run, and clear your worker object and itself up, the finally quit.
In case your "stuff" is a huge cycle of very fast operation like simple mathematics or directory navigation one-by-one that takes only a few milliseconds each, this will be enough.
In case your "stuff" contain huge blocking parts that you have no control of (an thus you can't place this flag checking call in it), you may need to wait in the main thread until the worker thread quits.
In case you use direct connect to set the flag, or you set it directly, be sure to protect the read/write access of the flag with a QMutex to prevent inconsistent reads, or user a queued connection to ensure single thread access of the flag.
While highly discouraged, optionally you can use QThread's terminate() method to instantaneously kill the thread. You should never do this as it may cause memory leak, heap corruption, resource leaking and any nasty stuff as destructors and clean-up codes will not run, and the execution can be halted at an undesired state.
I have read most of the similar questions in stackoverflow, but none see to solve my problem. I use ctypes to call a function from dll file. Therefore, I can't edit the source codes of the dll file to add any "end looping" conditions. Also, this function may last long (like some printing command). I need to design a "halt" command in case that something of emergency happens while printing is processed. The only way I can do is to kill the thread.
It is never good to forcibly kill a thread. Your program should be designed to cleanly exit from threads.
You can mark it as "daemon" before starting it. If you exit the main thread it will not wait on daemonized threads.
Terminating a thread can still be done in two ways. You can asynchronously raise a Python exception in a thread, via https://docs.python.org/2/c-api/init.html#c.PyThreadState_SetAsyncExc (as stated, this requires building a C module or using ctypes to make it work). The other approach on Windows is to call the Windows API TerminateThread():
TerminateThread is used to cause a thread to exit. When this occurs,
the target thread has no chance to execute any user-mode code. DLLs
attached to the thread are not notified that the thread is
terminating. The system frees the thread's initial stack.
[...]
TerminateThread is a dangerous function that should only be used in
the most extreme cases. You should call TerminateThread only if you
know exactly what the target thread is doing, and you control all of
the code that the target thread could possibly be running at the time
of the termination. For example, TerminateThread can result in the
following problems: ...
I think this should also be doable using ctypes.
You cannot safely terminate a thread without its cooperation. Threads are not isolated within a process, so unsafely terminating a thread contaminates the process. Please, don't go down this road.
If you need this kind of isolation, you need a process. You can safely terminate a process without its cooperation, though it may leave system objects (such as files) that the process was working on in an intermediate state. In your case, that may mean a print job half-done and a page halfway in the printer. Or it may mean temporary files that don't get removed.
I have the following code:
s.settimeout( 300 )
while notend:
data = s.recv()
datacheck()
...
The code above is running on a thread so if the user set the notend variable to False the thread will end. But in this case it won't exit immediately, it takes 300 second to exit, because of the recv function. How can exit from this thread?
# Wait all the threads
for thread in threading.enumerate():
if thread is not threading.currentThread():
thread.join()
The problem is that, if you have a blocking socket, recv() will block for the entire duration of the timeout. You could specify a shorter timeout, but that may not be what you want to do (what if the ping really is many thousands of milliseconds?).
Instead of simply trying to recv() all day, you could use a select.poll object to poll the socket of interest at a more frequent rate (smaller timeout), and call recv() only when the polling reports that there is some data to read. In between polls, you could break out of the polling loop if you discover at any point that notend is now false.
If your platform doesn't support poll(), then see if it supports select(). Then, you can use the select.select() function to accomplish more-or-less the same task.
Here's a reference for the described functionality:
http://docs.python.org/2/library/select.html#poll-objects
http://docs.python.org/2/library/select.html#select.select
Simple answer - close s from another thread. The recv() call will then return with an error, so allowing the thread that called it to clean up and terminate.
There is no need for timeouts, select() etc.
Make use of socket.settimeout because recv is a blocking method
I have been using Pyro 3 for a little while now, with great success, but occasionally I have noticed, that when a signal such as SIGHUP or SIGINT arrives while Pyro is doing some remote communications, the process hangs, hence the question, is Pyro signal safe?
Thanks in advance.
Seems the issue here is by default Python sets up a handlers for SIGINT and SIGTERM which raise exceptions. If you therfore receive a signal while doing some Pyro comms, the exception is raised, and off it goes to look for an appropriate except clause, not finishing what it was doing, if you then try and use Pyro again, for example in the except/finally clause, you can get issues. In my case it was sending some messages from finally to a log via a queue which was proxied to another process using Pyro.
What are the rules surrounding Python threads and how Unix signals are handled?
Is KeyboardInterrupt, which is triggered by SIGINT but handled internally by the Python runtime, handled differently?
First, when setting up signal handlers using the signal module, you must create them in the main thread. You will receive an exception if you try to create them in a separate thread.
Signal handlers registered via the signal.signal() function will always be called in the main thread. On architectures which support sending signals to threads, at the C level I believe the Python runtime ignores all signals on threads and has a signal handler on the main thread, which it uses to dispatch to your Python-code signal handler.
The documentation for the thread module states that the KeyboardInterrupt exception (which is ordinarily triggered by SIGINT) can be delivered to an arbitrary thread unless you have the signal module available to you, which all Unix systems should have. In that case, it's delivered to the main thread. If you're on a system without signal, you'll have to catch KeyboardInterrupt in your thread and call thread.interrupt_main() to re-raise it in the main thread.
More information can be found in the Python docs for the thread and signal modules.
From the signal documentation:
Some care must be taken if both signals and threads are used in the same program. The fundamental thing to remember in using signals and threads simultaneously is: always perform signal() operations in the main thread of execution. Any thread can perform an alarm(), getsignal(), pause(), setitimer() or getitimer(); only the main thread can set a new signal handler, and the main thread will be the only one to receive signals (this is enforced by the Python signal module, even if the underlying thread implementation supports sending signals to individual threads). This means that signals can’t be used as a means of inter-thread communication. Use locks instead.