We Keep Coding

Non blocking python class method executed with asyncio

I'm trying to initialize a non-blocking task, which shares data with its parent object. It is a websocket client, and it would not block the main execution, though still run "forever".
My humble expectations were this would do it, but sadly, it is blocking the main thread.
loop = asyncio.new_event_loop()
task = loop.create_task(self.initWS())
loop.run_forever()

self.initWS() is indeed not blocking the main thread, but loop.run_forever() is.
If you want to execute more tasks concurrently with self.initWS(), you have to add them to the asyncio loop, too.

How to shutdown server with open client connection?

I have been looking for some time for a solution to terminate a socketserver.StreamRequestHandler spawned as a separate thread by the class ThreadingMixIn when the main tread terminates. The problem was that the socketserver thread, although being daemonic, would not terminate as long as there was an open client connection, even when calling its shutdown() method at program shutdown. Therefore, the main tread also did not terminate. None of the suggestions in How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass? or similar posts worked for me.
I have now found a solution:
The class ThreadingMixIn has a property daemonic_threads, which is by default true:
class ThreadingMixIn:
"""Mix-in class to handle each request in a new thread."""
# Decides how threads will act upon termination of the
# main process
daemon_threads = False
In my subclass, which, of course, overrides the method handle(), I now also defines override in __init__:
daemon_threads = True
Now all client requests are daemonic and when the main thread terminates, they also terminate. Nice !
Hopefully this helps someone with a similar problem.

How to check if thread made by module thread (in Python 3 _thread) is running?

Yet the thread module works for me. How to check if thread made by module thread (in Python 3 _thread) is running? When the function the thread is doing ends, the thread ends too, or doesn't?
def __init__(self):
self.thread =None
......
if self.thread==None or not self.thread.isAlive() :
self.thread = thread.start_new_thread(self.dosomething,())
else:
tkMessageBox.showwarning("XXXX","There's no need to have more than two threads")
I know there is no function called isAlive() in "thread" module, is there any alternative?
But there isn't any reason why to use "threading" module, is there?

Unless you really need the low-level capabilities of the internal thread (_thread module, you really should use the threading module instead. It makes everything easier to use and does come with helpers such as is_alive.
Btw. the alternative of restarting a thread like you do in your example code would be to keep it running but have it wait for additional jobs. E.g. you could have a queue somewhere which keeps track of all jobs you want the thread to do, and the thread keeps working on them until the queue is empty—and then it will not terminate but wait for new jobs to appear. And only at the end of the application, you signalize the thread to stop waiting and terminate it.

Non-blocking server in Twisted

I am building an application that needs to run a TCP server on a thread other than the main. When trying to run the following code:
reactor.listenTCP(ServerConfiguration.tcpport, TcpCommandFactory())
reactor.run()
I get the following error
exceptions.ValueError: signal only works in main thread
Can I run the twisted servers on threads other than the main one?

Twisted can run in any thread - but only one thread at a time. If you want to run in the non-main thread, simply do reactor.run(installSignalHandlers=False). However, you cannot use a reactor on the non-main thread to spawn subprocesses, because their termination will never be detected. (This is a limitation of UNIX, really, not of Twisted.)

Do a callback from subclassed thread in main thread

I have a class that inherits from threading.Thread.
After processing an item from a queue I want it to call a function that I handed over to the thread when starting it. Doing that callback from the run-Method will have it run in my worker thread. Is there a way to run that callback inside the MainThread?
Thanks & Greetings,
Sean

You didnt mention intially that you are using pyqt. Qt has signals and slots built in just for this purpose. If you are launching your thread using QThread then it has the 'finished' signal that your main thread can be connected to on a slot. If you need even more control of the resulting signal you can emit a custom one from the thread with any value you want. When you construct the thread just connect its signal to the slot that you want to act as the callback.
Otherwise you would be blocking your main event loop if you are specifically waiting on the thread.
Here is a link specifically about the new style signal/slots: http://www.riverbankcomputing.co.uk/static/Docs/PyQt4/html/new_style_signals_slots.html
It makes it extremely easy now to create a python signal with whatever signature you want and connect it in an object oriented fashion to a slot.
Also as a side note about PyQt... Some of the other answers here have suggested creating a loop on your main thread that waits to collect responses in a queue from its dispatched threads. That is essentially what PyQt does. When you start your app the main thread goes into an event loop. This event loop is processing events that stack up in its event queue. There are different types of signal/slot connections. If your two endpoints are in the same thread, then a direct connection will be made and the emitting signal will call the slot. If they are in different threads, then I believe it goes through the event loop so that it can be processed outside of the emitters thread.

Push the method onto a queue and have the main thread sit and wait on that queue, running each method as it pulls them off.

You can use condition variables and have your main thread wait on a message from the sub-thread. The sub-thread can signal the main thread to call that method.