This is a simple example script from dev.deluge-torrent.org for interacting with the Deluge API.
Nothing happens after reactor.run() and I don't get the "Connection was successful" message, it just hangs forever.
I ran this on my Ubuntu machine where it works fine, but I couldn't get it to work on my Windows machine where i really want to put it to use.
from deluge.ui.client import client
# Import the reactor module from Twisted - this is for our mainloop
from twisted.internet import reactor
# Set up the logger to print out errors
from deluge.log import setupLogger
setupLogger()
# Connect to a daemon running on the localhost
# We get a Deferred object from this method and we use this to know if and when
# the connection succeeded or failed.
d = client.connect()
# We create a callback function to be called upon a successful connection
def on_connect_success(result):
print "Connection was successful!"
print "result:", result
# Disconnect from the daemon once we successfully connect
client.disconnect()
# Stop the twisted main loop and exit
reactor.stop()
# We add the callback to the Deferred object we got from connect()
d.addCallback(on_connect_success)
# We create another callback function to be called when an error is encountered
def on_connect_fail(result):
print "Connection failed!"
print "result:", result
# We add the callback (in this case it's an errback, for error)
d.addErrback(on_connect_fail)
# Run the twisted main loop to make everything go
reactor.run()
I have no idea of how to go about debugging this issue. I'm very new to Twisted, and from what I understand it's a huge library.
Related
How do you kill a websocket server programmatically? I'll be deploying this server to production along side other things. And I like to build a single python script that sends a kill signal to everything. I cannot figure out how to kill this thing without a user keyboard interrupt or a kill -9.
sys.exit() didn't work.
psutil and terminate() didn't work either
import os
import psutil
current_system_pid = os.getpid()
ThisSystem = psutil.Process(current_system_pid)
ThisSystem.terminate()
I'm out of ideas. For now I'm killing it on the command line with kill -9.
When I kill it varous ways, it tend to see this message below, but the scrip is still running
2020-12-12 12:24:54-0500 [autobahn.twisted.websocket.WebSocketServerFactory] (TCP Port 8080 Closed)
2020-12-12 12:24:54-0500 [-] Stopping factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x110680f28>
autobahn install:
pip install autobahn[twisted]
Code:
from autobahn.twisted.websocket import WebSocketServerProtocol, WebSocketServerFactory
import sys
from twisted.python import log
from twisted.internet import reactor
class MyServerProtocol(WebSocketServerProtocol):
def onConnect(self, request):
print("Client connecting: {0}".format(request.peer))
def onOpen(self):
print("WebSocket connection open.")
def onMessage(self, payload, isBinary):
print("Text message received: {0}".format(payload.decode('utf8')))
# echo back message verbatim
# self.sendMessage(payload, isBinary)
def onClose(self, wasClean, code, reason):
print("WebSocket connection closed: {0}".format(reason))
def StopWebsocketServer():
PrintAndLog_FuncNameHeader("Begin")
reactor.stop()
PrintAndLog_FuncNameHeader("End")
if __name__ == '__main__':
# TODO remove the logging that came in the example
log.startLogging(sys.stdout)
factory = WebSocketServerFactory("ws://127.0.0.1:8080")
factory.protocol = MyServerProtocol
# note to self: if using putChild, the child must be bytes...
reactor.listenTCP(Port_ws, factory)
reactor.run()
Solution using #Jean-Paul Calderone's answer:
import os
import signal
os.kill(os.getpid(), signal.SIGKILL)
I have an external python script that sends a kill signal to each of my python scripts. The kill signal is simply the existence of a file that every script knows to look for. Once that kill signal appears, each script knows it has x seconds before it will be killed. This way they have a few seconds to gracefully finish something.
twisted.internet.reactor.stop() is how you cause the reactor to shut down. This is usually what results in a Twisted-based program exiting (though of course it doesn't necessarily have to, if the program does more things after the reactor shuts down - but this is uncommon).
However, it sounds like you don't want to know what Python code to run inside the process to end it. You want to know what some other process can do to your Twisted-based process to make it exit. You gave two solutions - KeyboardInterrupt and SIGKILL. You didn't mention why either of these two solutions is inappropriate. They seem fine to me.
If you're uncomfortable with SIGKILL (which you shouldn't be, after all, your program might meet an untimely demise for many reasons and you should be prepared to deal with this) then what you might have overlooked about KeyboardInterrupt is that it is merely the exception that is raised inside a Python program by the default SIGINT handler.
If you send SIGINT to a Twisted-based process then, under normal usage, this will stop the reactor and allow an orderly shutdown.
I'm implementing a bi-directional ping-pong demo app between an electron app and a python backend.
This is the code for the python part which causes the problems:
import sys
import zerorpc
import time
from multiprocessing import Process
def ping_response():
print("Sleeping")
time.sleep(5)
c = zerorpc.Client()
c.connect("tcp://127.0.0.1:4243")
print("sending pong")
c.pong()
class Api(object):
def echo(self, text):
"""echo any text"""
return text
def ping(self):
p = Process(target=ping_response, args=())
p.start()
print("got ping")
return
def parse_port():
port = 4242
try:
port = int(sys.argv[1])
except Exception as e:
pass
return '{}'.format(port)
def main():
addr = 'tcp://127.0.0.1:' + parse_port()
s = zerorpc.Server(Api())
s.bind(addr)
print('start running on {}'.format(addr))
s.run()
if __name__ == '__main__':
main()
Each time ping() is called from javascript side it will start a new process that simulates some work (sleeping for 5 seconds) and replies by calling pong on nodejs server to indicate work is done.
The issue is that the pong() request never gets to javascript side. If instead of spawning a new process I create a new thread using _thread and execute the same code in ping_response(), the pong request arrives in the javascript side. Also if I manually run the bash command zerorpc tcp://localhost:4243 pong I can see that the pong request is received by the nodejs script so the server on the javascript side works ok.
What happens with zerorpc client when I create a new process and it doesn't manage to send the request ?
Thank you.
EDIT
It seems it gets stuck in c.pong()
Try using gipc.start_process() from the gipc module (via pip) instead of multiprocessing.Process(). It creates a new gevent context which otherwise multiprocessing will accidentally inherit.
I am using pyGTK and a python gRPC server. Both in a very basic setup.
I just create a gtk.Window(), show() it and run a gtk.main() loop.
My server starts like that:
def startServing():
global server
print("Starting server...")
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
laserClient_pb2_grpc.add_LComServicer_to_server(LCom(), server)
server.add_insecure_port('[::]:50053') # [::] is the same as 0.0.0.0 server.start()
print("Server is running...")
So I call:
try:
startServing()
gtk.main()
except KeyboardInterrupt:
server.stop(0)
This creates the window correctly but I never receive a request from my Java client. (The java client is not the problem.)
I read a lot on the internet and I do not understand all of the pyGTK thread handling but I tried gtk.gdk.threads_init() right before startServing() and I received the requests. However I receive just 1 request per second whereas my client sends a request every 50ms. If I delete gtk.main() and add a while loop:
while True:
time.sleep(60)
...I receive requests nearly every 50 to 100ms. This is the expected behaviour!
However my window won't get updated since there is no gtk.main() loop anymore. I even tried adding:
while True:
while gtk.events_pending():
gtk.main_iteration()
time.sleep(0.05)
But this gives, again, just 1 request per second.
I have no idea what I should do now. I really want to use gRPC and pyGTK together in the same program.
I have an app similar to a chat-room writing in python that intends to do the following things:
A prompt for user to input websocket server address.
Then create a websocket client that connects to server and send/receive messages. Disable the ability to create a websocket client.
After receiving "close" from server (NOT a close frame), client should drop connecting and re-enable the app to create a client. Go back to 1.
If user exits the app, it exit the websocket client if there is one running.
My approach for this is using a main thread to deal with user input. When user hits enter, a thread is created for WebSocketClient using AutoBahn's twisted module and pass a Queue to it. Check if the reactor is running or not and start it if it's not.
Overwrite on message method to put a closing flag into the Queue when getting "close". The main thread will be busy checking the Queue until receiving the flag and go back to start. The code looks like following.
Main thread.
def main_thread():
while True:
text = raw_input("Input server url or exit")
if text == "exit":
if myreactor:
myreactor.stop()
break
msgq = Queue.Queue()
threading.Thread(target=wsthread, args=(text, msgq)).start()
is_close = False
while True:
if msgq.empty() is False:
msg = msgq.get()
if msg == "close":
is_close = True
else:
print msg
if is_close:
break
print 'Websocket client closed!'
Factory and Protocol.
class MyProtocol(WebSocketClientProtocol):
def onMessage(self, payload, isBinary):
msg = payload.decode('utf-8')
self.Factory.q.put(msg)
if msg == 'close':
self.dropConnection(abort=True)
class WebSocketClientFactoryWithQ(WebSocketClientFactory):
def __init__(self, *args, **kwargs):
self.queue = kwargs.pop('queue', None)
WebSocketClientFactory.__init__(self, *args, **kwargs)
Client thread.
def wsthread(url, q):
factory = WebSocketClientFactoryWithQ(url=url, queue=q)
factory.protocol = MyProtocol
connectWS(Factory)
if myreactor is None:
myreactor = reactor
reactor.run()
print 'Done'
Now I got a problem. It seems that my client thread never stops. Even if I receive "close", it seems still running and every time I try to recreate a new client, it creates a new thread. I understand the first thread won't stop since reactor.run() will run forever, but from the 2nd thread and on, it should be non-blocking since I'm not starting it anymore. How can I change that?
EDIT:
I end up solving it with
Adding stopFactory() after disconnect.
Make protocol functions with reactor.callFromThread().
Start the reactor in the first thread and put clients in other threads and use reactor.callInThread() to create them.
Your main_thread creates new threads running wsthread. wsthread uses Twisted APIs. The first wsthread becomes the reactor thread. All subsequent threads are different and it is undefined what happens if you use a Twisted API from them.
You should almost certainly remove the use of threads from your application. For dealing with console input in a Twisted-based application, take a look at twisted.conch.stdio (not the best documented part of Twisted, alas, but just what you want).
So I'm writing a small script to use with Deluge. Deluge uses Twisted, and I really don't have a firm grasp on how it works. Normally I'd just look up more info on it, but getting started with Twisted would take a long time and is beyond the scope of this little project. So I figured I would just ask here.
Now, I have this code. I'll try to explain the specifig parts I need help with
import base64
import processargs
from deluge.ui.client import client
from twisted.internet import reactor
from deluge.log import setupLogger
setupLogger()
options = processargs.readConfig(os.path.expanduser("~/.deluge-automator"))
d = client.connect(
host=options['host'],
port=int(options['port']),
username=options['username'],
password=options['password']
)
def start():
#other code
t = client.core.add_torrent_file(tfile,
base64.encodestring(data), None)
t.addCallback(on_torrent_added_success, tfile)
t.addErrback(on_torrent_added_fail)
def handle_stop_signal(SIGNAL, stack):
client.disconnect()
reactor.stop()
def on_torrent_added_success(result, tfile):
#other code
start()
def on_torrent_added_fail(result):
print "Add torrent failed!"
print "result: ", result
def on_connect_success(result):
#other code
start()
d.addCallback(on_connect_success)
def on_connect_fail(result):
print "Connection failed!"
print "result: ", result
d.addErrback(on_connect_fail)
signal.signal(signal.SIGTERM, handle_stop_signal)
signal.signal(signal.SIGINT, handle_stop_signal)
reactor.run()
When a torrent is successfully added, it should go back to start(), and it does, but I think it loses the reactor or something. Because now whenever it recieves a SIGTERM or SIGINT, the reactor closes, but doesn't quit the program:
± % python2 main.py
Connection was successful!
result: 10
^C^CConnection failed!
result: [Failure instance: Traceback: <class 'twisted.internet.error.ReactorNotRunning'>: Can't stop reactor that isn't running.
/usr/lib/python2.7/site-packages/twisted/internet/defer.py:551:_runCallbacks
/usr/lib/python2.7/site-packages/deluge/ui/client.py:412:__on_login
/usr/lib/python2.7/site-packages/twisted/internet/defer.py:368:callback
/usr/lib/python2.7/site-packages/twisted/internet/defer.py:464:_startRunCallbacks
--- <exception caught here> ---
/usr/lib/python2.7/site-packages/twisted/internet/defer.py:551:_runCallbacks
main.py:70:on_connect_success
main.py:32:start
main.py:49:handle_stop_signal
/usr/lib/python2.7/site-packages/twisted/internet/base.py:577:stop
]
So the reactor gets stopped, but it doesn't quit the program. I have to keyboard interrupt twice. Once to stop the reactor, and a second time to throw the error. Is there a certain way to set up a loop like this?
reactor handles sigint, sigterm itself (there might be a parameter of reactor.run() that disables that). Install reactor.addSystemEventTrigger('before', 'shutdown', client.disconnect) instead.
See twisted: catch keyboardinterrupt and shutdown properly.