I am trying to figure out how to run my overloaded customized BaseHTTPServer instance in the background after running the "".serve_forever() method.
Normally when you run the method execution will hang until you execute a keyboard interrupt, but I would like it to serve requests in the background while continuing script execution. Please help!
You can start the server in a different thread: https://docs.python.org/3/library/_thread.html#thread.start_new_thread
So something like:
def start_server():
# Setup stuff here...
server.serve_forever()
# start the server in a background thread
thread.start_new_thread(start_server)
print('The server is running but my script is still executing!')
I was trying to do some long-term animation using async and thought I'd have to rewrite server to use aiohttp (https://docs.aiohttp.org/en/v0.12.0/web.html), but Olivers technique of using seperate thread saved me all that pain. My code looks like this, where MyHTTPServer is simply my custom sublass of HTTPServer
import threading
import asyncio
from http.server import BaseHTTPRequestHandler, HTTPServer
import socketserver
import io
import threading
async def tick_async(server):
while True:
server.animate_something()
await asyncio.sleep(1.0)
def start_server():
httpd.serve_forever()
try:
print('Server listening on port 8082...')
httpd = MyHTTPServer(('', 8082), MyHttpHandler)
asyncio.ensure_future(tick_async(httpd))
loop = asyncio.get_event_loop()
t = threading.Thread(target=start_server)
t.start()
loop.run_forever()
Related
I'm implementing a bi-directional ping-pong demo app between an electron app and a python backend.
This is the code for the python part which causes the problems:
import sys
import zerorpc
import time
from multiprocessing import Process
def ping_response():
print("Sleeping")
time.sleep(5)
c = zerorpc.Client()
c.connect("tcp://127.0.0.1:4243")
print("sending pong")
c.pong()
class Api(object):
def echo(self, text):
"""echo any text"""
return text
def ping(self):
p = Process(target=ping_response, args=())
p.start()
print("got ping")
return
def parse_port():
port = 4242
try:
port = int(sys.argv[1])
except Exception as e:
pass
return '{}'.format(port)
def main():
addr = 'tcp://127.0.0.1:' + parse_port()
s = zerorpc.Server(Api())
s.bind(addr)
print('start running on {}'.format(addr))
s.run()
if __name__ == '__main__':
main()
Each time ping() is called from javascript side it will start a new process that simulates some work (sleeping for 5 seconds) and replies by calling pong on nodejs server to indicate work is done.
The issue is that the pong() request never gets to javascript side. If instead of spawning a new process I create a new thread using _thread and execute the same code in ping_response(), the pong request arrives in the javascript side. Also if I manually run the bash command zerorpc tcp://localhost:4243 pong I can see that the pong request is received by the nodejs script so the server on the javascript side works ok.
What happens with zerorpc client when I create a new process and it doesn't manage to send the request ?
Thank you.
EDIT
It seems it gets stuck in c.pong()
Try using gipc.start_process() from the gipc module (via pip) instead of multiprocessing.Process(). It creates a new gevent context which otherwise multiprocessing will accidentally inherit.
I need to create a multithreaded web server in python 3, where each request gets a new thread. I followed a basic example from a blog. But, the server is always blocking, as sleep called in 1 thread blocks the other threads. Can someone help me on this?
Here is my code
import time
from http.server import HTTPServer, BaseHTTPRequestHandler
from socketserver import ThreadingMixIn
import threading
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
print('start-->')
time.sleep(5)
print('end-->')
self.wfile.write("Test".encode())
class ThreadingSimpleServer(ThreadingMixIn, HTTPServer):
pass
def run():
server = ThreadingSimpleServer(('0.0.0.0', 1234), Handler)
server.serve_forever()
if __name__ == '__main__':
run()
My problem is, If I send 2 requests to the server, the second one starts executing only after the first one is done. I need the sleep to be on a thread of it's own, without affecting the other threads. I will be grateful for any help.
In a project I'm working in I need to cover a Tornado service with Behave so I want to start an instance of my tornado service before running each scenario.
Naively trying to run the loop as part before all seems to lock the excecution:
from tornado import ioloop
from tornadoadapter.applications import APPLICATION
def before_all(context):
print "Service running on port 8000"
APPLICATION.listen(8000)
ioloop.IOLoop.instance().start()
So it's probably not what I need.
Your IOLoop is running in the main thread, so it's blocking. You could do it in a separate thread or process.
from multiprocessing import Process
from tornado import ioloop
from tornadoadapter.applications import APPLICATION
def run_server():
print "Service running on port 8000"
APPLICATION.listen(8000)
ioloop.IOLoop.instance().start()
def before_all(context):
context.server_thread = Process(target=run_server)
context.server_thread.deamon = True
context.server_thread.start()
I have a jabber client that is reading from its stdin and posting PubSub messages. If I get EOF on stdin, I want to terminate the client.
I first tried sys.exit(), but this causes an exception and the client does not exit. I then did some searching and found out that I should call reactor.stop(), but I am unable to make this work. The following code in my client:
from twisted.internet import reactor
reactor.stop()
Results in exceptions.AttributeError: 'module' object has no attribute 'stop'
What do I need to do to cause twistd to shut my application down and exit?
EDIT 2
The original problem was caused by some symlinks messing up the module import. After fixing that problem, I get a new exception:
twisted.internet.error.ReactorNotRunning: Can't stop reactor that isn't running.
After the exception, twistd shuts down. I think this may be caused by the call to MyClient.loop in MyClient.connectionInitialized. Perhaps I need to defer the call until later?
EDIT
Here's the .tac file for my client
import sys
from twisted.application import service
from twisted.words.protocols.jabber.jid import JID
from myApp.clients import MyClient
clientJID = JID('client#example.com')
serverJID = JID('pubsub.example.com')
password = 'secret'
application = service.Application('XMPP client')
xmppClient = client.XMPPClient(clientJID, password)
xmppClient.logTraffic = True
xmppClient.setServiceParent(application)
handler = MyClient(clientJID, serverJID, sys.stdin)
handler.setHandlerParent(xmppClient)
Which I'm invoking with
twistd -noy sentry/myclient.tac < input.txt
Here's the code for MyClient:
import os
import sys
import time
from datetime import datetime
from wokkel.pubsub import PubSubClient
class MyClient(PubSubClient):
def __init__(self, entity, server, file, sender=None):
self.entity = entity
self.server = server
self.sender = sender
self.file = file
def loop(self):
while True:
line = self.file.readline()
if line:
print line
else:
from twisted.internet import reactor
reactor.stop()
def connectionInitialized(self):
self.loop()
from twisted.internet import reactor
reactor.stop()
that should work. The fact that it doesn't means something else is wrong on your application. I can't figure out what's wrong from the information you provided.
Can you provide more (all) of the code?
EDIT:
Ok, now the problem is that you don't stop your own while True loop, so it will keep looping and eventually stop the reactor again.
Try this:
from twisted.internet import reactor
reactor.stop()
return
Now, I suspect your loop isn't very good thing for a event-driven framework. While you're just printing lines, it is fine, but depending on what you want to really do (I suspect you'll do more than just print lines) you'll have to refactor that loop to work with events.
Use reactor.callFromThread(reactor.stop) instead of reactor.stop. This should solve the issue.
I used to do this way (in sigint handler of a non-twistd called application):
reactor.removeAll()
reactor.iterate()
reactor.stop()
I'm not 100% sure it is the right way, but twisted is happy
the same application started in a tac is handled directly by twistd signal handler, I've found this question because I have some rpc client requests that I would to wait for and handle result before exiting and looks like twistd is just killing the reactor without letting the call finish
I have a class that I wish to test via SimpleXMLRPCServer in python. The way I have my unit test set up is that I create a new thread, and start SimpleXMLRPCServer in that. Then I run all the test, and finally shut down.
This is my ServerThread:
class ServerThread(Thread):
running = True
def run(self):
self.server = #Creates and starts SimpleXMLRPCServer
while (self.running):
self.server.handle_request()
def stop(self):
self.running = False
self.server.server_close()
The problem is, that calling ServerThread.stop(), followed by Thread.stop() and Thread.join() will not cause the thread to stop properly if it's already waiting for a request in handle_request. And since there doesn't seem to be any interrupt or timeout mechanisms here that I can use, I am at a loss for how I can cleanly shut down the server thread.
I had the same problem and after hours of research i solved it by switching from using my own handle_request() loop to serve_forever() to start the server.
serve_forever() starts an internal loop like yours. This loop can be stopped by calling shutdown(). After stopping the loop it is possible to stop the server with server_close().
I don't know why this works and the handle_request() loop don't, but it does ;P
Here is my code:
from threading import Thread
from xmlrpc.server import SimpleXMLRPCServer
from pyWebService.server.service.WebServiceRequestHandler import WebServiceRquestHandler
class WebServiceServer(Thread):
def __init__(self, ip, port):
super(WebServiceServer, self).__init__()
self.running = True
self.server = SimpleXMLRPCServer((ip, port),requestHandler=WebServiceRquestHandler)
self.server.register_introspection_functions()
def register_function(self, function):
self.server.register_function(function)
def run(self):
self.server.serve_forever()
def stop_server(self):
self.server.shutdown()
self.server.server_close()
print("starting server")
webService = WebServiceServer("localhost", 8010)
webService.start()
print("stopping server")
webService.stop_server()
webService.join()
print("server stopped")
Two suggestions.
Suggestion One is to use a separate process instead of a separate thread.
Create a stand-alone XMLRPC server program.
Start it with subprocess.Popen().
Kill it when the test is done. In standard OS's (not Windows) the kill works nicely. In Windows, however, there's no trivial kill function, but there are recipes for this.
The other suggestion is to have a function in your XMLRPC server which causes server self-destruction. You define a function that calls sys.exit() or os.abort() or raises a similar exception that will stop the process.
This is my way. send SIGTERM to self. (Works for me)
Server code
import os
import signal
import xmlrpc.server
server = xmlrpc.server.SimpleXMLRPCServer(("0.0.0.0", 8000))
server.register_function(lambda: os.kill(os.getpid(), signal.SIGTERM), 'quit')
server.serve_forever()
Client code
import xmlrpc.client
c = xmlrpc.client.ServerProxy("http://localhost:8000")
try:
c.quit()
except ConnectionRefusedError:
pass