I have an app using Uvicorn with FastAPI. I have also some connections open (e.g. to MongoDB). I want to gracefully close these connections once some signal occurs (SIGINT, SIGTERM and SIGKILL).
My server.py file:
import uvicorn
import fastapi
import signal
import asyncio
from source.gql import gql
app = fastapi.FastAPI()
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])
app.mount("/graphql", gql)
# handle signals
HANDLED_SIGNALS = (
signal.SIGINT,
signal.SIGTERM
)
loop = asyncio.get_event_loop()
for sig in HANDLED_SIGNALS:
loop.add_signal_handler(sig, _some_callback_func)
if __name__ == "__main__":
uvicorn.run(app, port=6900)
Unfortunately, the way I try to achieve this is not working. When I try to Ctrl+C in terminal, nothing happens. I believe it is caused because Uvicorn is started in different thread...
What is the correct way of doing this? I have noticed uvicorn.Server.install_signal_handlers() function, but wasn't lucky in using it...
FastAPI allows defining event handlers (functions) that need to be executed before the application starts up, or when the application is shutting down. Thus, you could use the shutdown event, as described here:
#app.on_event("shutdown")
def shutdown_event():
# close connections here
Related
So I have two microapp inside my app, there are:
fastapi for webservice
kafka for consuming message
I want to run both of them in a different thread within the main thread, I have run kafka on a separate thread, but I don't know how to run gunicorn fastapi, I have read that we can use subprocess in thread.
Is there a way to run gunicorn programmatically like uvicorn.run()?
Here is my main code:
def create_app():
logging.basicConfig(level=logging.DEBUG)
# register config
config = Config()
# register dependency
di_container = Container()
di_container.config.from_dict(config.get_config())
di_container.wire(modules=[runner_rating])
# start kafka consumer in a separate thread
kafka_client = di_container.kafka_consumer.client()
kafka_client.listen()
# how to run this inside thread?
app = FastAPI()
app.container = di_container
return app
Taking the standard Tornado demonstration and pushing the IOLoop into a background thread allows querying of the server within a single script. This is useful when the Tornado server is an interactive object (see Dask or similar).
import asyncio
import requests
import tornado.ioloop
import tornado.web
from concurrent.futures import ThreadPoolExecutor
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
pool = ThreadPoolExecutor(max_workers=2)
loop = tornado.ioloop.IOLoop()
app = make_app()
app.listen(8888)
fut = pool.submit(loop.start)
print(requests.get("https://localhost:8888"))
The above works just fine in a standard python script (though it is missing safe shutdown). Jupyter notebook are optimal environment for these interactive Tornado server environments. However, when it comes to Jupyter this idea breaks down as there is already a active running loop:
>>> import asyncio
>>> asyncio.get_event_loop()
<_UnixSelectorEventLoop running=True closed=False debug=False>
This is seen when running the above script in a Jupyter notebook, both the server and the request client are trying to open a connection in the same thread and the code hangs. Building a new Asyncio loop and/or Tornado IOLoop does not seem to help and I suspect I am missing something in Jupyter itself.
The question: Is it possible to have a live Tornado server running in the background within a Jupyter notebook so that standard python requests or similar can connect to it from the primary thread? I would prefer to avoid Asyncio in the code presented to users if possible due to its relatively complexity for novice users.
Based on my recent PR to streamz, here is something that works, similar to your idea:
class InNotebookServer(object):
def __init__(self, port):
self.port = port
self.loop = get_ioloop()
self.start()
def _start_server(self):
from tornado.web import Application, RequestHandler
from tornado.httpserver import HTTPServer
from tornado import gen
class Handler(RequestHandler):
source = self
#gen.coroutine
def get(self):
self.write('Hello World')
application = Application([
('/', Handler),
])
self.server = HTTPServer(application)
self.server.listen(self.port)
def start(self):
"""Start HTTP server and listen"""
self.loop.add_callback(self._start_server)
_io_loops = []
def get_ioloop():
from tornado.ioloop import IOLoop
import threading
if not _io_loops:
loop = IOLoop()
thread = threading.Thread(target=loop.start)
thread.daemon = True
thread.start()
_io_loops.append(loop)
return _io_loops[0]
To call in the notebook
In [2]: server = InNotebookServer(9005)
In [3]: import requests
requests.get('http://localhost:9005')
Out[3]: <Response [200]>
Part 1: Let get nested tornado(s)
To find the information you need you would have had to follow the following crumbtrails, start by looking at what is described in the release notes of IPython 7
It particular it will point you to more informations on the async and await sections in the documentation, and to this discussion,
which suggest the use of nest_asyncio.
The Crux is the following:
A) either you trick python into running two nested event loops. (what nest_asyncio does)
B) You schedule coroutines on already existing eventloop. (I'm not sure how to do that with tornado)
I'm pretty sure you know all that, but I'm sure other reader will appreciate.
There are unfortunately no ways to make it totally transparent to users – well unless you control the deployment like on a jupyterhub, and can add these lines to the IPython startups scripts that are automatically loaded. But I think the following is simple enough.
import nest_asyncio
nest_asyncio.apply()
# rest of your tornado setup and start code.
Part 2: Gotcha Synchronous code block eventloop.
Previous section takes only care of being able to run the tornado app. But note that any synchronous code will block the eventloop; thus when running print(requests.get("http://localhost:8000")) the server will appear to not work as you are blocking the eventloop, which will restart only when the code finish execution which is waiting for the eventloop to restart...(understanding this is an exercise left to the reader). You need to either issue print(requests.get("http://localhost:8000")) from another kernel, or, use aiohttp.
Here is how to use aiohttp in a similar way as requests.
import aiohttp
session = aiohttp.ClientSession()
await session.get('http://localhost:8889')
In this case as aiohttp is non-blocking things will appear to work properly. You here can see some extra IPython magic where we autodetect async code and run it on the current eventloop.
A cool exercise could be to run a request.get in a loop in another kernel, and run sleep(5) in the kernel where tornado is running, and see that we stop processing requests...
Part 3: Disclaimer and other routes:
This is quite tricky and I would advise to not use in production, and warn your users this is not the recommended way of doing things.
That does not completely solve your case, you will need to run things not in the main thread which I'm not sure is possible.
You can also try to play with other loop runners like trio and curio; they might allow you to do stuff you can't with asyncio by default like nesting, but here be dragoons. I highly recommend trio and the multiple blog posts around its creation, especially if you are teaching async.
Enjoy, hope that helped, and please report bugs, as well as things that did work.
You can make the tornado server run in background using the %%script --bg magic command. The option --bg tells jupyter to run the code of the current cell in background.
Just create a tornado server in one cell alongwith the magic command and run that cell.
Example:
%%script python --bg
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
loop = tornado.ioloop.IOLoop.current()
app = make_app()
app.listen(8000) # 8888 was being used by jupyter in my case
loop.start()
And then you can use requests in a separate cell to connect to the server:
import requests
print(requests.get("http://localhost:8000"))
# prints <Response [200]>
One thing to note here is that if you stop/interrupt the kernel on any cell, the background script will also stop. So you'll have to run this cell again to start the server.
I want to implement Server Side Events in my application. I'm using Flask with a gevent backend. I mostly follow the official snippet, with two important distinctions:
The function that fires events is spawned as a separate process (an instance of multiprocessing.Process), instead of gevent.spawn. It sends "inner" event
Event firing is done via logging, with QueueHandler from logutils
Here's a minimal example:
from flask import Flask, Response
import time
import logging
from gevent.wsgi import WSGIServer
from gevent.queue import Queue
import multiprocessing as mp
from logutils.queue import QueueHandler
app = Flask(__name__)
logger = logging.getLogger('events')
logger.setLevel(logging.INFO)
#app.route("/publish")
def publish():
logger = logging.getLogger('events')
def notify():
time.sleep(1)
logger.info('inner')
logger.info('outer')
p = mp.Process(target=notify)
p.start()
return "OK"
#app.route("/subscribe")
def subscribe():
def gen():
logger = logging.getLogger("events")
q = Queue()
handler = QueueHandler(q)
logger.addHandler(handler)
while True:
result = q.get().message
yield result
return Response(gen(), mimetype="text/event-stream")
if __name__ == "__main__":
root = logging.getLogger()
root.addHandler(logging.StreamHandler())
root.setLevel(logging.INFO)
app.debug = True
server = WSGIServer(("", 5000), app)
server.serve_forever()
The problem is, StreamHandler catches both "inner" and "outer" events and logs them to stdout. QueueHandler, on the other hand, only pushes the "outer" event to the queue, so it fails to catch the "inner" event, which originates in other process.
Now, if I use multiprocessing.Queue instead of gevent.queue, the "/subscribe" route blocks. If I use standard werkzeug server (with threading) in conjunction with multiprocessing.Queue, the code above works as expected.
What's going on and how to tackle this?
How to run a given module given I want to run some functions concurrently that are not necessarily using routing (could be daemon services) while at the same time running the app server?
For example:
#some other route functions app.post(...)
#some other concurrent functions
def alarm():
'''
Run this service every X duration
'''
ALARM = 21
try:
while 1:
#checking time and doing something. Then finding INTERVAL
gevent.sleep(INTERVAL)
except KeyboardInterrupt,e:
print 'exiting'
Do I have to use the above like this after main ?
gevent.joinall(gevent.spawn(alarm))
app.run(....)
or
gevent.joinall((gevent.spawn(alarm),gevent.spawn(app.run)))
The objective is run these alarm like daemon services, do their work and snooze while rest of service operations work as usual.
The server should start concurrently as well. correct me if im not on the right track.
Gevent comes with it's own WSGI servers, so it is really not necessary to use app.run. The servers are:
gevent.pywsgi.WSGIServer
gevent.wsgi.WSGIServer
Both provide the same interface.
You can use these to achieve what you want:
import gevent
import gevent.monkey
gevent.monkey.patch_all()
import requests
from gevent.pywsgi import WSGIServer
# app = YourBottleApp
def alarm():
'''
Run this service every X duration
'''
ALARM = 21
while 1:
#checking time and doing something. Then finding INTERVAL
gevent.sleep(INTERVAL)
if __name__ == '__main__':
http_server = WSGIServer(('', 8080), app)
srv_greenlet = gevent.spawn(http_server.serve_forever)
alarm_greenlet = gevent.spawn(alarm)
try:
gevent.joinall([srv_greenlet, alarm_greenlet])
except KeyboardInterrupt:
http_server.stop()
print 'Quitting'
I have a server running on tornado. I have a page that opens a websocket to the same server. Now I have observed that opening multiple instances of this page makes all of them wait except one. Only after that one has finished its websocket, does another one start. Is this normal tornado behaviour of I'm doing something wrong?
Earlier my server was running with django but I migrated to tornado for the websocket support. For that I use fallback server as django.
#!/usr/bin/env python
# Run this with
# PYTHONPATH=. DJANGO_SETTINGS_MODULE=testsite.settings testsite/tornado_main.py
# Serves by default at
# http://localhost:8080/hello-tornado and
# http://localhost:8080/hello-django
from tornado.options import options, define, parse_command_line
import django.core.handlers.wsgi
import tornado.httpserver
import tornado.ioloop
import tornado.web
import tornado.wsgi
define('port', type=int, default=8000)
class HelloHandler(tornado.web.RequestHandler):
def get(self):
self.write('Hello from tornado')
def main():
wsgi_app = tornado.wsgi.WSGIContainer(
django.core.handlers.wsgi.WSGIHandler())
tornado_app = tornado.web.Application(
[
('/hello-tornado', HelloHandler),
('.*', tornado.web.FallbackHandler, dict(fallback=wsgi_app)),
])
server = tornado.httpserver.HTTPServer(tornado_app)
server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == '__main__':
main()
Can I do something that can allow me to make multiple connections?
You need to look into the Asych facilities in tornado to get this to work properly. Tornado in it's normal state is a single threaded stack and thus you can only handle one connection at a time.
You can use the normal #asynchronous decorator or use their gen library to allow your code to handle multiple connections.
Decorator: http://www.tornadoweb.org/documentation/web.html#decorators
Gen: http://www.tornadoweb.org/documentation/gen.html
Read the documentation carefully if you choose to use the #asynchronous decorator as you need to close the connection when you are done with it.
Yes, this is normal Tornado behaviour in case when you trying run heavy blocking applications like Django in it.
You, definitely, should run django and tornado in separate OS processes. Especially if you use Django ORM.
Need I describe why?