We Keep Coding

How to deploy a scalable API using fastapi?

I have a complex API which takes around 7GB memory when I deploy it using Uvicorn.
I want to understand how I can deploy it, such a way that from my end I want to be able to make parallel requests. The deployed API should be capable of processing two or three requests at same time.
I am using FastAPI with uvicorn and nginx for deployment. Here is my deployed command.
uvicorn --host 0.0.0.0 --port 8888
Can someone provide some clarity on how people achieve that?

You can use gunicorn instead of uvicorn to handle your backend. Gunicorn offers multiple workers to effectively make load balancing of the arriving requests. This means that you will have as many gunicorn running process as you specify to receive and process requests. From the doc, gunicorn should only need 4-12 worker processes to handle hundreds or thousands of requests per second. However, the number of workers should be no more than (2 x number_of_cpu_cores) + 1 to avoid running out of memory errors. You can check this out in the doc.
For example, if you want to use 4 workers for your fastapi-based backend, you can specify it with the flag w:
gunicorn main:app -w 4 -k uvicorn.workers.UvicornWorker -b "0.0.0.0:8888"
In this case, the script where I have my backend functionalities is called main and fastapi is instantiated as app.

I'm working on something like this using Docker and NGINX.
There's a Docker official image created by the guy who developed FastAPI that deploys uvicorn/gunicorn for you that can be configured to your needs:
It took some time to get the hang of Docker but I'm really liking it now. You can build an nginx image using the below configuration and then build x amount of your app inside of separate containers for however many you need to serve as hosts.
The below example is running a weighted load balancer for two of my app services with a backup third if those two should fail.
https://hub.docker.com/r/tiangolo/uvicorn-gunicorn-fastapi
nginx Dockerfile:
FROM nginx
# Remove the default nginx.conf
RUN rm /etc/nginx/conf.d/default.conf
# Replace with our own nginx.conf
COPY nginx.conf /etc/nginx/conf.d/
nginx.conf:
upstream loadbalancer {
server 192.168.115.5:8080 weight=5;
server 192.168.115.5:8081;
server 192.168.115.5:8082 backup;
}
server {
listen 80;
location / {
proxy_pass http://loadbalancer;
}
}
app Dockerfile:
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7
RUN pip install --upgrade pip
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . /app

How to deploy Visual Studio Flask app to Elastic Beanstalk

I am using the Visual Studio 2019 Flask Web Project template. It runs in my local Python environment and is a good start (very much like the standard Asp.net) template. I created a Python Elastic Beanstalk Application to host this. I am attempting to deploy this on AWS Elastic Beanstalk. I created a repository at: https://github.com/jlongo62/Flask-Web-Project
I think one of these files (or a missing file) needs to be named
application.py(it may also need some special content). I included a
directory listing.
It appears that I need to zip this and upload it
through the portal/cli. I am not sure what zip should look like, but
I suspect requirements.txt needs to be at the root.). If AWS Toolkit
Extension cannot handle this job, it should be easy to script.
Is there a better template or sample project in GitHub ?
Is the fix something simple ?
\FlaskWebProject1\FlaskWebProject1
\FlaskWebProject1\FlaskWebProject1.pyproj
\FlaskWebProject1\FlaskWebProject1.pyproj.user
\FlaskWebProject1\requirements.txt
\FlaskWebProject1\runserver.py
\FlaskWebProject1\FlaskWebProject1\static
\FlaskWebProject1\FlaskWebProject1\templates
\FlaskWebProject1\FlaskWebProject1\views.py
\FlaskWebProject1\FlaskWebProject1\__init__.py
\FlaskWebProject1\FlaskWebProject1\__pycache__
\FlaskWebProject1\FlaskWebProject1\static\content
\FlaskWebProject1\FlaskWebProject1\static\fonts
\FlaskWebProject1\FlaskWebProject1\static\scripts
\FlaskWebProject1\FlaskWebProject1\static\content\bootstrap.css
\FlaskWebProject1\FlaskWebProject1\static\content\bootstrap.min.css
\FlaskWebProject1\FlaskWebProject1\static\content\site.css
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.eot
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.svg
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.ttf
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.woff
\FlaskWebProject1\FlaskWebProject1\static\scripts\bootstrap.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\bootstrap.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.intellisense.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.min.map
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate-vsdoc.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.unobtrusive.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.unobtrusive.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\modernizr-2.6.2.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\respond.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\respond.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\_references.js
\FlaskWebProject1\FlaskWebProject1\templates\about.html
\FlaskWebProject1\FlaskWebProject1\templates\contact.html
\FlaskWebProject1\FlaskWebProject1\templates\index.html
\FlaskWebProject1\FlaskWebProject1\templates\layout.html
\FlaskWebProject1\FlaskWebProject1\__pycache__\views.cpython-37.pyc
\FlaskWebProject1\FlaskWebProject1\__pycache__\__init__.cpython-37.pyc

Your application works on Python 3.7 running on 64bit Amazon Linux 2/3.0.3 EB environment.
Just change runserver.py into application.py. Also you can slightly modify (port and name) its content, unless you want to customize EB environment to match your application' settings.
application.py
"""
This script runs the FlaskWebProject1 application using a development server.
"""
from os import environ
from FlaskWebProject1 import app as application
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '8000'))
except ValueError:
PORT = 8000
application.run(HOST, PORT)
Zip it into app.zip (example name) with content:
application.py
FlaskWebProject1
requirements.txt
Deploy the app.zip.

With thanks to Marcin.
AWS Toolkit for Visual Studio will not support deployment or environment creation. It only applies to .Net environments.
I have updated https://github.com/jlongo62/Flask-Web-Project to reflect these notes
Create AWS Elastic Beanstalk Environment. Use the portal or CLI.
Environments > Create environment (Web server environment) > Select
Settings:
Platform: Python
Platform Branch: Python 3.7
Platform Version: 3.0.3
Add application.py with content (the port 8000 seems to be a key ingredient):
"""
This script runs the FlaskWebProject1 application using a development server.
"""
from os import environ
from FlaskWebProject1 import app as application
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '8000'))
except ValueError:
PORT = 8000
application.run(HOST, PORT)
Create a zip with the following root directory. Zip file name does not matter:
application.py
FlaskWebProject1
requirements.txt
Deploy zip using the portal (or cli)
Elastic Beanstalk > Environments > Flaskwebproject1-env
click Upload and deploy

How to setup Celery to talk ssl to Azure Redis Instance

Using the great answer to "How to configure celery-redis in django project on microsoft azure?", I can configure Celery to use Azure Redis Cache using the non-ssl port, 6379, using the following Python code:
from celery import Celery
# This one works
url = 'redis://:<access key>#<my server>.redis.cache.windows.net:6379/0'
# I want to use a url that specifies ssl like one of the following:
# url = 'redis://:<my key>=#<my server>.redis.cache.windows.net:6380/0'
# url = 'redis://:<my key>#<my server>.redis.cache.windows.net:6380/0?ssl=True'
app = Celery('tasks', broker=url)
#app.task
def add(x, y):
return x + y
However, I would like to have celery use ssl and communicate on port 3380 using ssl to the Azure Redis Cache. If I change the port to 6380, I get an "Error while reading from socket" after a few minutes of waiting after running the following command:
celery -A tasks worker --loglevel=INFO -Q "celery" -Ofair
Does anyone know how to configure this, on the Celery or Azure side, so that I can have celery communicate on the default 3380 port on Azure Redis Cache using ssl?
I am using the latest version of Celery (4.0.2)
Note that code like the following works with no problem when connecting directly from a Linux client (on Azure) using port 3380 and ssl using Python's redis library:
import redis
redis.StrictRedis(host='<my host>.redis.cache.windows.net', port=6380, db=0, password='<my key>', ssl=True)

It's already possible using rediss:// instead redis://.
url = 'rediss://:<access key>#<my server>.redis.cache.windows.net:6380/0'

For the broker, you should be able to set the broker_use_ssl configuration option.
For the backend, the option redis_backend_use_ssl was made available in the 4.1.0 release.
The ability to enable SSL via the URL isn't available yet: https://github.com/celery/celery/issues/2833
Also, note that official support for Windows was dropped in 4.0. However, you might be able to get it working by following the instructions at https://github.com/celery/celery/issues/4082

How do I use python Bottle and Gunicorn with custom settings ini file?

I'm running a simple bottle application with gunicorn as a webserver, the application is working fine. My code:
from bottle import route, run, template
#route('/hello/<name>')
def index(name):
return template('<b>Hello {{name}}</b>!', name=name)
bottle.run(server='gunicorn', workers="3")
The Problem
Now I would like to create my own gunicorn config file and use it with bottle. I want to add a lot of extra functionality to the gunicorn workers (like SSL for example) and using a config file is a great way to do this.
I've tried this:
bottle.run(server='gunicorn', config="settings.py.ini")
AND
bottle.run(server='gunicorn', -c="settings.py.ini")
I know that in the CLI this the settings file can be set as an extra option like so:
-c CONFIG, --config CONFIG
gunicorn --config="settings.py.ini"
Anyone knows how to achieve the same thing when using the bottle gunicorn controller?

Solved it by taking a different aproach.
I'm using the code from this question: Bottle with Gunicorn
This makes it possible to use gunicorn from the CLI and load the bottle app in gunicorn. Now I can use the CLI to use gunicorn with config file, it works with bottle. Bottle is now basicly an Gunicorn module.

Deploying CherryPy (daemon)

I've followed the basic CherryPy tutorial (http://www.cherrypy.org/wiki/CherryPyTutorial). One thing not discussed is deployment.
How can I launch a CherryPy app as a daemon and "forget about it"? What happens if the server reboots?
Is there a standard recipe? Maybe something that will create a service script (/etc/init.d/cherrypy...)
Thanks!

Daemonizer can be pretty simple to use:
# this works for cherrypy 3.1.2 on Ubuntu 10.04
from cherrypy.process.plugins import Daemonizer
# before mounting anything
Daemonizer(cherrypy.engine).subscribe()
cherrypy.tree.mount(MyDaemonApp, "/")
cherrypy.engine.start()
cherrypy.engine.block()
There is a decent HOWTO for SysV style here.
To summarize:
Create a file named for your application in /etc/init.d that calls /bin/sh
sudo vim /etc/init.d/MyDaemonApp
#!/bin/sh
echo "Invoking MyDaemonApp";
/path/to/MyDaemonApp
echo "Started MyDaemonApp. Tremble, Ye Mighty."
Make it executable
sudo chmod +x /etc/init.d/MyDaemonApp
Run update-rc.d to create our proper links in the proper runtime dir.
sudo update-rc.d MyDaemonApp defaults 80
sudo /etc/init.d/MyDaemonApp

There is a Daemonizer plugin for CherryPy included by default which is useful for getting it to start but by far the easiest way for simple cases is to use the cherryd script:
> cherryd -h
Usage: cherryd [options]
Options:
-h, --help show this help message and exit
-c CONFIG, --config=CONFIG
specify config file(s)
-d run the server as a daemon
-e ENVIRONMENT, --environment=ENVIRONMENT
apply the given config environment
-f start a fastcgi server instead of the default HTTP
server
-s start a scgi server instead of the default HTTP server
-i IMPORTS, --import=IMPORTS
specify modules to import
-p PIDFILE, --pidfile=PIDFILE
store the process id in the given file
As far as an init.d script goes I think there are examples that can be Googled.
And the cherryd is found in your:
virtualenv/lib/python2.7/site-packages/cherrypy/cherryd
or in: https://bitbucket.org/cherrypy/cherrypy/src/default/cherrypy/cherryd

I wrote a tutorial/project skeleton, cherrypy-webapp-skeleton, which goal was to fill the gaps for deploying a real-world CherryPy application on Debian* for a web-developer. It features extended cherryd for daemon privilege drop. There's also a number of important script and config files for init.d, nginx, monit, logrotate. The tutorial part describes how to put things together and eventually forget about it. The skeleton part proposes a way of possible arrangement of CherryPy webapp project assets.
* It was written for Squeeze but practically it should be same for Wheezy.

Info on Daemonizer options
When using Daemonizer, the docs don't state the options, e.g. how to redirect stdout or stderr. From the source of the Daemonizer class you can find the options. As a reference take this example from my project:
# run server as a daemon
d = Daemonizer(cherrypy.engine,
stdout='/home/pi/Gate/log/gate_access.log',
stderr='/home/pi/Gate/log/gate_error.log')
d.subscribe()