I have a complex API which takes around 7GB memory when I deploy it using Uvicorn.
I want to understand how I can deploy it, such a way that from my end I want to be able to make parallel requests. The deployed API should be capable of processing two or three requests at same time.
I am using FastAPI with uvicorn and nginx for deployment. Here is my deployed command.
uvicorn --host 0.0.0.0 --port 8888
Can someone provide some clarity on how people achieve that?
You can use gunicorn instead of uvicorn to handle your backend. Gunicorn offers multiple workers to effectively make load balancing of the arriving requests. This means that you will have as many gunicorn running process as you specify to receive and process requests. From the doc, gunicorn should only need 4-12 worker processes to handle hundreds or thousands of requests per second. However, the number of workers should be no more than (2 x number_of_cpu_cores) + 1 to avoid running out of memory errors. You can check this out in the doc.
For example, if you want to use 4 workers for your fastapi-based backend, you can specify it with the flag w:
gunicorn main:app -w 4 -k uvicorn.workers.UvicornWorker -b "0.0.0.0:8888"
In this case, the script where I have my backend functionalities is called main and fastapi is instantiated as app.
I'm working on something like this using Docker and NGINX.
There's a Docker official image created by the guy who developed FastAPI that deploys uvicorn/gunicorn for you that can be configured to your needs:
It took some time to get the hang of Docker but I'm really liking it now. You can build an nginx image using the below configuration and then build x amount of your app inside of separate containers for however many you need to serve as hosts.
The below example is running a weighted load balancer for two of my app services with a backup third if those two should fail.
https://hub.docker.com/r/tiangolo/uvicorn-gunicorn-fastapi
nginx Dockerfile:
FROM nginx
# Remove the default nginx.conf
RUN rm /etc/nginx/conf.d/default.conf
# Replace with our own nginx.conf
COPY nginx.conf /etc/nginx/conf.d/
nginx.conf:
upstream loadbalancer {
server 192.168.115.5:8080 weight=5;
server 192.168.115.5:8081;
server 192.168.115.5:8082 backup;
}
server {
listen 80;
location / {
proxy_pass http://loadbalancer;
}
}
app Dockerfile:
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7
RUN pip install --upgrade pip
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . /app
I am trying to use python image to build and test a very simple python project. But when I give Docker Image name in Jenkinsfile, it fails to pull the image.
[drone-python_test-jenk-NVHH77CLU5PUMV6UVRK62EARJB3DUVF5FWILYVRZDXOE54RACN2Q] Running shell script
+ docker pull python
Using default tag: latest
Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon. Is the docker daemon running on this host?). Using system default: https://index.docker.io/v1/
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
script returned exit code 1
JenkinsFile
pipeline {
agent {
docker {
image 'python'
}
}
stages {
stage('Build') {
steps {
sh '''virtualenv --no-site-packages .env
'''
sh '.env/bin/pip install -r dev-requirements.txt'
}
}
stage('Test') {
steps {
sh 'flake8 setup.py drone tests'
}
}
stage('test2') {
steps {
sh 'nosetests --with-coverage --cover-package drone -v'
}
}
}
Edit:
Tried mounting the docker host using a docker-compose file
version: '2'
services:
jenkins:
image: jenkinsci/blueocean
ports:
- 8888:8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock
restart: always
also added user jenkins to docker group(is this correct user?)
But it still did not work. DroneIO also used docker images to setup environment but it did not have this issue.
I have encountered the same problem.
The Docker daemon inside container seems not started. I started if manually by docker exec -u root jenkins /bin/sh -c 'dockerd >/var/log/docker.log 2>&1 &'. It seems to work.
Now, i try to build a Dockerfile that exends jenkins/blueocean:latest with a modification af entry point script to start docker daemon automatically.
I have a django application. I can't seem to get nginx to serve the static files correctly, but that's not this issue.
The current issue is that when following this guide: https://gist.github.com/evildmp/3094281
I try to use the unix sockets rather than the web socket.
e.g.
server unix:///tmp/uwsgi.sock; # for a file socket
#server 127.0.0.1:8001; # for a web port socket
using the /tmp/uwsgi.sock socket rather than the 127.0.0.1:8001 web port socket.
The issue is that when I use the web port socket, and I navigate to domainname.com:8001 I reach the website as served by uWSGI, but there are no staticfiles loaded. So that means that at least uwsgi is working. But when I switch over to the file socket, I cannot even get it to work at all.
What am I doing wrong?
Here is my nginx.conf:
# nginx.conf
upstream django {
# connect to this socket
# server unix:///tmp/uwsgi.sock; # for a file socket (TRYING TO USE)
server 127.0.0.1:8001; # for a web port socket (RATHER THAN THIS)
}
server {
# the port your site will be served on
listen 8000;
# the domain name it will serve for
server_name .cshenkan.com; # substitute your machine's IP address or FQDN
charset utf-8;
#Max upload size
client_max_body_size 75M; # adjust to taste
# Django media
location /media {
alias /home/ubuntu/sasite-rewrite/media; # your Django pro$
}
location /static {
alias /home/ubuntu/sasite-rewrite/static; # your Django pro$
}
location /assets {
alias /home/ubuntu/sasite-rewrite/assets
}
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params; # or the uwsgi_params you installe$
}
}
And my base.py settings file snippet:
STATIC_ROOT = normpath(join(SITE_ROOT, 'static'))
STATIC_URL = '/static/'
STATICFILES_DIRS = (
normpath(join(SITE_ROOT, 'static')),
normpath(join(SITE_ROOT, 'assets')),
)
And how I run uwsgi when using the web socket:
uwsgi --http :8001 --chdir /home/ubuntu/sasite-rewrite --wsgi-file /home/ubuntu/sasite-rewrite/sasite/wsgi.py
And how I run it when using the unix socket:
uwsgi --socket /tmp/uwsgi.sock --chdir /home/ubutnu/sasite-rewrite --wsgi-file /home/ubuntu/sasite-rewrite/sasite/wsgi.py
I just can't seem to get it to work when using the unix socket.
Neither can I get nginx to serve the correct static files, for example if I add a file to the media directory called 1.png or 1.txt and try to access them with domainname.com:8000/media/1.png I always get a hang or a no response from server error.
What am I doing wrong?
How can I get the unix socket to work correctly? And is the command I'm using correct?
I also don't understand this, when I use the web socket, I can connect to domainname.com:8001 and get the page with no static files, it looks like crap but loads from uwsgi. But with the unix sockets I have no idea how to access the page served by uwsgi to see if it works with the unix socket. Before I can move onto getting nginx working, I need to solve this issue, as I tried using web sockets with nginx and still had no luck there, I'm hoping once my unix sockets are set up correctly nginx will work better.
Any suggestions, or examples you could provide? Would be much appreciated, I am a long time django programmer, but I have very rarely deployed the sites myself. So I am really struggling here.
Any help is much appreciated, thanks.
Today I had the same problem with NGINX + Socket Unix for uwsgi protocol.
I discovered that version 1.8 of the NGINX is bugged.
Try installing version 1.10 that will work via UNIX SOCKET.
If you use open-SuSE, you can get the rpm here:
http://download.opensuse.org/repositories/home:/ghostlyrat/openSUSE_Leap_42.2/x86_64/nginx-1.10.1-3.3.x86_64.rpm
Or, directly in the official nginx downloads area:
http://nginx.org/en/download.html
At the moment I am running python manage.py test every once in a while after I make significant changes in my django project. Is it possible to run those tests automatically whenever I change and save a file in my project? It'll be useful to detect bugs earlier (I know rails has something like this with rspec). I am using nose and django-nose. Thanks in advance.
Use entr:
$ brew install entr
$ find . -name '*.py' | entr python ./manage.py test
Or, for extra credit, combine it with ack:
$ ack --python | entr python ./manage.py test
If you want it to even find new files as you add them:
$ until ack -f --python | entr -d python ./manage.py test; do sleep 1; done
py.test answer (which also works for nose):
pip install pytest-xdist
py.test -f # will watch all subfolders for changes, and rerun the tests
Since py.test understands nose, this works for nose too.
I'm a JavaScript developer so I used the tools JS developer have built with Node.js to achieve the same goal in my projects. It is very simple but you also need to install nodeJS to get it working.
I created a file called gruntfile.js in my project root directory:
//gruntfile.js
module.exports = function(grunt) {
grunt.initConfig({
watch: {
files: ['*.py'],
tasks: ['shell']
},
shell: {
test: {
command: 'python main_test.py'
}
}
});
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-shell');
grunt.registerTask('default', ['watch']);
};
What it's doing is basically watching any file in that directory that has a py extension and if they changed it execute a shell command which in this case is my python test (you might wanna change it, my test name was main_test.py). In order to run this grunt script you need to install Node.js and after that you will have npm in your global path. after that you need to insall a few node modules as well. All these modules except grunt-cli will be stored in your current folder so make sure you are at the root of your project or what ever folder you put that gruntfile.js in. then run the fallowing commands.
npm install grunt-cli -g
npm install grunt
npm install grunt-contrib-watch
npm install grunt-shell
Don't worry about the size, these are very small modules. Now that you have every thing setup you can simply run grunt and it will start watching your py files and when you saved them it will run your tests. It may not be best way for running python tests but as I said I'm a JavaScript developer and I think Grunt has provided a very simple way of executing tests even for other languages so I use it.
I just tried nose-watch and it worked fantastic! install the plugin and run the test with the --with-watch option.
Update: :( it does not seem to work well when running tests from django-nose's manage.py helper.
Eventually I opted to use tdaemon, which supports django, although might require a bit of fiddling as well for full fledged projects.
For example here is how I ran it for my django project:
tdaemon -t django --custom-args=a_specific_app_to_test -d --ignore-dirs logs
The --custom-args was to focus tests to specific app (same as you would do python manage.py test a_specific_app_to_test
The -d argument is to enable debug logging, which prints which file change triggered the run.
The --ignore-dirs was necessary because my tests wrote to the logs (which in itself is a problem!) and tdaemon went into an infinite loop.
Another Javascript dev here, I've found nodemon (https://github.com/remy/nodemon) to work pretty well. By default it watches *.js files but that's configurable with the --ext flag. To use it, do:
npm install -g nodemon
cd /your/project/dir/
nodemon --ext py --exec "python manage.py test"
Now, whenever a *.py file changes, it'll re-run your command. It even finds new files.
I'm used watchr, something like Watchr
I did this using gulp. Install gulp-shell:
npm install gulp-shell --save-dev
And in the gulpfile.js:
var shell = require('gulp-shell')
gulp.task('run-tests', shell.task([
'python3 manage.py test']))
gulp.task('watch', function(){
gulp.watch(['./your-project-name/**/*.html', './your-project-name/**/*.py'], ['run-tests']);
});
gulp.task('default',['run-tests','watch']);
And it runs the tests anytime there are changes saved to any .py or .html files!
You can use Django Supervisor on top of Django. This will avoid the use of a CI tool (which may be useful in any case, this isn't invalidating the other response - maybe just complementary).
I would recommend setting up django-nose and sniffer. It's quite easy to setup and works great. Something along the lines of this scent.py was the only customization I needed. Then you can just run sniffer -x myapp.tests.
Nose comes with some other goodies that make tests a bit nicer to work with as well.
if you use git control code, another way to is use git hook pre-commit
maybe error like remote: fatal: Not a git repository: '.', check this post https://stackoverflow.com/a/4100577/7007942
I've found the easiest way is to use gulp as recommended by this post. Note that gulp-shell (recommended by other answers) is actually blacklisted by gulp, but thankfully you don't even need a plugin. Try this instead:
// gulpfile.js
const { watch } = require('gulp')
var exec = require('child_process').exec
function test (cb) {
exec('python manage.py test', function (err, stdout, stderr) {
console.log(stdout)
console.log(stderr)
cb(err)
})
}
exports.default = function () {
watch('./**/*.py', test)
}
In the past, I've tried many of the options suggested in other answers. This one was comparatively painless. It's helpful if you have some knowledge of JavaScript.
I wrote a Gulp task to automatically run ./manage.py test whenever any specified Python files are changed or removed. You'll need Node for this.
First, install Gulp:
yarn add -D gulp#next
Then use the following gulpfile.js and make any necessary adjustments:
const { exec } = require('child_process');
const gulp = require('gulp');
const runDjangoTests = (done) => {
const task = exec('./manage.py test --keepdb', {
shell: '/bin/bash', // Accept SIGTERM signal. Doesn't work with /bin/sh
});
task.stdout.pipe(process.stdout);
task.stderr.pipe(process.stderr);
task.on('exit', () => {
done();
});
return task;
};
gulp.task('test', runDjangoTests);
gulp.task('watch', (done) => {
let activeTask;
const watcher = gulp.watch('**/*.py', {
// Ignore whatever directories you need to
ignored: ['.venv/*', 'node_modules'],
});
const runTask = (message) => {
if (activeTask) {
activeTask.kill();
console.log('\n');
}
console.log(message);
activeTask = runDjangoTests(done);
};
watcher.on('change', (path) => {
runTask(`File ${path} was changed. Running tests:`);
});
watcher.on('unlink', (path) => {
runTask(`File ${path} was removed. Running tests:`);
});
});
Then simply run node node_modules/gulp/bin/gulp.js watch to run the task :)
https://pycrunch.com/
One of key features - run only impacted tests on file changes. Available as plugin for PyCharm
On ubuntu this script works, after installing inotifywait (sudo apt install inotify-tools):
inotifywait -qrm -e close_write * |
while read -r filename event; do
python manage.py test accord
done
Similar to entr you can use watchexec with
brew install watchexec
watchexec -rc -e py -- python ./manage.py test
with options
-r # restart
-c # clear screen
-e extensions # list of file extensions to filter by
You'll need a continuous integration server, something like Jenkins.
am trying to configure uwsgi and in the process it says on a tutorial that I must run
uwsgi -s /tmp/uwsgi.sock -w myapp:app
the problem is -w is an invalid option. Can anyone help me point out why or what should I do?
Thanks
maybe you are using debian-supplied packages. They are fully modular so you need to install/load the required plugins:
http://projects.unbit.it/uwsgi/wiki/Quickstart
My uwsgi app configuration looks like that
/etc/uwsgi/apps-enabled/mysite.ini
[uwsgi]
socket=/tmp/uwsgi_mysite.sock
chmod-socket=666
abstract-socket=false
master=true
workers=2
uid=altera
gid=altera
chdir=/home/altera/www/mysite ; Current dir
pp=/home/altera/www/mysite ; Python Path (to your application)
pyhome=/home/altera/vpy/mysite ; Path to virtual environment
plugins=python3
module=main ; *.py file name application starting from
post-buffering=8192
/etc/nginx/sites-available/mysite
server {
server_name mysite;
root /home/altera/www/mysite;
location / {
include uwsgi_params;
uwsgi_pass unix:/tmp/uwsgi_mysite.sock;
}
}