We Keep Coding

uwsgi upstream prematurely closed connection while reading response header from upstream flask+uwsgi+nginx

I'm trying to deploy a flask project (uwsgi + Nginx) on ubuntu 20.04 server. For this I'm following Digital Ocean.
I get 502 bad gateway nginx/1.18.0 (Ubuntu) error.
After checking nginx error logs I find
upstream prematurely closed connection while reading response header from upstream, client: 93.37.157.148, server: 3.75.132.146, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://unix:/home/ubuntu/flask-app/webapp/flask-app.sock:", host: "3.75.132.146", referrer: "http://3.75.132.146/"
My .ini file includes:
[uwsgi]
module = wsgi:app
wsgi-file = /home/ubuntu/flask-app/webapp/wsgi.py
chdir = /home/ubuntu/flask-app/webapp/
protocol = http
#home = /usr/local/lib/python3.8/dist-packages/
master = true
processes = 5
limit-as = 512
uid = ubuntu
gid = www-data
socket = flask-app.sock
chown-socket = ubuntu
chmod-socket = 777
vacuum = true
die-on-term = true
And my nginx config files includes:
server {
listen 80;
server_name 3.75.132.146;
location /static {
alias /home/ubuntu/flask-app/webapp/static/;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/home/ubuntu/flask-app/webapp/flask-app.sock;
}
}
I have been searching online and no one seems to have a solid solution for this. Please help me if anyone already faced this and solved this issue.
Thanks

Do we need to use runserver command in production to start our django project?

Do we need to use runserver command in production to start our project? If yes, then how to do it, and If No then how does out project server starts?

The most common way starting Django in production for now is using UWSGI (https://uwsgi-docs.readthedocs.io/en/latest/) (or an asynchronous version ASGI)
You need config like this:
[uwsgi]
socket = 0.0.0.0:3300
chdir = /app/src
pythonpath=/app/src
wsgi-file = /app/src/django_app/wsgi.py
env = DJANGO_SETTINGS_MODULE=django_app.settings
module = django.core.wsgi:get_wsgi_application()
max-requests = 1000
harakiri = 60
buffer-size = 65535
no-orphans = true
touch-reload = /var/run/uwsgi-touch-reload
uid = www-data
gid = www-data
master = 1
workers = 8
And then you start your application with:
#!/usr/bin/env bash
uwsgi --ini /app/uwsgi.ini
And then your HTTP server, for example nginx, sends requests to uwsgi application. Nginx config my be like this:
upstream django {
server django:3300;
}
location / {
include uwsgi_params;
default_type text/html;
uwsgi_pass django;
uwsgi_param HTTP_X_FORWARDED_PROTOCOL 'https';
}

Timeout with Flask/uWSGI/nginx app using mongodb

I have a Flask python web app on uWSGI/nginx that works fine, except when I use pymongo, specifically when I initialize the MongoClient class. I get the following nginx error when I try to access the app while using pymongo:
019/02/19 21:58:13 [error] 16699#0: *5 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 127.0.0.1, server: example.com, request: "GET /api/test HTTP/1.1", upstream: "uwsgi://unix:/var/www/html/myapp/myapp.sock:”, host: “example.com”
My small test app:
from flask import Flask
from flask_cors import CORS
from bson.json_util import dumps
import pymongo
DEBUG = True
app = Flask(__name__)
app.config.from_object(__name__)
CORS(app)
client = pymongo.MongoClient() # This line
db = client.myapp
#app.route('/api/test')
def test():
item = db.items.find_one()
return item['name']
def create_app(app_name='MYAPP'):
return app
# if __name__ == '__main__':
# app.run(debug=True, threaded=True, host='0.0.0.0')
If I run this app from the command line (python app.py) it works fine accessing 0.0.0.0:5000/api/test, so I'm pretty sure it's just a uWSGI configuration issue. My first thought was to increase the uwsgi_read_timeout parameter in my nginx config file:
uwsgi_read_timeout 3600
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name example.com www.example.com;
location /api {
include uwsgi_params;
uwsgi_read_timeout 3600;
uwsgi_pass unix:/var/www/html/myapp/myapp.sock;
}
location / {
root /var/www/html/myapp;
try_files $uri $uri/ /index.html;
}
#return 301 https://$server_name$request_uri;
}
But it had no apparent effect. My uWSGI app is running as a service, using the following config (myapp.ini):
[uwsgi]
module = wsgi:app
master = true
processes = 4
enable-threads = True
socket = /var/www/html/myapp/myapp.sock
chmod-socket = 660
vacuum = true
die-on-term = true
Again, everything seems to work fine except for when I try to initialize pymongo. Finally, my app's service file:
[Unit]
Description=uWSGI Python container server
After=network.target
[Service]
User=pi
Group=www-data
WorkingDirectory=/var/www/html/myapp
ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/apps-available/myapp.ini
[Install]
WantedBy=multi-user.target

I believe the issue is that you're forking and this causes issues with PyMongo.
PyMongo is thread safe but not Fork safe. Once you run the app in daemon mode you are forking the process. You'll have to create a MongoClient inside the app so that your threads can see it after the process has started.
You can try this(I didn't try this out, I normally wrap stuff like this in a class and do this in the init method):
def create_app(app_name='MYAPP'):
app.client = pymongo.MongoClient(connect=False) # this will prevent connecting until you need it.
app.db = app.client.myapp
return app
Read this: http://api.mongodb.com/python/current/faq.html#id3

UWSGI crash when started as service and POST big file: ConnectionError

I'm facing an inexplicable problem with uwsgi: Crashs irregular happens when upload a big file size. Scenario:
Context
For a simple wsgi application, here a python Flask application in /home/bastien/Projects/test_upload/wsgi.py:
# -*- coding: utf-8 -*-
from flask import Flask, request
app = Flask(__name__)
#app.route('/', methods=['POST'])
def hello_world():
f = request.files['file'].read()
return 'Hello, World! {}'.format(len(f))
application = app
No crash when:
Use this configuration uwsgi file, /etc/uwsgi/apps-available/test_upload.ini:
[uwsgi]
plugins = python3
chdir = /home/bastien/Projects/test_upload/tracim
home = /home/bastien/Projects/test_upload/venv3.4
module = wsgi
callable = application
enable-threads = true
env = PYTHON_EGG_CACHE=/tmp
limit-post = 0
chmod-socket = 660
vacuum = true
Run uwsgi with:
uwsgi -i /etc/uwsgi/apps-available/test_upload.ini --http-socket :6543
And send file (~262Mo) with /httpie:
http -h -f POST :6543 'file#/home/bastien/Téléchargements/pycharm-professional-2017.2.3.tar.gz'
HTTP request can be repeated, no crash.
Crash when:
Use this configuration uwsgi file, /etc/uwsgi/apps-available/test_upload.ini with symbolic link into /etc/uwsgi/apps-enabled:
[uwsgi]
plugins = python3
chdir = /home/bastien/Projects/test_upload/tracim
home = /home/bastien/Projects/test_upload/venv3.4
module = wsgi
callable = application
http-socket = :4321
enable-threads = true
env = PYTHON_EGG_CACHE=/tmp
limit-post = 0
chmod-socket = 660
vacuum = true
Note: Only difference is http-socket = :4321
Run uwsgi with service uwsgi start (on debian 8.9) and send file with:
http -h -f POST :4321 'file#/home/bastien/Téléchargements/pycharm-professional-2017.2.3.tar.gz'
This request will work one time, sometimes two times:
HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Length: 5188
Content-Type: text/html; charset=utf-8
Pragma: no-cache
But finally crash with:
http: error: ConnectionError: ('Connection aborted.', BadStatusLine("''",)) while doing POST request to URL: http://localhost:4321/
Note: Any wsgi application can be used for reproduce
Note: No log is produced by uwsgi or application about this "error"
Summary:
The error is not conistent and only difference is uwsgi usage as service with:
Debian 8.9
uwsgi 2.0.7-1+deb8u1 apt installed
Question
How can produce this difefrence ? Where can i search to know how is started uwsgi by service command ?

Problem solved when use Debian 9 with uwsgi in apt version 2.0.14+20161117-3.

upstream timed out error in Python Flask web app using Fabric on nginx

I have my Python Flask web app hosted on nginx. While trying to execute a request it shows a timeout error in the nginx error log as shown below :
[error] 2084#0: *1 upstream timed out (110: Connection timed out)
while reading response header from upstream, client:
192.168.2.224, server: 192.168.2.131, request: "POST /execute HTTP/1.1", upstream: "uwsgi://unix:/hom
e/jay/PythonFlaskApp/app.sock", host: "192.168.2.131:9000", referrer:
"http://192.168.2.131:9000/"
If I try to run the app locally it works fine and responds fine.
Any one have any idea what might be wrong ?
the error found in browser console is :
Gateway Time-out
Here is the nginx config file:
server {
listen 9000;
server_name 192.168.2.131;
location / {
include uwsgi_params;
proxy_read_timeout 300;
uwsgi_pass unix:/home/jay/PythonFlaskApp/app.sock;
}
}
And here is the Python Fabric code that i trying to execute. i'm not sure if this is causing the issue, but any waz here is the code :
from fabric.api import *
#application.route("/execute",methods=['POST'])
def execute():
try:
machineInfo = request.json['info']
ip = machineInfo['ip']
username = machineInfo['username']
password = machineInfo['password']
command = machineInfo['command']
isRoot = machineInfo['isRoot']
env.host_string = username + '#' + ip
env.password = password
resp = ''
with settings(warn_only=True):
if isRoot:
resp = sudo(command)
else:
resp = run(command)
return jsonify(status='OK',message=resp)
except Exception, e:
print 'Error is ' + str(e)
return jsonify(status='ERROR',message=str(e))
I have a uWSGi config file for the web app and started it using an upstart script. Here is uwSGi conf file :
[uwsgi]
module = wsgi
master = true
processes = 5
socket = app.sock
chmod-socket = 660
vacuum = true
die-on-term = true
and here is upstart script
description "uWSGI server instance configured to serve Python Flask App"
start on runlevel [2345]
stop on runlevel [!2345]
setuid jay
setgid www-data
chdir /home/jay/PythonFlaskApp
exec uwsgi --ini app.ini
I have followed the below tutorial on running flask app on nginx

This is likely a problem with the Fabric task, not with Flask. Have you tried isolating / removing Fabric from the application, just for troubleshooting purposes? You could try stubbing out a value for resp, rather than actually executing the run/sudo commands in your function. I would bet that the app works just fine if you do that.
And so that would mean that you've got a problem with Fabric executing the command in question. First thing you should do is verify this by mocking up an example Fabfile on the production server using the info you're expecting in one of your requests, and then running it with fab -f <mock_fabfile.py>.
It's also worth noting that using with settings(warn_only=True): can result in suppression of error messages. I think that you should remove this, since you are in a troubleshooting scenario. From the docs on Managing Output:
warnings: Warning messages. These are often turned off when one expects a given operation to fail, such as when using grep to test existence of text in a file. If paired with setting env.warn_only to True, this can result in fully silent warnings when remote programs fail. As with aborts, this setting does not control actual warning behavior, only whether warning messages are printed or hidden.
As a third suggestion, you can get more info out of Fabric by using the show('debug') context manager, as well as enabling Paramiko's logging:
from fabric.api import env
from fabric.context_managers import show
# You can also enable Paramiko's logging like so:
import logging
logging.basicConfig(level=logging.DEBUG)
def my_task():
with show('debug'):
run('my command...')
The Fabric docs have some additional suggestions for troubleshooting: http://docs.fabfile.org/en/1.6/troubleshooting.html. (1.6 is an older/outdated version, but the concepts still apply.)