I want to integrate Flask-SocketIO with my Flask project. My app is running behind a Nginx reverse proxy:
location /plivo_api{
rewrite ^/plivo_api(.*) /$1 break;
proxy_pass http://127.0.0.1:8090;
}
So I undestand all trafic received in the /plivo_api will be rewrited as "/" port 8090. This part work well.
The problem starts when I want to connect to the socket. Direct connection to socket has no problem.
# all those examples work
# from localhost
var socket = io.connect()
var socket = io.connect('http://localhost:8090/')
# running the app out of the reverse proxy
var socket = io.connect('http://my_server:8090/')
But throught Nginx I cannot connect
# Bad Gateway
var socket = io.connect('http://my_server/plivo_api')
Question is Do I'm missing something to connect to my socketio app or there's something extra to add to Nginx config ?
The flask app code with socketio integration looks like
# this code work well, the flask app and socket io
# problem must be in Ngin settings.
The flask app code with socketio integration looks like
from flask import Flask, render_template
from flask_socketio import SocketIO, emit
HOST = '127.0.0.1'
PORT = 8090
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret'
app.config['DEBUG'] = True
app.config['SERVER_NAME'] = f'{HOST}:{PORT}'
socketio = SocketIO(app)
#app.route('/')
def index():
return render_template('index.html')
if __name__ == '__main__':
socketio.run(app, port=PORT, host=HOST)
You need to create a special location block in nginx for the Socket.IO endpoint. You can't use a regular URL like you do for your HTTP routes.
The documentation has an example:
server {
listen 80;
server_name _;
location /socket.io {
include proxy_params;
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass http://127.0.0.1:5000/socket.io;
}
}
Related
I edited my question:
I was trying to change my flask server to a production-level server.
Unfortunately, I am failing at the moment to do so even with the example flask app:
When trying to connect without HTTPS the site works fine, when connecting with HTTPS I get a "This site can't be reached" error.
My nginx config:
server {
listen 443 ssl;
ssl_certificate path/cert.pem;
ssl_certificate_key path/key.pem;
server_name 127.0.0.1;
location / {
proxy_pass http://127.0.0.1:5000;
proxy_set_header X-Real-IP $remote_addr;
}
}
server {
listen 80;
server_name 127.0.0.1;
return 302 https://$server_name$request_uri;
}
server {
listen 5000;
server_name 127.0.0.1;
return 302 https://$server_name$request_uri;
}
After edditing my config i used this command in order for nginx to recognize the new config:
sudo ln -s /etc/nginx/sites-available/default.conf /etc/nginx/sites-enabled
My flask app:
from flask import Flask
from waitress import serve
import logging
app = Flask(__name__)
#app.route("/")
def hello():
return "<h1 style='color:blue'> A very simple flask server !</h1>"
if __name__ == "__main__":
# app.run(host='0.0.0.0', port=8080)
logger = logging.getLogger('waitress')
logger.setLevel(logging.INFO)
serve(app, host='127.0.0.1', port=5000, url_scheme='https')
Thanks a lot in advance for all your help!
I have a Flask python web app on uWSGI/nginx that works fine, except when I use pymongo, specifically when I initialize the MongoClient class. I get the following nginx error when I try to access the app while using pymongo:
019/02/19 21:58:13 [error] 16699#0: *5 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 127.0.0.1, server: example.com, request: "GET /api/test HTTP/1.1", upstream: "uwsgi://unix:/var/www/html/myapp/myapp.sock:”, host: “example.com”
My small test app:
from flask import Flask
from flask_cors import CORS
from bson.json_util import dumps
import pymongo
DEBUG = True
app = Flask(__name__)
app.config.from_object(__name__)
CORS(app)
client = pymongo.MongoClient() # This line
db = client.myapp
#app.route('/api/test')
def test():
item = db.items.find_one()
return item['name']
def create_app(app_name='MYAPP'):
return app
# if __name__ == '__main__':
# app.run(debug=True, threaded=True, host='0.0.0.0')
If I run this app from the command line (python app.py) it works fine accessing 0.0.0.0:5000/api/test, so I'm pretty sure it's just a uWSGI configuration issue. My first thought was to increase the uwsgi_read_timeout parameter in my nginx config file:
uwsgi_read_timeout 3600
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name example.com www.example.com;
location /api {
include uwsgi_params;
uwsgi_read_timeout 3600;
uwsgi_pass unix:/var/www/html/myapp/myapp.sock;
}
location / {
root /var/www/html/myapp;
try_files $uri $uri/ /index.html;
}
#return 301 https://$server_name$request_uri;
}
But it had no apparent effect. My uWSGI app is running as a service, using the following config (myapp.ini):
[uwsgi]
module = wsgi:app
master = true
processes = 4
enable-threads = True
socket = /var/www/html/myapp/myapp.sock
chmod-socket = 660
vacuum = true
die-on-term = true
Again, everything seems to work fine except for when I try to initialize pymongo. Finally, my app's service file:
[Unit]
Description=uWSGI Python container server
After=network.target
[Service]
User=pi
Group=www-data
WorkingDirectory=/var/www/html/myapp
ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/apps-available/myapp.ini
[Install]
WantedBy=multi-user.target
I believe the issue is that you're forking and this causes issues with PyMongo.
PyMongo is thread safe but not Fork safe. Once you run the app in daemon mode you are forking the process. You'll have to create a MongoClient inside the app so that your threads can see it after the process has started.
You can try this(I didn't try this out, I normally wrap stuff like this in a class and do this in the init method):
def create_app(app_name='MYAPP'):
app.client = pymongo.MongoClient(connect=False) # this will prevent connecting until you need it.
app.db = app.client.myapp
return app
Read this: http://api.mongodb.com/python/current/faq.html#id3
I have a Flask app with bjoern as python server. An example url I have is something like:
http://example.com/store/junihh
http://example.com/store/junihh/product-name
Where "junihh" and "product-name" are arguments that I need to pass to python.
I try to use unix socket after reading about the performance against TCP/IP calls. But now I get a 502 error on the browser.
This is an snippet of my conf:
upstream backend {
# server localhost:1234;
# server unix:/run/app_stores.sock weight=10 max_fails=3 fail_timeout=30s;
server unix:/run/app_stores.sock;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name example.com www.example.com;
root /path/to/my/public;
location ~ ^/store/(.*)$ {
include /etc/nginx/conf.d/jh-proxy-pass.conf;
include /etc/nginx/conf.d/jh-custom-headers.conf;
proxy_pass http://backend/$1;
}
}
How to pass the url arguments to Flask through Nginx proxy_pass with unix socket?
Thanks for any help.
Here is my conf, it can works. 502 is because it cannot find route to the upstream server(ie. change http://127.0.0.1:5000/$1 to http://localhost:5000/$1) will cause 502.
nginx.conf
http {
server {
listen 80;
server_name localhost;
location ~ ^/store/(.*)$ {
proxy_pass http://127.0.0.1:5000/$1;
}
}
}
flask app.py
#!/usr/bin/env python3
from flask import Flask
app = Flask(__name__)
#app.route('/')
def world():
return 'world'
#app.route('/<name>/<pro>')
def shop(name, pro):
return 'name: ' + name + ', prod: ' + pro
if __name__ == '__main__':
app.run(debug=True)
Update
or you can use unix socket like this, but relay on uwsgi.
nginx.conf
http {
server {
listen 80;
location /store {
rewrite /store/(.+) $1 break;
include uwsgi_params;
uwsgi_pass unix:/tmp/store.sock;
}
}
}
flask app.py
like above, not change
uwsgi config
[uwsgi]
module=app:app
plugins=python3
master=true
processes=1
socket=/tmp/store.sock
uid=nobody
gid=nobody
vaccum=true
die-on-term=true
save as config.ini, then run uwsgi config.ini
after nginx reload, you can visit your page ;-)
Flask, uwsgi, nginx mongo socket application.
Periodically produces this errors
File "/home/username/venv/local/lib/python2.7/site-packages/engineio/server.py", line 277, in handle_request
start_response(r['status'], r['headers'] + cors_headers)
IOError: headers already sent
File "/home/username/venv/local/lib/python2.7/site-packages/flask_socketio/__init__.py", line 562, in _handle_event
app = self.server.environ[sid]['flask.app']
KeyError: 'flask.app'
All configs here:
uwsgi.ini
[uwsgi]
module = socket_app:app
chdir = /home/username/app_dir/
virtualenv = /home/usernamevenv/
touch-reload = /home/username/reload.txt
logdate = 1
logto = /home/usernamelogs/socket.log
procname = socket_username
log-maxsize = 204800
env = LANG=ru_RU.utf8
env = LC_ALL=ru_RU.utf8
env = LC_LANG=ru_RU.utf8
ANd extra args:
--http :3456 --gevent 100 --http-websockets
Nginx socket proxy
server {
listen 80;
server_name domain;
charset utf-8;
client_max_body_size 5M; # adjust to taste
location /socket.io {
proxy_pass http://upstream_name/socket.io;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
Application init sources:
init.py (using in another web app without sockets)
from flask_openid import OpenID
from flask_mongoengine import MongoEngine
from flask import Flask
from flask_socketio import SocketIO
app = Flask(__name__)
own_socketio = SocketIO()
db = MongoEngine()
base_app.py (COnfigure application && postforked database connection)
# coding: utf-8
from __future__ import unicode_literals
from flask import Flask, g
from flask_mongoengine import MongoEngine, MongoEngineSessionInterface
from blueprint.monitor import monitor
from init import oid, app, db
from flask_mongoengine import MongoEngine
from uwsgidecorators import postfork
def make_app():
app.register_blueprint(monitor)
app.config.from_pyfile('settings.py')
#if not UWSGI_ALLOWED:
# db.init_app(app)
oid.init_app(app)
app.session_interface = MongoEngineSessionInterface(db)
return app
#postfork
def setup_db():
db.init_app(app)
And socket_app.py
from gevent.monkey import patch_all
patch_all()
from init import own_socketio
from init import app
from base_app import make_app
make_app()
own_socketio.init_app(app, async_mode='gevent_uwsgi', message_queue=app.config['SOCKETIO_REDIS_URL'], cookie='session')
if __name__ == '__main__':
own_socketio.run(app, port=3456)
I am trying to deploy a Flask app on an Ubuntu server. I referenced this, this and this and found a lot of similar questions on SO, but I still can't figure it out.
I can run it manually from the source directory by doing uwsgi siti_uwsgi.ini and navigating to http://server_IP_address:8080/. But when I try uwsgi --socket 127.0.0.1:3031 --wsgi-file views.py --master --processes 4 --threads 2 and navigate to http://server_IP_address:3031, I get nothing.
If I go to siti.company.loc (the DNS name I set up), there is a standard Nginx 502 error page.
When I try to restart the supervisor process, it dies with a FATAL error:
can't find command "gunicorn"
What am I doing wrong? Let me know if I need to provide more info or background.
/webapps/patch/src/views.py (Flask app):
from flask import Flask, render_template, request, url_for, redirect
from flask_cors import CORS
app = Flask(__name__)
CORS(app, resources={r"/*": {'origins': '*'}})
#app.route('/')
def home():
return 'Hello'
#app.route('/site:<site>/date:<int:day>-<month>-<int:year>')
def application(site, month, day, year):
if request.method == 'GET':
# Recompile date from URL. todo: better way
dte = str(day) + "-" + str(month) + "-" + str(
print('about to run')
results = run_SITI(site, dte)
return results
def run_SITI(site, dte):
print('running SITI')
return render_template('results.html', site=site, dte=dte, results=None) # todo: Show results
if __name__ == '__main__':
app.run(debug=True)
/webapps/patch/siti_wsgi.ini (uWSGI ini):
[uwsgi]
http = :8008
chdir = /webapps/patch/src
wsgi-file = views.py
processes = 2
threads = 2
callable = app
/etc/nginx/sites-available/siti (Nginx config):
upstream flask_siti {
server 127.0.0.1:8008 fail_timeout=0;
}
server {
listen 80;
server_name siti.company.loc;
charset utf-8;
client_max_body_size 75M;
access_log /var/log/nginx/siti/access.log;
error_log /var/log/nginx/siti/error.log;
keepalive_timeout 5;
location /static {
alias /webapps/patch/static;
}
location /media {
alias /webapps/patch/media;
}
location / {
# checks for static file, if not found proxy to the app
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://flask_siti;
}
}
/etc/supervisor/conf.d/siti.conf (Supervisor config):
[program:webapp_siti]
command=gunicorn -b views:app
directory=/webapps/patch/src
user=nobody
autostart=true
autorestart=true
redirect_stderr=true
/var/log/nginx/siti/error.log (Nginx error log):
2016/08/30 11:44:42 [error] 25524#0: *73 connect() failed (111: Connection refused) while connecting to upstream, $
2016/08/30 11:44:42 [error] 25524#0: *73 connect() failed (111: Connection refused) while connecting to upstream, $
2016/08/30 11:44:42 [error] 25524#0: *73 no live upstreams while connecting to upstream, client: 10.1.2.195, serve$
You have errors in nginx config:
Instead of:
upstream flask_siti {
server 127.0.0.1:8008 fail_timeout=0;
server {
...
try:
upstream flask_siti {
server 127.0.0.1:8080 fail_timeout=0;
}
server {
...
You must "activate" virtualenv in supervisor config. To do this, add following line to Your supervisor config:
environment=PATH="/webapps/patch/venv/bin",VIRTUAL_ENV="/webapps/patch/venv",PYTHONPATH="/webapps/patch/venv/lib/python:/webapps/patch/venv/lib/python/site-packages"
Was able to get it working with the following changes:
/etc/supervisor/conf.d/siti.conf (Supervisor config):
[program:webapp_siti]
command=/webapps/patch/venv/bin/gunicorn -b :8118 views:app # didn't use uwsgi.ini after all
directory=/webapps/patch/src
user=nobody
autostart=true
autorestart=true
redirect_stderr=true
/etc/nginx/sites-enabled/siti (Nginx config):
upstream flask_siti {
server 127.0.0.1:8118 fail_timeout=0; # changed ports because 8008 was already in use by something else
}
# snip ...
Turns out I had set up uWSGI to listen on port 8008. I also had an extra file in /etc/nginx/sites-enabled called siti.save that was preventing Nginx from reloading. I deleted it, reloaded/restarted Nginx, restarted Supervisor, and it worked.