I'm currently trying to connect to a flask server running with gunicorn from outside the local network through a reverse proxy with nginx but I get Timeout errors every time I'm trying to connect.
So here's my setup:
Flask file:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "<h1 style='color:blue'>Hello There!</h1>"
if __name__ == "__main__":
app.run()
Then I'm starting it with Gunicorn like this:
netstat lists the process:
tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 16664/python3
The reverse proxy setup in Nginx (path: /etc/nginx/sites-available/flaskSrv) looks like this:
server {
listen 5000;
server_name mydomain.com www.mydomain.com;
access_log /var/log/nginx/accesslog.log
location / {
proxy_pass http://127.0.0.1:8000/;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Prefix /;
}
}
But if I'm trying to connect to https://example.com:5000/ I get an ERR_CONNECTION_TIMED_OUT. Same with http://example.com:5000/
I'm pretty sure my flask file is correct and think there is something wrong with Nginx.
Edit: I solved the problem by calling a PHP script from JS ajax and calling the python script from PHP via system
I think you run gunicorn at 127.0.0.1 that listen at local network. Try change to 0.0.0.0 listen on every available network interface and run again
Related
I have a Bokeh application embedded in Flask and Gunicorn (see flask_gunicorn_embed.py). This works fine when I access everything through the IP address of the web server, but not when everything is proxied through Nginx. It will load everything from Flask, but not from Bokeh (the autoload.js).
Example:
I start Flask through Gunicorn with
gunicorn --workers 9 --bind 0.0.0.0:5000 --timeout 120 --log-file /some/directory/to/gunicorn/logs/gunicorn.log -m 0700 flask:app
I can now access everything from Flask through http://xxx.xxx.xxx.xxx:5000. I have adapted the flask_gunicorn_embed.py and the most important bits are
script = server_document('http://xxx.xxx.xxx.xxx:%d/bkapp' % port, resources=None)
sockets, port = bind_sockets("0.0.0.0", 0)
bokeh_tornado = BokehTornado({'/bkapp': bkapp}, extra_websocket_origins=["xxx.xxx.xxx.xxx:5000"])
If I now access a page which uses this Bokeh server everything works fine. It loads
http://xxx.xxx.xxx.xxx:XXXXX/bkapp/autoload.js?bokeh-autoload-element=1001&bokeh-app-path=/bkapp&bokeh-absolute-url=http://xxx.xxx.xxx.xxx:XXXXX/bkapp&resources=none
and displays the graphs, and creates a web socket for callbacks.
This is without Nginx as a reverse proxy. I don't want to use the IP address of the web server because I need to use HTTPS, which requires a domain.
Thus I have the following Nginx configuration:
server {
listen 80;
server_name example.com;
return 301 https://$server_name/;
}
server {
listen 443;
server_name example.com;
ssl on;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
client_max_body_size 50M;
location / {
include proxy_params;
proxy_pass http://127.0.0.1:5000;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host:$server_port;
proxy_buffering off;
}
location /static {
alias /some/directory/to/flask/static;
}
}
And I start Flask through Gunicorn with
gunicorn --workers 9 --bind 127.0.0.1:5000 --timeout 120 --log-file /some/directory/to/gunicorn/logs/gunicorn.log -m 0700 flask:app
I can now access everything from Flask through https://example.com. The flask_gunicorn_embed.py now looks like
script = server_document('https://example.com:%d/bkapp' % port, resources=None)
sockets, port = bind_sockets("0.0.0.0", 0)
bokeh_tornado = BokehTornado({'/bkapp': bkapp}, extra_websocket_origins=["example.com"])
However, every request that is generated through Bokeh's server_document
https://example.com:XXXXX/bkapp/autoload.js?bokeh-autoload-element=1001&bokeh-app-path=/bkapp&bokeh-absolute-url=https://example.com:XXXXX/bkapp&resources=none
results in either a connection timed out or a connection refused and thus not loading the scripts.
How can I make it such that it will connect and load the script?
It must have something to do with Nginx because if I request the file through the IP address it still works (due to the bind_sockets("0.0.0.0", 0)). But I cannot figure out what is causing this issue.
Edit:
It appears to be an issue with the fact that it uses HTTPS. My Nginx configuration is the same as the one given in the Bokeh documentation. The documentation says to use --use_xheaders, which is not possible because I am not using bokeh serve.
I do have
conf = {'use_xheaders': True}
bokeh_tornado = BokehTornado({'/bkapp': bkapp}, extra_websocket_origins=["example.com"], **conf)
bokeh_http = HTTPServer(bokeh_tornado, xheaders=True)
but it still won't load the scripts for HTTPS pages.
http://example_no_https.com will load the pages through the ports and https://example.com won't.
use_xheaders is an argument for the Bokeh Server class (which passes it to a Tornado HTTPServer), not by BokehTornado. If you are not using Server because you are coordinating BokehTornado and an HTTPServer yourself, then you will have manually configure this option on the HTTP server, since you don't have a Server class doing it for you:
http_server_kwargs.setdefault('xheaders', opts.use_xheaders)
HTTPServer(..., **http_server_kwargs)
I was following this tutorial to setup my flask server.
https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-18-04#step-6-%E2%80%94-securing-the-application
When I got to step 6 I see that they are setting flask for the whole url but I would like to point it to a specific port.
This is the code I have for my nginx which points. This currenly produces a 404.
server {
listen 5000;
server_name site.com;
location / {
include proxy_params;
proxy_pass http://unix:/home/user/project/project.sock;
}
}
All the other files are the same as the tutorial. I have tried to modify the .sock file but it seems like it was generated automatically and it can't be modified. In addition I need to find a way for nginx to handle this before I worry about handling it from gunicorn.
My end goal is to have nginx foward requests to flask running when a request is sent to 0.0.0.0:5000 and have all other requests 0.0.0.0 , 0.0.0.0/* be handled by nginx.
Any help to undestand all this is really appreciated got lost at this point.
EDIT
my nginx configuration in sites-available
server {
server_name domain www.domain;
location / {
include proxy_params;
proxy_pass http://127.0.0.1:8080/;
}
}
If you want flask to be open to a port instead of a file you should override
[service] to
[service]
...
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind 127.0.0.1:8000 -m 007 wsgi:app
And change your nginx config to proxy_pass http://127.0.0.1:8000/;
This way you can have access to port 8000 for checking how are working gunicorn and flask. Remember to be careful with firewall rules to secure port 8000. For a good discussion on which one is better you can try: gunicorn + nginx: Server via socket or proxy?
I've read a lot of instructions since yesterday about this issue but all of them have similar steps. However I followed step by step but still can't get everything Ok.
Actually I can make Flask+Gunicorn+supervisor working but Nginx is not working well.
I connect my remote cloud server with SSH and I'm not deploying the site on my computer.
Nginx is installed correctly because when I visit the site via the domain name (aka. example.com) it shows the Nginx welcome page.
I use supervisor to start Gunicorn and the configuration is
[program:myapp]
command=/home/fh/test/venv/bin/gunicorn -w4 -b 0.0.0.0:8000 myapp:app
directory=/home/fh/test
startsecs=0
stopwaitsecs=0
autostart=false
autorestart=false
stdout_logfile=/home/fh/test/log/gunicorn.log
stderr_logfile=/home/fh/test/log/gunicorn.err
here I bind the server to port 8000 and I don't actually know what does 0.0.0.0 stand for but I think it doesn't mean the localhost because I can visit the site via http://example.com:8000 and it works well.
Then I tried to use Nginx as a proxy server.
I deleted /etc/nginx/sites-available/default' and '/etc/nginx/sites-enabled/default/ and created /etc/nginx/sites-available/test.com and /etc/nginx/sites-enabled/test.com and symlink them.
test.com
server {
server_name www.penguin-penpen.com;
rewrite ^ http://example/ permanent;
}
# Handle requests to example.com on port 80
server {
listen 80;
server_name example.com;
# Handle all locations
location / {
# Pass the request to Gunicorn
proxy_pass http://127.0.0.1:8000;
# Set some HTTP headers so that our app knows where the request really came from
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
To my understanding, what Nginx do is when I visit http://example.com it passes my request to http://example.com:8000.
I'm not quite sure that I should use proxy_pass http://127.0.0.1:8000 here because I don't know whether should Nginx pass the request to localhost But I 've tried to change it to 0.0.0.0:8000 but it still doesn't work.
Can anyone help?
0.0.0.0 means the server will accept connections from all IP address. See https://en.wikipedia.org/wiki/0.0.0.0 for more detail.
If the gunicorn server listens on 127.0.0.1, only you (or someone else on the same machine with gunicorn server) can access it through local loop https://en.wikipedia.org/wiki/Local_loop.
But since you use Nginx to accept connection from the internet, you can just proxy_pass http://127.0.0.1:8000; and change the command to command=/home/fh/test/venv/bin/gunicorn -w4 -b 127.0.0.1:8000 myapp:app. In this scenario, gunicorn itself only need to accept connections from Nginx which runs on the same machine with gunicorn.
The whole process looks like this
Connections from the Internet -> Nginx (reverse proxy, listen on 0.0.0.0:80) -> Gunicorn (which runs your Python code, listen on 127.0.0.1:8000)
I have a problem with my amazon ec2 instance. I cannot access my django application running on port 8004 from the browser.
I have setup an instance on amazon ec2, installed django and nginx. This is running on ubuntu trusty 14.04. I have a domain on external dns-nameserver and I have correctly pointed that domain to the public elastic ip of my amazon ec2 instance. When I ping the server it works and it shows the public ip.
I've seen similar threads before and in most cases problems with this have been because firewall rules have not been added for specific ports, that is they are not open to the outside world. In my case I made sure that port 22, 80 and 443 were open when I created the instance. I've even made ports 8000 - 8100 open.
Note: Eventually I will make django run with gunicorn but just to test it I'm simply running it by going: manage.py runserver 8004
Here is a snapshot of how my inbound rules look like for open ports on amazon ec2
Could it be because I'm editing inbound rules after I launched the instance? Isn't that what you're suppose to be able to do?
Nginx is running without problems, I'll post my config below. I have no idea why this doesn't work. I have followed any thread I find on the subject and nothing seems to fix it.
UPDATE: I can now confirm that I can access my django site directly through the ip, by going
ip:8004. So obviously this is not a problem with the ports but has likely something to do with nginx config or my DNS settings.
My nginx config (I've replaced ip's with x, and the domain with sub.domain.com. I then try to access the site from sub.domain.com)
upstream docko_server {
server 127.0.0.1:8004 fail_timeout=0;
}
server {
server_name sub.domain.com;
listen xx.xx.xx.xxx:80;
return 303 https://$host$request_uri;
}
# HTTPS server
#
server {
listen xx.xx.xx.xxx:443;
server_name sub.domain.com;
ssl on;
ssl_certificate ssl/server.crt;
ssl_certificate_key ssl/server.key;
ssl_session_timeout 5m;
ssl_protocols SSLv3 TLSv1;
ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
ssl_prefer_server_ciphers on;
client_max_body_size 25M;
access_log /webapps/docko/logs/nginx-access.log;
error_log /webapps/docko/logs/nginx-error.log;
location /assets/ {
alias /webapps/docko/docko/staticfiles/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://docko_server;
break;
}
}
}
If you want to access the django dev server from the Internet through the port 8004 try:
manage.py runserver 0.0.0.0:8004
The dev server will only listen by default on 127.0.0.1.
However you won't need this if you get to the dev server through Nginx, i.e. you connect to your.domain.com:80. In this case Nginx will act as a proxy, passing the request to the dev server.
BTW you can edit the rules of existing security group after launching the instances, this works perfectly.
To answer my own question I succeeded by not assigning the elastic ip but simply just listening on port 80, 443. So in my nginx conf instead of this.
server_name sub.domain.com;
listen xx.xx.xx.xxx:80;
I now just have
server_name sub.domain.com
listen 80;
same for 443.
Trying to setup a server on Rackspace.com.
Have done the following things:
Installed Centos 6.3
Installed Python 2.7
Installed gunicorn using the "Quick Start" on their home page: gunicorn.org/
In the quick start, a "hello world" application seems to be initialized:
Create file "myapp.py":
(tutorial) $ vi myapp.py
(tutorial) $ cat myapp.py
Contents of "myapp.py"
def app(environ, start_response):
data = "Hello, World!\n"
start_response("200 OK", [
("Content-Type", "text/plain"),
("Content-Length", str(len(data)))
])
return iter([data])
Since I know very little about servers, I do not know what to do next. I tried typing the server's IP address into the browser, but that seemed to result in a timeout.
I'm not sure if there is:
something else that needs to be installed. Nginx is mentioned under "deploy" on the gunicorn website. Looks like Nginx is a proxy server which is confusing to me because I thought gunicorn was a server. Not sure why I need two servers?
something that needs to be configured in gunicorn
something that needs to be configured on the server itself
something that else entirely that needs to be done in order to actually serve a request
What are the next steps?
Thanks so much!
since gunicorn is a Web server on your case Nginx will act as a back proxy passing the an HTTP request from Nginx to gunicorn.
So, I will put here the steps to take for a simple Nginx and Gunicorn configuration running on the same machine.
Starting with nginx configuration
Go to your /etc/nginx/nginx.conf and under the http{} make sure you have: include /etc/nginx/site-enabled/*;
http{
# other configurations (...)
include /etc/nginx/sites-enabled/*;
}
now, include a file on /etc/nginx/sites-enabled/mysite.conf where you will proxy your requests to your gunicorn app.
server {
listen 80 default; # this means nginx will be
# listening requests on port 80 and
# this will be the default nginx server
server_name localhost;
# declare proxy params and values to forward to your gunicorn webserver
proxy_pass_request_headers on;
proxy_pass_request_body on;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 120s;
location / {
# here is where you declare that every request to /
# should be proxy to 127.0.0.1:8000 (which is where
# your gunicorn will be running on)
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://127.0.0.1:8000/; # the actual nginx directive to
# forward the request
}
}
Ok, at this point all you have is an Nginx acting as a proxy where all the requests going to 127.0.0.1:80 will be passed to 127.0.0.1:8000.
Now is time to configure your Gunicorn webserver:
Usually the way I do I use a configuration file, Gunicorn config file can be an ordinary python file. So now, create a file at any location you like, I will assume this file will be /etc/gunicorn/mysite.py
workers = 3 # number of workers Gunicorn will spawn
bind = '127.0.0.1:8000' # this is where you declare on which address your
# gunicorn app is running.
# Basically where Nginx will forward the request to
pidfile = '/var/run/gunicorn/mysite.pid' # create a simple pid file for gunicorn.
user = 'user' # the user gunicorn will run on
daemon = True # this is only to tell gunicorn to deamonize the server process
errorlog = '/var/log/gunicorn/error-mysite.log' # error log
accesslog = '/var/log/gunicorn/access-mysite.log' # access log
proc_name = 'gunicorn-mysite' # the gunicorn process name
Ok, all set in configuration. Now all you have to do its to start the servers.
Starting the gunicorn and telling it which app to use and which config file.
from the command line and the folder where your myapp.py file is located run:
gunicorn -c /etc/gunicorn/mysite.py mysite:app
Now, only start nginx.
/etc/init.d/nginx start
or
service nginx start
Hope this helps.
looking at the quickstart guide, you probably should have run
(tutorial) $ ../bin/gunicorn -w 4 myapp:app
which should have produced a line that looks a bit like:
Listening at: http://127.0.0.1:8000
Among others. see if you can access your site at that address.
Also Note that 127.0.0.1 is the loopback address; accessible only from that host itself. To get gunicorn to bind to a different option, pass it --bind 0.0.0.0:80, as Jan-Philip suggests.
Since you mention rackspace, its possible that you may need to adjust the firewall settings to allow incoming connections to the desired ports.
Looks like you do not have a web application developed so far. So, I assume that your goal for now is to set up a development environment. For the time being, develop your web application using the development web server included in most frameworks, e.g. Flask.
Whatever framework you are using, make the development web server listen on 0.0.0.0 so that the service is listening on all configured network interfaces and make sure that the port is open to the outside (check the Rackspace settings).
When you are done developing your application or are looking into an existing one, you have to deploy it in a solid way. Then, gunicorn behind nginx is an option.
I will roughly go through your questions. It looks you have to read a bit more :-)
Nginx is mentioned under "deploy" on the gunicorn website. Looks like Nginx is a proxy server which is confusing to me because I thought gunicorn was a server. Not sure why I need two servers?
Nginx is a full-featured web server. It is appreciated for its performance and stability. People use it to serve static files (to not burden a dynamic web application with this task), to forward requests to web applications whenever necessary, for SSL-termination, and for load-balancing. Note that this is an incomplete picture.
gunicorn is a server for serving WSGI apps. Mainly, it manages worker processes that actually execute the web application.
something that needs to be configured in gunicorn.
something that needs to be configured on the server itself.
something that else entirely that needs to be done in order to actually serve a request.
Actually, you can optimize your linux box in endless ways (for performance, e.g. increase the file descriptor limit and for security). Within gunicorn, you can configure the number of worker processes and a lot more. If you have nginx as frontend or even another load balancer, this one has its own configuration. You see, your setup might become very complex for actual deployment in a real-world scenario. This is not trivial.
However, for playing around with a WSGI application, just set up your development framework properly, which is very simple in most cases, and make sure that there are no firewall issues. That's all.