Trying to setup a server on Rackspace.com.
Have done the following things:
Installed Centos 6.3
Installed Python 2.7
Installed gunicorn using the "Quick Start" on their home page: gunicorn.org/
In the quick start, a "hello world" application seems to be initialized:
Create file "myapp.py":
(tutorial) $ vi myapp.py
(tutorial) $ cat myapp.py
Contents of "myapp.py"
def app(environ, start_response):
data = "Hello, World!\n"
start_response("200 OK", [
("Content-Type", "text/plain"),
("Content-Length", str(len(data)))
])
return iter([data])
Since I know very little about servers, I do not know what to do next. I tried typing the server's IP address into the browser, but that seemed to result in a timeout.
I'm not sure if there is:
something else that needs to be installed. Nginx is mentioned under "deploy" on the gunicorn website. Looks like Nginx is a proxy server which is confusing to me because I thought gunicorn was a server. Not sure why I need two servers?
something that needs to be configured in gunicorn
something that needs to be configured on the server itself
something that else entirely that needs to be done in order to actually serve a request
What are the next steps?
Thanks so much!
since gunicorn is a Web server on your case Nginx will act as a back proxy passing the an HTTP request from Nginx to gunicorn.
So, I will put here the steps to take for a simple Nginx and Gunicorn configuration running on the same machine.
Starting with nginx configuration
Go to your /etc/nginx/nginx.conf and under the http{} make sure you have: include /etc/nginx/site-enabled/*;
http{
# other configurations (...)
include /etc/nginx/sites-enabled/*;
}
now, include a file on /etc/nginx/sites-enabled/mysite.conf where you will proxy your requests to your gunicorn app.
server {
listen 80 default; # this means nginx will be
# listening requests on port 80 and
# this will be the default nginx server
server_name localhost;
# declare proxy params and values to forward to your gunicorn webserver
proxy_pass_request_headers on;
proxy_pass_request_body on;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 120s;
location / {
# here is where you declare that every request to /
# should be proxy to 127.0.0.1:8000 (which is where
# your gunicorn will be running on)
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://127.0.0.1:8000/; # the actual nginx directive to
# forward the request
}
}
Ok, at this point all you have is an Nginx acting as a proxy where all the requests going to 127.0.0.1:80 will be passed to 127.0.0.1:8000.
Now is time to configure your Gunicorn webserver:
Usually the way I do I use a configuration file, Gunicorn config file can be an ordinary python file. So now, create a file at any location you like, I will assume this file will be /etc/gunicorn/mysite.py
workers = 3 # number of workers Gunicorn will spawn
bind = '127.0.0.1:8000' # this is where you declare on which address your
# gunicorn app is running.
# Basically where Nginx will forward the request to
pidfile = '/var/run/gunicorn/mysite.pid' # create a simple pid file for gunicorn.
user = 'user' # the user gunicorn will run on
daemon = True # this is only to tell gunicorn to deamonize the server process
errorlog = '/var/log/gunicorn/error-mysite.log' # error log
accesslog = '/var/log/gunicorn/access-mysite.log' # access log
proc_name = 'gunicorn-mysite' # the gunicorn process name
Ok, all set in configuration. Now all you have to do its to start the servers.
Starting the gunicorn and telling it which app to use and which config file.
from the command line and the folder where your myapp.py file is located run:
gunicorn -c /etc/gunicorn/mysite.py mysite:app
Now, only start nginx.
/etc/init.d/nginx start
or
service nginx start
Hope this helps.
looking at the quickstart guide, you probably should have run
(tutorial) $ ../bin/gunicorn -w 4 myapp:app
which should have produced a line that looks a bit like:
Listening at: http://127.0.0.1:8000
Among others. see if you can access your site at that address.
Also Note that 127.0.0.1 is the loopback address; accessible only from that host itself. To get gunicorn to bind to a different option, pass it --bind 0.0.0.0:80, as Jan-Philip suggests.
Since you mention rackspace, its possible that you may need to adjust the firewall settings to allow incoming connections to the desired ports.
Looks like you do not have a web application developed so far. So, I assume that your goal for now is to set up a development environment. For the time being, develop your web application using the development web server included in most frameworks, e.g. Flask.
Whatever framework you are using, make the development web server listen on 0.0.0.0 so that the service is listening on all configured network interfaces and make sure that the port is open to the outside (check the Rackspace settings).
When you are done developing your application or are looking into an existing one, you have to deploy it in a solid way. Then, gunicorn behind nginx is an option.
I will roughly go through your questions. It looks you have to read a bit more :-)
Nginx is mentioned under "deploy" on the gunicorn website. Looks like Nginx is a proxy server which is confusing to me because I thought gunicorn was a server. Not sure why I need two servers?
Nginx is a full-featured web server. It is appreciated for its performance and stability. People use it to serve static files (to not burden a dynamic web application with this task), to forward requests to web applications whenever necessary, for SSL-termination, and for load-balancing. Note that this is an incomplete picture.
gunicorn is a server for serving WSGI apps. Mainly, it manages worker processes that actually execute the web application.
something that needs to be configured in gunicorn.
something that needs to be configured on the server itself.
something that else entirely that needs to be done in order to actually serve a request.
Actually, you can optimize your linux box in endless ways (for performance, e.g. increase the file descriptor limit and for security). Within gunicorn, you can configure the number of worker processes and a lot more. If you have nginx as frontend or even another load balancer, this one has its own configuration. You see, your setup might become very complex for actual deployment in a real-world scenario. This is not trivial.
However, for playing around with a WSGI application, just set up your development framework properly, which is very simple in most cases, and make sure that there are no firewall issues. That's all.
Related
I was following this tutorial to setup my flask server.
https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-18-04#step-6-%E2%80%94-securing-the-application
When I got to step 6 I see that they are setting flask for the whole url but I would like to point it to a specific port.
This is the code I have for my nginx which points. This currenly produces a 404.
server {
listen 5000;
server_name site.com;
location / {
include proxy_params;
proxy_pass http://unix:/home/user/project/project.sock;
}
}
All the other files are the same as the tutorial. I have tried to modify the .sock file but it seems like it was generated automatically and it can't be modified. In addition I need to find a way for nginx to handle this before I worry about handling it from gunicorn.
My end goal is to have nginx foward requests to flask running when a request is sent to 0.0.0.0:5000 and have all other requests 0.0.0.0 , 0.0.0.0/* be handled by nginx.
Any help to undestand all this is really appreciated got lost at this point.
EDIT
my nginx configuration in sites-available
server {
server_name domain www.domain;
location / {
include proxy_params;
proxy_pass http://127.0.0.1:8080/;
}
}
If you want flask to be open to a port instead of a file you should override
[service] to
[service]
...
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind 127.0.0.1:8000 -m 007 wsgi:app
And change your nginx config to proxy_pass http://127.0.0.1:8000/;
This way you can have access to port 8000 for checking how are working gunicorn and flask. Remember to be careful with firewall rules to secure port 8000. For a good discussion on which one is better you can try: gunicorn + nginx: Server via socket or proxy?
I've read a lot of instructions since yesterday about this issue but all of them have similar steps. However I followed step by step but still can't get everything Ok.
Actually I can make Flask+Gunicorn+supervisor working but Nginx is not working well.
I connect my remote cloud server with SSH and I'm not deploying the site on my computer.
Nginx is installed correctly because when I visit the site via the domain name (aka. example.com) it shows the Nginx welcome page.
I use supervisor to start Gunicorn and the configuration is
[program:myapp]
command=/home/fh/test/venv/bin/gunicorn -w4 -b 0.0.0.0:8000 myapp:app
directory=/home/fh/test
startsecs=0
stopwaitsecs=0
autostart=false
autorestart=false
stdout_logfile=/home/fh/test/log/gunicorn.log
stderr_logfile=/home/fh/test/log/gunicorn.err
here I bind the server to port 8000 and I don't actually know what does 0.0.0.0 stand for but I think it doesn't mean the localhost because I can visit the site via http://example.com:8000 and it works well.
Then I tried to use Nginx as a proxy server.
I deleted /etc/nginx/sites-available/default' and '/etc/nginx/sites-enabled/default/ and created /etc/nginx/sites-available/test.com and /etc/nginx/sites-enabled/test.com and symlink them.
test.com
server {
server_name www.penguin-penpen.com;
rewrite ^ http://example/ permanent;
}
# Handle requests to example.com on port 80
server {
listen 80;
server_name example.com;
# Handle all locations
location / {
# Pass the request to Gunicorn
proxy_pass http://127.0.0.1:8000;
# Set some HTTP headers so that our app knows where the request really came from
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
To my understanding, what Nginx do is when I visit http://example.com it passes my request to http://example.com:8000.
I'm not quite sure that I should use proxy_pass http://127.0.0.1:8000 here because I don't know whether should Nginx pass the request to localhost But I 've tried to change it to 0.0.0.0:8000 but it still doesn't work.
Can anyone help?
0.0.0.0 means the server will accept connections from all IP address. See https://en.wikipedia.org/wiki/0.0.0.0 for more detail.
If the gunicorn server listens on 127.0.0.1, only you (or someone else on the same machine with gunicorn server) can access it through local loop https://en.wikipedia.org/wiki/Local_loop.
But since you use Nginx to accept connection from the internet, you can just proxy_pass http://127.0.0.1:8000; and change the command to command=/home/fh/test/venv/bin/gunicorn -w4 -b 127.0.0.1:8000 myapp:app. In this scenario, gunicorn itself only need to accept connections from Nginx which runs on the same machine with gunicorn.
The whole process looks like this
Connections from the Internet -> Nginx (reverse proxy, listen on 0.0.0.0:80) -> Gunicorn (which runs your Python code, listen on 127.0.0.1:8000)
I have a django application. I can't seem to get nginx to serve the static files correctly, but that's not this issue.
The current issue is that when following this guide: https://gist.github.com/evildmp/3094281
I try to use the unix sockets rather than the web socket.
e.g.
server unix:///tmp/uwsgi.sock; # for a file socket
#server 127.0.0.1:8001; # for a web port socket
using the /tmp/uwsgi.sock socket rather than the 127.0.0.1:8001 web port socket.
The issue is that when I use the web port socket, and I navigate to domainname.com:8001 I reach the website as served by uWSGI, but there are no staticfiles loaded. So that means that at least uwsgi is working. But when I switch over to the file socket, I cannot even get it to work at all.
What am I doing wrong?
Here is my nginx.conf:
# nginx.conf
upstream django {
# connect to this socket
# server unix:///tmp/uwsgi.sock; # for a file socket (TRYING TO USE)
server 127.0.0.1:8001; # for a web port socket (RATHER THAN THIS)
}
server {
# the port your site will be served on
listen 8000;
# the domain name it will serve for
server_name .cshenkan.com; # substitute your machine's IP address or FQDN
charset utf-8;
#Max upload size
client_max_body_size 75M; # adjust to taste
# Django media
location /media {
alias /home/ubuntu/sasite-rewrite/media; # your Django pro$
}
location /static {
alias /home/ubuntu/sasite-rewrite/static; # your Django pro$
}
location /assets {
alias /home/ubuntu/sasite-rewrite/assets
}
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params; # or the uwsgi_params you installe$
}
}
And my base.py settings file snippet:
STATIC_ROOT = normpath(join(SITE_ROOT, 'static'))
STATIC_URL = '/static/'
STATICFILES_DIRS = (
normpath(join(SITE_ROOT, 'static')),
normpath(join(SITE_ROOT, 'assets')),
)
And how I run uwsgi when using the web socket:
uwsgi --http :8001 --chdir /home/ubuntu/sasite-rewrite --wsgi-file /home/ubuntu/sasite-rewrite/sasite/wsgi.py
And how I run it when using the unix socket:
uwsgi --socket /tmp/uwsgi.sock --chdir /home/ubutnu/sasite-rewrite --wsgi-file /home/ubuntu/sasite-rewrite/sasite/wsgi.py
I just can't seem to get it to work when using the unix socket.
Neither can I get nginx to serve the correct static files, for example if I add a file to the media directory called 1.png or 1.txt and try to access them with domainname.com:8000/media/1.png I always get a hang or a no response from server error.
What am I doing wrong?
How can I get the unix socket to work correctly? And is the command I'm using correct?
I also don't understand this, when I use the web socket, I can connect to domainname.com:8001 and get the page with no static files, it looks like crap but loads from uwsgi. But with the unix sockets I have no idea how to access the page served by uwsgi to see if it works with the unix socket. Before I can move onto getting nginx working, I need to solve this issue, as I tried using web sockets with nginx and still had no luck there, I'm hoping once my unix sockets are set up correctly nginx will work better.
Any suggestions, or examples you could provide? Would be much appreciated, I am a long time django programmer, but I have very rarely deployed the sites myself. So I am really struggling here.
Any help is much appreciated, thanks.
Today I had the same problem with NGINX + Socket Unix for uwsgi protocol.
I discovered that version 1.8 of the NGINX is bugged.
Try installing version 1.10 that will work via UNIX SOCKET.
If you use open-SuSE, you can get the rpm here:
http://download.opensuse.org/repositories/home:/ghostlyrat/openSUSE_Leap_42.2/x86_64/nginx-1.10.1-3.3.x86_64.rpm
Or, directly in the official nginx downloads area:
http://nginx.org/en/download.html
I have a flask application running with uwsgi,nginx,and supervisor.
No matter what I try I cant seem to get the code changes to take effect on the server.
If I run the app locally the changes are there.
If I stop and start uwsgi, the changes take effect.
If I restart the supervisor service the changes don't take effect.
I know the code has the changes because I log in and see the changes that I made but its still running the old code at specific routes.
If I change the title of my page those changes take effect right away but for my webhook end points, they never seem to change.
Here are my config files.
app.ini
[uwsgi]
module = wsgi
master = true
processes = 5
socket = app.sock
chmod-socket = 660
vacuum = true
die-on-term = true
supervisor
[program:app.io]
command=/home/www/beta/v_env/bin/uwsgi --ini /home/www/beta/app.ini --chown-socket www-data:www-data
directory=/home/www/beta
autostart=true
autorestart=true
stdout_logfile=/home/logs/app_uwsgi.log
redirect=true
stopsignal=QUIT
nginx
server {
listen 80;
server_name beta.domain.io;
location / {
include uwsgi_params;
uwsgi_pass unix:/home/www/beta/app.sock;
uwsgi_read_timeout 1800;
}
}
When testing it I also get this error a lot
upstream prematurely closed connection while reading response header from upstream
If your changes to the code aren't reflected in your program's behavior, then your code either hasn't been reloaded, or it hasn't changed, or you misunderstand your changes. The most common time this happens to me is when .pyc files fail to get refreshed. You might try clearing all the pyc files from your project then launching it again, assuming you have any pyc files. This will also have the benefit of restarting the application.
Before doing anything I recommend using supervisor to restart your application. If your application is within supervisor, you can use the following relevant methods:
supervisorctl stop all
supervisorctl start all
supervisorctl restart all
Edit:
I forgot to mention that if you are viewing this in a browser then it is conceivable the browser is caching the older version. If this is the case, you would need to do a hard refresh.
I got a Web.py app and wanted to push it into production.
As recommended by the Web.py Community I decided to use uWSGI and Nginx for this.
My App uses Memcached for Session Storing and MySQL for other storing tasks. The app works fine on my MacBook.
I configured the uWSGI + Nginx setup previously which worked fine. But know I receive a 502 Bad Gateway when I try to access the Index Page on my Ubuntu Server.
BUT: when entering another page I receive all the content I wanted.
In general the app works fine in the Ubuntu environment, as I tested it by typing python app.py 8080. I was able to enter the page.tld:8080/ and receive all content.
My uWSGI config:
[uwsgi]
gid = www-data
uid = www-data
vhost = true
plugins = python
logdate
#socket = /tmp/uwsgi_vhosts.sock
socket = 127.0.0.1:3031
master = true
processes = 1
harakiri = 120
limit-as = 128
memory-report
no-orphans
The Nginx config:
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
# Make site accessible from http://localhost/
server_name page.tld;
location / {
include uwsgi_params;
uwsgi_pass 127.0.0.1:3031;
# This is the absolute path to the folder containing your application
uwsgi_param UWSGI_CHDIR /var/www/page.tld/apps;
# This is actually not necessary for our simple application,
# but you may need this in future
uwsgi_param UWSGI_PYHOME /var/www/page.tld/apps;
# This is the name of your application file, minus the '.py' extension
uwsgi_param UWSGI_SCRIPT test;
}
I keep getting this lines in the vhosts.log of uWSGI:
libgcc_s.so.1 must be installed for pthread_cancel to work
- DAMN ! worker 1 (pid: 1281) died, killed by signal 6 :( trying respawn ...
- Respawned uWSGI worker 1 (new pid: 1330)
Please let me know if you need to see other parts of the configuration.
And these lines in the error.log of nginx:
[error] 1233#0: *1 upstream prematurely closed connection while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: page.tld, request: "GET / HTTP/1.1", upstream: "uwsgi://127.0.0.1:3031", host: "page.tld"
Let me know if any other logs are needed to solve this.
Update: It seems that I get 502 Bad Gateway, when I want to access a page that has to load things from the MySQL Database. But as it is working without uWSGI & NGINX I guess that nginx kills the uwsgi instance for some reason when it tries to load things from the Database.
I recently fixed this problem by setting a higher memory limit within the uwsgi. You will need to restart the uwsgi. I am running uwsgi emperor at start up. So, in my case I rebooted.
[uwsgi]
...
limit-as = 512
System:
Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-43-generic x86_64)
mysqlclient==1.3.6