I'm running a Django server with Gunicorn and Nginx hosted on DigitalOcean. I've run into a problem where adding a new file through the administrator interface produces a 403 forbidden error. Specifically, the file in question works fine if I summon a query of it (e.g. Object.objects.all())but can't be rendered in my templates. I've previously fixed the problem by doing chmod/chown, but the fix only applies to existing files, not new ones. Does anyone know how to permanently apply the fix once?
TL;DR:
FILE_UPLOAD_PERMISSIONS = 0o644 in settings.py
in bash shell: find /path/to/MEDIA_ROOT -type d -exec chmod go+rx {} +
The explanation
The files are created with permissions that are too restrictive, so the user Nginx runs as, cannot read them. To fix this you need to make sure Nginx can read the files and can get to the files.
The goal
First you need FILE_UPLOAD_PERMISSIONS to allow reading by the Nginx user. Second, MEDIA_ROOT and all subdirectories must be readable by Nginx and writeable by Gunicorn.
How to
You must ensure the directories are world readable (and executable) or the group for the directories must be a group that the Nginx process belongs to and they must be at least group readable (and executable).
As a side note, you said you've used chmod and chown before, so I assumed you were familiar with the terminology used. Since you're not, I highly recommend fully reading the linked tutorial, so you understand what the commands you used can do and can screw up.
Related
I have a small Flask application that runs fine locally, however when I go to run the application on my server, it runs but I am not able to get the output to save to a public_html folder.
This is the area I believe I am having the issue, when I run the application remotely:
df.to_html('/home/mydomain/public_html/data/candles.html', index = False)
If I run the application locally, this location works fine:
df.to_html('candles.html', index = False)
I have ensured that the remote folder 'data' has full access - 0777.
What am I doing wrong?
If you don't have an exception occurring, then very likely the file was saved, but not where you think it should have. If you did not provide a full path, the destination will be relative to the application directory. The solution is to be explicit and provide a full path, unless you are using some Flask functions that already have a default setting.
You should never grant 0777 permissions on public_html, that is a potential vulnerability. For example, someone could upload a shell to that directory if they can leverage a security flaw on your website.
There is not enough context, but the user running the process (Apache, Nginx or whatever) should not have write permissions here. If you must grant write permissions, create a dedicated directory (preferably outside the webroot unless they have to be exposed to the user), then add some directives to stipulate that files present in the directory cannot be executed. So that even if a webshell is uploaded it cannot run.
I implemented a REST api in django with django-rest-framework and used oauth2 for authentication.
I tested with:
curl -X POST -d "client_id=YOUR_CLIENT_ID&client_secret=YOUR_CLIENT_SECRET&grant_type=password&username=YOUR_USERNAME&password=YOUR_PASSWORD" http://localhost:8000/oauth2/access_token/
and
curl -H "Authorization: Bearer <your-access-token>" http://localhost:8000/api/
on localhost with successful results consistent with the documentation.
When pushing this up to an existing AWS elastic beanstalk instance, I received:
{ "detail" : "Authentication credentials were not provided." }
I like the idea of just having some extra configuration on the standard place. In your .ebextensions directory create a wsgi_custom.config file with:
files:
"/etc/httpd/conf.d/wsgihacks.conf":
mode: "000644"
owner: root
group: root
content: |
WSGIPassAuthorization On
As posted here: https://forums.aws.amazon.com/message.jspa?messageID=376244
I thought the problem was with my configuration in django or some other error type instead of focusing on the differences between localhost and EB. The issue is with EB's Apache settings.
WSGIPassAuthorization is natively set to OFF, so it must be turned ON. This can be done in your *.config file in your .ebextensions folder with the following command added:
container_commands:
01_wsgipass:
command: 'echo "WSGIPassAuthorization On" >> ../wsgi.conf'
Please let me know if I missed something or if there is a better way I should be looking at the problem. I could not find anything specifically about this anywhere on the web and thought this might save somebody hours of troubleshooting then feeling foolish.
I use a slightly different approach now. sahutchi's solution worked as long as env variables were not changed as Tom dickin pointed out. I dug a bit deeper inside EB and found out where the wsgi.conf template is located and added the "WSGIPassAuthorization On" option there.
commands:
WSGIPassAuthorization:
command: sed -i.bak '/WSGIScriptAlias/ a WSGIPassAuthorization On' config.py
cwd: /opt/elasticbeanstalk/hooks
That will always work, even when changing environment variables. I hope you find it useful.
Edit: Seems like lots of people are still hitting this response. I haven't used ElasticBeanstalk in a while, but I would look into using Manel Clos' solution below. I haven't tried it personally, but seems a much cleaner solution. This one is literally a hack on EBs scripts and could potentially break in the future if EB updates them, specially if they move them to a different location.
Though the above solution is interesting, there is another way. Keep the wsgi.conf VirtualHost configuration file you want to use in .ebextensions, and overwrite it in a post deploy hook (you can't do this pre-deploy because it will get re-generated (yes, I found this out the hard way). If you do this, to reboot, make sure to use the supervisorctl program to restart so as to get all your environment variables set properly. (I found this out the hard way as well.)
cp /tmp/wsgi.conf /etc/httpd/conf.d/wsgi.conf
/usr/local/bin/supervisorctl -c /opt/python/etc/supervisord.conf restart httpd
exit 0
01_python.config:
05_fixwsgiauth:
command: "cp .ebextensions/wsgi.conf /tmp"
I run my application locally using dev_appserver.py
using the following command
dev_appserver.py mydir --port=xxxx
Is there a way to delete all the local data generated by app engine server.
Currently, I go to localhost:8000/datastore, and delete entity groups manually.
IS there a way to automate it
Where does dev_appserver.py write local data to? (File or directory? may be in can delete that)
Yes there is!
dev_appserver.py accepts some arguments that clears things before it starts up. Two arguments that I use consistently are:
dev_appserver.py --clear_datastore=true --clear_search_indexes=true ...
dev_appserver.py --help can probably give you more information about other things that you may want to clear -- But this always gets everything I need it too.
I'm working on a site on OpenShift (Django, Python 2.7, Apache), and I'd like it to always be served over HTTPS. I figured this would be a simple task using .htaccess, and OpenShift's knowledge base seems to show that it should be. However, I've been agonizing over this for days, and I can't figure out how to make it work. Basically, I can't get my .htaccess files to operate on anything but static files. I simply can't get requests for non-static resources to redirect to anything.
Here's my directory structure, inside ~/app-root/repo. It's using the "new" (March 2014 or so?) OpenShift directory structure.
/MyProject
settings.py
urls.py
[etc.]
/[app directories]
/static
/wsgi
wsgi.py
static -> ../static
manage.py
requirements.txt
Ok, so after much frustration, I've been testing with a minimum viable .htaccess to just see if I can get anything to work. It looks like this:
RewriteEngine on
RewriteRule .* http://www.google.com
Everything should go to Google. If I put this in the wsgi directory, all my static files get redirected to google.com, but nothing else does. If I put it in ~/app-root/repo, nothing at all happens. It's like the file doesn't exist.
Similarly, if I use the commands suggested by OpenShift and put the file in wsgi, my static files get redirected to use HTTPS, but nothing else does.
I've also tried WSGI and Django things like adding os.environ['HTTPS'] = "on" to wsgi.py as suggested by sites like this one. That's been ineffective, and I really can't say I'm knowledgable enough about those sorts of settings to know what I'd need to add, but ideally I'd like to do this through .htaccess. The request should be using SSL before it hits WSGI.
What do I need to do to make my .htaccess files work on all requests? And why is OpenShift ignoring .htaccess files not in wsgi?
I fought with this for a short time and quickly decided to use the django-ssl-redirect middleware package.
The whole reason I'm on OpenShift is to avoid fighting with these kinds of configuration issue, so I'm a little disappointed I had to resort to this.
Still, here goes:
Just add this to your MIDDLEWARE_CLASSES
'ssl_redirect.middleware.SSLRedirectMiddleware',
Add this to requirements.txt
django-ssl-redirect==1.0
Add something like this in settings (I put this in my base settings file that I share in multiple environments, hence the "if").
if os.environ.has_key('OPENSHIFT_REPO_DIR'):
SSL_ON = True
SSL_ALWAYS = True
More info here: https://bitbucket.org/nextscreenlabs/django-ssl-redirect
As the title states, I'm trying to figure out the best practice for where to store application files for a Python website on the server. Document root, or no?
I come from a land of PHP. :)
EDIT - To that end, links to any material describing the best practice differences between Python and PHP are hugely appreciated.
No. WSGI containers don't require the scripts to be in the document root, and so to increase security in case of a transient server error they shouldn't be placed in the document root.
There's no reason to store it in the document root.
While storing the app in the doc root isn't nessescarily a security problem - if configured correctly and handled carefully - storing it outside will remove a lot of headache and configuration work.
That's the main reason not to do it.
I personally use https://bitbucket.org/acoobe/django-starter/ layout with buildout. So developed apps goes to apps folder and just used apps goes to parts/eggs folders (parts for packets from git, mercurial or svn and eggs for pypi located apps).
So the answer is NO. Everything should be placed in separate tidied folders. All your server need to know were is wsgi script and where is var dir. Well just like everyone else here said.
Everything has been said I think, so I will only elaborate a bit. Here is an explanation of how Apache maps URLs to files on disk: http://httpd.apache.org/docs/2.2/urlmapping.html. As you can see, the base rule is that only the files within DocumentRoot are exposed to the outside world. You can change that by doing the explicit import of other files or folders using e.g. Alias directive.
Now, you obviously don't want your Python scripts to be exposed to everyone - which means that you should keep them outside DocumentRoot and any other folder "imported" to DocumentRoot (using e.g. the mentioned Alias directive). What you want to do instead is to merely hook given URL to your Python program - if you use mod_wsgi, this can be done with WSGIScriptAlias directive. In other words, you should map the effects (result) of your script to given URL, instead of mapping the files themselves.
So - where you should keep your Python files? I would say it's a matter of personal taste - some people advise to not keep them in user folder (i.e. /home/xyz/) because e.g. Apache configuration flaw may expose user folders to the outside world. What's left? E.g. /usr/local/, /var/www - there's really no magic in picking home folder for your scripts.