We Keep Coding

Static files namespaces in django

I can't get my static files in Django to work with "namespaced" urls - I wonder if I'm missing something here?
my project structure
Basically I have one app called "foo" in my project "static_test". Inside that app I have a "static" folder which contains "test.txt".
I haven't changed anything in settings.py, so it contains all the default settings: staticfiles is included in INSTALLED_APPS, and STATIC_URL is set to '/static/'.
All of this is in full accordance with official Django tutorial: https://docs.djangoproject.com/en/1.9/howto/static-files/
I'm using Django 1.9.2.
When I start the Django development server, this is what I get:
localhost:8000/static/test.txt - 200 OK
localhost:8000/static/foo/test.txt - 404 Page not found
This is despite the official tutorial claiming
In your templates, either hardcode the url like /static/my_app/myexample.jpg or, preferably, use the static template tag to build the URL for the given relative path by using the configured STATICFILES_STORAGE storage (this makes it much easier when you want to switch to a content delivery network (CDN) for serving static files).
Why is the second url (/static/foo/test.txt) not working for me? And why is the first url working? I do not have a project-wide static folder.
Thanks

from the docs
You should use 'django.contrib.staticfiles.finders.AppDirectoriesFinder' in your STATICIFILES_FINDERS variable in your settings.

First one is searching under static.
Second one under foo under static.
Your link should look like this.
href="{% static "test.txt" %}"
And settings.py like this
STATIC_URL = '/static/'

Minimal developer setup of sorl-thumbnail with Django 1.7

My initial goal is to use sorl-thumbnail in the most basic way to cache on my filesystem cropped images that are downloaded from external sites. I don't care about performance at the moment and don't want to yet setup extra systems (memcachedb, redis). I am using the development runserver.
On the one hand the docs make it sound like I must use one of these two options. I feel like other places I have read that it can be setup to not require these KV stores. As one evidence for that, I see the setting sorl.thumbnail.kvstores.dbm_kvstore.KVStore in the reference docs (which says A simple Key Value Store has no dependencies outside the standard Python library and uses the DBM modules to store the data.), but I cannot get that to work either (see below).
Using Python 2.7.5, Django 1.7.1, Pillow 2.6.1, and sorl-thumbnail 12.1c.
Added sorl.thumbnail as part of my INSTALLED_APPS.
Added to settings.py:
THUMBNAIL_DEBUG = True
import logging
from sorl.thumbnail.log import ThumbnailLogHandler
handler = ThumbnailLogHandler()
handler.setLevel(logging.DEBUG)
logging.getLogger('sorl.thumbnail').addHandler(handler)
I see no other logging in my web server console despite this.
Attempted to sync my db:
$ ./manage.py migrate thumbnail
Operations to perform:
Apply all migrations: thumbnail
Running migrations:
Applying thumbnail.0001_initial... FAKED
No tables appear to be added to my database.
At this point, I've added to my template the load directive and the following snippet, where item.image_url is a models.URLField which works fine apart from thumbnail.
{% thumbnail item.image_url "235x200" crop="center" as im %}
<img src="{{ im.url }}">
{% empty %}
<p>No image</p>
{% endthumbnail %}
When I try to view the page, I see broken image links:
http://127.0.0.1:8001/<myapp>/cache/cf/43/cf43126f1f961593650b5df4791e329f.jpg 404 (NOT FOUND)
My MEDIA_URL is not set, though I tried playing with that to no avail.
I further tried putting into the settings: THUMBNAIL_KVSTORE = 'sorl.thumbnail.kvstores.dbm_kvstore.KVStore' but this gives the DJANGO error in the browser: Error importing module sorl.thumbnail.kvstores.dbm_kvstore: "No module named dbm_kvstore".
Can I configure it in this way, not requiring memcached, and if so, which of my settings are wrong/missing? If I must use memcached, how many more settings must I configure in addition to its installation? Thanks.
Update
Here are my settings involving static assets.
STATIC_URL = '/static/'
STATIC_ROOT = '/tmp/static/'
STATICFILES_DIRS = (
PROJECT_ROOT.child("static"),
)
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
#'django.contrib.staticfiles.finders.DefaultStorageFinder', If enabled breaks my LESS CSS.
'static_precompiler.finders.StaticPrecompilerFinder',
)
# STATICFILES_STORAGE not set but should default to 'django.contrib.staticfiles.storage.StaticFilesStorage'
I am already serving some static images from mysite/static/img. And I forgot to mention that I am using the Django Static Precompiler for LESS CSS. My LESS files are at mysite/static/css and are compiled to /tmp/static/COMPILED/.
I see there is a cache dir in my project root mysite, and it does have the file which is trying to be served: cache/6a/a6/6aa6ebf6cef5bf481fd37d4947d25623.jpg.
I've read the documentation on serving static assets but it's unclear to me what settings to change. Seems that I either need to have that jpg produced in a different directory, or add this directory to the list of dirs from which I'm serving static assets. I tried adding this path to STATICFILES_DIRS but that didn't work.

When you go into this directory '/cache/cf/43/', do you actually see the file 'cf43126f1f961593650b5df4791e329f.jpg' in there?
If so, it may be returning a 404 because you are using the Django runserver (not sure if you are or not). If you are, it might be worth taking a look at how to serve media files in development mode, https://docs.djangoproject.com/en/1.7/howto/static-files/.

You don't need to setup the cache backend initialy, you may need to setup serving the static files, please look at the django docs about serving MEDIA and STATIC resources.
But the most important, Django 1.7 Support was introduced in the 12.1c release.
Try first:
pip install sorl-thumbnail==12.1c
It's also helpful that you set the debug thumbnail setting on your settings file:
THUMBNAIL_DEBUG = True

Both other answers correctly suggested problems with serving up media files. Here is the complete list of code changes that were required:
I had glossed over the fact that MEDIA assets are not the same as STATIC assets. This answer is the one that tipped me off to the fact that sorl-thumbnail is relying on MEDIA_URL to form its URL, and accordingly, MEDIA_ROOT. For my development, I set the following:
MEDIA_URL = '/media/'
MEDIA_ROOT = '/tmp/media/'
I used this snippet for URLconf changes for serving up the media files. At this point I put an image in the media directory and made sure my template could correctly reference it.
By this point, I had manually removed the sorl-thumbnail generated thumbnails, but no permutation of settings and activity would regenerate them. I remembered that getting your key value store/database/cached images out of sync requires manual cleanup. The management command ./manage.py thumbnail cleanup did the job, and it began regenerating again.
Also worth noting is that I did not have to set THUMBNAIL_KVSTORE at all, or setup any key value store.
I hope this helps get others get started faster in setting up their dev environment.

Can't load static files with bottle on elastic beanstalk

After some help, I got my bottle application running with Elastic Beanstalk. Well almost - I still can't see any static files. I have followed the instructions here by inserting a file called python.conf in directory .ebextensions with this in it:
option_settings:
"aws:elasticbeanstalk:container:python:staticfiles":
"/static/": "static/"
Unfortunately that did not work. My files are in folders under /static (for example, /static/js, /static/css etc). In my bottle application, I was serving static files as per the answer here, like this:
#route('/static/:path#.+#')
def server_static(path):
return static_file(path, root='./static')
It all works when I run it locally. Does anyone know what I'm doing wrong? Should I be following a different procedure because my static files are in subfolders, or because I'm using bottle instead of flask? I'm aware of a similar question that has been asked here for flask, but there has been no answer (and for all I know it could be a totally different issue).
Thanks a lot, Alex

I'm sure it isn't the most elegant solution, but I solved this by putting all of my static files in an S3 bucket and using that in all of my pages.
I didn't really have that many static files to begin with so it wasn't a very big deal. I just made a variable of the S3 bucket url:
S3Static = r'mystaticbucket.s3-us-west-2.amazonaws.com'
Passed it to my bottle templates, and changed the links from:
<link rel="stylesheet" type="text/css" href = "/static/css/MarmoStyle.css" >
to
<link rel="stylesheet" type="text/css" href = "{{S3Static}}/static/css/Style.css">
(ie just added {{S3Static}} before the path)
If you're using a static file in a separate %included header template like I am, you have to pass the S3Static variable to the template like this:
%include header.tpl S3Static=S3Static
And that was about it. I know this won't be an ideal solution for everyone and there are probably better ways to do it, but it's worked for me so far.
Thanks,
Alex

You should really look into python's whitenoise module. Then you only have one line you have to change and the URL routing is handled by whitenoise next to bottle. Essentially you can keep the url the same as your domain, even if whitenoise is pulling from someplace else. PLus it's designed to cache your static data. Definitely should be using it. It takes like 3 lines to setup, and you can remove bottle's static folder route.

Static assets don't show up for flask on elastic beanstalk

How do you get aws elastic beanstalk to recognize your static assets in your flask app? I did the standard /.ebextensions/python.config couple of YAML lines a la:
option_settings:
"aws:elasticbeanstalk:container:python:staticfiles":
"/static/": "static/"
All of my calls to assets in templates are wrapped in "{{url_for('static', filename='img/office.jpg')}}" type things.
But no images, styles or javascript shows up on page load. Here is an example log file 404.
IP - - [25/Feb/2013:21:48:13 +0000] "GET /static/css/bootstrap.css HTTP/1.1" 404 328 "http://xyz.elasticbeanstalk.com/"
Am I missing something obvious? Everything runs great on my local, just static assets dont load once i git aws.push

It can be done also through Elastic Beanstalk Panel:
Configuration -> Software Configuration - > Static Files
and then
just as an alternative option

As of this writing, after spending many hours fighting with AWS EB's config, I gave up trying to make the static files work the way we all expect and updated my Flask app creation to:
app = Flask(__name__, static_url_path='/s')
This renders urls like /s/scripts/my-script.js and since I always use url_for('static', ...) in my code and templates, everything continued to work outside of AWS as well.
Update on 9/30/2013: I can pretty much guarantee that the staticFiles settings are completely ignored in AWS EB's Python container.
The change I suggested above has the undesirable downside of routing all static file requests through Flask (maybe, more accurately, WSGI.) That's not very hard to fix, though.
Create an Apache conig file at your project root, named app-httpd.conf:
Alias /s /opt/python/current/app/static
<Directory /opt/python/current/app/static>
Order allow,deny
Allow from all
</Directory>
This config tells Apache to take over any requests for URLs starting with /s, same prefix we chose for our static files, and serve files from our app's static folder.
Create this file at .ebextensions/custom-apache.config:
container_commands:
add_apache_conf:
command: "cp app-httpd.conf /etc/httpd/conf.d"
This file will be used during the app deployment and will copy the new .config file to a directory from which Apache is configure to load all .config files it sees.

4+ years later, I'm able to get static files working using:
(file: .ebextensions/WHATEVER_NAME.config)
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: StaticFiles
value: /static/=PATH/FROM/MY/APP/BASE/DIR/TO/STATIC/DIR/
...in my case, this was
value: /static/=distrib/static/
I found that changing my
app = Flask(__name__)
to
app = Flask(__name__, static_url_path='/static')
was neither necessary nor sufficient. When I only set static_url_path but not StaticFiles, it didn't work; when I set StaticFiles but not static_url_path, it worked fine.
<sarcasm>Elastic Beanstalk is super straightforward and well documented!</sarcasm>

I spent a goodly amount of time trying to figure this out and sort through these suggestions. I've made comments on the answers that were still relevant in 2020. Here's the TL;DR of the solution I encountered:
The static files section in the Modify Software screen under the Configuration left navigation pane on the Elastic Beanstalk UI sets the property aws:elasticbeanstalk:container:python:staticfiles. This configuration does not map to the static_url_path which Flask needs to find the static assets.
The correct property to set is StaticFiles, which can be set by adding a .config file to the .ebextensions directory.
Read more here.

A strange thing that I found to solve this issue was editing my .gitignore file. It had included removing the /dist folders, and that included the dist folders that my CSS was generated into. So the css files were actually missing when I deployed.
Hope this may help anyone who might be in the same boat.

This problem baffled me for a bit because it worked locally. No need to change config. I change line 1 to line 2 in my html file and it worked. Keep static folder in root directory and also keep config at /static/:/static/.
1 <link rel="stylesheet" href={{url_for('static',filename='style.css')}}/>
2 <link rel= "stylesheet" type= "text/css" href= "{{ url_for('static',filename='style.css') }}">
may be the quotes around the href.

I tried Myles Baker's solution, but did not work as upon deployment I received the error -
ERROR Invalid option specification (Namespace: 'aws:elasticbeanstalk:container:python', OptionName: 'StaticFiles')
I ended up using the following code in my .ebextensions/WHATEVER_NAME.config -
option_settings:
aws:elasticbeanstalk:environment:proxy:staticfiles:
/html: statichtml
/images: staticimages
Which is a straight copy/paste from here - https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environment-cfg-staticfiles.html

I had a similar problem where, strangely enough, all files from static/img/ were being served, but anything in static/css/ or static/js/ were simply not being served.
To fix this I opened .elasticbeanstalk/optionsettings.app-env from the root directory of the Flask source and edited this block of code to be
[aws:elasticbeanstalk:container:python]
NumProcesses=1
NumThreads=15
StaticFiles=/static/.*=
WSGIPath=application.py
[aws:elasticbeanstalk:container:python:staticfiles]
Note that StaticFiles actually takes a regular expression, so in /static/.* the .* allows for any path after /static/

Django static files, and filepaths in settings.py

I'm trying my first proper webdev project and i'm learning the django framework.
I came here to ask about the cleanest way to use "static files", like the external CSS i'm referring to in one of my html templates. I tried reading through the official documentation on the subject but found it a little bit confusing as a beginner, i then tried googling but i noticed that most of the guides or stackoverflow answers differed slightly and i realised i needed a better understanding. Bit cheeky to ask but, could someone explain and summarise the process to me?
For reference, here is my project folder hierarchy. At the moment i'm trying to get the template base.html to use the sylesheet at CSS/base.css:
Also one of the things that keeps throwing me off is the use of absolute filepaths. So far i've managed to get away with just using relative filepaths, which makes more sense to me as the aim is to develop on the django test server then transfer it all onto my own server when i have one. (note: perhaps it's because i have no idea how complicated that transfer process is that i don't understand why absolute filepaths are prefered). What's the issue with using relative filepaths?
I realise that this has sort of become two questions, which is against the rules, but i really think both could be answered together and if i understood one it would probably aid my understanding of the other. Any help would be much appreciated.

Here is a set of code snippets from settings.py i have changed for including static files
import os
import django
DJANGO_ROOT = os.path.dirname(os.path.realpath(django.__file__))
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
Dynamically setting variables for SITE_ROOT directory
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(SITE_ROOT,'data.db'),
where the sqlite3 database is stored it needs absolute address
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
}
STATIC_ROOT = SITE_ROOT
STATIC_URL = '/static/'
store static files here
STATICFILES_DIRS = (
os.path.join(SITE_ROOT,'static'),
)
where the templates are stored
TEMPLATE_DIRS = (
os.path.join(SITE_ROOT,'templates'),
)
Now in templates use
{% load static %} --at beginning
<link href="{% static "css/bootstrap.min.css"%}" rel="stylesheet" media="screen">
this is an example showing how to access "css/bootstrap.min.css" in static directory

I'm going to assume you're using Django 1.3 or higher as that allows you to use staticfiles (docs) for static files. Here's the basic idea: each Django website consists of multiple apps (directories) that contain all the files needed for a particular part of the website (i.e., a models.py, views.py, a templates directory and also a static directory). This static directory contains the JavaScript and CSS files (as well as other static files for that app.
When deploying the website, you run the collectstatic command to gather all the static files into one directory. From that directory you then serve the static files to visitors of your site (i.e, Apache or a different web server can do that for you).
So the general structure of your web project could be:
manage.py
website/models.py
website/views.py
website/static/website/css/base.css
website/static/website/js/base.js
website/templates/website/base.html
static
settings.py
urls.py
You'll notice that the name of the Django app is repeated in the subdirectories of the static and templates directories. This is necessary because it keeps the files apart when they are moved to a general static directory. In the above example, the STATIC_ROOT is the top-level static directory and into this directory all your static files will be copied.
It's definitely worth reading the documentation of staticfiles as this project layout allows you to reuse Django apps in other projects with ease. For example, you can imagine a website that looks like this:
blog/models.py
blog/templates/blog/overview.html
blog/templates/blog/post.html
blog/urls.py
blog/views.py
photos/models.py
photos/templates/photos/overview.html
photos/templates/photos/photo.html
photos/urls.py
photos/views.py
manage.py
static
settings.py
urls.py
The apps blog and photos can now easily be reused in other websites if desired.

Really appreciate the time put into the #Simeon Visser's answer, and there was some really useful information there but it wasn't entirely what I was looking for. So I asked around and have sinister_user_name from reddit to thank for this, which I thought might help someone else:
Static files have changed a fair bit in recent releases so old blogs might be a bit of a mess.
I'll give it a shot:
In your templates you can use the static_url tag and django will fill in the path for you.
<link rel="stylesheet" href="{{ STATIC_URL }}css/bootstrap.min.css">
It fills in the static url by looking at your settings file. STATIC_URL defaults to /static/ I think in settings? If static files end up in a weird url on your corporate web server you can just change it in settings anyway and it will reflect in all templates. So it doesn't matter if they are absolute.
When you go to deploy you fill in STATIC_ROOT in your settings, this is just the path that you've set your webserver to find the static files. Then if you run manage.py collectstatic and it will copy them to that directory. That's the only function of STATIC_ROOT I think.
I keep the template directory separate from my static directory, mainly because you want your webserver just to serve static files, while templates are processed by python. So, static files are almost not part of your application as far as the web server views them. I keep my templates and static files in separate paths just in the base directory, mainly because both are not python source files.
So my project structure would be like:
manage.py
website/settings.py
app/models.py
app/views.py
templates/base.html
static/js/jquery.js
static/css/bootstrap.css
The problem with relative paths is that they will need to change if your url scheme changes.
Optional: I found in this blog which helps to not put absolute file system paths into the settings.py which is good if your working from multiple machines eg work and home :(
With the slightly different layout for 1.4 I use this at the top of my settings:
import os
import django
import manage
DJANGO_ROOT = os.path.dirname(os.path.realpath(django.__file__))
SITE_ROOT = os.path.dirname(os.path.realpath(manage.__file__))
instead of what they suggest (the location of settings has moved in 1.4 or 1.3) so SITE_ROOT is messed up. I think that might work.
Further on down in the settings I use this:
# Additional locations of static files
STATICFILES_DIRS = (
os.path.join(SITE_ROOT, 'static'),
)
This will tell the development server where to find the static files. You can do the same for templates or just use the full file path.