My initial goal is to use sorl-thumbnail in the most basic way to cache on my filesystem cropped images that are downloaded from external sites. I don't care about performance at the moment and don't want to yet setup extra systems (memcachedb, redis). I am using the development runserver.
On the one hand the docs make it sound like I must use one of these two options. I feel like other places I have read that it can be setup to not require these KV stores. As one evidence for that, I see the setting sorl.thumbnail.kvstores.dbm_kvstore.KVStore in the reference docs (which says A simple Key Value Store has no dependencies outside the standard Python library and uses the DBM modules to store the data.), but I cannot get that to work either (see below).
Using Python 2.7.5, Django 1.7.1, Pillow 2.6.1, and sorl-thumbnail 12.1c.
Added sorl.thumbnail as part of my INSTALLED_APPS.
Added to settings.py:
THUMBNAIL_DEBUG = True
import logging
from sorl.thumbnail.log import ThumbnailLogHandler
handler = ThumbnailLogHandler()
handler.setLevel(logging.DEBUG)
logging.getLogger('sorl.thumbnail').addHandler(handler)
I see no other logging in my web server console despite this.
Attempted to sync my db:
$ ./manage.py migrate thumbnail
Operations to perform:
Apply all migrations: thumbnail
Running migrations:
Applying thumbnail.0001_initial... FAKED
No tables appear to be added to my database.
At this point, I've added to my template the load directive and the following snippet, where item.image_url is a models.URLField which works fine apart from thumbnail.
{% thumbnail item.image_url "235x200" crop="center" as im %}
<img src="{{ im.url }}">
{% empty %}
<p>No image</p>
{% endthumbnail %}
When I try to view the page, I see broken image links:
http://127.0.0.1:8001/<myapp>/cache/cf/43/cf43126f1f961593650b5df4791e329f.jpg 404 (NOT FOUND)
My MEDIA_URL is not set, though I tried playing with that to no avail.
I further tried putting into the settings: THUMBNAIL_KVSTORE = 'sorl.thumbnail.kvstores.dbm_kvstore.KVStore' but this gives the DJANGO error in the browser: Error importing module sorl.thumbnail.kvstores.dbm_kvstore: "No module named dbm_kvstore".
Can I configure it in this way, not requiring memcached, and if so, which of my settings are wrong/missing? If I must use memcached, how many more settings must I configure in addition to its installation? Thanks.
Update
Here are my settings involving static assets.
STATIC_URL = '/static/'
STATIC_ROOT = '/tmp/static/'
STATICFILES_DIRS = (
PROJECT_ROOT.child("static"),
)
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
#'django.contrib.staticfiles.finders.DefaultStorageFinder', If enabled breaks my LESS CSS.
'static_precompiler.finders.StaticPrecompilerFinder',
)
# STATICFILES_STORAGE not set but should default to 'django.contrib.staticfiles.storage.StaticFilesStorage'
I am already serving some static images from mysite/static/img. And I forgot to mention that I am using the Django Static Precompiler for LESS CSS. My LESS files are at mysite/static/css and are compiled to /tmp/static/COMPILED/.
I see there is a cache dir in my project root mysite, and it does have the file which is trying to be served: cache/6a/a6/6aa6ebf6cef5bf481fd37d4947d25623.jpg.
I've read the documentation on serving static assets but it's unclear to me what settings to change. Seems that I either need to have that jpg produced in a different directory, or add this directory to the list of dirs from which I'm serving static assets. I tried adding this path to STATICFILES_DIRS but that didn't work.
When you go into this directory '/cache/cf/43/', do you actually see the file 'cf43126f1f961593650b5df4791e329f.jpg' in there?
If so, it may be returning a 404 because you are using the Django runserver (not sure if you are or not). If you are, it might be worth taking a look at how to serve media files in development mode, https://docs.djangoproject.com/en/1.7/howto/static-files/.
You don't need to setup the cache backend initialy, you may need to setup serving the static files, please look at the django docs about serving MEDIA and STATIC resources.
But the most important, Django 1.7 Support was introduced in the 12.1c release.
Try first:
pip install sorl-thumbnail==12.1c
It's also helpful that you set the debug thumbnail setting on your settings file:
THUMBNAIL_DEBUG = True
Both other answers correctly suggested problems with serving up media files. Here is the complete list of code changes that were required:
I had glossed over the fact that MEDIA assets are not the same as STATIC assets. This answer is the one that tipped me off to the fact that sorl-thumbnail is relying on MEDIA_URL to form its URL, and accordingly, MEDIA_ROOT. For my development, I set the following:
MEDIA_URL = '/media/'
MEDIA_ROOT = '/tmp/media/'
I used this snippet for URLconf changes for serving up the media files. At this point I put an image in the media directory and made sure my template could correctly reference it.
By this point, I had manually removed the sorl-thumbnail generated thumbnails, but no permutation of settings and activity would regenerate them. I remembered that getting your key value store/database/cached images out of sync requires manual cleanup. The management command ./manage.py thumbnail cleanup did the job, and it began regenerating again.
Also worth noting is that I did not have to set THUMBNAIL_KVSTORE at all, or setup any key value store.
I hope this helps get others get started faster in setting up their dev environment.
Related
I use django and my application works, when I change the style.css the app changes etc.
However, I recently found out that I didn't define STATICFILES_DIR, I only defined
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, "static/")
Which is only for collectstatic.
But now I'm wondering, how does django know where to look for the static files?
Specifically in the polls/static/polls directory?
Django will look for a directory named static in each of your apps by default. More on this later.
From the documentation:
Now we might be able to get away with putting our static files
directly in my_app/static/ (rather than creating another my_app
subdirectory), but it would actually be a bad idea. Django will use
the first static file it finds whose name matches, and if you had a
static file with the same name in a different application, Django
would be unable to distinguish between them.
The key part being
...Django will use the first static file it finds whose name matches...
There are Django libraries called "static file finders" that help with this process. You can configure them by modifying the installed apps in your Django configuration file.
According to the documentation, they're actually undocumented, because they are considered private. However, you can look at the source code for those "finders" on GitHub.
Django projects are composed of modules called apps. Each app contains some default files when you generate it. Although there is configuration, there's plenty of convention about Django too. For example, the views.py file. Another such convention is your app's static directories folder.
Your STATICFILES_DIR array allows you to add additional places for you to put your static files. This is useful, as you'll see when you prepare to deploy, because of the way Django handles static files in a real-world environment.
It's totally optional, but it's there to make your life easier later.
I'm attempting to setup appropriate STATIC_URL and STATIC_ROOT values for a new Django project, and I'm running into a problem with using an absolute URL for STATIC_URL.
My project is structured like:
<project root>
static
media
src
<apps>
static
js
css
custom.css
i
settings.py
In my settings.py, I have
STATIC_ROOT = os.path.abspath(os.path.join(os.path.dirname(os.path.realpath(__file__)), '../static/'))
If I set STATIC_URL = '/static/', then I can access http://localhost:8000/static/css/custom.css perfectly.
However, if I use an absolute URL like STATIC_URL = 'http://localhost:8000/static/', as though I were using a CDN, then http://localhost:8000/static/css/custom.css returns a 404 error.
Shouldn't these settings be effectively identical? The docs state STATIC_URL can be absolute. Why does the later setting fail to load static media?
I just tested the same thing, using http://localhost:8000/static/ doesn't work for me either.
In my settings I keep a development variable, so on localhost the url is '/static/' and when I'm deployed (and I set DEVELOPMENT = False), its a full url.
if DEVELOPMENT == True:
STATIC_URL = '/static/'
else:
STATIC_URL = 'https://www.mywebsite.com/static/'
I assume that you are using the static function from the django.conf.urls.static to serve your static files in the development environment (that means DEBUG=True). Now as you can read in the documentation this function is not working with the absolute urls with DEBUG enabled.
So from the docs:
https://docs.djangoproject.com/en/dev/howto/static-files/#serving-static-files-during-development
"This helper function works only in debug mode and only if the given prefix is local (e.g. /static/) and not a URL (e.g. http://static.example.com/)."
The cause and solution was simple, but discovering it was unintuitive based on how I described the problem. I failed to mention that my absolute STATIC_URL actually had a different domain (http://localmedia:8000/static/ not http://localhost:8000/static/).
I was doing this as part of a hack to simplify media hosting among several external development Windows machines I was using to test various versions of IE.
However, when there's a mismatch between the domain in STATIC_URL and runserver's local domain, Django only serves content from STATIC_ROOT and doesn't attempt to retrieve media from app folders. Not sure why, but that's the way it is.
If I ran manage.py collectstatic, it would effectively fix the 404 error by copying all app folder media into my STATIC_ROOT. Alternatively, I could just use the same domain in my STATIC_URL, and Django would automatically find any uncollected media in the app folders.
I'm trying my first proper webdev project and i'm learning the django framework.
I came here to ask about the cleanest way to use "static files", like the external CSS i'm referring to in one of my html templates. I tried reading through the official documentation on the subject but found it a little bit confusing as a beginner, i then tried googling but i noticed that most of the guides or stackoverflow answers differed slightly and i realised i needed a better understanding. Bit cheeky to ask but, could someone explain and summarise the process to me?
For reference, here is my project folder hierarchy. At the moment i'm trying to get the template base.html to use the sylesheet at CSS/base.css:
Also one of the things that keeps throwing me off is the use of absolute filepaths. So far i've managed to get away with just using relative filepaths, which makes more sense to me as the aim is to develop on the django test server then transfer it all onto my own server when i have one. (note: perhaps it's because i have no idea how complicated that transfer process is that i don't understand why absolute filepaths are prefered). What's the issue with using relative filepaths?
I realise that this has sort of become two questions, which is against the rules, but i really think both could be answered together and if i understood one it would probably aid my understanding of the other. Any help would be much appreciated.
Here is a set of code snippets from settings.py i have changed for including static files
import os
import django
DJANGO_ROOT = os.path.dirname(os.path.realpath(django.__file__))
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
Dynamically setting variables for SITE_ROOT directory
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(SITE_ROOT,'data.db'),
where the sqlite3 database is stored it needs absolute address
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
}
STATIC_ROOT = SITE_ROOT
STATIC_URL = '/static/'
store static files here
STATICFILES_DIRS = (
os.path.join(SITE_ROOT,'static'),
)
where the templates are stored
TEMPLATE_DIRS = (
os.path.join(SITE_ROOT,'templates'),
)
Now in templates use
{% load static %} --at beginning
<link href="{% static "css/bootstrap.min.css"%}" rel="stylesheet" media="screen">
this is an example showing how to access "css/bootstrap.min.css" in static directory
I'm going to assume you're using Django 1.3 or higher as that allows you to use staticfiles (docs) for static files. Here's the basic idea: each Django website consists of multiple apps (directories) that contain all the files needed for a particular part of the website (i.e., a models.py, views.py, a templates directory and also a static directory). This static directory contains the JavaScript and CSS files (as well as other static files for that app.
When deploying the website, you run the collectstatic command to gather all the static files into one directory. From that directory you then serve the static files to visitors of your site (i.e, Apache or a different web server can do that for you).
So the general structure of your web project could be:
manage.py
website/models.py
website/views.py
website/static/website/css/base.css
website/static/website/js/base.js
website/templates/website/base.html
static
settings.py
urls.py
You'll notice that the name of the Django app is repeated in the subdirectories of the static and templates directories. This is necessary because it keeps the files apart when they are moved to a general static directory. In the above example, the STATIC_ROOT is the top-level static directory and into this directory all your static files will be copied.
It's definitely worth reading the documentation of staticfiles as this project layout allows you to reuse Django apps in other projects with ease. For example, you can imagine a website that looks like this:
blog/models.py
blog/templates/blog/overview.html
blog/templates/blog/post.html
blog/urls.py
blog/views.py
photos/models.py
photos/templates/photos/overview.html
photos/templates/photos/photo.html
photos/urls.py
photos/views.py
manage.py
static
settings.py
urls.py
The apps blog and photos can now easily be reused in other websites if desired.
Really appreciate the time put into the #Simeon Visser's answer, and there was some really useful information there but it wasn't entirely what I was looking for. So I asked around and have sinister_user_name from reddit to thank for this, which I thought might help someone else:
Static files have changed a fair bit in recent releases so old blogs might be a bit of a mess.
I'll give it a shot:
In your templates you can use the static_url tag and django will fill in the path for you.
<link rel="stylesheet" href="{{ STATIC_URL }}css/bootstrap.min.css">
It fills in the static url by looking at your settings file. STATIC_URL defaults to /static/ I think in settings? If static files end up in a weird url on your corporate web server you can just change it in settings anyway and it will reflect in all templates. So it doesn't matter if they are absolute.
When you go to deploy you fill in STATIC_ROOT in your settings, this is just the path that you've set your webserver to find the static files. Then if you run manage.py collectstatic and it will copy them to that directory. That's the only function of STATIC_ROOT I think.
I keep the template directory separate from my static directory, mainly because you want your webserver just to serve static files, while templates are processed by python. So, static files are almost not part of your application as far as the web server views them. I keep my templates and static files in separate paths just in the base directory, mainly because both are not python source files.
So my project structure would be like:
manage.py
website/settings.py
app/models.py
app/views.py
templates/base.html
static/js/jquery.js
static/css/bootstrap.css
The problem with relative paths is that they will need to change if your url scheme changes.
Optional: I found in this blog which helps to not put absolute file system paths into the settings.py which is good if your working from multiple machines eg work and home :(
With the slightly different layout for 1.4 I use this at the top of my settings:
import os
import django
import manage
DJANGO_ROOT = os.path.dirname(os.path.realpath(django.__file__))
SITE_ROOT = os.path.dirname(os.path.realpath(manage.__file__))
instead of what they suggest (the location of settings has moved in 1.4 or 1.3) so SITE_ROOT is messed up. I think that might work.
Further on down in the settings I use this:
# Additional locations of static files
STATICFILES_DIRS = (
os.path.join(SITE_ROOT, 'static'),
)
This will tell the development server where to find the static files. You can do the same for templates or just use the full file path.
I've recently have taken a new server under my wing that has an interesting installation of django on it. The previous developer mixed in the media uploads with the static content and in other modules created it's own directory on the root file level of the project. My first reaction to this was general annoyance. ( I'm a huge fan of modular development. ) However after working to 'correct,' it's raised a question.
Even though this question is tagged with django, feel free to post response according to java and asp.net.
How do you set up your static files? Do you stack everything inside a static directory or do you take the time link each modular independently?
One of my tricks for every django app I start is, in the init.py of said app I put the following.
import os
from django.conf import settings as djsettings
TEMPLATES_DIR = (os.path.join(os.path.dirname(__file__),'./templates'),)
djsettings.TEMPLATES_DIR += TEMPLATES_DIR
I don't think your trick is really needed (anymore).
If you use django.template.loaders.app_directories.Loader (docs)
you can put a template dir in your app dir and Django will check it for your app-specific templates
And starting with Django 1.3 you can use the staticfiles app to do something similar with all your static media files. Check the docs for the staticfiles-finders
So finally, thanks to the collectstatic management command, you're now able to keep your static media files modularized (for easier development and distribution), yet you still can bundle them at a centralized place once it is time to deploy and serve your project.
Right now I'm in the habit of putting a static folder in each app directory containing its static files. I keep templates in each app directory under templates. I alias the static path when putting the app behind nginx or apache.
One thing that I'm starting to do more of is putting static files such as javascript, css, or images behind a CDN like S3.
Background:
I'm starting to use Django for the first time, which is also my first foray into web development. I just got stuck on the whole "serving static media" problem. After spending a while looking at all the documentation and StackOverflow questions, I think I understand how it's supposed to work (i.e. MEDIA_ROOT, MEDIA_URL, updating the urls file, etc).
My Question:
Ok, so here's the part I'm not sure about. Django applications are supposed to be "pluggable", i.e. I can move an application from one project to another. So, how should these applications bundle static media?
For example, let's say I have a "foo" application, which has templates that load some css/image files. Where am I supposed to put these files, so that they'll automatically get served once I include the application?
The only solution I see, is that installing an application has to include the extra step of copying its static media to some place on your own server that serves that media.
Is this the accepted way to do it? It includes an extra step, but maybe that's standard when dealing with web-dev (I'm new so I don't really know).
Also, if this is the way, is there a standard way to collect all my static media to make it easy to know what I need to serve? (I.e., is it standard to have a folder named "media" or something inside the app?).
Thanks,
Convention is to put static media in either media/appname/ or static/appname/ within the app (similar to templates).
For using apps in your project that come with media, I strongly recommend using django-staticfiles. It will automatically serve media (including media within apps) in development through a view that replaces django.views.static.serve, and it comes with a build_static management command that will copy media from all apps into a single directory for serving in production.
Update: django-staticfiles has become part of Django 1.3. It now expects app media to live in a "static/" subdirectory of the app, not "media/". And the management command is now "collectstatic."
The only app I know of that deals with this without any intervention is the rather wonderful django-debug-toolbar, though it's arguable that this isn't a great example, since it's an app specifically designed for debug mode only.
The way it deals with it is that it serves its media through Django itself - see the source for urls.py:
url(r'^%s/m/(.*)$' % _PREFIX, 'debug_toolbar.views.debug_media'),
In general, this is a bad idea (you don't want to serve static files through Django), per this comment from the documentation:
[Serving static files through Django] is inefficient and
insecure. Do not use this in a
production setting. Use this only for
development.
Obviously, the django-debug-toolbar is only used for development, so I think its method of deployment makes sense, but this is very much an exception.
In general, the best way I know to do it is to create symbolic links wherever your media is stored to the media inside your app code. For example, create a folder called media within your app, and then require users installing your app to either add a symbolic link from their media directory, or copy the whole thing.
i usually put apps media in ./apps/appname/static (my apps resides in an apps subfolder)
then i have something similar in the vhost in apache :
AliasMatch ^/apps/([^/]+)/static/(.*) /home/django/projectname/apps/$1/static/$2
<DirectoryMatch "^/home/django/projectname/apps/([^/]+)/static/*">
Order deny,allow
Options -Indexes
deny from all
Options +FollowSymLinks
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|swf|js|css|pdf|txt|htm|html|json)$">
allow from all
</FilesMatch>
</DirectoryMatch>
i also have this in my urls.py for dev server (use only for debug) :
def statics_wrapper(request, **dict):
from django.views import static
return static.serve(request, dict['path'], document_root = os.path.join(settings.BASE_DIR, 'apps', dict['app'], 'static'), show_indexes=True)
urlpatterns += patterns('', (r'^apps/(?P<app>[^/]+)/static/(?P<path>.+)$', statics_wrapper))
this is very handy because statics url are simply mapped to filesystem, eg :
http://wwww.ecample.com/apps/calendar/static/js/calendar.js resides in [BASE_DIR]/apps/calendar/static/js/calendar.js
hope this helps