I'm starting to program in Django and need some advice from you.
My project is a catalog containing more than 1000 products. Each product has an image.
I need to optimize the loading time and management of these images and my question is: Where do I store these images? In static folder, in the media folder or another solution?
I read the Django documentation and searched on google on static files and media files, but only found recommendations on "store user uploaded data such profile image in Media folder". But I do not know if this recommendation would apply to my project to optimize loading time and management of these image files.
What are your thoughts on the best way to store these images in my project based on your experience?
I'm using Nginx, DigitalOcean VPS.
Thank you!
Static files are part of your theme / skin ("packaging"). These files should generally not change and should be served based upon when the browser last retrieved them and set far ahead in the future:
location /static {
expires max;
}
Media files are part of the content, not the packaging of the site. It's much more likely that an image changes: the first one uploaded doesn't look that good, better manufacturer supplied media, etc. So it makes more sense to base it's expiration upon the modification date:
location /media {
# Optional: if rollbacks are frequent
# if_modified_since before;
expires modified+1w;
}
Regarding the rollbacks: if it's likely that older versions of an image is restored from backup or versioning systems, you'll want to set this. Note that Django will not set the modification time of an uploaded image to the modification time on the uploader's computer, so a new upload will be a new image, even if it's a previous version.
Now here comes the tricky part: page optimization tools will tell warn you about the expiration time of images not being far in the future and thus will complain about the media files. This is because they can't make a distinction between packaging and content when it comes to images.
Another thing to note is that after that week, the browser will request the images each time and nginx will serve a 304 response when it hasn't changed. Thus your page will generate many more requests. This and the page optimization tools are the two reasons why modified isn't used much for images in the wild and instead only the first policy is used. To deal with modified images, you will then associate a differently named image with the same resource (product) and thus is the responsibility of the content managers or a middleware layer that renames uploaded images to a unique name. One simple trick is to name media images as their md5 or sha1 hash.
Even if you apply a single expiration policy for serving media and static, I would still serve them from different directories. The reason is that static files do not have to be writable by the user running Django (only for the user running the collectstatic management command). This prevents configuration errors or a compromized Django user from messing with the static directory.
Related
I want to create the file(.wav) by django and let it be downloaded by user.
Currently I create the file under /myproj/static directory.
However it is mixed with the other jpg/img files, so,, it's not good design?
then, I read the document about /myproj/media directory but it is said used for user upload files.
So,,, how is the good directory design for server created file?
Should I create such as /myproj/create_wav?
but how can user access this url in template?
Thank you for any idea or helps.
According to Django's documentation,
By default, Django stores files locally, using the MEDIA_ROOT and MEDIA_URL settings. (...) However, Django provides ways to write custom file storage systems that allow you to completely customize where and how Django stores files.
So, in OP's shoes I'd simply use the media folder, since that's used not only for user uploads but also for user downloads.
I am working on a django project that provides an API to generate thumbnails of images, and the basic logic is like the following:
when the source image URL comes for the first time, the django would do some sort of image manipulation, and return the thumbnail image
when the same image URL comes again, django would simply serve the previous thumbnail image (stored as static media) again.
basically, case 2 happened much often than case 1. Now I used django to serve the images all the time, which I believe is a bad practice.
I wonder if it's possible to do a better way of image serving for case 2? For example, is there some sort of way to ask django to send proxy requests to apache and ask apache to serve the file?
I know I could use HTTP redirect to do that, but that seems to generate too much redirect requests on the client side (one HTML page would contain a lot of links to this API).
thx.
The simplest solution of the top of my head would be to use an Apache rewrite rule with a condition.
RewriteCond %(REQUEST_URI) ^media
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule #Some rewrite rule to redirect from '/media/filename' to '/image_generator/filename'
This basically just checks to see whether the file exists in the media directory, and if it doesn't it sends the user to the image generator, which can then generate and save the file to /media where it can be found for the next request.
NB: I've never actually tried this sort of redirection with Django, so it may need some measure of tweaking..
For example, is there some sort of way to ask django to send proxy requests to apache and ask apache to serve the file?
You have that exactly backwards.
Read the Django deployment guide. https://docs.djangoproject.com/en/1.3/howto/deployment/modwsgi/#serving-files
Apache should be serving all static files (images, for example) all the time. Always.
Django should never, ever serve an image file (or a .css or .js or anything other than .html).
See later part of this section in documentation:
http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines#The_Apache_Alias_Directive
Using Alias/AddHandler/mod_rewrite allows Django to overlay static files in filesystem. In other words, static files take precedence.
When localising Django application the makemessages command simply parses all the TXT, HTML and PY files and generates PO files for them but when localising JS files, you need to run the djangojs command. I haven't delved into the Django source to figure out why this done differently. Could someone explain?
I've read that in production environments, Apache is used to serve the application files while a simple proxy like Nginx is used to serve static files as this greatly reduces the load on the application server. With this scenario, I guess it works like this: when rendering a template, Django checks the requested locale, loads the appropriate localisation file and serves the template but JS on the other hand being served as static media doesn't get parsed by Django. Is this it?
(Its my first foray in to the world of localisation with Django and I'm packed full of question, many of who's answers I can't seem to find and therefore this post.)
Thanks
The reason why it's handled differently is in the docs.
Adding translations to JavaScript poses some problems:
JavaScript code doesn't have access to a gettext implementation.
JavaScript code doesn't have access to .po or .mo files; they need to be delivered by the server.
The translation catalogs for JavaScript should be kept as small as possible.
So essentially, the internal Python translation is done on the server. But for JS, there's another file served by the server, which contains all the required translations for user's language. And the translation is done on the user's side. So as you can see, it's a completely different strategy. Django helps by adding similar interface for JS files, even though they're handled in a completely different way.
I guess it works like this: when rendering a template, Django checks
the requested locale, loads the appropriate localisation file and
serves the template but JS on the other hand being served as static
media doesn't get parsed by Django. Is this it?
You are right in the first part, about handling templates. Handling JS works as I've explained above.
Note that Django JS translation mechanism, doesn't treat JS translations as static files. It uses a Django view to generate the JS file everytime (javascript_catalog mentioned in the docs linked in the first line).
That is one of the problems I've encountered. Such files don't need to be generated on every request. There are some projects that actually let you pack those JS translations as static files and enable you to cache them properly (like django-mediagenerator).
I currently have a django app which generates PDFs and saves them to a specific directory. From the admin interface I want to have some way to view the list of files within that directory (similar to models.FilePathField()), but also be able to download them. I realize that django was never intended to actually serve files, and I have been tinkering with django-sendfile, as a possible option. It just doesn't seem like there is any way to create a dynamic list of files other than with FilePathField (which I don't believe can suite my purposes).
Would this project fit your needs? http://code.google.com/p/django-filebrowser/
I ended up realizing that I was going about the problem in a more complicated manner than was necessary. Using two separate views trivializes the issue. I was just under the impression that the admin interface would include such a basic feature.
What I did was create a download_list view to display the files in the directory, and a download_file view which uses django-sendfile to serve the file to the end-user. Download_file simply parses through the directory with listdir(), checks if the extension is valid and sends the complete file path to the download_file function (after the user selects one).
Are the files in a directory that is served by your webserver? If all you want to do is list and download the files, it may be easier just to have a link to the directory with all the files in it and let the webserver take care of listing and serving the files. I understand this may not be the ideal solution for you, but it does avoid having Django serve static files, which is a task best left to the webserver.
Background:
I'm starting to use Django for the first time, which is also my first foray into web development. I just got stuck on the whole "serving static media" problem. After spending a while looking at all the documentation and StackOverflow questions, I think I understand how it's supposed to work (i.e. MEDIA_ROOT, MEDIA_URL, updating the urls file, etc).
My Question:
Ok, so here's the part I'm not sure about. Django applications are supposed to be "pluggable", i.e. I can move an application from one project to another. So, how should these applications bundle static media?
For example, let's say I have a "foo" application, which has templates that load some css/image files. Where am I supposed to put these files, so that they'll automatically get served once I include the application?
The only solution I see, is that installing an application has to include the extra step of copying its static media to some place on your own server that serves that media.
Is this the accepted way to do it? It includes an extra step, but maybe that's standard when dealing with web-dev (I'm new so I don't really know).
Also, if this is the way, is there a standard way to collect all my static media to make it easy to know what I need to serve? (I.e., is it standard to have a folder named "media" or something inside the app?).
Thanks,
Convention is to put static media in either media/appname/ or static/appname/ within the app (similar to templates).
For using apps in your project that come with media, I strongly recommend using django-staticfiles. It will automatically serve media (including media within apps) in development through a view that replaces django.views.static.serve, and it comes with a build_static management command that will copy media from all apps into a single directory for serving in production.
Update: django-staticfiles has become part of Django 1.3. It now expects app media to live in a "static/" subdirectory of the app, not "media/". And the management command is now "collectstatic."
The only app I know of that deals with this without any intervention is the rather wonderful django-debug-toolbar, though it's arguable that this isn't a great example, since it's an app specifically designed for debug mode only.
The way it deals with it is that it serves its media through Django itself - see the source for urls.py:
url(r'^%s/m/(.*)$' % _PREFIX, 'debug_toolbar.views.debug_media'),
In general, this is a bad idea (you don't want to serve static files through Django), per this comment from the documentation:
[Serving static files through Django] is inefficient and
insecure. Do not use this in a
production setting. Use this only for
development.
Obviously, the django-debug-toolbar is only used for development, so I think its method of deployment makes sense, but this is very much an exception.
In general, the best way I know to do it is to create symbolic links wherever your media is stored to the media inside your app code. For example, create a folder called media within your app, and then require users installing your app to either add a symbolic link from their media directory, or copy the whole thing.
i usually put apps media in ./apps/appname/static (my apps resides in an apps subfolder)
then i have something similar in the vhost in apache :
AliasMatch ^/apps/([^/]+)/static/(.*) /home/django/projectname/apps/$1/static/$2
<DirectoryMatch "^/home/django/projectname/apps/([^/]+)/static/*">
Order deny,allow
Options -Indexes
deny from all
Options +FollowSymLinks
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|swf|js|css|pdf|txt|htm|html|json)$">
allow from all
</FilesMatch>
</DirectoryMatch>
i also have this in my urls.py for dev server (use only for debug) :
def statics_wrapper(request, **dict):
from django.views import static
return static.serve(request, dict['path'], document_root = os.path.join(settings.BASE_DIR, 'apps', dict['app'], 'static'), show_indexes=True)
urlpatterns += patterns('', (r'^apps/(?P<app>[^/]+)/static/(?P<path>.+)$', statics_wrapper))
this is very handy because statics url are simply mapped to filesystem, eg :
http://wwww.ecample.com/apps/calendar/static/js/calendar.js resides in [BASE_DIR]/apps/calendar/static/js/calendar.js
hope this helps