I have a file on my computer that I'm trying to serve up as JSON from a django view.
def serve(request):
file = os.path.join(BASE_DIR, 'static', 'files', 'apple-app-site-association')
response = HttpResponse(content=file)
response['Content-Type'] = 'application/json'
What I get back is the path to the file when navigating to the URL
/Users/myself/Developer/us/www/static/files/apple-app-site-association
What am I doing wrong here?
os.path.join returns a string, it's why you get a path in the content of the response. You need to read the file at that path first.
For a static file
If the file is static and on disk, you could just return it using the webserver and avoid using python and django at all. If the file needs authenticating to be downloaded, you could still handle that with django, and return a X-Sendfile header (this is dependant on the webserver).
Serving static files is a job for a webserver, Nginx and Apache are really good at this, while Python and Django are tools to handle application logic.
Simplest way to read a file
def serve(request):
path = os.path.join(BASE_DIR, 'static', 'files', 'apple-app-site-association')
with open(path , 'r') as myfile:
data=myfile.read()
response = HttpResponse(content=data)
response['Content-Type'] = 'application/json'
This is inspired by How do I read a text file into a string variable in Python
For a more advanced solution
See dhke's answer on StreamingHttpResponse.
Additional information
Reading and writing files
Managing files with Django
If you feed HttpResponse a string a content you tell it to serve that string as HTTP body:
content should be an iterator or a string. If it’s an iterator, it should return strings, and those strings will be joined together to form the content of the response. If it is not an iterator or a string, it will be converted to a string when accessed.
Since you seem to be using your static storage directory, you might as well use staticfiles to handle content:
from django.contrib.staticfiles.storage import staticfiles_storage
from django.http.response import StreamingHttpResponse
file_path = os.path.join('files', 'apple-app-site-association')
response = StreamingHttpResponse(content=staticfiles_storage.open(file_path))
return response
As noted in #Emile Bergeron's answer, for static files, this should already be overkill, since those are supposed to be accessible from outside, anyway. So a simple redirect to static(file_path) should do the trick, too (given your webserver is correctly configured).
To serve an arbitrary file:
from django.contrib.staticfiles.storage import staticfiles_storage
from django.http.response import StreamingHttpResponse
file_path = ...
response = StreamingHttpResponse(content=open(file_path, 'rb'))
return response
Note that from Django 1.10 and on, the file handle will be closed automatically.
Also, if the file is accessible from your webserver, consider using django-sendfile, so that the file's contents don't need to pass through Django at all.
Related
I'm using the following django/python code to stream a file to the browser:
wrapper = FileWrapper(file(path))
response = HttpResponse(wrapper, content_type='text/plain')
response['Content-Length'] = os.path.getsize(path)
return response
Is there a way to delete the file after the reponse is returned? Using a callback function or something?
I could just make a cron to delete all tmp files, but it would be neater if I could stream files and delete them as well from the same request.
You can use a NamedTemporaryFile:
from django.core.files.temp import NamedTemporaryFile
def send_file(request):
newfile = NamedTemporaryFile(suffix='.txt')
# save your data to newfile.name
wrapper = FileWrapper(newfile)
response = HttpResponse(wrapper, content_type=mime_type)
response['Content-Disposition'] = 'attachment; filename=%s' % os.path.basename(modelfile.name)
response['Content-Length'] = os.path.getsize(modelfile.name)
return response
temporary file should be deleted once the newfile object is evicted.
For future references:
I just had the case in which I couldn't use temp files for downloads.
But I still needed to delete them after it; so here is how I did it (I really didn't want to rely on cron jobs or celery or wossnames, its a very small system and I wanted it to stay that way).
def plug_cleaning_into_stream(stream, filename):
try:
closer = getattr(stream, 'close')
#define a new function that still uses the old one
def new_closer():
closer()
os.remove(filename)
#any cleaning you need added as well
#substitute it to the old close() function
setattr(stream, 'close', new_closer)
except:
raise
and then I just took the stream used for the response and plugged into it.
def send_file(request, filename):
with io.open(filename, 'rb') as ready_file:
plug_cleaning_into_stream(ready_file, filename)
response = HttpResponse(ready_file.read(), content_type='application/force-download')
# here all the rest of the heards settings
# ...
return response
I know this is quick and dirty but it works. I doubt it would be productive for a server with thousands of requests a second, but that's not my case here (max a few dozens a minute).
EDIT: Forgot to precise that I was dealing with very very big files that could not fit in memory during the download. So that is why I am using a BufferedReader (which is what is underneath io.open())
Mostly, we use periodic cron jobs for this.
Django already has one cron job to clean up lost sessions. And you're already running it, right?
See http://docs.djangoproject.com/en/dev/topics/http/sessions/#clearing-the-session-table
You want another command just like this one, in your application, that cleans up old files.
See this http://docs.djangoproject.com/en/dev/howto/custom-management-commands/
Also, you may not really be sending this file from Django. Sometimes you can get better performance by creating the file in a directory used by Apache and redirecting to a URL so the file can be served by Apache for you. Sometimes this is faster. It doesn't handle the cleanup any better, however.
One way would be to add a view to delete this file and call it from the client side using an asynchronous call (XMLHttpRequest). A variant of this would involve reporting back from the client on success so that the server can mark this file for deletion and have a periodic job clean it up.
This is just using the regular python approach (very simple example):
# something generates a file at filepath
from subprocess import Popen
# open file
with open(filepath, "rb") as fid:
filedata = fid.read()
# remove the file
p = Popen("rm %s" % filepath, shell=True)
# make response
response = HttpResponse(filedata, content-type="text/plain")
return response
Python 3.7 , Django 2.2.5
from tempfile import NamedTemporaryFile
from django.http import HttpResponse
with NamedTemporaryFile(suffix='.csv', mode='r+', encoding='utf8') as f:
f.write('\uFEFF') # BOM
f.write('sth you want')
# ref: https://docs.python.org/3/library/tempfile.html#examples
f.seek(0)
data=f.read()
response = HttpResponse(data, content_type="text/plain")
response['Content-Disposition'] = 'inline; filename=export.csv'
I have a page where users can upload PDF / image files to their profile. The model for these files is relativly straightforward:
class ResumeItemFile(models.Model):
item = models.ForeignKey(ResumeItem, related_name='attachment_files')
file = models.FileField(
max_length=255, upload_to=RandomizedFilePath('resume_attachments'),
verbose_name=_('Attachment'))
name = models.CharField(max_length=255, verbose_name=_('Naam'), blank=True)
I am creating a view where all files linked to a profile (item) are gathered in a .zip file. I've got this working locally, but in production I run in the following error NotImplementedError: This backend doesn't support absolute paths.
The main difference is that on production the mediafiles are served through S3
MEDIA_URL = 'https://******.s3.amazonaws.com/'
STATIC_URL = MEDIA_URL
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
In my view I created a list of the ResumeItemFile in the attachments variable, which is a list of dicts that look like this: {'filename', ResumeItemFileObject}
for file in attachments:
storage = DefaultStorage()
filename = file[1]
file_extension = str(file[0].file).split('.')[-1]
file_object = storage.open(file[0].file.path, mode='rb')
filename, file_object.read())
file_object.close()
Though this works fine locally, on staging it crashes on the file_object = storage.open(file[0].file.path, mode='rb') line.
If the backend does not support absolute paths, how I am to select the correct file? Does anyone have an idea of what I am doing wrong?
I think that problem comes because in the s3boto storage class, the path() method is not implemented. As per the Django documentation,
For storage systems that aren’t accessible from the local filesystem,
this will raise NotImplementedError instead.
Instead of file.path use file.name in your code.
# file_object = storage.open(file[0].file.path, mode='rb')
file_object = storage.open(file[0].file.name, mode='rb')
You may want to look into the File object. It allows you to manipulate files in a largely Pythonic manner, but leverages the Django project's storage settings. In my case, this allows me to use local, on-disk storage locally and S3 in production:
https://docs.djangoproject.com/en/2.0/ref/files/file/
This will abstract away a lot of the boilerplate you're writing. There is an example here:
https://docs.djangoproject.com/en/2.0/topics/files/#the-file-object
Good luck!
This might perhaps be a simple question, but I somehow just can not find the solution. Django offers a lot about uploading file, but how do I do to download a file.
Let's assume we have a button on HTML on uploads/something.txt as a file.
I tried with django.views.static.serve, however what this did it would open a file on webpage.
My question is simple: What is the best and most pythonic way for user of our website to download a file?
You need to read that file.
Serve it using HttpResponse along with proper content type.
Here's some sample code:
content = open("uploads/something.txt").read()
return HttpResponse(content, content_type='text/plain')
This should serve a text file.
But as you described, on some browser, it will not ask to download the file, rather, it would show it in the browser. If you want to show a download prompt, use this:
response = HttpResponse(open("uploads/something.txt", 'rb').read())
response['Content-Type'] = 'text/plain'
response['Content-Disposition'] = 'attachment; filename=DownloadedText.txt'
return response
However, please note that it might be a better idea to serve static contents or uploaded files via nginx or the reverse proxy of your choice. Sending large files through Django might not be the most optimum way of doing that.
import os
from django.conf import settings
from django.http import HttpResponse, Http404
def download(request, path):
file_path = os.path.join(settings.MEDIA_ROOT, path)
if os.path.exists(file_path):
with open(file_path, 'rb') as fh:
response = HttpResponse(fh.read(), content_type="application/vnd.ms-excel")
response['Content-Disposition'] = 'inline; filename=' + os.path.basename(file_path)
return response
raise Http404
Maybe a little late but here is my solution:
def render(self, value):
return format_html('<a href="/media/{0}" download>{0}</a>', value)
my users can upload an image of themselves and use that as an avatar. Now I am struggling how to retrieve a default fallback image if they haven't uploaded an image themselves.
The path to the avatar is "//mysite.com/avatar/username".
So far I have this code, which works fine when the user has uploaded an avatar themselves, but it gives me the following error when I try to retrieve the default image:
raise IOError(errno.EACCES, 'file not accessible', filename)
IOError: [Errno 13] file not accessible: '/Users/myuser/Documents/github/mysite/static/images/profile.png'
def get(self):
path = self.request.path.split('/')
action = self.get_action(path)
if action:
e = employees.filter('username = ', action).get()
if e.avatar:
self.response.headers['Content-Type'] = "image/png"
self.response.out.write(e.avatar)
else:
self.response.headers['Content-Type'] = 'image/png'
path = os.path.join(os.path.split(__file__)[0], 'static/images/profile.png')
with open(path, 'r') as f:
print self.response.out.write(f.read())
I have defined the "/static"-folder as a static_dir in my app.yaml.
I know I can place the profile.png in the root-folder, but I prefer to have it in the "/static/images"-folder.
Any ideas?
If you declared the file itself as a static_file or its directory or any directory in its filepath a static_dir inside your app/service's .yaml config file then, by default, it's not accessible to the application code.
You need to also configure it as application_readable. From Handlers element:
application_readable
Optional. Boolean. By default, files declared in static file handlers
are uploaded as static data and are only served to end users. They
cannot be read by an application. If this field is set to true, the
files are also uploaded as code data so your application can read
them. Both uploads are charged against your code and static data
storage resource quotas.
I need to get the content of the resources received in command line. The user can write a relative path to a file or an URL. Is it possible to read from this resource regardless if it is a path to a file or an URL?
In Ruby I have something like the next, but I'm having problems finding a Python alternative:
content = open(path_or_url) { |io| io.read }
I don't know of a nice way to do it, however, urllib.request.urlopen() will support opening normal URLs (http, https, ftp, etc) as well as files on the file system. So you could assume a file if the URL is missing a scheme component:
from urllib.parse import urlparse
from urllib.request import urlopen
resource = input('Enter a URL or relative file path: ')
if urlparse(resource).scheme == '':
# assume that it is a file, use "file:" scheme
resource = 'file:{}'.format(resource)
data = urlopen(resource).read()
This works for the following user input:
http://www.blah.com
file:///tmp/x/blah
file:/tmp/x/blah
file:x/blah # assuming cwd is /tmp
/tmp/x/blah
x/blah # assuming cwd is /tmp
Note that file: (without slashes) might not be a valid URI, however, this is the only way to open a file specified by relative path, and urlopen() works with such URIs.