I have included an excel file in my project directory. I want to created a Django view that allows a user to download that file. Please how best do I handle that?
import csv
from django.http import StreamingHttpResponse
# create an echo handler, returns what is put into for the writer
psuedo_buffer = Echo()
#Build csv writer ontop of echo filelike instance
writer = csv.writer(psuedo_buffer)
#Stream the response row by row using the psuedo_writer
response = StreamingHttpResponse((
writer.writerow(row) for row in query_data),
content_type="text/csv"
)
response['Content-Disposition'] = 'attachment; filename="Something.csv"'
return response
This is a code snippet that I use to return a streaming HTTP response with the data. The data that would be in query_data can either be raw CSV data from a file handler which you can pretty easily find a few ways to open the data and drop it into this function, or you can use arrays of data from query sets to pass in. Just format your data for query_set and return this Response handler in either API views or Template views. Hope this helps!
Data should be formatted in arrays of data, which you can use .dict to get parsable data from most models or simply parsing the csv into memory with the CSV library will accomplish the same thing.
Is your file associated with a Model? I prefer to create a Model to store general resources.
models.py
class Resources(models.Model):
description = models.CharField(max_length=200)
file = models.FileField(upload_to='uploads/resources/')
def __str__(self):
return self.description
views.py
...
some_file = Resources.objects.get(description='Your description')
...
return render(request, "template.html", {
"some_file": some_file,
}
template.html
...
<p>Download file.</p>
...
Related
What I'm trying to implement is this:
User sends query parameters from React FE microservice to the Django BE microservice.
URI is something like /api/reports?startingPage=12&dataView=Region
These PDFs are way too big to be generated in FE, so doing it server side
Request makes its way into the view.py where the data related to dataView=Region is queried from the database, each row is iterated through and a PDF report is generated for each item
Each dataView=Region can consist of a few hundred items and each of those items is its own report that can be a page long or several pages long
As the reports are generated, they should be saved to the server persistent volume claim and not be sent back to FE until they have all run.
When they have all run, I plan to use pypdf2 to combine all of the PDFs into one large file.
At that point, the file is sent back to the FE to download.
I'm only working on 1. and 3. at this point and I'm unable to:
Get the files to save to storage
Prevent the default behavior of the PDF being sent back to the FE after it has been generated
The PDFs are being generated, so that is good.
I'm trying to implement the suggestions as found here, but I'm not getting the desired results:
Save pdf from django-wkhtmltopdf to server (instead of returning as a response)
This is what I currently have on the Django side:
# urls.py
from django.urls import path
from .views import GeneratePDFView
app_name = 'Reports'
urlpatterns = [
path('/api/reports',
GeneratePDFView.as_view(), name='generate_pdf'),
]
# views.py
from django.conf import settings
from django.views.generic.base import TemplateView
from rest_framework.permissions import IsAuthenticated
from wkhtmltopdf.views import PDFTemplateResponse
# Create your views here.
class GeneratePDFView(TemplateView):
permission_classes = [IsAuthenticated]
template_name = 'test.html'
filename = 'test.pdf'
def generate_pdf(self, request, **kwargs):
context = {'key': 'value'}
# generate response
response = PDFTemplateResponse(
request=self.request,
template=self.template_name,
filename=self.filename,
context=context,
cmd_options={'load-error-handling': 'ignore'})
self.save_pdf(response.rendered_content, self.filename)
# Handle saving the document
# This is what I'm using elsewhere where files are saved and it works there
def save_pdf(self, file, filename):
with open(settings.PDF_DIR + '/' + filename, 'wb+') as destination:
for chunk in file.chunks():
destination.write(chunk)
# settings.py
...
DOWNLOAD_ROOT = '/mnt/files/client-downloads/'
MEDIA_ROOT = '/mnt/files/client-submissions/'
PDF_DIR = '/mnt/files/pdf-sections/'
...
I should note the other DOWNLOAD_ROOT and MEDIA_ROOT are working fine where the app uses them. I've even tried using settings.MEDIA_ROOT because I know it works, but still nothing is saved there. But as you can see, I'm starting out super basic and haven't added a query, loops, etc.
My save_pdf() is different than the SO question I linked to because that is what I'm using in other parts of my application and it is saving files fine there. I did try what they provided in the SO question, but had the same results with it not saving. That being:
with open("file.pdf", "wb") as f:
f.write(response.rendered_content)
So what do I need to do to get these PDFs to save to disk?
Perhaps I need to be using a different library for my needs as django-wkhtmltopdf seems to do a number of things out of the box that I don't want that I'm not clear I can override.
OK, my smooth brain gained a few ripples overnight and figured it out this morning:
# views.py
class GeneratePDFView(TemplateView):
permission_classes = [IsAuthenticated]
def get(self, request, *args, **kwargs):
template_name = 'test.html'
filename = 'test.pdf'
context = {'key': 'value'}
# generate response
response = PDFTemplateResponse(
request=request,
template=template_name,
filename=filename,
context=context,
cmd_options={'load-error-handling': 'ignore'})
# write the rendered content to a file
with open(settings.PDF_DIR + '/' + filename, "wb") as f:
f.write(response.rendered_content)
return HttpResponse('Hello, World!')
This saved the PDF to disk and also did not respond with the PDF. Obviously a minimally functioning example that I can expand on, but at least got those two issues figured out.
I have a search function that returns json via ajax, but I also want to export the data to a csv file.
It occurred to me to do another function to export the data, which brings the search function
I'm doing it like this:
def search(request):
date = request.GET.get('date')
queryset = List.objects.filter(date=date)
data = serializers.serialize('json', queryset)
return HttpResponse(data, content_type='application/json')
def export_to_csv(request):
data = search(request)
# But that does not bring the search data
print(data)
# <HttpResponse status_code=200, "application/json">
I hope you understand my question, some ideas or suggestions?
Maybe you can try with simply extracting the logic which retrieves and serializes the data into a helper function, which can be a part of your views.py (or, probably a better approach, moved to a helper/utility module) e.g.:
def get_search_data(date=None):
queryset = List.objects.all()
if date:
queryset = queryset.filter(date=date)
return serializers.serialize('json', queryset)
def search(request):
data = get_search_data(request.GET.get('date'))
return HttpResponse(data, content_type='application/json')
def export_to_csv(request):
data = get_search_data()
...
print(data)
# <HttpResponse status_code=200, "application/json">
i have "problems" validating external xml. What i want is to fetch data from an external rss site, validate if all fields are present and save it to the database then.
The problem i am having is that i am not sure how can i send the xml data to my validator. I tried in different ways but it was not working.
https://api.foxsports.com/v1/rss?partnerKey=zBaFxRyGKCfxBagJG9b8pqLyndmvo7UU&tag=nba
This is an example of the rss i am trying to parse.
Here is my code:
import json
import requests
import xml.etree.ElementTree as ET
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status, generics
class Test(APIView):
def get(self, request, format=None):
response = requests.get(
channel.url
)
print(response.status_code)
if response.status_code == 200:
xml = response.text.encode("utf-8")
tree = ET.fromstring(xml)
for child in tree.iter("item"):
serializer = RssSerializer(data=child)
if serializer.is_valid():
serializer.save(parsed_xml)
The problem here is that my serializer is always not valid, no matter what i do. I kind of got around this problem when i wrote a small helper function that is manually extracting fields from the request.
It looks like this:
def parse_xml(self, node):
parsed_json = {
"title": node.find("title").text,
"description": node.find("description").text,
"link": node.find("link").text,
"pub_date": node.find("pubDate").text,
"guid": node.find("guid").text,
}
return parsed_json
Basically i just added this line parsed_xml = self.parse_xml(child)
and i am sending the parsed_xml to my serializer. That works fine but it seems like a hackhish way to me, but i am not able to process the data in any other way.
class RssSerializer(serializers.Serializer):
title = serializers.CharField(min_length=2)
link = serializers.CharField(min_length=2)
description = serializers.CharField(min_length=2)
pub_date = serializers.CharField(min_length=2)
guid = serializers.CharField(min_length=2)
def save(self, data):
new_feed = RssFeed()
new_feed.title = data["title"]
new_feed.description = data["description"]
new_feed.pub_date = data["pub_date"]
new_feed.link = data["link"]
new_feed.guid = data["guid"]
new_feed.save()
What i want to know is there any way i can fetch the xml from an external source and directly pass it to my validator? Thanks in advance for your help
Serializers do not expect xml nodes as data, so the short answer is no, unfortunately there is no direct way to pass it to validator. However, you could reuse one of the existing methods to either change whole xml to dictionary, and then feed the items to the serializer, or to change single nodes to dictionaries.
For inspiration, I would look into available solution (no need to reinvent the wheel):
This problem was already touched How to convert an xml string to a dictionary in Python?
I used once this DRF XML parser to parse XML requests, and it did the job: https://github.com/jpadilla/django-rest-framework-xml/blob/master/rest_framework_xml/parsers.py#L40
and if you don't mind additional library, xmltodict did the job for me with pretty similiar problem as well https://docs.python-guide.org/scenarios/xml/
E.g. your code could look like this:
xml = response.text.encode("utf-8")
xml_dict = xmltodict.parse(xml)
for item in xml_dict["rss"]["channel"]["item"]:
serializer = RssSerializer(data=item)
I am creating a webservice with django using django rest framework.
Users are able to upload some images and videos. Uploading media is a two step action, first user uploads the file and receives an ID then in a separate request uses that ID to refer to the media (for example (s)he can use it as profile picture or use it in a chat message).
I need to know who is uploading the media for both HMAC authentication middleware and setting owner of media in database. All other requests are in JSON format and include a username field that it used by HMAC middleware to retrieve the secret shared key.
It first came to my mind that media upload api may look like this:
{
"username":"mjafar",
"datetime":"2015-05-08 19:05",
"media_type":"photo",
"media_data": /* base64 encoded image file */
}
But i thought that base64 encoding may have significant overhead for larger files like videos; or there may be some restrictions on size of data that can be parsed in json or be created in user side. (This webservice is supposed to communicate with a Android/iOS app, they have limited memory)! Is this a good solution? Are my concerns real problems or i shouldn't worry? Better solutions?
You could separate the two. Meta data at one interface with a URL pointing to the actual file. Depending on how you store the actual file you could then reference the file directly via URL at a later point.
You could then have the POST API directly accept the file and simply return the JSON meta data
{
"username":"mjafar", // inferred from self.request.user
"datetime":"2015-05-08 19:05", // timestamp on server
"media_type":"photo", // inferred from header content-type?
// auto-generated hashed location for file
"url": "/files/1dde/2ecf/4075/f61b/5a9c/1cec/53e0/ca9b/4b58/c153/09da/f4c1/9e09/4126/271f/fb4e/foo.jpg"
}
Creating such an interface using DRF would be more along the lines of implementing rest_framework.views.APIView
Here's what I'm doing for one of my sites:
class UploadedFile(models.Model):
creator = models.ForeignKey(auth_models.User,blank=True)
creation_datetime = models.DateTimeField(blank=True,null=True)
title = models.CharField(max_length=100)
file = models.FileField(max_length=200, upload_to=FileSubpath)
sha256 = models.CharField(max_length=64,db_index=True)
def save(self,*args,**kw_args):
if not self.creation_datetime:
self.creation_datetime = UTC_Now()
super(UploadedFile,self).save(*args,**kw_args)
serializer:
class UploadedFileSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = UploadedFile
fields = ('url', 'creator','creation_datetime','title','file')
And the view to use this:
from rest_framework.views import APIView
from qc_srvr import serializers,models
from rest_framework.response import Response
from rest_framework import status
from rest_framework import parsers
from rest_framework import renderers
import django.contrib.auth.models as auth_models
import hashlib
class UploadFile(APIView):
'''A page for uploading files.'''
throttle_classes = ()
permission_classes = ()
parser_classes = (parsers.FormParser, parsers.JSONParser,)
renderer_classes = (renderers.JSONRenderer,)
serializer_class = serializers.UploadedFileSerializer
def calc_sha256(self,afile):
hasher = hashlib.sha256()
blocksize=65536
hasher.update('af1f9847d67300b996edce88889e358ab81f658ff71d2a2e60046c2976eeebdb') # salt
buf = afile.read(blocksize)
while len(buf) > 0:
hasher.update(buf)
buf = afile.read(blocksize)
return hasher.hexdigest()
def post(self, request):
if not request.user.is_authenticated():
return Response('User is not authenticated.', status=status.HTTP_401_UNAUTHORIZED)
uploaded_file = request.FILES.get('file',None)
if not uploaded_file:
return Response('No upload file was specified.', status=status.HTTP_400_BAD_REQUEST)
# calculate sha
sha256 = self.calc_sha256(uploaded_file)
# does the file already exist?
existing_files = models.UploadedFile.objects.filter(sha256=sha256)
if len(existing_files):
serializer = self.serializer_class(instance=existing_files[0],context={'request':request})
else:
instance = models.UploadedFile.objects.create(
creator = request.user,
title= uploaded_file.name,
file = uploaded_file,
sha256 = sha256)
serializer = self.serializer_class(instance=instance,context={'request':request})
#import rpdb2; rpdb2.start_embedded_debugger('foo')
#serializer.is_valid()
return Response(serializer.data)
FYI, this is a bit of security-through-obscurity since all the uploaded files are retrievable if you have the URL to the file.
I'm still using DRF 2.4.4, so this may not work for you on 3+. I haven't upgraded due to the dropped nested-serializers support.
dear colleagues.
In my project I have successfully saved data(image) in mongodb using mongoengine. I have problem with getting image from mongodb and displaying it on the client. I'm not using local storage, so this is causing the problem with loading the page (the page will not be shown until photo_view function has been loaded the file and stored to temp file.
This is a short cut of my saved object in mongodb
{
"_id" : ObjectId("53c123edcb596046fdf6c746"),
"main_photo" : ObjectId("53c123e1cb596046fdf6c6fc"),
"create_date" : ISODate("2014-07-12T12:02:41.036Z")
}
The actual file is saved in db.fs.files using GridFS.
This is the models.py file
class Photo(Document):
def __unicode__(self):
return self.create_date
main_photo = FileField()
create_date = DateTimeField(default=datetime.datetime.now)
The problem is in photo_view function
#login_required
def photo_view(request, id, template_name):
params = {}
obj = Photo.objects.get(id = id)
print obj.main_photo.read()
params['object'] = obj
return render(request, template_name, params)
The obj.main_photo.read() is an actual file. How can I achieve in loading page without storing data in temporary file on my server. Even if I will load the data, the function photo_view will not finish until it is loaded/copied from GridFS .