Python convert zip file to bytes stream - python

I have a zip file that when I open it locally it looks great. I want to convert it to a bytes stream buffer and then return it as HttpResponse(buffer) using django. The code is,
studies_zip = zipfile.ZipFile('./studies.zip', 'r')
buffer = io.BytesIO()
bytes = [zipfile.Path(studies_zip, at=file.filename).read_bytes()
for file in studies_zip.infolist()]
buffer = io.BytesIO()
buffer_writer = io.BufferedWriter(buffer)
[buffer_writer.write(b) for b in bytes]
buffer.seek(0)
response = HttpResponse(buffer)
response['Content-Type'] = 'application/zip'
response['Content-Disposition'] = 'attachment;filename=studies.zip'
At the front-end/UI I get this,
that looks fine i.e. the displayed size of 34.9MB is a little bit less than the actual 36.6MB. Also, when I try to open the file either on the spot or after saving it locally, I get
What's wrong?

You are sending the contents of the compressed files, omitting the metadata contained in the zip archive.
There is no reason to open the file as a zip file as no changes are made to the contents, so just open the file in byes mode and send it. I haven't tested this, but try this:
with open('./studies.zip', 'rb') as f:
response = HttpResponse(f)
response['Content-Type'] = 'application/zip'
response['Content-Disposition'] = 'attachment;filename=studies.zip'

Related

How to save image which sent via flask send_file

I have this code for server
#app.route('/get', methods=['GET'])
def get():
return send_file("token.jpg", attachment_filename=("token.jpg"), mimetype='image/jpg')
and this code for getting response
r = requests.get(url + '/get')
And i need to save file from response to hard drive. But i cant use r.files. What i need to do in these situation?
Assuming the get request is valid. You can use use Python's built in function open, to open a file in binary mode and write the returned content to disk. Example below.
file_content = requests.get('http://yoururl/get')
save_file = open("sample_image.png", "wb")
save_file.write(file_content.content)
save_file.close()
As you can see, to write the image to disk, we use open, and write the returned content to 'sample_image.png'. Since your server-side code seems to be returning only one file, the example above should work for you.
You can set the stream parameter and extract the filename from the HTTP headers. Then the raw data from the undecoded body can be read and saved chunk by chunk.
import os
import re
import requests
resp = requests.get('http://127.0.0.1:5000/get', stream=True)
name = re.findall('filename=(.+)', resp.headers['Content-Disposition'])[0]
dest = os.path.join(os.path.expanduser('~'), name)
with open(dest, 'wb') as fp:
while True:
chunk = resp.raw.read(1024)
if not chunk: break
fp.write(chunk)

Passing Binary file over HTTP POST

I have a local python file that decodes binary files. This python file first reads from the file, opens it as binary and then saves it in a buffer and interprets it. Reading it is simply:
with open(filepath, 'rb') as f:
buff = f.read()
read_all(buff)
This works fine locally. Now I'd like to setup a Azure Python job where I can send the file, approx. 100kb, over a HTTP POST and then read the interpreted meta data which my original python script does well.
I've first removed the read function so that I'll now work with the buffer only.
In my Azure Python Job I have the following, triggered by a HttpRequest
my_data = reader.read_file(req.get_body())
To test my sending I've tried the following in python
import requests
url = 'http://localhost:7071/api/HttpTrigger'
files = {'file': open('test', 'rb')}
with open('test', 'rb') as f:
buff = f.read()
r = requests.post(url, files=files) #Try using files
r = requests.post(url, data=buff) #Try using data
I've also tried in Postman adding the file to the body as a binary and setting the headers to application/octet-stream
All this doesn't send the binary file the same way as the original f.read() did. So I'm getting a wrong interpretation of the binary file.
What is file.read doing differently to how I'm sending it over as a HTTP Body message?
Printing out the first line from the local python read file gives.
b'\n\n\xfe\xfe\x00\x00\x00\x00\\\x18,A\x18\x00\x00\x00(\x00\x00\x00\x1f\x00\x00\
Whereas printing it out at the req.get_body() shows me
b'\n\n\xef\xbf\xbd\xef\xbf\xbd\x00\x00\x00\x00\\\x18,A\x18\x00\x00\x00(\x00\x00\x00\x1f\x00\
So something is clearly wrong. Any help why this could be different?
Thanks
EDIT:
I've implemented a similar function in Flask and it works well.
The code in flask is simply grabbing the file from a POST. No encoding/decoding.
if request.method == 'POST':
f = request.files['file']
#f.save(secure_filename(f.filename))
my_data = reader.read_file(f.read())
Why is the Azure Function different?
You can try UTF-16 to decode and do the further action in your code.
Here is the code for that:
with open(path_to_file,'rb') as f:
contents = f.read()
contents = contents.rstrip("\n").decode("utf-16")
Basically after doing re.get_body, perform the below operation:
contents = contents.rstrip("\n").decode("utf-16")
See if it gives you the same output as your receive in local python file.
Hope it helps.

Can't Render base64 file using FileResponse or StreamingHttpResponse, works using HttpResponse

I am saving my file in base64 in database and then trying to render it using Django view.
file_obj = AttachmentData.objects.filter(id=file_id)
file_data = base64.b64decode(file_obj.attachment_file)
bytes_out = BytesIO()
bytes_out.write(file_obj)
response = HttpResponse(string_out.getvalue(),content_type=email_file_obj.mime_type)
response["Content-Disposition"] = "attachment; filename={}".format(name)
return response
This code works fine, but When file is large then i need to use StreamingHttpResponse or FileResponse. The problem is these responses need file object as parameter. To create file i need to save it which i don't want as it will consume up my disk space.
temp_file_obj = open(file_obj.file_name, 'wb')
temp_file_obj.write(file_data)
temp_file_obj.close()
response = FileResponse(open(file_obj.file_name, 'rb'), content_type=email_file_obj.mime_type)
I need a solution so that i send file without saving file on disk.
You do not need to pass a true file object to a StreamingHttpResponse, you can yield the data itself as the parameter. The documentation provides this example, where they show passing a large CSV file as a response. You can do something similar with the data you have stored as well.
You can just send data like that when it's base64 :
file_obj = AttachmentData.objects.filter(id=file_id)
file_data = base64.b64decode(file_obj.attachment_file)
return file_data

Fetch multiple images from the server using Django

I am trying to download multiple image files from the server. I am using Django for my backend.
Question related to single image has already been answered and I tried the code and it works on single image. In my application, I want to download multiple images in a single HTTP connection.
from PIL import Image
img = Image.open('test.jpg')
img2 = Image.open('test2.png')
response = HttpResponse(content_type = 'image/jpeg')
response2 = HttpResponse(content_type = 'image/png')
img.save(response, 'JPEG')
img2.save(response2, 'PNG')
return response #SINGLE
How can I fetch both img and img2 at once. One way I was thinking is to zip both images and unzip it on client size but I dont think that is good solution. Is there a way to handle this?
I looked around and find an older solution using a temporary Zip file on disk: https://djangosnippets.org/snippets/365/
It needed some updating, and this should work (tested on django 2.0)
import tempfile, zipfile
from django.http import HttpResponse
from wsgiref.util import FileWrapper
def send_zipfile(request):
"""
Create a ZIP file on disk and transmit it in chunks of 8KB,
without loading the whole file into memory. A similar approach can
be used for large dynamic PDF files.
"""
temp = tempfile.TemporaryFile()
archive = zipfile.ZipFile(temp, 'w', zipfile.ZIP_DEFLATED)
for index in range(10):
filename = 'C:/Users/alex1/Desktop/temp.png' # Replace by your files here.
archive.write(filename, 'file%d.png' % index) # 'file%d.png' will be the
# name of the file in the
# zip
archive.close()
temp.seek(0)
wrapper = FileWrapper(temp)
response = HttpResponse(wrapper, content_type='application/zip')
response['Content-Disposition'] = 'attachment; filename=test.zip'
return response
Right now, this takes my .png and writes it 10 times in my .zip, then sends it.
You could add your files/images to a ZIP file and return that one in the response. I think that is the best approach.
Here is some example code of how you could achieve that (from this post):
def zipFiles(files):
outfile = StringIO() # io.BytesIO() for python 3
with zipfile.ZipFile(outfile, 'w') as zf:
for n, f in enumarate(files):
zf.writestr("{}.csv".format(n), f.getvalue())
return outfile.getvalue()
zipped_file = zip_files(myfiles)
response = HttpResponse(zipped_file, content_type='application/octet-stream')
response['Content-Disposition'] = 'attachment; filename=my_file.zip'
Otherwise (if you don't like ZIP files) you could make individual requests from the client.

How to download a file uploaded using django-filebrowser?

I am trying to create a download of a file object. the file was added using django-filebrowser which means it is turn in to a string path to the the file. I have tried the following:
f = Obj.objects.get(id=obj_id)
myfile = FileObject(os.path.join(MEDIA_ROOT, f.Audio.path))
...
response = HttpResponse(myfile, content_type="audio/mpeg")
response['Content-Disposition'] = 'attachment; filename=myfile.mp3'
return response
The file that is downloaded contains the string of the path to the file location and not the file. Could anyone be of assistance on how to access the file object?
f = Obj.objects.get(id=obj_id)
myfile = open(os.path.join(MEDIA_ROOT, f.Audio.path)).read()
...
response = HttpResponse(myfile, content_type="audio/mpeg")
response['Content-Disposition'] = 'attachment; filename=myfile.mp3'
return response
NOTE! This is not memory friendly! Since the whole file is put into memory. You're better of using a webserver for file serving or if you want to use Django for file serving you could use xsendfile or have a look at this thread
You need to open the file and send it's binary contents back in the response. So something like:
fileObject = FileObject(os.path.join(MEDIA_ROOT, f.Audio.path))
myfile = open(fileObject.path)
response = HttpResponse(myfile.read(), mimetype="audio/mpeg")
response['Content-Disposition'] = 'attachment; filename=myfile.mp3'
return response
Hope that gets what you're looking for.

Categories