Right now I have a flask app in which part of the functionality allows me to select a date range and see data from a sql database from that selected date range. I then can click a button and it exports this to a csv file which is just saved in the flask project directory. I want the user to be able to download this csv file. I want to know what the best practice for a user to download a dynamic csv file. Should I send_file() and then delete the file after user has downloaded since this data shouldn't be saved and the user won't be using that file again. Should the file be saved to the database and then deleted out of the db? Or can I just keep it within the flask directory? Please provide insight if possible, thank you so much.
#brunns pointed it in very right direction.
You don't have to save the file in your database or in your file structure or anywhere. It will get created in memory on user request.
I've done this with django for pdf and for csv files it'll work in the same way with flask too. Basics are all same.
for python3 use io.StringIO, for python2 use StringIO
from io import StringIO
import csv
from flask import make_response
#app.route('/download')
def post(self):
si = StringIO.StringIO()
cw = csv.writer(si)
cw.writerows(csvList)
output = make_response(si.getvalue())
output.headers["Content-Disposition"] = "attachment; filename=export.csv"
output.headers["Content-type"] = "text/csv"
return output
Courtesy: vectorfrog
Based on #xxbinxx's answer, used with pandas
from io import StringIO
import csv
from flask import make_response
#app.route('/download')
def download_csv(self, df: pd.DataFrame):
si = StringIO()
cw = csv.writer(si)
cw.writerows(df.columns.tolist())
cw.writerows(df.values.tolist())
output = make_response(si.getvalue())
output.headers["Content-Disposition"] = "attachment; filename=export.csv"
output.headers["Content-type"] = "text/csv"
return output
Related
I am trying to provide the client side the option of downloading some files in Flask. There can be multiple files or a single file available for the user/client to download.
However I am not able to understand how to provide the user the option to download multiple files.
Here is what I have tried so far:
#app.route('/download_files')
def download():
count=0
download_list=[]
for path in pathlib.Path("dir1/dir2").iterdir():
if path.is_file():
for i in names:
if pathlib.PurePosixPath(path).stem == i:
count += 1
download_list.append(path)
return send_file(download_list, as_attachment=True, mimetype="text/plain", download_name="Downloaded Files", attachment_filename="Generated Files")
This does not work properly even with a single file. The file type I am trying to download is text file with the extension .sql .
Will I somehow have to zip multiple files and then provide the download option? Please guide with my available options.
In order to offer several files together as a download, you only have the option of compressing them in an archive.
In my example, all files that match the specified pattern are listed and compressed in a zip archive. This is written to the memory and sent by the server.
from flask import Flask
from flask import send_file
from glob import glob
from io import BytesIO
from zipfile import ZipFile
import os
app = Flask(__name__)
#app.route('/download')
def download():
target = 'dir1/dir2'
stream = BytesIO()
with ZipFile(stream, 'w') as zf:
for file in glob(os.path.join(target, '*.sql')):
zf.write(file, os.path.basename(file))
stream.seek(0)
return send_file(
stream,
as_attachment=True,
download_name='archive.zip'
)
You haven't provided a code sample where you actually getting these files or this file. Minimum working example would be like this:
from flask import Flask, request
app = Flask(__name__)
#app.route('/download_files', methods=['POST'])
def download():
file = request.files['file'] # for one file
files = request.files.getlist("file[]") # if there're multiple files provided
if __name__ == "__main__":
app.run()
After what your file variable will be an object of werkzeug.FileStorage and files variable will be a list of these objects.
And to download all these files you can check this question.
I am new to Python development.
I am developing a flask API in Python that will help to download one excel file in .xlsx format.
my code is generating the file in .xlsx format but when I downloading the report I am getting the error: "File format or file extensions are not valid. Verify the file has not been corrupted".
Please help me on this.
import io
import pandas as pd
from flask import send_file
def get_data():
buf = io.BytesIO()
with pd.ExcelWriter(buf, date_format='dd/mm/yyyy', datetime_format='dd/mm/yyyy') as test:
dtl_ext = detail.to_excel(test, index=False,encoding='utf-16')
detail_rec.save()
excel_data = buf.getvalue()
buf.seek(0)
return send_file(
buf,
mimetype='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
attachment_filename='test11.xlsx',
as_attachment=True,
cache_timeout=0
)
You can use the following libraries to make it easier:
http://flask.pyexcel.org/en/latest/
https://github.com/pyexcel/pyexcel-xls
https://github.com/pyexcel/pyexcel-xlsx
After you have installed them (check the links for that). Import them in Python in the following manner:
import flask_excel as excel
import pyexcel.ext.xls
import pyexcel.ext.xlsx
Then:
#app/route('/download', methods=['GET'])
def download_data():
sample_data=[0, 1, 2]
excel.init_excel(app)
extension_type = "xlsx"
filename = "test123" + "." extension_type
d = {'colName': sample_data}
return excel.make_response_from_dict(d, file_type=extension_type, file_name=filename)
# check the flask-excel library from different export option.
# This one uses a dictionary of lists.
# You can do a lists of dictionary, simply an array, so on and so forth
Is it possible to define a route in bottle which would return a file?
I have a mongo database which is accessed by pandas.
Pandas generates a xls file based on a request parameters.
Two steps above are clear and easy to implement.
The third step is the one I have a problem with.
Define a bottle route which would return a file to download by user.
I don't want to use static previously generated files.
Thanks in advance.
I'm not familiar with Pandas but you need to get binary contents of a xls file to send to a user via a Bottle route. Modified example from here for Python 3:
from io import BytesIO
from bottle import route, response
from pandas import ExcelWriter
#route('/get-xlsx')
def get_xlsx():
output = BytesIO()
writer = ExcelWriter(output, engine='xlsxwriter')
# Do something with your Pandas data
# ...
pandas_dataframe.to_excel(writer, sheet_name='Sheet1')
writer.save()
response.contet_type = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
response.add_header('Content-Disposition', 'attachment; filename="report.xlsx"')
return output.getvalue()
When a user click a link that corresponds to this route, a file download dialog for "report.xlxs" will open in their browser.
Python: 2.7.11
Django: 1.9
Pandas: 0.17.1
How should I go about creating a potentially large xlsx file download? I'm creating a xlsx file with pandas from a list of dictionaries and now need to give the user possibility to download it. The list is in a variable and is not allowed to be saved locally (on server).
Example:
df = pandas.DataFrame(self.csvdict)
writer = pandas.ExcelWriter('pandas_simple.xlsx', engine='xlsxwriter')
df.to_excel(writer, sheet_name='Sheet1')
writer.save()
This example would just create the file and save it where the executing script is located. What I need is to create it to a http response so that the user would get a download prompt.
I have found a few posts about doing this for a xlsxwriter but non for pandas. I also think that I should be using 'StreamingHttpResponse' for this and not a 'HttpResponse'.
I will elaborate on what #jmcnamara wrote. This if for the latest versions of Excel, Pandas and Django. The import statements would be at the top of your views.py and the remaining code could be in a view:
import pandas as pd
from django.http import HttpResponse
try:
from io import BytesIO as IO # for modern python
except ImportError:
from io import StringIO as IO # for legacy python
# this is my output data a list of lists
output = some_function()
df_output = pd.DataFrame(output)
# my "Excel" file, which is an in-memory output file (buffer)
# for the new workbook
excel_file = IO()
xlwriter = pd.ExcelWriter(excel_file, engine='xlsxwriter')
df_output.to_excel(xlwriter, 'sheetname')
xlwriter.save()
xlwriter.close()
# important step, rewind the buffer or when it is read() you'll get nothing
# but an error message when you try to open your zero length file in Excel
excel_file.seek(0)
# set the mime type so that the browser knows what to do with the file
response = HttpResponse(excel_file.read(), content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
# set the file name in the Content-Disposition header
response['Content-Disposition'] = 'attachment; filename=myfile.xlsx'
return response
Jmcnamara is pointing you in the rigth direction. Translated to your question you are looking for the following code:
sio = StringIO()
PandasDataFrame = pandas.DataFrame(self.csvdict)
PandasWriter = pandas.ExcelWriter(sio, engine='xlsxwriter')
PandasDataFrame.to_excel(PandasWriter, sheet_name=sheetname)
PandasWriter.save()
sio.seek(0)
workbook = sio.getvalue()
response = StreamingHttpResponse(workbook, content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
response['Content-Disposition'] = 'attachment; filename=%s' % filename
Notice the fact that you are saving the data to the StringIO variable and not to a file location. This way you prevent the file being saved before you generate the response.
Maybe a bit off-topic, but it's worth pointing out that the to_csv method is generally faster than to_excel, since excel contains format information of the sheets. If you only have data and not formatting information, consider to_csv. Microsoft Excel can view and edit csv files with no problem.
One gain by using to_csv is that to_csv function can take any file-like object as the first argument, not only a filename string. Since Django response object is file-like, to_csv function can directly write to it. Some codes in your view function will look like:
df = <your dataframe to be downloaded>
response = HttpResponse(content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename=<default filename you wanted to give to the downloaded file>'
df.to_csv(response, index=False)
return response
Reference:
https://gist.github.com/jonperron/733c3ead188f72f0a8a6f39e3d89295d
https://docs.djangoproject.com/en/2.1/howto/outputting-csv/
With Pandas 0.17+ you can use a StringIO/BytesIO object as a filehandle to pd.ExcelWriter. For example:
import pandas as pd
import StringIO
output = StringIO.StringIO()
# Use the StringIO object as the filehandle.
writer = pd.ExcelWriter(output, engine='xlsxwriter')
# Write the data frame to the StringIO object.
pd.DataFrame().to_excel(writer, sheet_name='Sheet1')
writer.save()
xlsx_data = output.getvalue()
print len(xlsx_data)
After that follow the XlsxWriter Python 2/3 HTTP examples.
For older versions of Pandas you can use this workaround.
Just wanted to share a class-based view approach to this, using elements from the answers above. Just override the get method of a Django View. My model has a JSON field which contains the results of dumping a dataframe to JSON with the to_json method.
Python version is 3.6 with Django 1.11.
# models.py
from django.db import models
from django.contrib.postgres.fields import JSONField
class myModel(models.Model):
json_field = JSONField(verbose_name="JSON data")
# views.py
import pandas as pd
from io import BytesIO as IO
from django.http import HttpResponse
from django.views import View
from .models import myModel
class ExcelFileDownloadView(View):
"""
Allows the user to download records in an Excel file
"""
def get(self, request, *args, **kwargs):
obj = myModel.objects.get(pk=self.kwargs['pk'])
excel_file = IO()
xlwriter = pd.ExcelWriter(excel_file, engine='xlsxwriter')
pd.read_json(obj.json_field).to_excel(xlwriter, "Summary")
xlwriter.save()
xlwriter.close()
excel_file.seek(0)
response = HttpResponse(excel_file.read(),
content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
response['Content-Disposition'] = 'attachment; filename="excel_file.xlsx"'
return response
# urls.py
from django.conf.urls import url
from .views import ExcelFileDownloadView
urlpatterns = [
url(r'^mymodel/(?P<pk>\d+)/download/$', ExcelFileDownloadView.as_view(), name="excel-download"),]
You're mixing two requirements that should be separate:
Creating a .xlsx file using python or pandas--it looks like you're good on this part.
Serving a downloadable file (django); see this post or maybe this one
I'm trying to return a CSV from an action in my webapp, and give the user a prompt to download the file or open it from a spreadsheet app. I can get the CSV to spit out onto the screen, but how do I change the type of the file so that the browser recognizes that this isn't supposed to be displayed as HTML? Can I use the csv module for this?
import csv
def results_csv(self):
data = ['895', '898', '897']
return data
To tell the browser the type of content you're giving it, you need to set the Content-type header to 'text/csv'. In your Pylons function, the following should do the job:
response.headers['Content-type'] = 'text/csv'
PAG is correct, but furthermore if you want to suggest a name for the downloaded file you can also set response.headers['Content-disposition'] = 'attachment; filename=suggest.csv'
Yes, you can use the csv module for this:
import csv
from cStringIO import StringIO
...
def results_csv(self):
response.headers['Content-Type'] = 'text/csv'
s = StringIO()
writer = csv.writer(s)
writer.writerow(['header', 'header', 'header'])
writer.writerow([123, 456, 789])
return s.getvalue()