I'm currently building an application that will be used and maintained by another developer shortly in the future. What I want is the ability to upload a zip file, unzip and process the contents, and discard the file without ever actually storing it in the Django file storage system. These are the relevant parts of what I have right now:
views.py:
def upload(request):
if request.method == 'POST' and request.FILES['myfile']:
myfile = request.FILES['myfile']
if str(myfile.name).endswith('.zip'):
## THIS STORES THE FILE -- NOT WHAT I WANT
#fs = FileSystemStorage()
#filename = fs.save(myfile.name, myfile)
uploaded_file_url = str(myfile.name)
return render(request, 'webapp/upload.html', {
'uploaded_file_url': uploaded_file_url
})
file_error = "There was an error processing the file"
return render(request, 'webapp/upload.html', {
'file_error': file_error
})
return render(request, 'webapp/upload.html')
upload.html
{% extends "./base.html" %}
{% block content %}
<body>
<form method="post" enctype="multipart/form-data">
{% csrf_token %}
<input type="file" name="myfile">
<button type="submit">Upload</button>
</form>
{% if uploaded_file_url %}
<p>File uploaded at: {{ uploaded_file_url }}</p>
{% endif %}
{% if file_error %}
<p>There was an error in the file submission.</p>
{% endif %}
</body>
{% endblock %}
I know that checking if a file ends with .zip is not necessarily indicative of whether it's actually a zip file or not but it is sufficient for my purposes right now. myfile is a UploadedFile and I'm trying to find a way to unzip and process the contents but I'm not sure how to go about this. I could just store it in the FileSystem and then process it from there but I'd like to avoid storing it at all if possible. Any suggestions on how to do this would be greatly appreciated.
Related
Iam trying to loop thru images in static/folder. I can loop thru images in main 'static' folder but when i put them in 'static/folder' iam not sure how to do it in html.
my html lines(works when img in main 'static' folder)
{% for file in files %}
<img src=" {% static file %}" height="800">
<p>File name:{{ file }}</p>
{% endfor %}
my views.py
def album1(request):
images = '/home/michal/PycharmProjects/Family/gallery/static/'
files = os.listdir(os.path.join(images))
context = {'files': files}
return render(request, 'gallery/album1/main.html', context)
If i change views to:
def album1(request):
images = '/home/michal/PycharmProjects/Family/gallery/static/'
files = os.listdir(os.path.join(images, 'folder'))
context = {'files': files}
return render(request, 'gallery/album1/main.html', context)
It loops filenames in 'static/folder/' as expected, but then i cant figure out how to change it in html ,as its adding file names to: /static/{{ file }} instead to /static/folder/{{ file }}.
I think iam missing something or something need changing in load static on this particular page?
{% load static %} # either here
<img src=" {% static file %}"> # or here?
You prepend the filenames with the name of the folder:
from os.path import join
def album1(request):
images = '/home/michal/PycharmProjects/Family/gallery/static/'
files = os.listdir(join(images, 'folder'))
context = {'files': [join(folder, file) for file in files]}
return render(request, 'gallery/album1/main.html', context)
You can slice in the template with the |slice template filter [Django-doc]:
{% for file in files|slice:':21' %}
<img src=" {% static file %}" height="800">
<p>File name:{{ file }}</p>
{% endfor %}
But it is more efficient to do this in the view, since you save cycles on by performing less join calls, and furthermore the templates are less efficient than Python code.
This question already has an answer here:
What is the cause of the Bad Request Error when submitting form in Flask application?
(1 answer)
Closed 4 years ago.
I'm trying for the first time to upload file with python, I tried using flask and werkzeug libraries, here is my code:
Here I'm creating the function to upload files:
#app.route('/upload')
def upload_file():
return render_template('load.html')
#app.route('/uploader', methods=['GET','POST'])
#login_required(must=[be_admin, have_approval])
def uploaderV():
if request.method == 'POST':
file = request.files['file']
if file:
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'],filename))
return 'file uploaded'
return render_template('load.html')
Than this is my load.html page:
{% extends 'base.html' %}
{% block title %}Secret{% endblock %}
{% block page_body %}
<div class="row">
<form action="{{ url_for('uploaderV') }}" method="POST" enctype="multipart/form-data">
<p>
<input type='file' name='file[]' multiple=''>
<input type="submit" value="Upload">
</p>
</form>
</div>
{% endblock %}
Every time I try to upload a file the server gives me werkzeug.exceptions.BadRequestKeyError
BadRequestKeyError: 400 Bad Request: KeyError: 'file'
I tried in different ways, and now I really don't know what to do.
Your input name is file[] , not file. Try something like:
file = request.files['file[]']
Or just change your input name to file.
I have a simple form in my template, index.html:
{% if stripped_thumbnail_file_list %}
{% for thumbnail_name in stripped_thumbnail_file_list %}
<div class="">
<div class="">
This is my form
<form class="" action="{% url 'index' %}" method="post">
{% csrf_token %}
<input type="image" value="{{ thumbnail_name }}" src="{{ MEDIA_URL}}thumbnails/{{ thumbnail_name }}.jpg">
</form>
</div>
</div>
{% endfor %}
{% else %}
<p>No videos are available.</p>
{% endif %}
I want the index view to pull the {{ thumbnail_name }} value from this form and use it as a variable when the index view redirects to a different view that will use that name to play a matching video.
I have been unsuccessful in trying to pull that value from the form as I have it. I suspect this may because I'm not creating a Django form object. I tried to create that object, but I can't find any examples of a Django form object as an image like I have in my form.
What should that look like? Or, can someone make a recommendation on how to pull the value from the form as is?
EDIT: adding views.py snippet:
def index(request):
# if this is a POST request we need to process the form data
if request.POST:
# get thumbnail_name from form
# redirect to a new URL (hardcode thumbnail name for now):
return HttpResponseRedirect('2017-02-01_04-29-10/video/')
thumbnail_file_list = get_file_list(target_directory, ".jpg")
stripped_thumbnail_file_list = strip_file_extension(thumbnail_file_list)
template = loader.get_template('dash/index.html')
context = {
'stripped_thumbnail_file_list': stripped_thumbnail_file_list,
}
return HttpResponse(template.render(context, request))
def video(request, file_name):
print("this is the file name passed: " + file_name)
template = loader.get_template('dash/video.html')
context = {
'file_name': file_name,
}
return HttpResponse(template.render(context, request))
First: you need to declare the 'name' attribute on your form imput.
<input name="thumbnail_name" type="image" value="{{ thumbnail_name }}" src="{{ MEDIA_URL}}thumbnails/{{ thumbnail_name }}.jpg">
Second: Why don't you just set the 'action' of the form to your 'video' function (when you perform a redirect, you are losing all your POST data).Then, from there you could retrieve the value: something like that
def video(request):
file_name = request.POST.get('thumbnail_name')
print("this is the file name passed: " + file_name)
template = loader.get_template('dash/video.html')
context = {
'file_name': file_name,
}
return HttpResponse(template.render(context, request))
Hope it helps
(Flask novice alert)
Given the following to upload and save a file in Flask:
#app.route('/upload', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
file = request.files['file']
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return render_template_string('''
{% extends "base.html" %}
{% block content %}
<h4>File uploaded</h4>
<p><a href={{ url_for('members_page') }}>Back</a></p>
{% endblock %}
''')
elif not allowed_file(file.filename):
return render_template_string('''
{% extends "base.html" %}
{% block content %}
<h3>Please try again</h3>
<h4>File must be a .csv</h4>
<p><a href={{ url_for('upload_file') }}>Back</a></p>
{% endblock %}
''')
return render_template_string('''
{% extends "base.html" %}
{% block content %}
<h4>Upload CSV of Company/URL data</h2>
<form action="" method="post" enctype="multipart/form-data">
<input type="file" name="file" />
<input type="submit" />
</form>
{% endblock %}
''')
I wish to make filename available within another function:
#app.route('/scrape', methods=['GET', 'POST'])
#login_required # Use of #login_required decorator
def scrape():
parser = ConfigParser()
parser.read('config.ini')
keywords = parser.get('scrape', 'keywords').replace(' ', '').split(',')
jobs = scraper.scrape(os.path.join(app.config['UPLOAD_FOLDER'], filename), keywords)
The above is the desired intent, where filename is known by the scrape fucnction. Obviously that is not yet the case. With upload_file() already having a return value in the positive case (a confirmation page), how can I make filename available? UPLOAD_FOLDER will contain more than just the uploaded file, so I can't just join this path with whatever is in there.
Where this a non-Flask program, I would probably return locals() and then access the appropriate key, but I imagine that's not possible here if I want to maintain the serving up of the confirmation page.
You need to somehow connect two requests. If many users request the first one, a then someone requests a /scrape, how do you know which one is requesting, and which filename does he belong to?
You can use a session (a cookie session for example, see http://pythonhosted.org/Flask-Session/) to keep track of the uploaded file. Store the filename in the session, and the when the same user (with the same cookie) requests /scrape, you can retrieve the filename from the user session.
You can include the filename to use in the second request. This way, the user himself has to keep track of the files that he uploaded.
In either case, but especially in the latter, it's important to think about ownership of files: which user has access to which file on your system?
Pickle the filename in upload_file(), unpickle it in scrape().
PICKLED_CSV_FILENAME = 'pickled_csv_file_name'
def pickle_filename(filename, pickle_file):
with open(os.path.join(UPLOAD_FOLDER, pickle_file),'wb') as p:
pickle.dump(filename, p)
def load_pickled_filename(pickle_file):
return pickle.load(open(os.path.join(UPLOAD_FOLDER, pickle_file), 'rb'))
in upload_file():
pickle_filename(filename, PICKLED_CSV_FILENAME)
then in scrape():
jobs = scraper.scrape(os.path.join(app.config['UPLOAD_FOLDER'], load_pickled_filename(PICKLED_CSV_FILENAME)), keywords)
pickle_filename(filename, PICKLED_CSV_FILENAME)
Obviously not a sustainable solution in the case of many users/files, but it is a single user, single file scenario so it's acceptable.
Python: 2.7.11
Django: 1.9
I want to upload a csv file to Django and analyze it with a Python class. No saving is allowed and the file is only needed to reach the class to be analyzed. I'm using Dropzone.js for the form but I don't understand how I should configure/program the views to achieve this.
<form action="/upload/" method="post" enctype="multipart/form-data" class="dropzone" id="dropzone">
{% csrf_token %}
<div class="fallback">
<input name="file" type="file" multiple />
</div>
</form>
I have found an article about this but it describes saving and is based on Django 1.5.
view.py
def upload(request):
if request.method == 'POST':
file = FileUploadForm(request.POST)
if file.is_valid():
return HttpResponseRedirect('/upload/')
else:
file = FileUploadForm()
return render(request, 'app/upload.html', {'file': file})
forms.py
from django import forms
class FileUploadForm(forms.Form):
file = forms.FileField()
Closing Update:
The most important difference between the helping answer and my situation is that I had to decode my input.
See the following line as mine csv_file in handle_csv_data:
StringIO(content.read().decode('utf-8-sig'))
Access the csv file in the view function. If you are using python 3, you must wrap the InMemoryUploadedFile in a TextIOWrapper to parse it with the csv module.
In this example the csv is parsed and passed back as a list named 'content' that will be displayed as a table.
views.py
import csv
import io # python 3 only
def handle_csv_data(csv_file):
csv_file = io.TextIOWrapper(csv_file) # python 3 only
dialect = csv.Sniffer().sniff(csv_file.read(1024), delimiters=";,")
csv_file.seek(0)
reader = csv.reader(csv_file, dialect)
return list(reader)
def upload_csv(request):
csv_content=[]
if request.method == 'POST':
csv_file = request.FILES['file'].file
csv_content = handle_csv_data(csv_file)
return render(request, 'upload.html', {'content':content})
Your original code did not use django's form framework correctly, so I just dropped that from this example. So you should implement error handling when the uploaded file is invalid or missing.
upload.html
<form action="/upload/"
method="post"
enctype="multipart/form-data"
class="dropzone"
id="dropzone">
{% csrf_token %}
<div class="fallback">
<input name="file" type="file"/>
<input type="submit"/>
</div>
</form>
{% if content %}
<table>
{% for row in content %}
<tr>
{% for col in row %}
<td>{{ col }}</td>
{% endfor %}
</tr>
{% endfor %}
</table>
{% endif %}
I've added a 'submit' button so this works without the dropzone thing. I also removed 'multiple' from the file input, to keep the example simple. Finally there's a table if the template receives content from a parsed csv. But when using dropzone.js, you have to use a javascript callback function to display the table.