I have the following code (taken from the sample code) to upload files directly to GCS but it does not preserve the file name of the original file. Instead it stored using a hash as the filename like
'L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlVxd3BLMk1ZVVg0SVVzSTlUQTMzRzEyVzQ3ZHQtQy1JSjh6ZU5aT0hWc3FzZ2k0NXpfNXZBUTZDRThSQTFNNi0xX0dEY1M4MEFwUFJYbGt4cUkxNkRpeGp6M0VUQS51QlBNM3BJNG9qXy0zVHZF'
The problem is that I need to show all the files uploaded to GCS to users so need to preserve the file identity to distinguish the files. They won't be able to identify their uploads from hashes
Here's the sample code... how do I ensure that it uploads with the file name provided? For instance, if I upload a file 'log.csv', it is uploaded as 'log.csv' rather than a hash which doesn't say if it's the file or not.
import os
import urllib
import webapp2
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
class MainHandler(webapp2.RequestHandler):
def get(self):
upload_url = blobstore.create_upload_url('/upload', gs_bucket_name='uploads')
self.response.out.write('<html><body>')
self.response.out.write('<form action="%s" method="POST" enctype="multipart/form-data">' % upload_url)
self.response.out.write("""Upload File: <input type="file" name="file"><br> <input type="submit"
name="submit" value="Submit"> </form></body></html>""")
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file') # 'file' is file upload field in the form
blob_info = upload_files[0]
self.redirect('/serve/%s' % blob_info.key())
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info)
app = webapp2.WSGIApplication([('/', MainHandler),
('/upload', UploadHandler),
('/serve/([^/]+)?', ServeHandler)],
debug=True)
The filename and other metadata are in the BlobInfo object in the datastore, not in the blobstore per se. See https://cloud.google.com/appengine/docs/python/blobstore/blobinfoclass for all details of the BlobInfo class (and https://cloud.google.com/appengine/docs/python/tools/webapp/blobstorehandlers for how the provided handlers deal with it on both upload and download).
So, instead of requiring the user to pass you the blob key (so you can use blobstore.BlobInfo.get(resource) directly on the request), you can allow the user to pass you the filename, use BlobInfo.gql to give you all the blobinfo objects for the given filename (no guarantee of uniqueness, of course -- unless you enforce it yourself in the app at upload time by rejecting duplicate filenames), then use the key() method of those blob info objects to get the required blob keys.
Access to the datastore (for blobs' metadata) is faster than access all the way to the blobstore, so e.g to present a list of uploaded files, BlobInfo.all() (returning a query to which you can add filters) will let you present filenames & other metadata (content type, creation datetime, size in bytes) to your users, for them to select which blob(s) they want to retrieve.
Related
I am trying to upload some picture to Google App Engine using the Blobstore.
And some of the files contain non-ascii characters.
When I download these files, the filename for these downloaded file appeared to show the "key" in blobstore, rather than the original file name.
My site is http://wlhunaglearn.appspot.com/
I already add a save_as=blob_info.filename in my BlobstoreDownloadHandler , but it failed when the file name contains non-ascii characters.
Any suggestions?
Thanks in advance.
The following is my main.py file
# -*- encoding: utf-8 -*-
import os
import urllib
import webapp2
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
class MainHandler(webapp2.RequestHandler):
def get(self):
upload_url = blobstore.create_upload_url('/upload')
self.response.out.write('<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body>')
self.response.out.write('<form action="%s" method="POST" enctype="multipart/form-data">' % upload_url)
self.response.out.write("""Upload File: <input type="file" multiple name="file"><br> <input type="submit"
name="submit" value="Submit"> </form></body></html>""")
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file') # 'file' is file upload field in the form
blob_info = upload_files[0]
self.redirect('/serve/%s' % blob_info.key())
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info, save_as=blob_info.filename)
app = webapp2.WSGIApplication([('/', MainHandler),
('/upload', UploadHandler),
('/serve/([^/]+)?', ServeHandler)],
debug=True)
Searching all the posts, I finally got a hint from an Answer to another Question
I figure out the correct way to display non-ascii file name is to
add a urllib.quote function to the last line of ServeHandler class
Therefore the ServeHandler class will be:
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info, save_as=urllib.quote(blob_info.filename.encode('utf-8')))
I have an HTML form that allows file uploads. I need to send the uploaded file via email. I am following the Google appengine Request class example.
from google.appengine.ext import webapp
from google.appengine.api import mail
class MyRequestHandler(webapp.RequestHandler):
def get(self):
self.response.out.write('''
<html>
<body>
<form method="post">
<p>File: <input type="file" name="upload" /></p>
<p><input type="submit" /></p>
</form>
</body>
</html>
''')
def post(self):
uploaded_file = self.request.get("upload")
uploaded_name = "file.doc"
message = mail.EmailMessage(sender="Me <me#mydomain.com>",
subject="Email from me")
message.to = "Me <me#mydomain.com>"
message.body = "See attached file."
message.attachments = [(uploaded_name, uploaded_file)]
message.send()
The above code works. I need to replace the hardcoded file name (file.doc) in the message.attachments tuple with the actual name of the file uploaded. I have two questions:
How do I get the uploaded file name from the webapp Request class?
Where can I find the documentation that describes how to do this? I have done a bunch of searching and can not find reference material or examples.
UPDATE:
Looks like the solution is:
if uploaded_file != "":
uploaded_name = self.request.params["upload"].filename
...as posted
here.
Instead of going self.request.get("upload"), try examining the self.request.POST["upload"] dict.
Specifically, self.request.POST['upload'] will return a cgi.FieldStorage instance. The attribute you are looking for is filename. Also of interest would be mimetype - and of course data.
For a complete how-to on handling uploads, take a look at this ancient post over at Nick Johnson's blog.
For an under-the-hood look at how webapp2 request data works, consult this portion of webapp2 docs.
I'm using this link as an example to uploading images:
https://gist.github.com/jdstanhope/5079277
My HTML code:
<form action="/upload_image" method="post" id="form1" runat="server">
<div class="fileButtons">
<input type='file' id="imgInp" name="imgInput" accept="image/*"/><br><br>
<input type='button' id='remove' value='Remove' />
</div></form>
main.py:
class SetImage(webapp2.RequestHandler):
def post(self):
logging.debug("test")
id = str(self.request.get('id'))
image = self.request.get('imgInput')
app = webapp2.WSGIApplication([('/upload_image', SetImage),
('/', MainPage)], debug=True)
But when I add an image, nothing is being done, and the log console doesn't print:
logging.debug("test")
The recommended way of uploading images to GAE is by using blobstore.
Here is a quick breakdown of the doc to help you achieve this fast:
Imports:
import os
import urllib
import webapp2
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
Serve the form HTML. this form performs the POST to GAE with the selected files data:
class MainHandler(webapp2.RequestHandler):
def get(self):
upload_url = blobstore.create_upload_url('/upload')
self.response.out.write('<html><body>')
self.response.out.write('<form action="%s" method="POST"
enctype="multipart/form-data">' % upload_url)
self.response.out.write("""Upload File:
<input type="file" name="file"><br> <input type="submit"
name="submit" value="Submit"> </form></body></html>""")
Add a handler for receiving the POST data (binary content of the file). the last line in this function is to redirect the response to the location from where the file could be downloaded:
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
# 'file' is file upload field in the form
upload_files = self.get_uploads('file')
blob_info = upload_files[0]
self.redirect('/serve/%s' % blob_info.key())
Add handler to serve the image that you uploaded using the UploadHandler described above:
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info)
And finally, define the routes for the app:
app = webapp2.WSGIApplication([('/', MainHandler),
('/upload', UploadHandler),
('/serve/([^/]+)?', ServeHandler)],
debug=True)
Regarding the issue of your log statement not firing: The default log level for dev_appserver.py is info, which you can override with the --dev_appserver_log_level flag.
Reference the new dev_appserver documentation.
To access uploaded files from a multipart/form-data POST request, the webapp2 documentation states that they're available in request.POST.
In my GAE app I want to store the uploaded file in a ndb.BlobProperty . How can I assign the uploaded file content to this property.
Further more is BlobProperty the standard approach to store user uploaded files.
Link to similar question
Simply put, assign a serving url that you pass to a handler setup by routes
handlers
class GetBlobstoreUrl(BaseHandler):
def get(self):
upload_url = blobstore.create_upload_url('/upload/')
self.response.out.write(upload_url)
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads()
blob_info = upload_files[0]
model
class SomeModel(ndb.Model):
avatar = ndb.BlobProperty()
I have been banging my head against the wall on this one, for some reason I am having trouble tying the different aspects of Google App Engine together to make this work.
Basically I want to let a user upload a photo to the Blobstore, which I have working in the below code, and then I want to put the BlobKey into a list which will be stored in a database entity. So here is my code to upload the image and where in here can I get the BlobKey so that I can store it?
class MainHandler(BlogHandler):
def get(self):
upload_url = blobstore.create_upload_url('/upload')
self.response.out.write('<html><body>')
self.response.out.write('<form action="%s" method="POST" enctype="multipart/form-data">' % upload_url)
self.response.out.write("""Upload File: <input type="file" name="file"><br> <input type="submit" name="submit" value="Submit"> </form></body></html>""")
#there is a lot more code in here where I get all the following info but it isn't relevant
location_db = Location(
description=description,
submitter=submitter,
user_id=user_id,
title=title,
locationtype = locationtype)
#This is what I would like to do but I don't know where to get thr BlobKey
location_db.blobRefs.append(BlobKey)
location_db.put()
for b in blobstore.BlobInfo.all():
self.response.out.write('<li>' + str(b.filename) + '')
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file')
blob_info = upload_files[0]
self.redirect('/main')
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, blob_key):
blob_key = str(urllib.unquote(blob_key))
if not blobstore.get(blob_key):
self.error(404)
else:
self.send_blob(blobstore.BlobInfo.get(blob_key), save_as=True)
Here:
blob_info = upload_files[0]
self.redirect('/serve/%s' % blob_info.key())
I think it's that
blob_info.key()
you are missing. Grab that, stuff it into your list. Docs also note:
In this handler, you can store the blob key with the rest of your application's data model. The blob key itself remains accessible from the blob info entity in the datastore.
https://developers.google.com/appengine/docs/python/blobstore/overview#Serving_a_Blob