I am trying to get a simple image upload app working on Heroku using Flask. I'm following the tutorial here: http://flask.pocoo.org/docs/patterns/fileuploads/
However, I want to use S3 to store the file instead of a temporary directory, since Heroku does not let you write to disk. I cannot find any examples of how to do this specifically for Heroku and Flask.
It seems to me that in the example code that stores the uploaded file to a temporary file, you would just replace file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename)) with code that uploads the file to S3 instead.
For example, from the linked page:
def upload_file():
if request.method == 'POST':
file = request.files['file']
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
s3 = boto.connect_s3()
bucket = s3.create_bucket('my_bucket')
key = bucket.new_key(filename)
key.set_contents_from_file(file, headers=None, replace=True, cb=None, num_cb=10, policy=None, md5=None)
return 'successful upload'
return ..
Or if you want to upload to S3 asynchrnously, you could use whatever queuing mechanism is provided by Heroku.
A bit of an old question, but I think ever since Amazon introduced CORS support to S3, the best approach is to upload directly to S3 from the user's browser - without the bits ever touching your server.
This is a very simple flask project that shows exactly how to do that.
Using boto library it will look something like this:
import boto
from boto.s3.connection import S3Connection
from boto.s3.key import Key
def upload_file():
if request.method == 'POST':
file = request.files['file']
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
conn = S3Connection('credentials', '')
bucket = conn.create_bucket('bucketname')
k = Key(bucket)
k.key = 'foobar'
k.set_contents_from_string(file.readlines())
return "Success!"
Instead of storing the file on the disk directly, you could also store its data in the database (base64 encoded for example).
Anyway, to interact with Amazon S3 using Python, you should consider using the boto library (the same is true for any other Amazon service).
To know how to use it, you could have a lookat the related documentation.
I'm working on something similar for a website I'm developing now. Users will be uploading very large files. I'm looking at using Plupload to upload directly to S3 following the advice here.
An alternative is to use the direct-to-S3 uploader in Boto.
Related
On a regular (non-flexible) instance of Google App Engine, you can use the Blobstore API and create a URL to allow a user to upload a file directly into your Blobstore. When it is uploaded, your app engine application is notified of the location of the file and can process it. An example of the python code is:
from google.appengine.ext import blobstore
upload_url = blobstore.create_upload_url('/upload_photo')
See the Blobstore docs.
Switching to Google App Engine Flexible Environment, usage of the Blobstore has been largely replaced by Cloud Storage. In such a case, is there an equivalent of create_upload_url?
My current implementation takes a standard file upload to a python Flask application. Then proceeds with something like:
from flask import request
from google.cloud import storage
uploaded_file = request.files.get('file')
gcs = storage.Client()
bucket = gcs.get_bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.upload_from_string(
uploaded_file.read(),
content_type=uploaded_file.content_type
)
This seems like it is doubling the network load compared with create_upload_url because the file is coming into my app engine instance and then immediately being copied out. So the uploader will be made to wait extra time whilst this is happening. Presumably I will also incur extra App Engine charges for this. Is there a better way?
I have workers that later process the uploaded file, but I tend to download the file from Cloud Storage again in their code because I don't think you can assume that the worker will still have access to a file stored in the instance file system. Therefore I don't get any benefit of having the file uploaded to my instance rather than direct to it's storage location.
I have started using create_resumable_upload_session to create a signed URL that our client side application can upload a file to. Something like:
gcs = storage.Client()
bucket = gcs.get_bucket(BUCKET)
blob = bucket.blob(blob_name)
signed_url = blob.create_resumable_upload_session(content_type=content_type)
Then when the client has successfully uploaded a file to our storage, I subscribe to a Pub/Sub notification of the creation using this Cloud Pub/Sub Notifications for Cloud Storage.
Each blob created with the new Google Cloud Storage Client has a public_url property:
from flask import request
from google.cloud import storage
uploaded_file = request.files.get('file')
gcs = storage.Client()
bucket = gcs.get_bucket(bucket_name)
blob = bucket.blob('blob_name')
blob.upload_from_string(
uploaded_file.read(),
content_type=uploaded_file.content_type
)
url = blob.public_url
--
With the Blobstore, a GAE system handler in your instance takes care of the uploaded file you pass to the upload url created. I'm not sure if it's an issue handling it yourself in your code. If your current approach is problematic, you might want to consider doing the upload client side and not pass the file through App Engine at all. GCS has a REST API and the cloud storage client uses it underneath, so you can read and upload the file directly to GCS on the client side if it's more convenient. There's firebase.google.com/docs/storage/web/upload-files to ease you through the process
We currently use blobstore.create_upload_url to create upload urls to be used on the frontend see Uploading a blob.
However, with the push toward Google Cloud Storage (GCS) by Google, I'd like to use GCS instead of the blobstore. We use currently blobstore.create_upload_url but I can't find anything equivalent in the GCS documentation. Am I missing something? Is there a better way to upload files to GCS from the frontend?
Thanks
Rob
If you will provide gs_bucket_name to blobstore.create_upload_url file will be stored in GCS instead of blobstore, this is described in official documentation: Using the Blobstore API with Google Cloud Storage
blobstore.create_upload_url(
success_path=webapp2.uri_for('upload'),
gs_bucket_name="mybucket/dest/location")
You can take a look at simple upload handler implementation made in webapp2
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
import webapp2
import cloudstorage as gcs
class Upload(blobstore_handlers.BlobstoreUploadHandler):
"""Upload handler
To upload new file you need to follow those steps:
1. send GET request to /upload to retrieve upload session URL
2. send POST request to URL retrieved in step 1
"""
def post(self):
"""Copy uploaded files to provided bucket destination"""
fileinfo = self.get_file_infos()[0]
uploadpath = fileinfo.gs_object_name[3:]
stat = gcs.stat(uploadpath)
# remove auto generated filename from upload path
destpath = "/".join(stat.filename.split("/")[:-1])
# copy file to desired location with proper filename
gcs.copy2(uploadpath, destpath)
# remove file from uploadpath
gcs.delete(uploadpath)
def get(self):
"""Returns URL to open upload session"""
self.response.write(blobstore.create_upload_url(
success_path=uri_for('upload'),
gs_bucket_name="mybucket/subdir/subdir2/filename.ext"))
All the examples I can find on google endpoint api (e.g., tic-tac-toe sample) show strings, integers, enums, etc fields. None of the examples say anything about how to specify document (e.g., image or zip files) uploads or downloads using the API. Is this not possible?
If this is possible, can anyone share a code snippet on how to define google endpoint api on the server to allow downloads and uploads of files? For example, is there a way to set HTTPResponse headers to specify that an endpoint response will serve a zip file? How do we include the zip file in the response?
An example with python or php would be appreciated. If anyone from the endpoints-proto-datastore team is watching this discussion, please say whether or not file downloads are supported in endpoints at the moment. We hate to waste our time trying to figure this out if it is simply impossible. Thanks.
We are seeking a complete example for upload and download. We need to store the key for the uploaded file in our database during upload and retrieve it for download. The client app sends a token that the API needs to use to figure out what file to download. Hence, we would need to store the blob key generated during the upload process in our database. Our database would have the mapping between the token and the blob file's key.
class BlobDataFile(models.Model):
data_code = models.CharField(max_length=10) # Key used by client app to request file
blob_key = models.CharField()
By the way, our app is written in Django 1.7 with a mysql (modeled with models.Model) database. It is infuriating that all the examples for Google App Engine upload I can find is written for a standalone webapp Handlers (no urls.py/views.py solutions could be found anywhere). Hence, building a standalone uploader is as much of a challenge as writing the API code. If your solution has full urls.py/views.py example for uploading files and saving the blob_key in our BlobDataFile, it would be good enough for us.
f you use the blobstore use the get_serving_url function to read the images from url in the client, or use the messages.ByteField in the ResourceContainer and serialize the image with base64.b64decode
#the returned class
class Img(messages.Message):
message = messages.BytesField (1)
#The api class
#endpoints.api(name='helloImg', version='v1')
class HelloImgApi(remote.Service):
ID_RESOURCE = endpoints.ResourceContainer(
message_types.VoidMessage,
id=messages.StringField(1, variant=messages.Variant.STRING))
#endpoints.method(ID_RESOURCE, Img,
path='serveimage/{id}', http_method='GET', #ID is the blobstore key
name='greetings.getImage')
def image_get(self, request):
try:
blob_reader = blobstore.BlobReader(blob_key)
value = blob_reader.read()
return Img(message=value)
except:
raise endpoints.NotFoundException('image %s not found.' %
(request.id,))
APPLICATION = endpoints.api_server([HelloImgApi])
And this is the response (save it in the client with the proper format)
{
"message": "/9j/4AAQSkZJRgABAQAAAQABAAD//gA+Q1JFQVRPUjogZ2QtanBlZyB2MS4wICh1c2luZyBJSkcgSlBFRyB2NjIpLCBkZWZhdWx0IHF1YWxpdHkK/9sAQwAIBgYHBgUIBwcHCQkICgwUDQwLCwwZEhMPFB0aHx4dGhwcICQuJyAiLCMcHCg3KSwwMTQ0NB8nOT04MjwuMzQy/9sAQwEJCQkMCwwYDQ0YMiEcITIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIy/8AAEQgBZwKAAwEiAAIRAQMRAf/EAB8AAAEFAQEBAQEBAAAAAAAAAAABAgMEBQYHCAkKC//EALUQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz9PX29/j5+v/EAB8BAAMBAQEBAQEBAQEAAAAAAAABAgMEBQYHCAkKC//EALURAAIBAgQEAwQHBQQEAAECdwABAgMRBAUhMQYSQVEHYXETIjKBCBRCkaGxwQkjM1LwFWJy0QoWJDThJfEXGBkaJicoKSo1Njc4OTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoKDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uLj5OXm5+jp6vLz9PX29/j5+v/aAAwDAQACEQMRAD8A9/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACkpaKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoopKACkZgilmIAHUntWTr3iTTfD1qZr2YBiPkjHLN9BXjniTxzqniSRreItb2ZOBDGeWH+0e/wBK5a+KhRWu/Y48TjadBa6vsei6z8TNF0udreDzL2VeD5ONgPpuP9M1zF18U9VuVZrWzt7OH/npKS5/DpmvP9kVqMyYkl/uDoPrUEs0kzbnbPoOwryZ46tPZ2PDqZlXm9HZHp/g74g3V54gFlqU2+Kf5Y3YBcN+HTNeqCvlqORopFkRiroQykdiK+k/D+oDVNBsrwHJkiUn645rvwFdzTjLdHpZZiZVIuEndo06KKK9E9YKKKKACikpaACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACkpaKAEoopaACkpaKACkpaKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApKWoLu7gsrZ7i5lSKJBlnc4AFF7CbS1ZKSAMk1wPi74j22lb7PSytxeDhn6pH/ia5bxh8RrjVTJY6SzwWf3Wl6PJ/gK4iOAbfOnJVP1b6V5WJx9vdp/eeLjMzt7lL7ya4uLzWLt7q7naRzy0jngVE9wsSmO34z1c9TTZpzLhQNkY6KKhryG23dnhtuTuxOvWilPSu68DeApdbdNR1JGj04HKJ0M3+C+/etaVKVWXLE2oUZ1pcsEUvB/ge68STC4nDQ6ep5fHMnsv+Ne4afYW2mWMVnaRiOGJcKoqWCCK2hWGGNY40GFVRgAVLXv4fDxoxstz6fC4WGHjZbhRRRXQdQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUlFAC0UUlAC0UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFJRmsLxN4psfDViZbhg07D91CDyx/wqZSUVdkznGEeaT0Les65Y6FYvdXsoRR91e7H0Arw3xT4wvvE9zhyYrNT+7gU8fVvU1Q1zXr/wAQ37XN3IWOcJGPuoPQCqyqtoMsA03Ydl/+vXh4rGOp7sdj5vG5hKr7sNI/mIkSQKJJhlj91P8AGopZWlfcx+ntTWYuxZjkmkrhPNCg0V3PgPwO+tzLqN+hWwQ5RT/y1P8AhWlKlKrLlibUKEq0+WJJ4E8Btq8ialqcZWxU5SM8GU+/t/OvZo41iRURQqqMAAYAFEcSRRrHGoVFGAoHAFPr6GhQjRjZH1WGw0KEOWIUUUVudAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRSUALRRRQAlFLRQAUUUUAFJS0UAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUlFcn4y8Z23hq1MUZWW/kHyR5+77n2qJzjCPNIipUjTjzSehL4u8Y2nhizxxNfSD91CD+p9BXheo6lea1qD3V5K0s8h/AewHYUy8vLrVL6S5uZGmuJTksf5fSnZW2XauDKerf3fpXgYrFSrO3Q+YxmNlXlboHy2gwMGY9T2X/wCvVckkkk5JoPNFchwBSUtb/hLwvceJtTES5S0jOZpfQeg9zV04SnLliaUqcqklGO5f8D+DZfEd4Lm5Vk02Fvnb/nof7o/rXuUEEVtCkMKKkaDaqqMACo7Cxt9OsorS1jWOGJdqqKsV9FhsPGjGy3Pq8JhY4eFlv1FoooroOoKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigBKWiigApKWkoAWiiigAopKWgAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigApM0ZrlvGXjC38M2O1dsl9KD5UWen+0faonNQjzSIqVI04uUthnjTxlb+G7QxRFZL+Qfu4/7v+0a8Muru51K9e4uJGlnlbJJ5JNF5eXOpXsl1dStLPK2WY08AWy4HMp6n+7XgYnEyrS8j5fGYyVeXkHFspVcGU/eb+77CoOtFFchwCUUtSWtrNe3UdtbxtJNIwVVHUmhJt2RSTbsi5omjXWvapFY2q/MxyzdkXuTX0BoWiWug6ZFZWq4VR8zd2Pcms7wd4Wg8NaWqEK15KA00nv6D2FdJ3r6DB4VUo3e7PqMBg1QjzS+JgKWiiu09AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAEopaKAEopaKAEopaKAEopaSgBaKKKAEpaKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKSsjxH4htfDulveXLAt0jjB5dvQUpSUVdkykormlsVfFnim28NaaZXIe5cYhizyx9T7V4HqOoXWrX8t5dyGSeQ5JPb2HtU2s6zd67qUl7dyFnY8L2UegqCNBAokcfOfuqe3vXz+LxTqystj5jHYx15WXwoVVFuuT/AK0/+O1ETk5pSSxJPU0lcR51wpKWg0AJ1r2T4deDhpdqNVvo/wDTZl/dow/1Sn+prm/hz4Q/tK5XV76P/RYW/cow/wBYw7/QV7GABwK9nAYW372XyPfyzB2/fT+QUUUYr1T2xaKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAopKWgAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAopKiuLiK1t3nncJFGpZmY4AFFxN2K+q6pa6Np8t7dyBIoxnnqT6D3r5+8S+I7rxLqjXU5KxDiGLPCL/jV/xt4ul8S6iViZl0+EkRJ/eP8AeNc5BEGy7/cH614eNxXtHyx2PnMwxvtXyR+FfiPhjCL5sgz/AHV9TSMxdixOSaV3Ltk8DsPSm15tzyb3EopaSgQVveE/Dc3iTV1gAK2yfNM/oPT6msmxsp9RvYbO2QvNK21QK+gfDHh+Dw7pEdpEAZD80sn95q7cFhvayu9kell+E9vPml8KNO1tYbK1itreMJFGoVVHYCpqMUV9ClY+oSsrIWiiigYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAJS0UUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFJRQAhOASTwK8a+IvjM6ncNpFhJ/okTfvXU/6xh2+grofiR4y/s+3bR7CT/SpV/eup/1ant9TXjyqXYKOSa8rHYq37uPzPEzLGW/dQfqOiiMr46KOSfQVM7g4VRhF6ChsRp5aHj+I+ppleM2eA3cKKKKQgpDS123w88K/2zqI1C7TNlbNkAjiR/T6CtaNJ1ZqKNqFGVaahE634ceE/wCy7Iarex4vLhfkVhzGn+JrvhQAAMDpS19NSpRpwUYn2FGjGjBQj0CiiitDUKKKKACiiigAooooAKSlooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAEpaSigBaSiigBaKYzqv3mA+tNE8RbAlQn0zSuhXRLRSUUxi0UlLQAUUUUAFFJRQAVzfjPxRD4Z0hpAQ13KCsEfqfX6CtnU9St9J0+a9u3CQxLlj6+w96+d/EevXHiPWJb64JCn5Yo88IvYVx4vEeyjZbs4Mdi1QhZfEzPubia8uZLidzJNIxZmPUmpUXyU5++36Co4EAHmN/wEe9PJJ5718/KVz5acrsSiiioMxKKWgKWIABJPAAosNK5oaHo8+u6tDY24OXOXbsq9zX0HpWm2+k6dDZWyhY41AHv71zvgLwuNB0oT3Cf6dcANJkcoOy119fQ4LDeyhzPdn1OXYT2NPml8TFoooruPSCiiigAooooAKKKKAEpaKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAopKKACiiuJ8Y+P7bQVezsys+oYwR/DH9ff2rOpUjTjzSZnVqwpR5ps6PWde07QrYz31wqD+FOrN9BXmOt/FPULxjDpEIto+0jjc5/DoK4m6vL3Wbt7q9naRicl3PA9hTfMWIbYRg93PU14uIzCcnaGiPncVmlSbtT0Rcu7/U79vM1DUZj6B3P8hVdZkhkDpLcGQchw+0iqxJJyTk0lcDqSbu2eY6k27tnZaR8RdU00qkrNdQjtMcsPxr0bw9430zX3ECMYbnH+rk7/Q968Hp8UjwyrJG7I6nKspwQa6qGOq03q7o7cPmVak7N3R9OClrhfAnjP+2YRp984F9GPlY/8tB/jXc179KrGpHmifT0a0K0FOAtFJS1oaiUhIUEk4ApTXA/ErxZ/ZGnf2baSYvLlfmI6onr+NZ1KipxcmZVqsaUHORxvxG8WnWdROnWkn+hW7ckHiR/X6CuJij8x/8AZHJNRjLN6k1bACJtH4185WqucnJnyWIryqTc31FY54HAHApKKK5zlCkpaSgA+ld78NfC/wDaN9/a93Hm1t2xEpHDv6/Qfzrk9D0efXdXgsIAQZD87f3V7mvoTTdPg0vT4LK2QJFEoVRXpZfhueXtJbI9fK8J7SftJbL8y3RS0V7p9KFFFFABRRRQAUUUUAFFFFABRRRQAlLRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUlFV72CS5spoIpmhd0KrIo5U+tJiZwfj3x8NKV9L0qQNesMSyjkQj0H+1/KvI0Rp3aaZ2bJyzE5LGtjxD4W1LQb9xfK0kTNlbgch8+/rWUTkY6AdBXzuLrVJztLQ+Tx9erOpaenkK7lgABhR0AptFFcR54UUUUAFFFFAEttczWdzHc27lJY2DIw7GvffCniGLxFo0d0uFmX5Jk/ut/h3r59rpfBPiFtA11DIx+yXGI5h2Ho34V3YHEOlOz2Z6WW4t0anLLZnvNLTVYOoZTkEZBFBIAya+iPqzO17WbfQdIn1C5b5I1+Ve7N2A+pr5w1TVLjWNTnv7pt0srbj6AdgPYV1HxJ8V/wBuax9htpM2NoxAweJH7t/QVxkKGR/bvXiY2vzy5Vsj53McT7SXKtkWIEwN569qlo4xxRXlt3Z4zd2FFFFIQUdeBRXYfD7w3/bWsC6nTNpakMc9GbsK1pUnUmoo2oUZVqihE7v4e+GRouk/a7hP9MugGbI5RewrtKAAAAOlFfT06apxUUfZUaUaUFCPQWikpa0NAooooAKKKKACiiigAooooAKKSloAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooASloooAKSlooAr3dnBfW7QXMSyxMMFWGa8y8RfDF0L3GjtuXr5LdR9DXqtJWFbD06ytJHNiMJSrq00fM9zaz2czQ3EbRyKcFWGMVDXt/jTwjHrdm91axgX0YyB08wen19DXis0LQuykEYJBBGCD6H3rwMThZUJeR8vjMHPDSs9V3IqKWiuU4xKKKKACiiigD2v4da/wD2roQtZnzc2mEOTyy9j/SoPiZ4rGgaKbO3fF7dgquDyidz/SvNvCmv/wDCO65HeOW+zkFZlHda5bxT4lm8SeILi/lJCs22JP7qDoK9qlinOhbrsfRUca54bl+1sVlYu/qTWnDH5ceO561S02LcPNYfStGvLqy1seLXlryoSiiisTAKKKKAJrO0mvryG1t0LzTMEUD1NfQnh7RYdB0aCxiAyoy7f3m7muF+F3hwBX125Tk5jtgR27t/T869Pr3svw/JD2j3Z9NleF9nD2kt3+QUtFFekesFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABSUtFABRRRQAlLRRQAlFLRQAlFLRQAleX/Evwx5W7X7OLKHAvI1HbtIPcd69QqOeGO4geGVA8bqVZSOCDWValGrBxZjiKEa1NwkfM7LtIwcqeQfUUlbHiXQn8O69NprZNu+ZLVz/AHT/AA/hWMRivmKtN05OLPja1J0puEgooorMyCiiigA61zlzZFdV8sfdc7hXR0xokaVZCPmXoa1pVeRs3oVvZthEgijVB0Ap9FFZN3MW7u4lFFFABWn4f0eXXdagsIwdrnMjD+FB1NZley/DXw9/ZukHUJ0xc3YyMjlU7CuvB0PbVEuiO3AYb29VJ7Lc7O0tYrK0itoECRRKFUDsBU9FLX0iVtEfXJWVkFFFFMYUUUUAFFFFABRRRQAUUUlAC0UUUAFFFFABRRSUALRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUlAC0UUUAFFFFABRRSUALSUtJQBx/xE8Of274eeWFP9MtP3sRHU+orxFX86MSdG6MPQ19PMAykHoa8B8b6J/wAI94qlVVxZ3v7yP0BPUfnXlZjQuvaI8TNsNzL2qOfooIIJB60V4h86FFFFABRRSUDFpKKKACiijNOwG/4P0Jtf8QQwMD5EZ8yY/wCyO34178iLHGqIAFUYAHYVyPw90D+x9AWeVMXN1iR8jkL2FdhX0eCoeyp67s+sy7DexpK+7ClpM0V2HoC0UlFAC0UlFAC0UlFAC0UmaKAClpKKAFopKM0ALRSZozQAtFJmigBaKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigBKWiigAooooAKKKKACiiigAooooAKKKKACiiigBK474keH/AO2/DEskSZubTMseOpA6j8v5V2OaawDKQRkHqDUTipxcWRUgpxcX1PmGKTzoFc/eHyt9adWr4s0Y+HPF11ZgYtbj95Ce20nj8jkVknivmK1Nwm0z4zEUnTqOLFpKKM1iYhRSZpKYDs0maTNJmgY7NdF4J0M694khidc20H72Y9sDoPxP9a5vNe3/AA60P+yPDi3Eq4ubzEr56hf4R+XP412YKj7Sqr7I78vw/tqyvsjsRhQABgClzTM0Zr6I+sH5ozTM0ZoAfmjNMzRmgB9JTc0ZoAdmjNNzRmmA/NGaZmlzQA7NGabmjNADs0U3NGaAHZopM0ZoAWikzRmgB1FNpc0ALRSUUALRmkzRmgBc0uabS0AFLSUUALRSUUALRSUUALRSZpaACiiigApKWigAopKWgAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiikoAWikzRmgBaKbmjNAC0ZpM0maQDqSm5o3UAOzSZpu6k3UAcD8WdB/tHw6upQrm4sG3HHUxn735cH8DXkCSiaJJO54P1r6YuIo7m3kglUNHIpVlPcEV816jp76F4gvtIlziNz5ZPdeqn8q8rMKN/fR4ea0L2qIZmjNMzRurx7Hg2HZpM03dSbqdh2HZozTN1a+geHNQ8R3YhtI8RA/vJm+6g/x9quEHJ2ii4U5TfLFalzwfoEmv65FGUJtYmDzt2wO34174uEUKowAMAVi+HtBtPDumraW3zMeZJD1c1rb69/CYf2MLPdn1GBwvsKdnu9yXNGai30b66jtJc0ZqLdRuoAlzRmo91G6gZJmjNR7qXdTAkzRmo80uaAH5ozTM0uaAH5ozTM0uaAHZpc0zNLmgB2aM03NGaAHZpc03NGaAHZopKM0AOopuaWgBc0ZpM0uaAClpKKAFopM0tABS0lFAC0UmaWgAooooAKKKKAFooooAKKSloAKKKKACiiigAooooAKKKKACiiigAopKKAFpKKSgBc0UmaTNAC5ozTc0maQDs0hNNzSFqAHk0m6oy1IWoAk3Um6oi1NL0CJS1IXqEyUwyUATl6QyVWMvvUZmHrQBbMleSfGDSvLmstchXn/Uykfmp/mK9NM/vWH4rsl1rw3fWRALNGWT2Ycisq0OeDRhXh7Sm4nh/mBwHHRhmk3VQsZiY3hfh4z0/nVkvXz86fLKx8tUpcsmiXdSFqiDFmCgEknAA716B4U8FoDHf6uv8AtR25/m3+FXSoSqOyNaGFnWlaJT8J+CLnW3W6vd0FiDnOMNJ9P8a9fsbW1020S1s4VhhQYCr/ADNU1uFVQq4VQMADoKcLkHvXt0MPGitNz6HDYWFBabmn5lHmVni4HrThN71udRe30u+qYl96eJPegC1vpd1Vw9ODUAT7qN1RBqUGmMlzS5qPNKDQBJmlzUeaXNAEmaM0zNLmgB+aXNMzS0AOzS5plLQA7NLTaWgBaXNNzS0ALS5ptLQAtLSUUALmlpKKAFzS0lFAC0UlLQAUtJRQAtFJS0AFFFFAC0UlLQAtFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFJRRQAUlFFABSZopKACkzRSGgAJpM0E000ABNNJpTTTSAQmmlqDTTQAhamF6GqJjQKwM9RNJQxqu5NAWHNNUD3GO9RSM1UZnftQFi292B3qpLqKL1cD8azLkzHOM1iXkFy+fvUCscZ4s0iHTNZOoWThrW4Y+YmeY2P9KxtzFtoGTXU6jo9xcIylGYHtXM3HhvVzN8ofb2rjq4RTlc4a2BjUlc6rw6unaawurh1luv4R1Cf/XrrU8Swt0avObLw3qPG8NXRWfh+4XG4muqFOMFaJ2U6UaceWKOsj11W6Grceqbqw7bR3XGTWrBp+3FWaGlHfFqspck1TitdtXI4cUAWUlJqdXNQJHU6rQBKrGpQaiUVKKAJAacDTBTxQA4GnA00U4UAOpRTRS0AOpaQUtAC0tJS0ALS0lLQAUtIKWgBaBRQKAFooooAWlpKBQAtFFFAC0UUUALRSUtABS0lLQAUUUUALRSUtABRRRQA6iiigBKWiigAooooAKKKKACiiigApM0UUAFJS0lABSGlpDQAUlFFACUlLSUANNIadSUANppp+KaRQAwimEVIRSEUAQsKjK1YK0wrQBVZKiaOrpSmGOgDPaEHtUL2wPatQx0wxe1IRkNZqe1RNYqf4RW0Yfak8n2pCMM2Cf3R+VNOnIf4B+VbvkD0pPIHpQBg/2Yn90U9dPA/hrc8gelHke1AGQtmB2qVbUDtWn5A9KcIfagDPW39qlWH2q6IvanCL2oAqLFipBHVkR04R0wIAlPC1KEpwSgZEFpwFSbKXbQAwClAp+2l20ANApcU7FLigY3FLilxS4oATFLilxRigBKXFLilxTASilxS4oASloxRQAUUtFABS0UUAFFFLQAUUUUAApaKKAClpKWgAooooAKWkpaACiiigB1FFFABRRRQAUUUUAFFFFABSUtFACUUUUAFJS0UAJRRRQAlJTqTFACUlOpMUANpMU6jFADMUmKfikxSAZikxT8UYoAj20m2pMUYoEQ7aQrU2KTbQBDspNlT7aTbQBBspPLqxto20AV/Lo8urG2jbQBX8ul8up9tG2gCDZS7Km20baAsRbKNlTbaNtAEWyl21JtpcUAR7aXbT8UYoAZtpcU/FGKAGYpcU7FGKAG4pcU7FGKAG4oxTsUYoATFGKdijFAxMUYpcUtADcUtLijFACYpaMUtMBKKWigBKWjFFABRS0UAFFFGKACjFLRQAUUUUAFFFLQAUUUUAFFFFADqKKKACiiigAooooAKKKKACiiigAooooATFFLRQAlFFFACUUtFACUlOpKQCYpMU7FFADcUmKdijFADcUmKfijFAhmKMU7FGKAG4pMU/FGKAGYoxT8UmKAG4pMU/FGKAGYoxT8UYoAZilxTsUYoAbijFOxS4oAZijFPxSYoAbijFPxRigBmKMU/FGKBjcUYp2KMUANxRinYoxQIbijFOxRigY3FGKdRigBMUYpaKAExRS0UAJRilooASlpaKAEopaKYCUYpaKAExS0UYoAKKKWgBMUUtFABRRRQAUUUUAFFLRQAmKWiigBaKKKAEooooAWiiigAooooAKKKKACiiigAooooAKKKKACiiigBKKKKADFFFFABRRRSATFGKKKADFFFFABijFFFABijFFFABiiiigAxRiiigAxRiiigAxRiiigAxRiiimAYoxRRQAUUUUAFFFFACUuKKKQCUtFFACYoxRRQAYoxRRQAUUUUALSUUUAFLRRQAUUUUAFFFFABRRRQAUUUUwCiiigAxRRRQAtFFFABRRRQAUUUUAf/9k="
}
in the client you can do this (in python for continuity)
import base64
myFile = open("mock.jpg", "wb")
img = base64.b64decode(value) #value is the returned string
myFile.write(img)
myFile.close()
Did you try converting the image to base64 string and send it as an argument of your request on the client side?
So you will be able to do that on the server side :
#strArg is the Base64 string sent from the client
img = base64.b64decode(strArg)
filename = 'someFileName.jpg'
with open(filename, 'wb') as f:
f.write(img)
#then you can save the file to your BlobStore
I'm trying to figure out how to display an image with the Google Cloud Storage client library on the dev server. I figured out how to upload a photo. But not sure how to get it to display while using Flask.
If I deploy to App Engine, I can get the images to display if I link directly to them. Like if I put the following in the template:
<img src="http://storage.googleapis.com/foo/{{ userid }}.jpg">
But that doesn't work on the dev server, because the uploads aren't actually going up to App Engine. So I was wondering how I can read the objects using the client library and display them in Flask.
The following code works for the POST (upload) part. But on the GET, I'm not sure where to go. To get the image to put on the template. I was hoping to just be able to use a URL.
BUCKET = 'foo'
def account():
if request.method == "GET":
#not sure how to get the image to then display on the template
filename = "/".join(['', BUCKET, g.user.key.id()])
gcs_file = gcs.open(filename)
return render_template('users/account.html')
else:
image = request.files.get('file', None)
filename = "/".join(['', BUCKET, g.user.key.id()])
write_retry_params = gcs.RetryParams(backoff_factor=1.1)
with gcs.open(filename, 'w', content_type=image.mimetype,
retry_params=write_retry_params) as imageFile:
image.save(imageFile)
return redirect(url_for("users.account"))
I think what you're looking for is get_serving_url, which now works with Google Cloud Storage. It should also work when using the dev server.
See this question for more information.
I'm currently resizing images on the fly when a user uploads a picture. The original picture is stored on Amazon S3 in a bucket called djangobucket. Inside this bucket, contains thousands of folders.
Each folder is named after the user. I don't have to worry about bucket creation or folder creation since all of that is handled from the client side.
Here is a diagram:
djangobucket ------------> bob ---------> picture1.jpg
picture2.jpg
picture3.jpg
picture4.jpg
As you can see, Bob has many pictures. Once a user uploads a picture to S3, I download it through Django via a URL, in this case it would be: http://s3.amazonaws.com/djangobucket/bob/picture1.jpg
I download the image and perform image processing and save the processed image on my django app server.
I would like to send this processed image back into bob's folder so that it can be publicly reached at http://s3.amazonaws.com/djangobucket/bob/picture1_processed.jpg
The client already has access to the amazon key and secret key so that he or she can upload pictures to this bucket. All users on the service use the same keys. I too will be using the same keys.
I've heard of something called Boto, but it involves Bucket creation and I'm unsure of how to do just the uploading part. I'm only concerned about uploading the picture to the appropriate user folder.
I've been researching this for hours so I've turned to the experts here.
Here is my code, just so that you can get a better understanding of what I'm doing.
user = 'bob'
url = 'http://s3.amazonaws.com/djangobucket/bob/picture1.jpg'
filename = url.split('/')[-1].split('.')[0]
download_photo = urllib.urlretrieve(url, "/home/ubuntu/Desktop/Resized_Images/%s.jpg" % (filename))
downloaded_photo = Image.open("/home/ubuntu/Desktop/Resized_Images/%s.jpg" % (filename))
resized_photo = downloaded_photo.resize((300, 300), Image.ANTIALIAS)
new_filename = filename + "_processed"
resized_photo.save("/home/ubuntu/Desktop/Resized_Images/%s.jpg" % (new_filename))
I would like to send the resized photo saved in /home/ubuntu/Desktop/Resized_Images/
to Bob's folder in the djangobucket on Amazon S3 and make it publicly visible.
Thanks for your help.
EDIT
I found this link: http://www.laurentluce.com/posts/upload-and-download-files-tofrom-amazon-s3-using-pythondjango/
Not quite how to use it for my case, but I think I'm on the right track.
boto is the best way to do this.
You can get an existing bucket using:
get_bucket(bucket_name, validate=True, headers=None)
After installing boto, this code should do what you need to do
from boto.s3.connection import S3Connection
from boto.s3.key import Key
conn = S3Connection('<aws access key>', '<aws secret key>')
bucket = conn.get_bucket('<bucket name>')
k = Key(bucket)
k.key = 'file_path_on_s3' # for example, 'images/bob/resized_image1.png'
k.set_contents_from_file(resized_photo)
Here are some links to the Boto API and Boto S3 Doc