I am reading the documentation of passing a querystring to Amazon's S3 for authentication, and can't seem to grok how exactly the StringToSign is created and used. I am looking for a concrete example to illustrate (1) how to construct the StringToSign, and (2) once I have the signature, how to call the form.
For example's sake, let's say the following is my information:
Content-type='image/jpeg'
Bucket='test-bucket'
Key = 'filename'
ACL = 'public-read'
Expiration = '(never expires)'
Access Key = '12345'
Secret Password = '1a2b3c'
File = <file the user uploads>
How would I get the StringToSign value from this? And once I have that, how would I create the following form:
<form action="??" method="post" enctype='multipart/form-data' class="upload-form">
<input name="file" type="file">
</form>
And for reference: http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?RESTAuthentication.html#RESTAuthenticationQueryStringAuth. Thank you.
Based on what you described, it seems like you want to support browser-based uploads using POST. There is a section of the AWS documentation which talks about this.
As an overview keep in mind you'll either have to make your bucket publically writeable or include a policy document. I'll assume you'll be including a policy document (check the docs if you don't want to):
A policy document is just a fragment of JSON that is used to authenticate the request, and gives a bunch of conditions that must be met before data is uploaded. E.g:
"expiration": "2020-12-01T12:00:00.000Z",
"conditions": [
{"acl": "public-read" },
{"bucket": "test-bucket" },
["eq", "$key", "filename"],
]
}
This says the action to upload will be allowed until 2020, given that the bucket is only publically readable, the bucket name is 'test-bucket' and the key is exactly equal to 'filename'.
Now, to construct your signature you take the above JSON doc, UTF-8 encode it and then base64 that and then sign the whole thing using your secret access key (using hmac sha1) and finally base64 that whole thing
policy_data = ... # stuff above
enc_policy = base64.b64_encode(policy_data.encode('utf8'))
signed = base64.b64_encode(hmac.new(AWS_SECRET, enc_policy, hashlib.sha1))
Then finally, your form would look something like this:
<form action="http://test-bucket.s3.amazonaws.com/" method="post" enctype="multipart/form-data">
Key to upload: <input type="input" name="key" value="filename" /><br />
<input type="hidden" name="acl" value="public-read" />
<input type="hidden" name="success_action_redirect" value="http://test-bucket.s3.amazonaws.com/successful_upload.html" />
Content-Type: <input type="input" name="Content-Type" value="image/jpeg" /><br />
<input type="hidden" name="AWSAccessKeyId" value="YOUR_ACCESS_KEY_ID" />
<input type="hidden" name="Policy" value="<enc_policy from above>" />
<input type="hidden" name="Signature" value="<signed from above>" />
File: <input type="file" name="file" /> <br />
<!-- The elements after this will be ignored -->
<input type="submit" name="submit" value="Upload to Amazon S3" />
</form>
Related
I'm using django-paypal and paypalrestsdk to integrate Paypal payment & subscriptions to my website.
I've looked over django-paypal and other modules but I wasn't able to fully understand the process of handling a webhook.
I'm getting a 405 Error in my console when I'm completing a payment.
I have successfully created a paypal sandbox account for testing purposes ( two user accounts were automatically created on it for tests ).
In my settings.py:
PAYPAL_RECEIVER_EMAIL = "the email"
PAYPAL_IDENTITY_TOKEN = "_BB3dqp-crOrUo2uh84g0zN2alX0LwWPAT85r0g-2Eo0"
In my index.html:
<form action="https://www.sandbox.paypal.com/cgi-bin/webscr" method="post">
<input type="hidden" name="business" value="my_email" id="id_business" />
<input type="hidden" name="amount" value="1" id="id_amount" />
<input type="hidden" name="item_name" value="Subscription Package" id="id_item_name" />
<input type="hidden" name="notify_url" value="website/page" id="id_notify_url" />
<input type="hidden" name="cancel_return" value="website/page" id="id_cancel_return" />
<input type="hidden" name="return" value="website/page" id="id_return_url" />
<input type="hidden" name="invoice" value="UID" id="id_invoice" />
<input type="hidden" name="cmd" value="_xclick" id="id_cmd" />
<input type="hidden" name="charset" value="utf-8" id="id_charset" />
<input type="hidden" name="currency_code" value="USD" id="id_currency_code" />
<input type="hidden" name="no_shipping" value="1" id="id_no_shipping" />
<input type="image" src="https://www.sandbox.paypal.com/en_US/i/btn/btn_buynowCC_LG.gif" border="0" name="submit" alt="Buy it Now" />
</form>
And my views.py:
class PayPalWebhook(View):
#staticmethod
def post(request):
event_json = json.loads(request.body)
print '=========='
print event_json.type
print '=========='
print event_json
return HttpResponse(status=200)
class PaypalAPI(View):
#staticmethod
def post(request):
pass
shall I pass anything in my PaypalAPI class in order to get my form working as it should ? ( for the moment it correctly sends the payment but it uses the values inside the form parameters and that's not what I want)
how can I get rid of that 405 error ? What am I doing wrong ?( as a mention, I created a webhook in dashboard)
I would just like to see the webhooks in my console, that's all.
Hope this helps: 405 is usually when the API/Endpoint you are calling is used with an incorrect method. For example using a POST for a HTTP Call which only supports GET or not using the correct endpoint.
Python SDKs are a good point to start with as well https://github.com/paypal/PayPal-Python-SDK
Also, its a good practice to not share any tokens in forums/QnA sites. :)
You need to add
from django.views.decorators.csrf import csrf_exempt
#csrf_exempt
def PayPalReturnView(request):
# Do something here
to the view that PayPal posts too. I think I got your problem right but anyway try adding more info: your urls for example which are crucial here.
My python app lets people upload files on a Third-party platform.
They need to ask my server for a token for the platform to verify
It looks like:
when people upload files they post to the platform
<form method="post" action="http://upload.qiniu.com/"
enctype="multipart/form-data">
<input name="key" type="hidden" value="<resource_key>">
<input name="x:<custom_name>" type="hidden" value="<custom_value>">
<input name="token" type="hidden" value="<upload_token>">
<input name="file" type="file" />
<input name="crc32" type="hidden" />
<input name="accept" type="hidden" />
</form>
I wonder if it's ok to ask for the token each time someone tries to upload files, or just let people get the token when they login, set the token into the cookie and make its lifetime as long as the cookie's?
Thanks.
It is normal practice to store client access tokens as cookies for returning users.
This and also this SO post also back this idea up.
This is my form
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<form action="http://mybucket.s3.amazonaws.com/" method="post" enctype="multipart/form-data">
Key to upload: <input type="input" name="key" value="user/eric/" /><br />
<input type="hidden" name="acl" value="public-read" />
<input type="hidden" name="AWSAccessKeyId" value="myAWSId" />
<input type="hidden" name="Policy" value="Base64EncodingOfPolicy"/>
<input type="hidden" name="Signature" value="Signature Calculated as urlencode(base64(HMAC-SHA1(secret, policy base64 encoded string same as utf-8 encoded)))" />
File: <input type="file" name="file" /> <br />
<!-- The elements after this will be ignored -->
<input type="submit" name="submit" value="Upload to Amazon S3" />
</form>
</html>
This is my policy
{ "expiration": "2014-12-01T12:00:00.000Z",
"conditions": [
{"acl": "public-read" },
{"bucket": "mybucket" },
["starts-with", "$key", "user/eric/"],
]
}
This is the base64 encoded policy
eyAiZXhwaXJhdGlvbiI6ICIyMDE0LTEyLTAxVDEyOjAwOjAwLjAwMFoiLA0KDQogICJjb25kaXRp
b25zIjogWw0KDQogICAgeyJhY2wiOiAicHVibGljLXJlYWQiIH0sDQoNCiAgICB7ImJ1Y2tldCI6
ICJoYWJpdHN1c2VybWVkaWEiIH0sDQoNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNl
ci9lcmljLyJdLA0KDQogIF0NCg0KfQ==
I have tried using the base64 encoded string with and without the newlines. Should I keep something specific in mind when using the above encoding and newlines?
I keep getting the SignatureDoesNotMatch error even after several attempts to account for all possible permutations.
I also used the signature verification tool and checked what AWS accepts and it matches. http://aws.amazon.com/code/199
How do I go about debugging this? If you have successfully POSTed using the REST API to S3 can you share the snippet?
Ok I was able to solve this after a while.
This is my python code to generate the signature
import base64
import hmac
from hashlib import sha1
import urllib
input = open("policy.txt", "rb")
policy = input.read()
policy_encoded = base64.b64encode(policy).encode("UTF-8")
secret = "<my_aws_secret>"
print 'Encoded Policy %s' %(policy_encoded)
hashed = hmac.new(secret,policy_encoded, sha1)
#This is the required value
signature = base64.b64encode(hashed.digest())
#This is not required for a HTTP POST form based request, only when it has to be passed in urlencoded format
signature_urlencoded = urllib.quote_plus(base64.b64encode(hashed.digest()))
print 'Signature urlencoded %s' %(signature)
My policy.txt is
{ "expiration": "2014-12-01T12:00:00.000Z",
"conditions": [
{"acl": "public-read" },
{"bucket": "<mybucket>" },
{"success_action_status" : "201"},
["starts-with", "$key", "uploads/"],
]
}
And my form looks like
<form action="http://habitsusermedia.s3.amazonaws.com/" method="post" enctype="multipart/form-data">
<input type="hidden" name="acl" value="public-read" />
<input type="input" name="key" value="uploads/${filename}" />
<input type="hidden" name="success_action_status" value="201" />
<input type="hidden" name="AWSAccessKeyId" value="<aws_access_key>" />
<input type="hidden" name="Policy" value="<policy_encoded_as_base64>"/>
<input type="hidden" name="Signature" value="<signature>" />
File: <input type="file" name="file" /> <br />
<!-- The elements after this will be ignored -->
<input type="submit" name="submit" value="Upload to Amazon S3" />
</form>
Make sure your AWS bucket is configured to accept POSTS, PUT from any domain. Use the policy generating tool for that.
There is a helpful article explaining these concepts on AWS
I have the following template form, containing several variables.
<form action="https://me.s3.amazonaws.com/" method="post" enctype='multipart/form-data' class="upload-form">
<input type="hidden" name="key" value="videos/{{filename}}">
<input type="hidden" name="AWSAccessKeyId" value="{{access_key}}">
<input type="hidden" name="acl" value="public-read">
<input type="hidden" name="policy" value="{{policy}}">
<input type="hidden" name="signature" value="{{signature}}">
<input type="hidden" name="Content-Type" value="{{content_type}}">
<input name="file" type="file">
<input type="submit" value="Upload" name="upload">
</form>
However, as soon as the submit button is hit, the form is sent to amazon, and I'm not able to pass it variables. This is what I've been trying to do, unsuccessfully --
if 'upload' in request.POST:
policy = base64.b64encode(...)
signature = base64.b64encode(
hmac.new('secret_key', policy, sha).digest())
file = request.POST['files']
filename=file.name
content_type=mimetypes.guess_type(filename)[0]
What do I need to do to pass the variables to the form after the POST request but BEFORE amazon processes the form? Thank you.
You should change your form's action to your django view and in your view you can re-post to https://me.s3.amazonaws.com/:
In your template
<form action="http://mywebsite/upload" method="post" ...
In your view.py:
def upload(request):
# Your treatment here.
# Post the data to amazon S3.
urllib2.urlopen("https://me.s3.amazonaws.com/", your_data)
...
You could change the form to POST to one of your own views, then do your post-processing in your view, and then within your view code, issue a POST to Amazon with the correct values using, say, urllib2 or similar.
I'm trying to upload multiple files in a form to the BlobStore.
Form:
<form action="{{upload_url}}" method="POST" enctype="multipart/form-data">
<label>Key Name</label><input type="text" name="key_name" size="50"><br/>
<label>name</label><input type="text" name="name" size="50"><br/>
<label>image</label><input type="file" name="image" size="50"><br/>
<label>thumb</label><input type="file" name="thumb" size="50"><br/>
<input type="submit" name="submit" value="Submit">
</form>
I'm then trying to fetch the BlobInfo objects for each of those files uploaded:
def post(self):
image_upload_files = self.get_uploads('image')
thumb_upload_files = self.get_uploads('thumb')
image_blob_info = image_upload_files[0]
thumb_blob_info = thumb_upload_files[0]
I'm seeing some weird behavior. Both files are making it into the BlobStore, but I cannot figure out how to get the Keys so that I can store those on another Entity. The code above manages to get the key for image_blob_info, but not thumb_blob_info. I don't understand how to use get_uploads. I want to pass multiple files through the form and then fetch them by name so I can store them in the appropriate BlobReferenceProperties on another Entity.
Each file needs its own unique upload url, so my guess is something wacky is happening when all three files are posted to the same url.
The best solution for supporting multiple file uploads is described in Nick Johnson's blog post:
http://blog.notdot.net/2010/04/Implementing-a-dropbox-service-with-the-Blobstore-API-part-3-Multiple-upload-support
You could post the files to the same name, followed by [], which will post an array:
<form action="{{upload_url}}" method="POST" enctype="multipart/form-data">
<label>Key Name</label><input type="text" name="key_name" size="50"><br/>
<label>name</label><input type="text" name="files[]" size="50"><br/>
<label>image</label><input type="file" name="files[]" size="50"><br/>
<label>thumb</label><input type="file" name="thumb" size="50"><br/>
<input type="submit" name="submit" value="Submit">
</form>
Then in your form handler, you can something like this (depending on your web framework):
for uploaded_file in request.FILES.getlist('files'):
#do something with uploaded_file
Using the latest version of plupload, I was able to get the UploadQueue to work with GAE with this bit of code. Note, it is CoffeeScript, but should be easy to convert back to JavaScript if you really need to. It assumes you get a bit of json back from your server as {url:"gae generated url"}
$("#fileUploader").pluploadQueue
runtimes : 'html5,html4'
use_query_string : false
max_file_size : '3mb'
multipart: true
unique_names : true
multiple_queues : true
filters : [{title : "Image files", extensions : "jpg,gif,png"}]
preinit:
UploadFile: (up, file) ->
$.ajax
url: '/api/upload/url'
async: false
success: (data) ->
up.settings.url = data.url