Curl -d : Contents of the file must be URL Encoded - python

I have an app running in Python using Flask.
The endpoint of the API looks like this:
#app.route('/postIt', methods =['POST'])
def postReview():
#print flask.request
if flask.request.method == 'POST':
posts = flask.request.get_json()
print posts
return str(posts)
I am trying to send it request using CURL:
curl http://127.0.0.1:5000/postIt -d #post.json -H "Content-Type: application/json"
where post.json looks like this:
{"post1":"3", "post2": "2", "post3":"3", "post4":"4" "post5": "5"}
This is not working well. Output on server side prints nothing implying that it is unable to get the json file that I am sending.
I looked into -d flag of CURL and found this thing:
The contents of the file must already be URL-encoded.
So I am guessing there must be encoding issues with my post.json file. I don't exactly know which encoding should be used here. Please help!!

When I tried your code, I got 400 Bad Request:
!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>The browser (or proxy) sent a request that this server could not understand.</p>
Then I found that your json file is actually not valid. There is a missing , between post4 and post5. After fixing it, I got the correct response:
{u'post5': u'5', u'post4': u'4', u'post3': u'3', u'post2': u'2', u'post1': u'3'}%

Related

Can't upload raw string (csv or json) using S3 pre-signed url

Following instructions on this link using Lambda and API Gateway: https://sookocheff.com/post/api/uploading-large-payloads-through-api-gateway/ I have a setup that allows me to get a pre-signed URL and upload files. I've tested using CURL and it has worked.
But when I try to send raw string (csv format or json format) it fails!
Example of what works
curl --request PUT --upload-file Testing.csv "**pre signed upload url**"
Example of what doesn't work
curl --request PUT -H "Content-Type: text/plain" --data "this is raw data" "**pre signed upload url**"
curl --request PUT --data "this is raw data" "**pre signed upload url**"
Am I making the call incorrectly? Should I be switching to POST and what would the call look like then?
It is not becoz of self signed url, it is becoz of content type with the API Gateway set to,
consumes:
- application/json
produces:
- application/json
You add additional content types, it should make it through.
Hope it helps.
So the solution was specifying the content-type during the pre-signed url generation and then the same one in the CURL put command. Figured out thanks to answer here: S3 PUT doesn't work with pre-signed URL in javascript and pointer from #Kannaiyan in the right direction regarding content-types

How to read file sent from curl to Flask?

Flask code -
#app.route('/messages', methods = ['POST'])
def api_message():
if request.headers['Content-Type'] == 'text/plain':
return "Text Message: " + request.data
elif request.headers['Content-Type'] == 'application/json':
f = open(filename,'r')
l = f.readlines()
f.close()
return len(l)
On running, I get error as -
curl -H "Content-Type:application/json" -X POST http://127.0.0.1:5000/messages --data filename=#hello.json
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
Am I accessing the curl param wrong (filename)? Or I am sending the file in wrong way?
Also Upload a file to a python flask server using curl
Tried doing
f = request.files['filename']
Still, same error.
What your curl command code is doing is reading the file hello.json and putting it in the body of the request. (This feature is actually very useful if you have a large chunk of JSON you need to send to the server).
Normally in application/json requests you send the JSON as the body of the request, so this may be what you want. You can use request.get_json to get this data as a Python dictionary.
If you want to upload an actual file - like uploading a picture - you want multi part form encoding, which you tell curl to send via the -F parameter. (See also: an SO answer about this: https://stackoverflow.com/a/12667839/224334 ).

Django JWT Auth, why one request works and the other not

I was trying Django JWT Auth and noticed that the URL responds well to one type of post but doesn't respond well to another, but i can figure out why.
Basically, if i use the cURL POST referred in the readme.md, everything goes accordingly to planned:
$ curl -X POST -H "Content-Type: application/json" -d '{"username":"admin","password":"abc123"}' http://localhost:8000/api-token-auth/
but if you i use another type of cURL POST with the same info, it doesn't work:
$ curl -d 'username=admin&password=abc123' http://localhost:8000/api-token-auth/
I know that the "Content-Type" is diferent, but shouldn't the request be accepted in the same manner, they are both well formed posts?
Curl's -d option actually sends the request like it's a web browser. My guess is that the URL you're testing against doesn't have a standard web form, so it can't actually process the request.
TL;DR Pretty sure Django JWT Auth doesn't support the application/x-www-form-urlencoded content type.
From curl manual:
-d --data
(HTTP) Sends the specified data in a POST request to the HTTP
server, in the same way that a browser does when a user has
filled in an HTML form and presses the submit button. This will
cause curl to pass the data to the server using the content-type
application/x-www-form-urlencoded. Compare to -F, --form.
Hope this helps!

An example of POST request in Flask API gives "Method Not Allowed"

I'm using FLASK API and I want to use POST requests.
I want just to do an example with POST requests that will return something, I keep getting an error message "Method Not Allowed".
I want to give a parameter(e.g query_params = 'name1' ) to search for a user and to return a JSON, actually I don't know where to give this parameter and I don't understand why I'm getting that message.
Here I did a simple route:
#mod_api.route('/show-user', methods=['POST'])
def show_user():
query_params = 'name1'
query = {query_params: 'Myname' }
json_resp = mongo.db.coordinates.find(query)
return Response(response=json_util.dumps(json_resp), status=200, mimetype='application/json')
Any help please?
The likely reason is that you are probably not doing a POST request against the route, which only accepts POST requests. Here is a simplified example with the mongodb details removed to illustrate this.
from flask import Flask
app = Flask(__name__)
#app.route('/show-user', methods=('POST',))
def show_user():
return "name info"
if __name__ == "__main__":
app.run(debug=True)
Now if we do a POST request it works, but if we do A GET request it raises the error you saw:
curl -H "Content-Type: application/json" -X POST -d '{}' http://127.0.0.1:5000/show-user
name info
curl -H "Content-Type: application/json" -X GET http://127.0.0.1:5000/show-user
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method is not allowed for the requested URL.</p>

Spaces in a URL when using requests and python

I hope I can explain myself. with out making an arse of myself.
I am trying to use python 3.4 to send a url to a sparkcore api.
I have managed to use curl direcly from the windows command line:-
curl https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyy -d params=l1,HIGH
All works fine. there is a space between the led and -d, but that is not a problem.
I have read that reting to do this within python using libcurl is a big pain and I saw lots of messaged about using Requests, so I though I would give it a go.
So I wrote a small routine:
import requests
r = requests.get('https://api.spark.io/v1/devices/xxxxxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyyy -d params=l1,HIGH')
print(r.url)
print(r)
I get as return:
<Response [400]>
When I examine the URL which actually got sent out the spaces in the URL are replaced with %20. This seems to be my actual problem, because the %20 being added by requests are confusing the server which fails
"code": 400,
"error": "invalid_request",
"error_description": "The access token was not found"
I have tried reading up on how to inpractice have the spaces with out having a %20 being added by the encoding, but I really could do with a pointer in the right direction.
Thanks
Liam
URLs cannot have spaces. The curl command you are using is actually making a request to the url https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led with some command line arguments (using -d)
The curl man (manual) page says this about the -d command line argument
-d, --data
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. To post data purely binary, you should instead use the --data-binary option. To URL-encode the value of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command line, the data pieces specified will be merged together with a separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data #foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out.
So that says -d is for sending data to the URL with the POST request using the content-type application/x-www-form-urlencoded
The requests documentation has a good example of how to do that using the requests library: http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests
So for your curl command, I think this should work
import requests
payload = {'access_token': 'yyyyyyyyyyyyyyyy', 'params': 'l1,HIGH'}
r = requests.post("https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led", data=payload)
print(r.text)

Categories