I hope I can explain myself. with out making an arse of myself.
I am trying to use python 3.4 to send a url to a sparkcore api.
I have managed to use curl direcly from the windows command line:-
curl https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyy -d params=l1,HIGH
All works fine. there is a space between the led and -d, but that is not a problem.
I have read that reting to do this within python using libcurl is a big pain and I saw lots of messaged about using Requests, so I though I would give it a go.
So I wrote a small routine:
import requests
r = requests.get('https://api.spark.io/v1/devices/xxxxxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyyy -d params=l1,HIGH')
print(r.url)
print(r)
I get as return:
<Response [400]>
When I examine the URL which actually got sent out the spaces in the URL are replaced with %20. This seems to be my actual problem, because the %20 being added by requests are confusing the server which fails
"code": 400,
"error": "invalid_request",
"error_description": "The access token was not found"
I have tried reading up on how to inpractice have the spaces with out having a %20 being added by the encoding, but I really could do with a pointer in the right direction.
Thanks
Liam
URLs cannot have spaces. The curl command you are using is actually making a request to the url https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led with some command line arguments (using -d)
The curl man (manual) page says this about the -d command line argument
-d, --data
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. To post data purely binary, you should instead use the --data-binary option. To URL-encode the value of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command line, the data pieces specified will be merged together with a separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data #foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out.
So that says -d is for sending data to the URL with the POST request using the content-type application/x-www-form-urlencoded
The requests documentation has a good example of how to do that using the requests library: http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests
So for your curl command, I think this should work
import requests
payload = {'access_token': 'yyyyyyyyyyyyyyyy', 'params': 'l1,HIGH'}
r = requests.post("https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led", data=payload)
print(r.text)
Related
I'm trying to make a request, in python, from a VM running Ryu controller to a VM running a openvswitch. I've tested said request, and it works when I execute it in a terminal (a string is supposed to be returned):
curl -X POST -d '{"priority": 500, "match": {"in_port": 3}}' http://localhost:8080/stats/flow/8796748823560
This was my first try in python:
import subprocess
proc = subprocess.run(['curl', '-X', 'POST', '-d', '\'{"priority": 500, "match": { "in_port": 3} }\'','http://localhost:8080/stats/flow/8796748823560'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
response = proc.stdout.decode('utf-8')
A simple POST, as you can see. However, the response is always " ", showing the error:
curl: (3) URL using bad/illegal format or missing URL
Then, I decided to use the requests python library and wrote the POST the following way:
import requests
data = '{ "priority": 500, "match": {"in_port": 3}}'
response = requests.post('http://localhost:8080/stats/flow/234', data=data)
However, I don't know where to put the option -X. In the library documentation, I cannot find the right place to put it, if there is any.
I need help to understand where I should place that -X option in the code and, if it is not possible, how could I execute that curl on python (I was trying to avoid the shell=True flag on subprocess, as I don't think it is safe).
The -X/--request in curl is an option that is followed by the HTTP verb to be used in the request. Since this is followed by POST it means that a POST request should be used. In fact -X POST is not needed since the mere presence of -d should cause curl to make a HTTP POST request.
Thus, use of request.post with data containing the body should be sufficient.
After writing a file with the snippet below
with open("temp.trig", "wb") as f:
f.write(data)
I use curl to load it into the server
curl -X POST -F file=#"temp.trig" -H "Accept: application/json" http://localhost:8081/demo/upload
which works fine.
I am trying to replace the curl with python requests, as follows:
with open("temp.trig", "rb") as f:
result = requests.post("http://localhost:8081/demo/upload", files={'file': f},
headers = {"Accept": "application/json"})
which attempted to follow the curl as closely as possible. This code results in an error 500 from the server. I suspect it must be something related to the request, because the same server is ok via `curl. Any ideas?
There probably is nothing wrong with your python script.
Differences I've noticed between curl and requests are the following:
obviously, User-Agent headers are different — curl/7.47.0 vs. python-requests/2.22.0
multipart boundary format in Content-Type header is different — ------------------------6debaa3504bbc177 in curl vs. c1e9f4f617de4d0dbdb48fcc5aab67e0 in requests
therefore Content-Length value will almost certainly be different
multipart/form-data format in body is slightly different — curl adds an extra line (Content-Type: text/plain) before file contents
So depending on your file format, server may not be able to parse requests HTTP request format.
I think the best solution for you now is to compare raw HTTP requests from curl and requests and find what differences are significant.
For example:
Open terminal
Launch netcat with nc -l -p 1234 command. This will listen to HTTP requests on localhost on port 1234 and output raw HTTP requests to terminal.
Send your curl request as it is to localhost:1234 in another tab
Execute your python script as it is using URL localhost:1234 in another tab
Compare raw requests from your netcat output
Here's my attempt:
import requests
headers = {
'Accept': 'application/json',
}
files = {
'file': ('temp.trig', open('temp.trig', 'rb')),
}
response = requests.post('http://localhost:8081/demo/upload', headers=headers, files=files)
In case this doesn't work we really need to read more data on the server side, as Ivan Vinogradov explained well.
Following instructions on this link using Lambda and API Gateway: https://sookocheff.com/post/api/uploading-large-payloads-through-api-gateway/ I have a setup that allows me to get a pre-signed URL and upload files. I've tested using CURL and it has worked.
But when I try to send raw string (csv format or json format) it fails!
Example of what works
curl --request PUT --upload-file Testing.csv "**pre signed upload url**"
Example of what doesn't work
curl --request PUT -H "Content-Type: text/plain" --data "this is raw data" "**pre signed upload url**"
curl --request PUT --data "this is raw data" "**pre signed upload url**"
Am I making the call incorrectly? Should I be switching to POST and what would the call look like then?
It is not becoz of self signed url, it is becoz of content type with the API Gateway set to,
consumes:
- application/json
produces:
- application/json
You add additional content types, it should make it through.
Hope it helps.
So the solution was specifying the content-type during the pre-signed url generation and then the same one in the CURL put command. Figured out thanks to answer here: S3 PUT doesn't work with pre-signed URL in javascript and pointer from #Kannaiyan in the right direction regarding content-types
I was trying Django JWT Auth and noticed that the URL responds well to one type of post but doesn't respond well to another, but i can figure out why.
Basically, if i use the cURL POST referred in the readme.md, everything goes accordingly to planned:
$ curl -X POST -H "Content-Type: application/json" -d '{"username":"admin","password":"abc123"}' http://localhost:8000/api-token-auth/
but if you i use another type of cURL POST with the same info, it doesn't work:
$ curl -d 'username=admin&password=abc123' http://localhost:8000/api-token-auth/
I know that the "Content-Type" is diferent, but shouldn't the request be accepted in the same manner, they are both well formed posts?
Curl's -d option actually sends the request like it's a web browser. My guess is that the URL you're testing against doesn't have a standard web form, so it can't actually process the request.
TL;DR Pretty sure Django JWT Auth doesn't support the application/x-www-form-urlencoded content type.
From curl manual:
-d --data
(HTTP) Sends the specified data in a POST request to the HTTP
server, in the same way that a browser does when a user has
filled in an HTML form and presses the submit button. This will
cause curl to pass the data to the server using the content-type
application/x-www-form-urlencoded. Compare to -F, --form.
Hope this helps!
I am making a python build script for a phonegap project.
I need to open the ios key before i build
I am trying to do this with a http put request through the requests module for python.
If i do it with cURL from command line, it works fine
curl -vvv -d 'data={"password":"myPassWord"}' -X PUT https://build.phonegap.com/api/v1/keys/ios/193686?auth_token=passwordlesstokenphg
But from python like this.
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686', data=password_for_key, params=authentication_token)
It just returns the json you would recieve if you did a cURL without the data.
For me it seems like the data is not being sent to phonegap correctly.
API reference from build.phonegap.com
docs.build.phonegap.com/en_US/2.9.0/developer_api_write.md.html
Please help :)
So when you do
curl -d "..." -X PUT https://example.com
curl sends exactly what's in that string. requests does not translate so directly to curl. To do something similar in requests you need to do the following:
import json
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686',
data={'data': json.dumps(password_for_key)},
params=authentication_token)
What requests will do is build data={"password":"myPassword"} for you if you use the above. First you have to JSON encode the data in password_for_key then pass it in the dictionary to data.