I'm trying to make a request, in python, from a VM running Ryu controller to a VM running a openvswitch. I've tested said request, and it works when I execute it in a terminal (a string is supposed to be returned):
curl -X POST -d '{"priority": 500, "match": {"in_port": 3}}' http://localhost:8080/stats/flow/8796748823560
This was my first try in python:
import subprocess
proc = subprocess.run(['curl', '-X', 'POST', '-d', '\'{"priority": 500, "match": { "in_port": 3} }\'','http://localhost:8080/stats/flow/8796748823560'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
response = proc.stdout.decode('utf-8')
A simple POST, as you can see. However, the response is always " ", showing the error:
curl: (3) URL using bad/illegal format or missing URL
Then, I decided to use the requests python library and wrote the POST the following way:
import requests
data = '{ "priority": 500, "match": {"in_port": 3}}'
response = requests.post('http://localhost:8080/stats/flow/234', data=data)
However, I don't know where to put the option -X. In the library documentation, I cannot find the right place to put it, if there is any.
I need help to understand where I should place that -X option in the code and, if it is not possible, how could I execute that curl on python (I was trying to avoid the shell=True flag on subprocess, as I don't think it is safe).
The -X/--request in curl is an option that is followed by the HTTP verb to be used in the request. Since this is followed by POST it means that a POST request should be used. In fact -X POST is not needed since the mere presence of -d should cause curl to make a HTTP POST request.
Thus, use of request.post with data containing the body should be sufficient.
Related
I am very new to APIs (still learning) and I encountered a very weird issue with Python requests library when trying to initiate an OAuth Authentication flow with Client Credentials Grant Type.
For some reason, whenever I used my Python script (with the help of requests library) to send the HTTP request to the authentication endpoint, I always get
Response Status Code: 400
Response Body/Data returned: {"error":"unsupported_grant_type"}
However, if I tried using curl command line tool to send the request, I will get a successful response with status code 200 with the access token in the response body like this:
{'access_token': 'some access token',
'expires_in': 'num_of_seconds',
'token_type': 'Bearer'}
As a matter of fact, if I tried sending the request using Curl command line tool WITHIN my Python Script (with subprocess.Popen function), I can get the response with status code 200 and the access token with no problem.
Now, with that said, here's the Python script that I used to send the request to initiate the OAuth authentication flow:
import requests
import os
import base64
clientCredentialEndpoint = "https://base_url/path/token"
client_id = os.environ.get('CLIENT_ID')
client_secret = os.environ.get('CLIENT_SECRET')
# -- Encode the <client_id:client_secret> string to base64 --
auth_value = f'{client_id}:{client_secret}'
auth_value_bytes = auth_value.encode('ascii')
auth_value_b64 = base64.b64encode(auth_value_bytes).decode('ascii')
queryParams ={
'grant_type':'client_credentials',
'scope':'get_listings_data'
}
headers = {
'Authorization':f'Basic {auth_value_b64}',
'Content-Type':'application/x-www-form-urlencoded'
}
# send the post request to Authorisation server
response = requests.post(
clientCredentialEndpoint,
params=queryParams,
headers=headers,
)
print(response.status_code)
print(response.text)
whereas the curl command that I used (and worked) to send the request is:
curl -X POST -u '<client_id>:<client_secret>' \
-H "Content-Type: application/x-www-form-urlencoded" \
-d 'grant_type=client_credentials&scope=get_listings_data' \
'https://base_url/path/token'
Again, like I said, if I execute this curl command inside a Python script, it will successfully return the access token with no issue.
Does anyone know what I did wrong in my Python script which caused my request to always fail?
Thanks in advance!
My goodness me, I just realised that the -d in the curl command does not correspond to query params, it stands for 'data'.
Hence, I just need to change my Python script requests.post() a bit so that it looks like this:
response = requests.post(
clientCredentialEndpoint,
data=queryParams,
headers=headers,
)
Hope this helps others.
After writing a file with the snippet below
with open("temp.trig", "wb") as f:
f.write(data)
I use curl to load it into the server
curl -X POST -F file=#"temp.trig" -H "Accept: application/json" http://localhost:8081/demo/upload
which works fine.
I am trying to replace the curl with python requests, as follows:
with open("temp.trig", "rb") as f:
result = requests.post("http://localhost:8081/demo/upload", files={'file': f},
headers = {"Accept": "application/json"})
which attempted to follow the curl as closely as possible. This code results in an error 500 from the server. I suspect it must be something related to the request, because the same server is ok via `curl. Any ideas?
There probably is nothing wrong with your python script.
Differences I've noticed between curl and requests are the following:
obviously, User-Agent headers are different — curl/7.47.0 vs. python-requests/2.22.0
multipart boundary format in Content-Type header is different — ------------------------6debaa3504bbc177 in curl vs. c1e9f4f617de4d0dbdb48fcc5aab67e0 in requests
therefore Content-Length value will almost certainly be different
multipart/form-data format in body is slightly different — curl adds an extra line (Content-Type: text/plain) before file contents
So depending on your file format, server may not be able to parse requests HTTP request format.
I think the best solution for you now is to compare raw HTTP requests from curl and requests and find what differences are significant.
For example:
Open terminal
Launch netcat with nc -l -p 1234 command. This will listen to HTTP requests on localhost on port 1234 and output raw HTTP requests to terminal.
Send your curl request as it is to localhost:1234 in another tab
Execute your python script as it is using URL localhost:1234 in another tab
Compare raw requests from your netcat output
Here's my attempt:
import requests
headers = {
'Accept': 'application/json',
}
files = {
'file': ('temp.trig', open('temp.trig', 'rb')),
}
response = requests.post('http://localhost:8081/demo/upload', headers=headers, files=files)
In case this doesn't work we really need to read more data on the server side, as Ivan Vinogradov explained well.
I'm trying to post to a hipchat room via Python and the v2 API.
I can post without problems via curl in a shell script:
ROOM_ID=123
AUTH_TOKEN=123456789
MESSAGE="Hello World"
curl -H "Content-Type: application/json" \
-X POST \
-d "{ \"from\": \"GTM\",
\"notify\": \"true\",
\"color\": \"red\",
\"message_format\": \"text\",
\"message\": \"$MESSAGE\"
}" \
https://hipchat.server.domain.com/v2/room/$ROOM_ID/notification?auth_token=$AUTH_TOKEN
However sending a message with the same payload via Python fails. I have used ready-made clients as well as simple requests via various http modules, and examples like this:
https://gist.github.com/bdclark/4bc8ed06643e077fa620 (also of course I searched SO itself and tested examples like this one).
As a basic example I tried e.g. this:
host = 'hipchat.host.domain.com'
room = '123'
message = "Hello World"
headers = {'Content-type: application/json'}
color = "yellow"
format = "text"
notify=False
AUTH_TOKEN="123456789"
url = "https://{0}/v2/room/{1}/notification?auth_token={2}".format(host, room, AUTH_TOKEN)
h = httplib2.Http()
payload = {
'from':'FROM',
'message': message,
'notify': notify,
'message_format': format,
'color': color
}
resp, content = h.request(url,"POST",urllib.urlencode(payload),headers=headers)
httplib2 (and clients based on it) return a responseNotReady error. requests (and clients based on it) return a Connection reset by peer error.
Since Curl sends without problems it's probably not an issue with Hipchat itself. I assume that there might be an problem with my Python installation (this is the default 2.7 on MacOs Sierra). So the question would be, how do I find the underlying cause for the errors.
Any help much appreciated.
I hope I can explain myself. with out making an arse of myself.
I am trying to use python 3.4 to send a url to a sparkcore api.
I have managed to use curl direcly from the windows command line:-
curl https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyy -d params=l1,HIGH
All works fine. there is a space between the led and -d, but that is not a problem.
I have read that reting to do this within python using libcurl is a big pain and I saw lots of messaged about using Requests, so I though I would give it a go.
So I wrote a small routine:
import requests
r = requests.get('https://api.spark.io/v1/devices/xxxxxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyyy -d params=l1,HIGH')
print(r.url)
print(r)
I get as return:
<Response [400]>
When I examine the URL which actually got sent out the spaces in the URL are replaced with %20. This seems to be my actual problem, because the %20 being added by requests are confusing the server which fails
"code": 400,
"error": "invalid_request",
"error_description": "The access token was not found"
I have tried reading up on how to inpractice have the spaces with out having a %20 being added by the encoding, but I really could do with a pointer in the right direction.
Thanks
Liam
URLs cannot have spaces. The curl command you are using is actually making a request to the url https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led with some command line arguments (using -d)
The curl man (manual) page says this about the -d command line argument
-d, --data
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. To post data purely binary, you should instead use the --data-binary option. To URL-encode the value of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command line, the data pieces specified will be merged together with a separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data #foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out.
So that says -d is for sending data to the URL with the POST request using the content-type application/x-www-form-urlencoded
The requests documentation has a good example of how to do that using the requests library: http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests
So for your curl command, I think this should work
import requests
payload = {'access_token': 'yyyyyyyyyyyyyyyy', 'params': 'l1,HIGH'}
r = requests.post("https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led", data=payload)
print(r.text)
I am making a python build script for a phonegap project.
I need to open the ios key before i build
I am trying to do this with a http put request through the requests module for python.
If i do it with cURL from command line, it works fine
curl -vvv -d 'data={"password":"myPassWord"}' -X PUT https://build.phonegap.com/api/v1/keys/ios/193686?auth_token=passwordlesstokenphg
But from python like this.
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686', data=password_for_key, params=authentication_token)
It just returns the json you would recieve if you did a cURL without the data.
For me it seems like the data is not being sent to phonegap correctly.
API reference from build.phonegap.com
docs.build.phonegap.com/en_US/2.9.0/developer_api_write.md.html
Please help :)
So when you do
curl -d "..." -X PUT https://example.com
curl sends exactly what's in that string. requests does not translate so directly to curl. To do something similar in requests you need to do the following:
import json
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686',
data={'data': json.dumps(password_for_key)},
params=authentication_token)
What requests will do is build data={"password":"myPassword"} for you if you use the above. First you have to JSON encode the data in password_for_key then pass it in the dictionary to data.