The following curl command works perfectly (private data anonymized):
curl -X POST 'https://api.twilio.com/2010-04-01/Accounts/abc/SMS/Messages.json' \
-d 'From=%2B14155551234' \
-d 'To=%2B17035551212' \
-d 'Body=This+is+a+test' \
-u foo:bar
How can I send out this exact same HTTPS POST request in the proper Python3.3 way? I don't want to have to use anything other than Python 3.3's standard library if I can avoid it (in other words, not using the twilio python module, or "requests", or pycurl, or anything outside the plain vanilla Python 3.3 installation).
The preferred Python approach seems to keep evolving from version to version, the snippets I find by Googling never mention which version they're using or don't do the login part, the Python docs are full of "deprecated since 3.x" but never include code examples of the new way to do things....
If curl can do this so easily, so can standard Python 3.3. But how exactly is this supposed to be done now?
Here's a version that works both on Python 2 and 3:
import requests # pip install requests
url = 'https://api.twilio.com/2010-04-01/Accounts/abc/SMS/Messages.json'
r = requests.post(url, dict(
From='+17035551212',
To='+17035551212',
Body='This is a test'), auth=('foo', 'bar'))
print(r.headers)
print(r.text) # or r.json()
To make https post request with basic http authentication on Python 3.3:
from base64 import b64encode
from urllib.parse import urlencode
from urllib.request import Request, urlopen
user, password = 'foo', 'bar'
url = 'https://api.twilio.com/2010-04-01/Accounts/abc/SMS/Messages.json'
data = urlencode(dict(From='+17035551212', To='+17035551212',
Body='This is a test')).encode('ascii')
headers = {'Authorization': b'Basic ' +
b64encode((user + ':' + password).encode('utf-8'))}
cafile = 'cacert.pem' # http://curl.haxx.se/ca/cacert.pem
response = urlopen(Request(url, data, headers), cafile=cafile)
print(response.info())
print(response.read().decode()) # assume utf-8 (likely for application/json)
Related
I am very new to APIs (still learning) and I encountered a very weird issue with Python requests library when trying to initiate an OAuth Authentication flow with Client Credentials Grant Type.
For some reason, whenever I used my Python script (with the help of requests library) to send the HTTP request to the authentication endpoint, I always get
Response Status Code: 400
Response Body/Data returned: {"error":"unsupported_grant_type"}
However, if I tried using curl command line tool to send the request, I will get a successful response with status code 200 with the access token in the response body like this:
{'access_token': 'some access token',
'expires_in': 'num_of_seconds',
'token_type': 'Bearer'}
As a matter of fact, if I tried sending the request using Curl command line tool WITHIN my Python Script (with subprocess.Popen function), I can get the response with status code 200 and the access token with no problem.
Now, with that said, here's the Python script that I used to send the request to initiate the OAuth authentication flow:
import requests
import os
import base64
clientCredentialEndpoint = "https://base_url/path/token"
client_id = os.environ.get('CLIENT_ID')
client_secret = os.environ.get('CLIENT_SECRET')
# -- Encode the <client_id:client_secret> string to base64 --
auth_value = f'{client_id}:{client_secret}'
auth_value_bytes = auth_value.encode('ascii')
auth_value_b64 = base64.b64encode(auth_value_bytes).decode('ascii')
queryParams ={
'grant_type':'client_credentials',
'scope':'get_listings_data'
}
headers = {
'Authorization':f'Basic {auth_value_b64}',
'Content-Type':'application/x-www-form-urlencoded'
}
# send the post request to Authorisation server
response = requests.post(
clientCredentialEndpoint,
params=queryParams,
headers=headers,
)
print(response.status_code)
print(response.text)
whereas the curl command that I used (and worked) to send the request is:
curl -X POST -u '<client_id>:<client_secret>' \
-H "Content-Type: application/x-www-form-urlencoded" \
-d 'grant_type=client_credentials&scope=get_listings_data' \
'https://base_url/path/token'
Again, like I said, if I execute this curl command inside a Python script, it will successfully return the access token with no issue.
Does anyone know what I did wrong in my Python script which caused my request to always fail?
Thanks in advance!
My goodness me, I just realised that the -d in the curl command does not correspond to query params, it stands for 'data'.
Hence, I just need to change my Python script requests.post() a bit so that it looks like this:
response = requests.post(
clientCredentialEndpoint,
data=queryParams,
headers=headers,
)
Hope this helps others.
After writing a file with the snippet below
with open("temp.trig", "wb") as f:
f.write(data)
I use curl to load it into the server
curl -X POST -F file=#"temp.trig" -H "Accept: application/json" http://localhost:8081/demo/upload
which works fine.
I am trying to replace the curl with python requests, as follows:
with open("temp.trig", "rb") as f:
result = requests.post("http://localhost:8081/demo/upload", files={'file': f},
headers = {"Accept": "application/json"})
which attempted to follow the curl as closely as possible. This code results in an error 500 from the server. I suspect it must be something related to the request, because the same server is ok via `curl. Any ideas?
There probably is nothing wrong with your python script.
Differences I've noticed between curl and requests are the following:
obviously, User-Agent headers are different — curl/7.47.0 vs. python-requests/2.22.0
multipart boundary format in Content-Type header is different — ------------------------6debaa3504bbc177 in curl vs. c1e9f4f617de4d0dbdb48fcc5aab67e0 in requests
therefore Content-Length value will almost certainly be different
multipart/form-data format in body is slightly different — curl adds an extra line (Content-Type: text/plain) before file contents
So depending on your file format, server may not be able to parse requests HTTP request format.
I think the best solution for you now is to compare raw HTTP requests from curl and requests and find what differences are significant.
For example:
Open terminal
Launch netcat with nc -l -p 1234 command. This will listen to HTTP requests on localhost on port 1234 and output raw HTTP requests to terminal.
Send your curl request as it is to localhost:1234 in another tab
Execute your python script as it is using URL localhost:1234 in another tab
Compare raw requests from your netcat output
Here's my attempt:
import requests
headers = {
'Accept': 'application/json',
}
files = {
'file': ('temp.trig', open('temp.trig', 'rb')),
}
response = requests.post('http://localhost:8081/demo/upload', headers=headers, files=files)
In case this doesn't work we really need to read more data on the server side, as Ivan Vinogradov explained well.
I'm trying to post to a hipchat room via Python and the v2 API.
I can post without problems via curl in a shell script:
ROOM_ID=123
AUTH_TOKEN=123456789
MESSAGE="Hello World"
curl -H "Content-Type: application/json" \
-X POST \
-d "{ \"from\": \"GTM\",
\"notify\": \"true\",
\"color\": \"red\",
\"message_format\": \"text\",
\"message\": \"$MESSAGE\"
}" \
https://hipchat.server.domain.com/v2/room/$ROOM_ID/notification?auth_token=$AUTH_TOKEN
However sending a message with the same payload via Python fails. I have used ready-made clients as well as simple requests via various http modules, and examples like this:
https://gist.github.com/bdclark/4bc8ed06643e077fa620 (also of course I searched SO itself and tested examples like this one).
As a basic example I tried e.g. this:
host = 'hipchat.host.domain.com'
room = '123'
message = "Hello World"
headers = {'Content-type: application/json'}
color = "yellow"
format = "text"
notify=False
AUTH_TOKEN="123456789"
url = "https://{0}/v2/room/{1}/notification?auth_token={2}".format(host, room, AUTH_TOKEN)
h = httplib2.Http()
payload = {
'from':'FROM',
'message': message,
'notify': notify,
'message_format': format,
'color': color
}
resp, content = h.request(url,"POST",urllib.urlencode(payload),headers=headers)
httplib2 (and clients based on it) return a responseNotReady error. requests (and clients based on it) return a Connection reset by peer error.
Since Curl sends without problems it's probably not an issue with Hipchat itself. I assume that there might be an problem with my Python installation (this is the default 2.7 on MacOs Sierra). So the question would be, how do I find the underlying cause for the errors.
Any help much appreciated.
Could I anyone explain how I can use requests in Python to send this CURL:
curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'
I dont know how to parse it into a sample code. Thanks a lot
import requests
url_string = 'http://localhost:8086/write?db=mydb'
data_string = 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'
r = requests.post(url_string, data=data_string)
To create a InfluxDB via Python3:
#!/usr/bin/env python3
import urllib.request as requests
url = "http://localhost:8086/query"
params = {
"q=CREATE DATABASE mydb"
}
r = requests.Request(url, data=params)
I am making a python build script for a phonegap project.
I need to open the ios key before i build
I am trying to do this with a http put request through the requests module for python.
If i do it with cURL from command line, it works fine
curl -vvv -d 'data={"password":"myPassWord"}' -X PUT https://build.phonegap.com/api/v1/keys/ios/193686?auth_token=passwordlesstokenphg
But from python like this.
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686', data=password_for_key, params=authentication_token)
It just returns the json you would recieve if you did a cURL without the data.
For me it seems like the data is not being sent to phonegap correctly.
API reference from build.phonegap.com
docs.build.phonegap.com/en_US/2.9.0/developer_api_write.md.html
Please help :)
So when you do
curl -d "..." -X PUT https://example.com
curl sends exactly what's in that string. requests does not translate so directly to curl. To do something similar in requests you need to do the following:
import json
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686',
data={'data': json.dumps(password_for_key)},
params=authentication_token)
What requests will do is build data={"password":"myPassword"} for you if you use the above. First you have to JSON encode the data in password_for_key then pass it in the dictionary to data.