We Keep Coding

How to curl in Python with header, data?

I'm trying replace the following curl command by a Python script:
curl --request POST \
--url https://xx.com/login \
--header 'Content-Type: application/json' \
--data '{
"email": "user#domain.com",
"password": "PASSWORD"
}'
Script that I tried:
import urllib.request
import json
body = {"email": "xxx#xx.com","password": "xxx"}
myurl = "https://xx.com/login"
req = urllib.request.Request(myurl)
req.add_header('Content-Type', 'application/json; charset=utf-8')
jsondata = json.dumps(body)
jsondataasbytes = jsondata.encode('utf-8') # needs to be bytes
req.add_header('Content-Length', len(jsondataasbytes))
response = urllib.request.urlopen(req, jsondataasbytes)
When I tried to run this script, it doesn't return me anything and show processed completed. Is my code logic correct? Or there is something wrong with my code?

For HTTP and HTTPS URLs, urllib.request.urlopen function returns a http.client.HTTPResponse object. It has different attributes and methods you can use,
For example,
HTTPResponse.read([amt]) - Reads and returns the response body, or up to the next amt bytes.
HTTPResponse.getheaders() - Return a list of (header, value) tuples.
HTTPResponse.status - Status code returned by server.
So in your case you could do check the status using status attribute . If it's successful read the response body using read method.
status_code = response.status
if status_code == 200: # request succeeded
response_body = response.read() # This will be byte object
response_body_as_string = response_body.decode('utf-8')

you can simply just use requests:
import requests
headers = {
'Content-Type': 'application/json',
}
data = '{\n"email": "user#domain.com",\n"password": "PASSWORD"\n}'
response = requests.post('https://xx.com/login', headers=headers, data=data)

Transform Curl Oauth2 to Python Script

Below bash script works and returns a token.
curl -X POST --user <id>:<key> 'https://<name>.auth.eu-west-1.amazoncognito.com/oauth2/token?grant_type=client_credentials' -H 'Content-Type: application/x-www-form-urlencoded'
I would now like to generate a token via a Python script. I currently struggle with using requests library, but without success. Below generates Response 400 (bad request).
import requests
parameters = {"user":"<id>:<key>", "Content-Type": "application/x-www-form-urlencoded"}
response = requests.get("https://<name>.auth.eu-west-1.amazoncognito.com/oauth2/token?grant_type=client_credentials", params=parameters)
print(response)

To fix your usage of requests:
Use post() instead of get().
Use an auth object to construct a basic auth header. (Or send credentials in the body)
Remove the content type header; requests will set this for you.
Take the grant_type out of the query string. This belongs in the request body.
While the documentation for the /oauth2/token endpoint says you need to send a basic auth header for client_credentials grant, it also works if you send the client id and secret in the request body (with no auth header).
As such, these python examples both work, and are essentially equivalent:
Credentials in request body:
import requests
url = 'https://<COGNITO_DOMAIN>.auth.<REGION>.amazoncognito.com/oauth2/token'
params = {
"client_secret": "1ixxxxxxxxxxxxxxxxc2b",
"grant_type": "client_credentials",
"client_id": "7xxxxxxxxxxxxxxxt"
}
response = requests.post(url, data=params)
print(response)
print(response.text)
Credentials in basic auth header:
import requests
url = 'https://<COGNITO_DOMAIN>.auth.<REGION>.amazoncognito.com/oauth2/token'
params = {
"grant_type": "client_credentials"
}
auth = ('7xxxxxxxxxxt', '1ixxxxxxxxxxxxxxxc2b')
r = requests.post(url, data=params, auth=auth)
print(r)
print(r.text)

Requests with headers and data

I'm trying to convert the following cURL command into a Request in Python.
curl -H 'customheader: value' -H 'customheader2: value' --data "Username=user&UserPassword=pass" https://thisismyurl
From what I understand, one can GET headers and POST data. So how do I do both like a cURL?
This is what I'm trying:
url = 'myurl'
headers = {'customheader': 'value', 'customheader2': 'value'}
data = 'Username=user&UserPassword=pass'
requests.get(url, headers=headers, data=data)
Which returns: HTTPError: HTTP 405: Method Not Allowed
If I use post: MissingArgumentError: HTTP 400: Bad Request

When you use the --data command line argument, curl switches to a POST request. Do the same in Python:
requests.post(url, headers=headers, data=data)
From the curl manual:
-d, --data <data>
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded.
You may want to manually set the Content-Type header accordingly, or use a dictionary for the data parameter (and have requests encode those to the right format for you; the Content-Type header is set for you as well):
url = 'myurl'
headers = {'customheader': 'value', 'customheader2': 'value'}
data = {'Username': 'user', 'UserPassword': 'pass'}
requests.post(url, headers=headers, data=data)

Python requests doesn't upload file

I am trying to reproduce this curl command with Python requests:
curl -X POST -H 'Content-Type: application/gpx+xml' -H 'Accept: application/json' --data-binary #test.gpx "http://test.roadmatching.com/rest/mapmatch/?app_id=my_id&app_key=my_key" -o output.json
The request with curl works fine. Now I try it with Python:
import requests
file = {'test.gpx': open('test.gpx', 'rb')}
payload = {'app_id': 'my_id', 'app_key': 'my_key'}
headers = {'Content-Type':'application/gpx+xml', 'Accept':'application/json'}
r = requests.post("https://test.roadmatching.com/rest/mapmatch/", files=file, headers=headers, params=payload)
And I get the error:
<Response [400]>
{u'messages': [], u'error': u'Invalid GPX format'}
What am I doing wrong? Do I have to specify data-binary somewhere?
The API is documented here: https://mapmatching.3scale.net/mmswag

Curl uploads the file as the POST body itself, but you are asking requests to encode it to a multipart/form-data body. Don't use files here, pass in the file object as the data argument:
import requests
file = open('test.gpx', 'rb')
payload = {'app_id': 'my_id', 'app_key': 'my_key'}
headers = {'Content-Type':'application/gpx+xml', 'Accept':'application/json'}
r = requests.post(
"https://test.roadmatching.com/rest/mapmatch/",
data=file, headers=headers, params=payload)
If you use the file in a with statement it'll be closed for you after uploading:
payload = {'app_id': 'my_id', 'app_key': 'my_key'}
headers = {'Content-Type':'application/gpx+xml', 'Accept':'application/json'}
with open('test.gpx', 'rb') as file:
r = requests.post(
"https://test.roadmatching.com/rest/mapmatch/",
data=file, headers=headers, params=payload)
From the curl documentation for --data-binary:
(HTTP) This posts data exactly as specified with no extra processing whatsoever.
If you start the data with the letter #, the rest should be a filename. Data is posted in a similar manner as --data-ascii does, except that newlines and carriage returns are preserved and conversions are never done.

how to make post request in python

Here is the curl command:
curl -H "X-API-TOKEN: <API-TOKEN>" 'http://foo.com/foo/bar' --data #
let me explain what goes into data
POST /foo/bar
Input (request JSON body)
Name Type
title string
body string
So, based on this.. I figured:
curl -H "X-API-TOKEN: " 'http://foo.com/foo/bar' --data '{"title":"foobar","body": "This body has both "double" and 'single' quotes"}'
Unfortunately, I am not able to figure that out as well (like curl from cli)
Though I would like to use python to send this request.
How do i do this?

With the standard Python httplib and urllib libraries you can do
import httplib, urllib
headers = {'X-API-TOKEN': 'your_token_here'}
payload = "'title'='value1'&'name'='value2'"
conn = httplib.HTTPConnection("heise.de")
conn.request("POST", "", payload, headers)
response = conn.getresponse()
print response
or if you want to use the nice HTTP library called "Requests".
import requests
headers = {'X-API-TOKEN': 'your_token_here'}
payload = {'title': 'value1', 'name': 'value2'}
r = requests.post("http://foo.com/foo/bar", data=payload, headers=headers)