We Keep Coding

PYTHON: requests.post() how to send request_body encoded as application/x-www-form-urlencoded

I'm doign an app with the Spotify API. My problem is I'm trying to get an access_token but It's not working. In the docs it says i need to send the body encoded as application/x-www-form-urlencoded so I search a little bit and It should work just setting request_body as a dictionary.
This is the code of my function:
def get_access_token(self):
auth_code, code_verifier = self.get_auth_code()
endpoint = "https://accounts.spotify.com/api/token"
# as docs said data should be encoded as application/x-www-form-urlencoded
# as internet says i just need to send it as a dictionary. However it's not working
request_body = {
"client_id": f"{self.client_ID}",
"grant_type": "authorization_code",
"code": f"{auth_code}",
"redirect_uri": f"{self.redirect_uri}",
"code_verifier": f"{code_verifier}"
}
response = requests.post(endpoint, data=request_body)
print(response)
The response I'm getting is always <Response [400]>
Here are the docs, step 4 https://developer.spotify.com/documentation/general/guides/authorization-guide/#authorization-code-flow-with-proof-key-for-code-exchange-pkce
NOTE: I tryed executing this as a curl and it works fine I'm not sure what I'm doing wrong in the python code
here's the command:
curl -d client_id={self.client_ID} -d grant_type=authorization_code -d code={auth_code} -d redirect_uri={self.redirect_uri} -d code_verifier={code_verifier} https://accounts.spotify.com/api/token

You can specify the request type in the request header.
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
response = requests.post(endpoint, data=request_body, headers=headers)
print(response)

REST post query to GO server from bash works but fails for Python

I have go server that unmarshals the json it receives it.
It works when I do it using the curl but fails in case of python.
Go server Unmarshal code:
type Data struct {
Namespace string `json:"namespace"`
ContainerId string `json:"containerId"`
}
func notify(w http.ResponseWriter, r *http.Request) {
decoder := json.NewDecoder(r.Body)
var data Data
err := decoder.Decode(&data)
if err != nil {
glog.Errorf("Failed to decode the request json %s \n", err.Error())
return
}
...
}
If I Do curl command it works without complaining:
curl -i -H "Accept: application/json" -H "Content-Type:application/json" -X POST --data '{"namespace": "default", "containerId": "2f7c58d399f2dc35fa1be2abea19301c8e74973ddd72f55a778babf01db5ac26"}' http://mysvc:8080/notify
but if I do the same thing with Python it complains:
jsonPrep['containerId'] = "2f7c58d399f2dc35fa1be2abea19301c8e74973ddd72f55a778babf01db5ac26"
jsonPrep['namespace'] = "default"
headers = {'Content-type': 'application/json', 'Accept': 'application/json'}
r = requests.post('http://mysvc:8080/notify', json=json.dumps(jsonPrep), headers=headers)
the go server complains :
E1026 15:49:48.974117 1 main.go:59] Failed to decode the request json json: cannot unmarshal string into Go value of type main.Data
I don't see what is different when I do curl vs rest query in python.
Can anyone help me identify the issue?

The json argument to requests.post() is for passing a value that has not had json.dumps() called on it yet. requests calls json.dumps() on the json argument itself, so because you're passing json=json.dumps(jsonPrep), jsonPrep will end up being JSONified twice, which is not what you want.
Either use data:
requests.post(..., data=json.dumps(jsonPrep), ...)
or get rid of the json.dumps():
requests.post(..., json=jsonPrep, ...)

How to Translate CURL to Python Requests

I am currently trying to integrate Stripe Connect and have come across the flowing CURl POST request:
curl https://connect.stripe.com/oauth/token \
-d client_secret=SECRET_CODE \
-d code="{AUTHORIZATION_CODE}" \
-d grant_type=authorization_code
However I am very new to CURL and have been doing some research and trying to use the requests package to do it. This is what my current code looks like:
data = '{"client_secret": "%s", "code": "%s", "grant_type": "authorization_code"}' % (SECRET_KEY, AUTHORIZATION_CODE)
response = requests.post('https://connect.stripe.com/oauth/token', json=data)
However this always returns a response code 400. I have no idea where I am going wrong and any guidance would be thoroughly appreciated.

The error is because you are passing your data as string, instead json param of requests.post call expects it to be dict. Your code should be:
import requests
data = {
"client_secret": SECRET_KEY,
"code": AUTHORIZATION_CODE,
"grant_type": "authorization_code"
}
response = requests.post('https://connect.stripe.com/oauth/token', json=data)
Take a look at request library's More complicated POST requests document.

Convert curl with --form to python requests

I have a functioning curl request:
curl --form "bio[0]=#images/12345.jpg" --form "bio[1]=#images/6789.jpg" --form "Verify=Verfiy" http://192.45.35.89/test.php
that I'm trying to convert to python requests:
requests.post('http://192.45.35.89/test.php', data = {
'bio' : [
('12345.jpg', open('images/12345.jpg', 'rb'), 'image/png'),
('6789.jpg', open('images/6789.jpg', 'rb'), 'image/png')
],
'Verify' : 'Verify'
})
The python version of the request yields a server error about "files not found or not accessible".
Am I correctly transforming the curl request to python requests? Any pointer would be greatly appreciated.

You need to send the files in a files dictionary to send the data as multipart/form-data as curl does:
requests.post('http://192.45.35.89/test.php',
files = {
'bio[0]' : ('12345.jpg', open('images/12345.jpg', 'rb'), 'image/png'),
'bio[1]' : ('6789.jpg', open('images/6789.jpg', 'rb'), 'image/png')
},
data = {'Verify' : 'Verify'})
Reference: POST a Multipart-Encoded File

Curl POST request into pycurl code

I'm trying to convert following curl request into pycurl:
curl -v
-H Accept:application/json \
-H Content-Type:application/json \
-d "{
name: 'abc',
path: 'def',
target: [ 'ghi' ]
}" \
-X POST http://some-url
I have following python code:
import pycurl, json
c = pycurl.Curl()
c.setopt(pycurl.URL, 'http://some-url')
c.setopt(pycurl.HTTPHEADER, ['Accept: application/json'])
data = json.dumps({"name": "abc", "path": "def", "target": "ghi"})
c.setopt(pycurl.POST, 1)
c.setopt(pycurl.POSTFIELDS, data)
c.setopt(pycurl.VERBOSE, 1)
c.perform()
print curl_agent.getinfo(pycurl.RESPONSE_CODE)
c.close()
Executing this I had an error 415: Unsupported media type, so I have changed:
c.setopt(pycurl.HTTPHEADER, ['Accept: application/json'])
into:
c.setopt(pycurl.HTTPHEADER, [ 'Content-Type: application/json' , 'Accept: application/json'])
This time I have 400: Bad request. But bash code with curl works. Do you have any idea what should I fix in python code?

PycURL is a wrapper on the libcurl library written in C language so its Python API can be bit puzzling. As some people are advocating use of python requests instead I just want to point out that it isn't a perfect replacement. For me, its lack of DNS resolution timeout was a deal breaker. I also find it much slower on my Raspberry Pi. This comparison may be relevant:
Python Requests vs PyCurl Performance
So here's something that doesn't evade OP's question:
import pycurl
import json
from cStringIO import StringIO
curl = pycurl.Curl()
curl.setopt(pycurl.URL, 'http://some-url')
curl.setopt(pycurl.HTTPHEADER, ['Accept: application/json',
'Content-Type: application/json'])
curl.setopt(pycurl.POST, 1)
# If you want to set a total timeout, say, 3 seconds
curl.setopt(pycurl.TIMEOUT_MS, 3000)
## depending on whether you want to print details on stdout, uncomment either
# curl.setopt(pycurl.VERBOSE, 1) # to print entire request flow
## or
# curl.setopt(pycurl.WRITEFUNCTION, lambda x: None) # to keep stdout clean
# preparing body the way pycurl.READDATA wants it
# NOTE: you may reuse curl object setup at this point
# if sending POST repeatedly to the url. It will reuse
# the connection.
body_as_dict = {"name": "abc", "path": "def", "target": "ghi"}
body_as_json_string = json.dumps(body_as_dict) # dict to json
body_as_file_object = StringIO(body_as_json_string)
# prepare and send. See also: pycurl.READFUNCTION to pass function instead
curl.setopt(pycurl.READDATA, body_as_file_object)
curl.setopt(pycurl.POSTFIELDSIZE, len(body_as_json_string))
curl.perform()
# you may want to check HTTP response code, e.g.
status_code = curl.getinfo(pycurl.RESPONSE_CODE)
if status_code != 200:
print "Aww Snap :( Server returned HTTP status code {}".format(status_code)
# don't forget to release connection when finished
curl.close()
There are some more interesting features worth checking out in the libcurl curleasy setopts documentation

In your bash example, the property target is an array, in your Python example it is a string.
Try this:
data = json.dumps({"name": "abc", "path": "def", "target": ["ghi"]})
I also strongly advise you to check out the requests library which has a much nicer API:
import requests
data = {"name": "abc", "path": "def", "target": ["ghi"]}
response = requests.post('http://some-url', json=data)
print response.status_code

I know this is over a year old now, but please try removing the whitespace in your header value.
c.setopt(pycurl.HTTPHEADER, ['Accept:application/json'])
I also prefer using the requests module as well because the APIs/methods are clean and easy to use.

I had similar problem, and I used your code example but updated the httpheader section as follows:
c.setopt(pycurl.HTTPHEADER, ['Content-Type:application/json'])

It's better simple to use requests library. (http://docs.python-requests.org/en/latest)
I append python code for your original curl custom headers.
import json
import requests
url = 'http://some-url'
headers = {'Content-Type': "application/json; charset=xxxe", 'Accept': "application/json"}
data = {"name": "abc", "path": "def", "target": ["ghi"]}
res = requests.post(url, json=data, headers=headers)
print (res.status_code)
print (res.raise_for_status())