I'm trying to write a request using Python Requests which sends a request to Docusign. I need to use the legacy authorization header, but unfortunately it seems most documentation for this has been removed. When I send the request I get an error as stated in the title.
From research, I found that special characters in the password can cause this issue, so I've confirmed that my password has no special characters, and that my API key is correct. I am currently sending the header as a stringified dictionary as shown below. I have tried it several other ways, and this seems to be the closest, but it still results in the error. Other ways I've tried include attempting to write out the header as a single string (not forming a dictionary first), but that didn't seem to work any better.
docusign_auth_string = {}
docusign_auth_string["Username"] = docusign_user
docusign_auth_string["Password"] = docusign_password
docusign_auth_string["IntegratorKey"] = docusign_key
docusign_auth_string = str(docusign_auth_string)
headers = {'X-DocuSign-Authentication': docusign_auth_string}
response = requests.post(docusign_url, headers=headers, data=body_data)
The above code returns a 401 with the message, INVALID_TOKEN_FORMAT "The security token format does not conform to expected schema." The header I am sending looks as follows:
{'X-DocuSign-Authentication': "{'Username': 'test#test.com', 'Password': 'xxxxxxxxxx', 'IntegratorKey': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'}"}
When I send the request via Postman, it works just fine. In Postman I enter the header name as X-Docusign-Authentication, and the value as: {"Username":"{{ds_username}}","Password":"{{ds_password}}","IntegratorKey":"{{ds_integrator_key}}"} (subbing the same variable values as in the python code).
Therefore it definitely has something to do with the way Requests is sending the header.
Does anyone know why I might be getting the above error?
I'm able to reproduce this behavior: It looks like DocuSign doesn't accept Single Quotes around the sub-parameters of the x-DocuSign-Authentication header value.
Your example fails:
{'Username': 'test#test.com', 'Password': 'xxxxxxxxxx', 'IntegratorKey': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'}
This has more success:
{"Username": "test#test.com", "Password": "xxxxxxxxxx", "IntegratorKey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"}
I'm not familiar enough with Python to advise if there's a different code structure you can follow to use double quotes instead of single. Worst case scenario, you may need to manually set the Header Value to follow that format.
I found a solution to this issue. The response that mentioned double quotes is correct, but in Python I was unable to send a string with the proper format for docusign to understand. Next I found the following Stack overflow question, which ultimately provided the solution:
How to send dict in Header as value to key 'Authorization' in python requests?
I used json.dumps and that resolved the issue. My code is as follows:
docusign_auth_string = {}
docusign_auth_string["Username"] = docusign_user
docusign_auth_string["Password"] = docusign_password
docusign_auth_string["IntegratorKey"] = docusign_key
headers = {"X-DocuSign-Authentication": json.dumps(docusign_auth_string), "Content-Type": "application/json"}
Since you are having success using Postman, it will help to get exactly what is being sent via your request. For this use:
response = requests.get(your_url, headers=your_headers)
x = response.request.headers()
print(x)
This will show you exactly what requests is preparing and sending off. If you post that response here id be happy to help more.
How can I see the entire HTTP request that's being sent by my Python application?
The 2nd answer shows all the possible parameters of your response object.
This is more of a python question than Splunk but would be helpful if anyone had done this... specifically here, there's a discussion of sending multiple metrics in a single POST to the server. The example they provide is using curl and wrapping the entire payload in single quotes ('), e.g.
curl -k http://<IP address or host name or load balancer name>:8088/services/collector \
-H "Authorization: Splunk 98a1e071-bc35-410b-8642-78ce7d829083"
\
-d '{"time": 1505501013.000,"source":"disk","host":"host_99","fields":
{"region":"us-west-1","datacenter":"us-west- 1a","rack":"63","os":"Ubuntu16.10","arch":"x64","team":"LON","service":"6","service_version":"0","service_environment":"test","path":"/dev/sda1","fstype":"ext3","_value":999311222774,"metric_name":"total"}}
{"time": 1505511013.000,"source":"disk","host":"host_99","fields":
{"region":"us-west-1","datacenter":"us-west-1a","rack":"63","os":"Ubuntu16.10","arch":"x64","team":"LON","service":"6","service_version":"0","service_environment":"test","path":"/dev/sda1","fstype":"ext3","_value":1099511627776,"metric_name":"total"}}'
My question is how to do the same thing in python – i.e. you can't wrap multiple JSON objects in single quotes like in the curl command - that just makes the entire payload a string. Is there some other wrapper that can be used for this purpose?
So, this works:
payload = {"time": 1505501013.000,"source":"disk","host":"host_99","fields":
{"region":"us-west-1","datacenter":"us-west- 1a","rack":"63","os":"Ubuntu16.10","arch":"x64","team":"LON","service":"6","service_version":"0","service_environment":"test","path":"/dev/sda1","fstype":"ext3","_value":999311222774,"metric_name":"total"}}
But this does not:
payload = {"time": 1505501013.000,"source":"disk","host":"host_99","fields":
{"region":"us-west-1","datacenter":"us-west- 1a","rack":"63","os":"Ubuntu16.10","arch":"x64","team":"LON","service":"6","service_version":"0","service_environment":"test","path":"/dev/sda1","fstype":"ext3","_value":999311222774,"metric_name":"total"}}
{"time": 1505511013.000,"source":"disk","host":"host_99","fields":
{"region":"us-west-1","datacenter":"us-west-1a","rack":"63","os":"Ubuntu16.10","arch":"x64","team":"LON","service":"6","service_version":"0","service_environment":"test","path":"/dev/sda1","fstype":"ext3","_value":1099511627776,"metric_name":"total"}}
FYI, then the POST looks like:
resp = requests.post(splunkurl,json=payload,headers=headers)
Well, "multiple json objects" is not a valid json, until it's a list of objects.
Generally, python doesn't care (just like any other network tool), json is just data format, and you need a different one. So you need to construct text payload yourself, i.e. json.dumps(payload1) + json.dumps(payload2), and send it via your network client as "raw" data.
I highly doubt that mainstream http libraries provide such usecase out of the box.
Not sure on reason for downvotes, i.e. requests library (which is kinda standard de-facto for high-level networking) have smart processing for payloads:
requests.post(url, data={'v1': 1, 'v2': 2}) # will encode it as form data
requests.post(url, json={'v1': 1, 'v2': 2}) # will encode as json
requests.post(url, data="{'v1': 1}{'v2': 2}") # will send as-is
Json has nothing to do with http itself, it's just a way to serialize data. Most clients will eventually use urllib, which doesn't care at all, the only question is if library gives easy way to send data raw
Here is a small quote from this answer:
import requests
import json
data = {"data" : "24.3"}
data_json = json.dumps(data)
headers = {'Content-type': 'application/json'}
response = requests.post(url, data=data_json, headers=headers)
Does anyone know for sure whether it matters whether you have
data = {"data" : "24.3"}
or
data = {"data" : 24.3} ?
You are already giving a string to requests.post(), because you convert your dictionary to a JSON document with json.dumps(). It doesn't matter to requests what this string contains.
It only matters to whatever server you are sending this data; it is that server that will decode the JSON document and use your number or string.
Note that requests can do the JSON conversion for you. There is no need to use json.dumps() here, just pass your dictionary to the json keyword argumnet:
import requests
data = {"data" : "24.3"}
response = requests.post(url, json=data)
This also takes care of setting the Content-Type header to application/json.
There are two unrelated questions in your post.
The first is:
Does anyone know for sure whether it matters whether you have
data = {"data" : "24.3"}
or
data = {"data" : 24.3} ?
Yes, it does matter!
They are completely different things.
Treating them the same would make JSON format usage obsolete.
If server expects key "data" to be JSON data type number and you send it as a JSON data type string instead, a HTTP status code 400 should be responded.
If server does not report any error it means that this particular key is not being used in server and/or it is not being validated in server.
If server does treat them the same it is idiotic rather than redundant. That is why JSON format is being used in the first place.
The second is:
Does Python Requests POST need numerical data to be a string rather
than a float?
This question title is explained in Martijn Pieters's answer.
When sending data through python-requests a GET request, I have a need to specifically add something at the beginning of the query string. I have tried passing the data in through dicts and json strings with no luck.
The request as it appears when produced by requests:
/apply/.../explain?%7B%22......
The request as it appears when produced by their interactive API documentation (Swagger):
/apply/.../explain?record=%7B%22....
Where the key-value pairs of my data follow the excerpt above.
Ultimately, I think the missing piece is the record= that gets produced by their documentation. It is the only piece that is different from what is produced by Requests.
At the moment I've got it set up something like this:
import requests
s = requests.Session()
s.auth = requests.auth.HTTPBasicAuth(username,password)
s.verify = certificate_path
# with data below being a dictionary of the values I need to pass.
r = s.get(url,data=data)
I am trying to include an image of the documentation below, but don't yet have enough reputation to do so:
apply/model/explain documentation
'GET' requests don't have data, that's for 'POST' and friends.
You can send the query string arguments using params kwarg instead:
>>> params = {'record': '{"'}
>>> response = requests.get('http://www.example.com/explain', params=params)
>>> response.request.url
'http://www.example.com/explain?record=%7B%22'
From the comments i felt the need to explain this.
http://example.com/sth?key=value&anotherkey=anothervalue
Let's assume you have a url like the above in order to call with python requests you only have to write
response = requests.get('http://example.com/sth', params={
'key':'value',
'anotherkey':'anothervalue'
})
Have in mind that if your value or your keys have any special character in them they will be escaped thats the reason for the %7B%2 part of url in your question.
how do I get the whole raw http request in the python framework bottle?
I need something like this:
GET\n
myurl.com\n
/\n
attribute=value
&att2=value2
I need this to sign my http api requests
As far as I can tell from the docs you can't get the data in raw format.
What you can do is reconstruct it using bottle.request.data and bottle.request.headers. That may be enough for your purposes.
If you just want to print the request you can do the following:
headers_string = ['{}: {}'.format(h, request.headers.get(h)) for h in request.headers.keys()]
print('URL={}, method={}\nheaders:\n{}'.format(request.url, request.method, '\n'.join(headers_string)))