When sending data through python-requests a GET request, I have a need to specifically add something at the beginning of the query string. I have tried passing the data in through dicts and json strings with no luck.
The request as it appears when produced by requests:
/apply/.../explain?%7B%22......
The request as it appears when produced by their interactive API documentation (Swagger):
/apply/.../explain?record=%7B%22....
Where the key-value pairs of my data follow the excerpt above.
Ultimately, I think the missing piece is the record= that gets produced by their documentation. It is the only piece that is different from what is produced by Requests.
At the moment I've got it set up something like this:
import requests
s = requests.Session()
s.auth = requests.auth.HTTPBasicAuth(username,password)
s.verify = certificate_path
# with data below being a dictionary of the values I need to pass.
r = s.get(url,data=data)
I am trying to include an image of the documentation below, but don't yet have enough reputation to do so:
apply/model/explain documentation
'GET' requests don't have data, that's for 'POST' and friends.
You can send the query string arguments using params kwarg instead:
>>> params = {'record': '{"'}
>>> response = requests.get('http://www.example.com/explain', params=params)
>>> response.request.url
'http://www.example.com/explain?record=%7B%22'
From the comments i felt the need to explain this.
http://example.com/sth?key=value&anotherkey=anothervalue
Let's assume you have a url like the above in order to call with python requests you only have to write
response = requests.get('http://example.com/sth', params={
'key':'value',
'anotherkey':'anothervalue'
})
Have in mind that if your value or your keys have any special character in them they will be escaped thats the reason for the %7B%2 part of url in your question.
Related
I was to automate action using python requests, when encountered strange behavior.
I want to generate reports from a certain domain, to do so I have to send a POST request providing parameters in form of an XML. In devtools it looks like this:
xmlWe: <REPORTPARS><PROC_ID>11</PROC_ID>
..... and so on
when I send report data by python requests, it works perfectly whe provided a with dictionary:
data = dict(xmlWe = '<REPORTPARS><PROC_ID>11</PROC_ID>(...) '
r = s.post(URL_generate, data=data))
IMO, its kind of strange, dictionary is a type in python so why would this server handle or expect this type?
I was trying to send this XML as text or JSON, adding the corresponding header 'Content-type' but without success. The server will return 777 Java nullptr exception, which is the same as in case of sending any other nonsense datatype. So it looks like he was expecting some sort of dictionary.
My problem is, my company uses pl/sql for automation, so I will finally have to use this language. There, in http_utils data can be sent merely by write_text, so I'm restricted to sending string (can be dumped JSON).
What do You think, Is there an option for this server will accept report_pars as a string by any chance?
You need to send a xml directly:
import requests
headers = {'Content-Type': 'application/xml'}
r = requests.post(URL_generate, data=xmlWe, headers=headers)
Long story short, i get the query from spotify api which is JSON that has data about newest albums. How do i get the specific info from that like let's say every band name or every album title. I've tried a lot of ways to get that info that i found on the internet and nothing seems to work for me and after couple of hours im kinda frustrated
JSON data is on jsfiddle
here is the request
endpoint = "https://api.spotify.com/v1/browse/new-releases"
lookup_url = f"{endpoint}"
r = requests.get(lookup_url, headers=headers)
print(r.json())
you can find the
When you make this request like the comments have mentioned you get a dictionary which you can then access the keys and values. For example if you want to get the album_type you could do the following:
print(data["albums"]["items"][0]["album_type"])
Since items contains a list you would need to get the first values 0 and then access the album_type.
Output:
single
Here is a link to the code I used with your json.
I suggest you look into how to deal with json data in python, this is a good place to start.
I copied the data from the jsfiddle link.
Now try the following code:
import ast
pyobj=ast.literal_eval(str_cop_from_src)
later you can try with keys
pyobj["albums"]["items"][0]["album_type"]
pyobj will be a python dictionary will all data.
I'm trying to write a request using Python Requests which sends a request to Docusign. I need to use the legacy authorization header, but unfortunately it seems most documentation for this has been removed. When I send the request I get an error as stated in the title.
From research, I found that special characters in the password can cause this issue, so I've confirmed that my password has no special characters, and that my API key is correct. I am currently sending the header as a stringified dictionary as shown below. I have tried it several other ways, and this seems to be the closest, but it still results in the error. Other ways I've tried include attempting to write out the header as a single string (not forming a dictionary first), but that didn't seem to work any better.
docusign_auth_string = {}
docusign_auth_string["Username"] = docusign_user
docusign_auth_string["Password"] = docusign_password
docusign_auth_string["IntegratorKey"] = docusign_key
docusign_auth_string = str(docusign_auth_string)
headers = {'X-DocuSign-Authentication': docusign_auth_string}
response = requests.post(docusign_url, headers=headers, data=body_data)
The above code returns a 401 with the message, INVALID_TOKEN_FORMAT "The security token format does not conform to expected schema." The header I am sending looks as follows:
{'X-DocuSign-Authentication': "{'Username': 'test#test.com', 'Password': 'xxxxxxxxxx', 'IntegratorKey': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'}"}
When I send the request via Postman, it works just fine. In Postman I enter the header name as X-Docusign-Authentication, and the value as: {"Username":"{{ds_username}}","Password":"{{ds_password}}","IntegratorKey":"{{ds_integrator_key}}"} (subbing the same variable values as in the python code).
Therefore it definitely has something to do with the way Requests is sending the header.
Does anyone know why I might be getting the above error?
I'm able to reproduce this behavior: It looks like DocuSign doesn't accept Single Quotes around the sub-parameters of the x-DocuSign-Authentication header value.
Your example fails:
{'Username': 'test#test.com', 'Password': 'xxxxxxxxxx', 'IntegratorKey': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'}
This has more success:
{"Username": "test#test.com", "Password": "xxxxxxxxxx", "IntegratorKey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"}
I'm not familiar enough with Python to advise if there's a different code structure you can follow to use double quotes instead of single. Worst case scenario, you may need to manually set the Header Value to follow that format.
I found a solution to this issue. The response that mentioned double quotes is correct, but in Python I was unable to send a string with the proper format for docusign to understand. Next I found the following Stack overflow question, which ultimately provided the solution:
How to send dict in Header as value to key 'Authorization' in python requests?
I used json.dumps and that resolved the issue. My code is as follows:
docusign_auth_string = {}
docusign_auth_string["Username"] = docusign_user
docusign_auth_string["Password"] = docusign_password
docusign_auth_string["IntegratorKey"] = docusign_key
headers = {"X-DocuSign-Authentication": json.dumps(docusign_auth_string), "Content-Type": "application/json"}
Since you are having success using Postman, it will help to get exactly what is being sent via your request. For this use:
response = requests.get(your_url, headers=your_headers)
x = response.request.headers()
print(x)
This will show you exactly what requests is preparing and sending off. If you post that response here id be happy to help more.
How can I see the entire HTTP request that's being sent by my Python application?
The 2nd answer shows all the possible parameters of your response object.
Here is a small quote from this answer:
import requests
import json
data = {"data" : "24.3"}
data_json = json.dumps(data)
headers = {'Content-type': 'application/json'}
response = requests.post(url, data=data_json, headers=headers)
Does anyone know for sure whether it matters whether you have
data = {"data" : "24.3"}
or
data = {"data" : 24.3} ?
You are already giving a string to requests.post(), because you convert your dictionary to a JSON document with json.dumps(). It doesn't matter to requests what this string contains.
It only matters to whatever server you are sending this data; it is that server that will decode the JSON document and use your number or string.
Note that requests can do the JSON conversion for you. There is no need to use json.dumps() here, just pass your dictionary to the json keyword argumnet:
import requests
data = {"data" : "24.3"}
response = requests.post(url, json=data)
This also takes care of setting the Content-Type header to application/json.
There are two unrelated questions in your post.
The first is:
Does anyone know for sure whether it matters whether you have
data = {"data" : "24.3"}
or
data = {"data" : 24.3} ?
Yes, it does matter!
They are completely different things.
Treating them the same would make JSON format usage obsolete.
If server expects key "data" to be JSON data type number and you send it as a JSON data type string instead, a HTTP status code 400 should be responded.
If server does not report any error it means that this particular key is not being used in server and/or it is not being validated in server.
If server does treat them the same it is idiotic rather than redundant. That is why JSON format is being used in the first place.
The second is:
Does Python Requests POST need numerical data to be a string rather
than a float?
This question title is explained in Martijn Pieters's answer.
I'm trying to figure out what's the correct URL format for the Intersango API (which is poorly documented). I'm programming my client in C#, but I'm looking at the Python example and I'm a little confused as to what is actually being placed in the body of the request:
def make_request(self,call_name,params):
params.append(('api_key',self.api_key)) // <-- How does this get serialized?
body = urllib.urlencode(params)
self.connect()
try:
self.connection.putrequest('POST','/api/authenticated/v'+self.version+'/'+call_name+'.php')
self.connection.putheader('Connection','Keep-Alive')
self.connection.putheader('Keep-Alive','30')
self.connection.putheader('Content-type','application/x-www-form-urlencoded')
self.connection.putheader('Content-length',len(body))
self.connection.endheaders()
self.connection.send(body)
response = self.connection.getresponse()
return json.load(response)
//...
I can't figure out this piece of code: params.append(('api_key',self.api_key))
Is it some kind of a dictionary, something that gets serialized to JSON, comma delimited, or exactly how does it get serialized? What would the body look like when the parameters are encoded and assigned to it?
P.S. I don't have anything that I can run the code with so I can debug it, but I'm just hoping that this is simple enough to understand for somebody that knows Python and they would be able to tell me what's happening on that line of code.
params is a list of 2-element lists. The list would look like ((key1, value1), (key2, value2), ...)
params.append(('api_key',self.api_key)) adds another 2-element list to the existing params list.
Finally, urllib.urlencode takes this list and converts it into a propert urlencoded string. In this case, it will return a string key1=value1&key2=value2&api_key=23423. If there are any special characters in your keys or values, urlencode will %encode them. See documentation for urlencode
I tried to get the C# code working, and it kept failing with exception {"The remote server returned an error: (417) Expectation Failed."}. I finally found what the problem is. You could read about it in depth here
In short, the way to make C# access Intersango API is to add following code:
System.Net.ServicePointManager.Expect100Continue = false;
This code needs to only run once. This is a global setting, so it affects your full application, so beware that something else could break as a result.
Here's a sample code:
System.Net.ServicePointManager.Expect100Continue = false;
var address = "https://intersango.com/api/authenticated/v0.1/listAccounts.php";
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
var postBytes = Encoding.UTF8.GetBytes("api_key=aa75***************fd65785");
request.ContentLength = postBytes.Length;
var dataStream = request.GetRequestStream();
dataStream.Write(postBytes, 0, postBytes.Length);
dataStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Piece of cake
instead of params.append(('api_key',self.api_key))
just write:
params['api_key']=self.api_key