I'm working on an API wrapper. The spec I'm trying to build to has the following request in it:
curl -H "Content-type:application/json" -X POST -d data='{"name":"Partner13", "email":"example#example.com"}' http://localhost:5000/
This request produces the following response from a little test server I setup to see exatly what headers/params etc are sent as. This little script produces:
uri: http://localhost:5000/,
method: POST,
api_key: None,
content_type: application/json,
params: None,
data: data={"name":"Partner13", "email":"example#example.com"}
So that above is the result I want my python script to create when it hits the little test script.
I'm using the python requests module, which is the most beautiful HTTP lib I have ever used. So here is my python code:
uri = "http://localhost:5000/"
headers = {'content-type': 'application/json' }
params = {}
data = {"name":"Partner13", "email":"example#exmaple.com"}
params["data"] = json.dumps(data)
r = requests.post(uri, data=params, headers=headers)
So simple enough stuff. Set the headers, and create a dictionary for the POST parameters. That dictionary has one entry called "data" which is the JSON string of the data I want to send to the server. Then I call the post. However, the result my little test script gives back is:
uri: http://localhost:5000/,
method: POST,
api_key: None,
content_type: application/json,
params: None,
data: data=%7B%22name%22%3A+%22Partner13%22%2C+%22email%22%3A+%22example%40example.com%22%7D
So essentially the json data I wanted to send under the data parameter has been urlendcoded.
Does anyone know how to fix this? I have looked through the requests documentation and cannot seem to find a way to not auto urlencode the send data.
Thanks very much,
Kevin
When creating the object for the data keyword, simply assign a variable the result of json.dumps(data).
Also, because HTTP POST can accept both url parameters as well as data in the body of the request, and because the requests.post function has a keyword argument named "params", it might be better to use a different variable name for readability. The requests docs use the variable name "payload", so thats what I use.
data = {"name":"Partner13", "email":"example#exmaple.com"}
payload = json.dumps(data)
r = requests.post(uri, data=payload, headers=headers)
Requests automatically URL encodes dictionaries passed as data here. John_GG's solution works because rather than posting a dictionary containing the JSON encoded string in the 'data' field it simply passes the JSON encoded string directly: strings are not automatically encoded. I can't say I understand the reason for this behaviour in Requests but regardless, it is what it is. There is no way to toggle this behaviour off that I can find.
Best of luck with it, Kevin.
Related
I'm trying to write API client for Jira with Python requests lib according reference:
https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/
Request to be generated:
http://localhost:8080/rest/api/2/search?jql=assignee=charlie&startAt=2&maxResults=2
As I know, parameters to GET request should be passed as dictionary like:
params = {'assignee':'charlie', 'startAt':'2'}
But all main parameters are nested in jql parameter, so I assume there is should be a nested dict like:
params = {'jql': {'assignee': 'charlie'}}
But that's doesn't work - as a result I've got request to
/rest/api/2/search?jql=assignee
As expect /rest/api/2/search?jql=assignee=charlie
using
r = requests.get(url, params=params)
How to manage such request?
UPD:
To be more clear, I'd like to wrap request in a method with kwargs, like:
search_query(assignee='charlie', startAt=1, etc...)
And then generate a query using this params, but maybe there are any other ideas.
You are missing couple of key parameters, mainly if you are pushing data via requests, the data go into the data argument. Also the moment you push JSON data, you need to set the headers correctly as well. The last thing is authentication. Have you tried to post it in this manner?
import json
requests.post(url=url, headers={"Content-Type": "application/json"},
auth=('username', 'password'), # your username and password
data=json.dumps(params)
)
Also by the JIRA documentation you've provided (https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/) if you want to push query as data, the url you want is /rest/api/2/search.
I'm trying to post to a server using the following script:
import requests
data = {
'query': 'GetProcess',
'getFrom': '2018-12-06 10:10:10.000',
}
response = requests.post('http://localhost/monitor', data=data)
I cannot find where exactly, but the space character in the getFrom element is being replaced with a +: '2018-12-06+10:10:10.000'
This doesn't match the syntax SQL expects on our server, so the query fails.
I read here (https://stackoverflow.com/a/12528097) that setting the Content-type might help. I tried text/html, text/plain, application/json, and nothing seems to change.
Interestingly, the following (equivalent?) bash command succeeds:
curl -d 'query=GetProcess&getFrom=2018-12-06 10:10:10.000' localhost/monitor
I'm looking for a way to make my server receive "getFrom" : "2018-12-06 10:10:10.000" in the header.
I found a way to make this work: the problem I was having was due to the use of the urlencode function used in requests. In the requests documentation, it is shown how to go around this default behavior using PreparedRequests: http://docs.python-requests.org/en/master/user/advanced/#prepared-requests
Essentially, instead of using the requests.post() wrapper, make the function calls explicitly. This way, you will be able to control exactly what is sent. In my case, the solution was to do:
import requests
data = {
'query': 'GetProcess',
'getFrom': '2018-12-06 10:10:10.000'
}
s = requests.Session()
req = requests.Request('POST', 'http://'+ipAddress+'/monitor', data=data)
prepped = s.prepare_request(req)
prepped.body = prepped.body.replace("+", " ")
response = s.send(prepped)
Before downvoting/marking as duplicate, please note:
I have already tried out this, this, this, this,this, this - basically almost all the methods I could find pointed out by the Requests documentation but do not seem to find any solution.
Problem:
I want to make a POST request with a set of headers and form data.
There are no files to be uploaded. As per the request body in Postman, we set the parameters by selecting 'form-data' under the 'Body' section for the request.
Here is the code I have:
headers = {'authorization': token_string,
'content-type':'multipart/form-data; boundary=----WebKitFormBoundaryxxxxxXXXXX12345'} # I get 'unsupported application/x-www-form-url-encoded' error if I remove this line
body = {
'foo1':'bar1',
'foo2':'bar2',
#... and other form data, NO FILE UPLOADED
}
#I have also tried the below approach
payload = dict()
payload['foo1']='bar1'
payload['foo2']='bar2'
page = ''
page = requests.post(url, proxies=proxies, headers=headers,
json=body, files=json.dump(body)) # also tried data=body,data=payload,files={} when giving data values
Error
{"errorCode":404,"message":"Required String parameter 'foo1' is not
present"}
EDIT:
Adding a trace of the network console. I am defining it in the same way in the payload as mentioned on the request payload.
There isn't any gui at all? You could get the network data from chrome, although:
Try this:
headers = {'authorization': token_string}
Probably there is more authorization? Or smthng else?
You shouldn't add Content-Type as requests will handle it for you.
Important, you could see the content type as WebKitFormBoundary, so for the payload you must take, the data from the "name" variable.
Example:
(I know you won't upload any file, it just an example) -
So in this case, for my payload would look like this: payload = {'photo':'myphoto'} (yea there would be an open file etc etc, but I try to keep it simple)
So your payload would be this-> (So always use name from the WebKit)
payload = {'foo1':'foo1data',
'foo2':'foo2data'}
session.post(url,data = payload, proxies etc...)
Important! As I can see you use the method from requests library. Firstly you always should create a session like this
session = requests.session() -> it will handle cookies, headers, etc, and won't open a new session, or plain requests with every requests.get/post.
With the firefox plugin "HttpFox" i'm getting the POST request which looks like this:
{'json':'{"command":"SEARCH","data":{"someData":"someValue","otherData":"otherData"}}'}
Now i need to send a http request build with python to get the same data as i would get via browser. See code:
headers = {'Content-type': 'application/json; charset=utf-8'}
payload = ?
req = requests.post(url, data=json.dumps(payload), headers = headers)
My problem is:
I'm not sure how to build the payload. It should be a dictionary as well, but im confused because of the POST type which is delivered with HttpFox. There are two dictionaries inside the main dictionary.
How should i handle this ?
Appreciate any help.
Ok, i found the solution:
It was necessary to build a dict like this:
valueString = """{"command":"SEARCH","data":{"someData":"someValue","otherData":"otherData"}}"""
/// the """ ensures that the whole text between """ is handled as a string.
payload = {'json': valueString}
The key 'json' requieres a string. In this case the string looks like a dictionary.
That's it.
I need to send this JSON array in a GET request
{"user": "jähn", "id": 3}
I tried to use
data = '{"user": "jähn", "id": 3}'
headers = {
'Content-type': 'application/json',
'Accept': 'text/plain'
}
request = urllib.request.Request(self.update_url, data=data,
headers=headers, method='GET')
response = urllib.request.urlopen(request)
But its failing with: TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.
Another thing that I find quite weird is that it tells me about POST data although I set the method on the Request to GET.
Since this is a simple script I'd prefer not to use a library like python-requests
You cannot make a GET request with a JSON-encoded body, as a GET request only ever consists of the URL and the headers. Parameters are encoded into the URL using URL encoding instead, there is no option to encode such parameters to JSON instead.
You create URL-encoded parameters with the urllib.parse.urlencode() function, then appended to the URL with ?.
from request.parse import urlencode
data = {"user": "jähn", "id": 3} # note, a Python dictionary, not a JSON string
parameters = urlencode(data)
response = urllib.request.urlopen('?'.join((self.update_url, parameters)))
Do not use the data parameter; using that keyword argument forces a request to use the POST method:
data must be a bytes object specifying additional data to send to the server, or None if no such data is needed. Currently HTTP requests are the only ones that use data; the HTTP request will be a POST instead of a GET when the data parameter is provided. data should be a buffer in the standard application/x-www-form-urlencoded format.