I need to PUT data from a string (no dict) as the body of the call to a REST API.
When I call
r = requests.put(url, data = string)
then I can see in r.request.body after this call that it is None. Also, the server responds with a "411 Length Required" error.
However, when I try it with a dict instead of a string then it works, as the server responds with the correct JSON data. Moreover, in that case I can see in r.request.body the correct data.
Any ideas?
PS: I am using Python 2.7.3 and Python-requests 1.2.0
Even after three attempts to clarify the question, it still isn't clear what you're asking here, but I can try to throw out enough info that you can figure out the answer.
First, r = requests.put(url, data = string) returns a Response, which doesn't have a body, but it does have a request, and a history of 0 or more redirect requests, all of which are PreparedRequest objects, which do have body attributes.
On the other hand, if you did r.requests.Request(method='PUT', url=url, data=string), that would return a Request, which has to be prepare()d before it has a body.
Either way, if I do a simple test and look at the results, I find that the body is always correct:
>>> resp = requests.put('http://localhost/nosuchurl', data='abc')
>>> resp.request.body
'abc'
>>> resp = requests.put('http://localhost/redirect_to_https', data='abc')
>>> resp.history[-1].request.body
'abc'
>>> req = requests.Request(method='PUT', url='http://localhost/nosuchurl', data='abc')
>>> preq = req.prepare()
>>> preq.body
'abc'
My best guess is that you need to be looking at resp.history[0].request.body, but you're looking at resp.request.body, or something similar.
If Redirection and History in the quickstart tutorial doesn't help, read the detailed API docs, or just experiment with all of them until you figure it out.
Or do this:
resp = request.put('http://localhost/nosuchurl', data='abc', allow_redirects=False)
And then do the redirect-handling manually.
Related
If I make the request
api-key = 'asdfklhsdfkjahsdlgkjahlkdjahfsa'
url = 'https://www.website.com'
headers = {'api-key': api-key,
'Content-Type': 'application/json'}
request_data = {'foo': 'bar', 'egg': 'spam'}
result = requests.post(url, headers=headers, data=request_data)
The server is contacted. Suppose that instead I want to do something like
request_string = requests.foobar(url, headers=headers, data=request_data)
import os
os.system('curl ' + request_string)
So that I can look to see what the request is doing without bothering the server (possibly to the point that I could c&p it into curl), what would foobar be? Or in general, what is a way to inspect the contents of the request without making it?
Here's another post that implies that you can use Request().prepare() to observe the request without actually sending the request.
Furthermore the official documentation reads "In some cases you may wish to do some extra work to the body or headers (or anything else really) before sending a request. The simple recipe for this is the following" and then it illustrates Request.prepare()
I am trying to run my csv data thru "https://rxnav.nlm.nih.gov/REST/interaction" to identify any drug interactions using python. What else do I need in order to have the program be ready?
I got 200 when print status_code is that mean my code is up and ready?
import requests
response = requests.get("https://rxnav.nlm.nih.gov/REST/interaction")
print(response.status_code)
Here's how you'd hit this API, using requests and the details in their example:
import requests
uri = "https://rxnav.nlm.nih.gov/REST/interaction/interaction.json"
params = {'rxcui': 341248}
r = requests.get(uri, params)
Now you can check that r.status_code is 200, and get at the result of the request. For example:
r.json()
As you may realize, this returns a Python dictionary.
The general idea is that requsts.get() takes the base URL, followed by the query parameters, given as a dictionary. What you get back depends on the API endpoint you're querying, and/or on the parameters. In this, it's giving you JSON. Others might give you text (see r.text for this representation), or bytes (r.content).
I'm looking for a way to build urls in python3 without having to do string concatenation. I get that I can
import requests
url_endpoint = 'https://www.duckduckgo.com'
mydict = {'q': 'whee! Stanford!!!', 'something': 'else'}
resp = requests.get(url_endpoint, params=mydict)
print(resp.url) # THIS IS EXACTLY WHAT I WANT
or
from requests import Request, Session
s = Session()
req = Request('GET', url, params={'q': 'blah'})
print(req.url)
# I didn't get this to work, but from the docs
# it should build the url without making the call
or
url = baseurl + "?" + urllib.urlencode(params)
I like that the request library intelligently decides to drop ? if it isn't needed, but that code actually makes a full GET request so instead of just building a full text url (which I plan to dump to an html tag). I am using django, but I didn't see anything to help with that in the core library.
Django comes with QueryDicts which basically do everything you want.
def make_url(url, args=None):
query = QueryDict(mutable=True)
query.update(args or {})
return '{}{}{}'.format(url, '?' if query else '', query.urlencode())
It supports multiple values per argument just like you can encounter in a url: example.com/foo?a=1&a=2&a=3.
I'm working on an API wrapper. The spec I'm trying to build to has the following request in it:
curl -H "Content-type:application/json" -X POST -d data='{"name":"Partner13", "email":"example#example.com"}' http://localhost:5000/
This request produces the following response from a little test server I setup to see exatly what headers/params etc are sent as. This little script produces:
uri: http://localhost:5000/,
method: POST,
api_key: None,
content_type: application/json,
params: None,
data: data={"name":"Partner13", "email":"example#example.com"}
So that above is the result I want my python script to create when it hits the little test script.
I'm using the python requests module, which is the most beautiful HTTP lib I have ever used. So here is my python code:
uri = "http://localhost:5000/"
headers = {'content-type': 'application/json' }
params = {}
data = {"name":"Partner13", "email":"example#exmaple.com"}
params["data"] = json.dumps(data)
r = requests.post(uri, data=params, headers=headers)
So simple enough stuff. Set the headers, and create a dictionary for the POST parameters. That dictionary has one entry called "data" which is the JSON string of the data I want to send to the server. Then I call the post. However, the result my little test script gives back is:
uri: http://localhost:5000/,
method: POST,
api_key: None,
content_type: application/json,
params: None,
data: data=%7B%22name%22%3A+%22Partner13%22%2C+%22email%22%3A+%22example%40example.com%22%7D
So essentially the json data I wanted to send under the data parameter has been urlendcoded.
Does anyone know how to fix this? I have looked through the requests documentation and cannot seem to find a way to not auto urlencode the send data.
Thanks very much,
Kevin
When creating the object for the data keyword, simply assign a variable the result of json.dumps(data).
Also, because HTTP POST can accept both url parameters as well as data in the body of the request, and because the requests.post function has a keyword argument named "params", it might be better to use a different variable name for readability. The requests docs use the variable name "payload", so thats what I use.
data = {"name":"Partner13", "email":"example#exmaple.com"}
payload = json.dumps(data)
r = requests.post(uri, data=payload, headers=headers)
Requests automatically URL encodes dictionaries passed as data here. John_GG's solution works because rather than posting a dictionary containing the JSON encoded string in the 'data' field it simply passes the JSON encoded string directly: strings are not automatically encoded. I can't say I understand the reason for this behaviour in Requests but regardless, it is what it is. There is no way to toggle this behaviour off that I can find.
Best of luck with it, Kevin.
There's a lot of stuff out there on urllib2 and POST calls, but I'm stuck on a problem.
I'm trying to do a simple POST call to a service:
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
content = urllib2.urlopen(url=url, data=data).read()
print content
I can see the server logs and it says that I'm doing GET calls, when I'm sending the data
argument to urlopen.
The library is raising an 404 error (not found), which is correct for a GET call, POST calls are processed well (I'm also trying with a POST within a HTML form).
Do it in stages, and modify the object, like this:
# make a string with the request type in it:
method = "POST"
# create a handler. you can specify different handlers here (file uploads etc)
# but we go for the default
handler = urllib2.HTTPHandler()
# create an openerdirector instance
opener = urllib2.build_opener(handler)
# build a request
data = urllib.urlencode(dictionary_of_POST_fields_or_None)
request = urllib2.Request(url, data=data)
# add any other information you want
request.add_header("Content-Type",'application/json')
# overload the get method function with a small anonymous function...
request.get_method = lambda: method
# try it; don't forget to catch the result
try:
connection = opener.open(request)
except urllib2.HTTPError,e:
connection = e
# check. Substitute with appropriate HTTP code.
if connection.code == 200:
data = connection.read()
else:
# handle the error case. connection.read() will still contain data
# if any was returned, but it probably won't be of any use
This way allows you to extend to making PUT, DELETE, HEAD and OPTIONS requests too, simply by substituting the value of method or even wrapping it up in a function. Depending on what you're trying to do, you may also need a different HTTP handler, e.g. for multi file upload.
This may have been answered before: Python URLLib / URLLib2 POST.
Your server is likely performing a 302 redirect from http://myserver/post_service to http://myserver/post_service/. When the 302 redirect is performed, the request changes from POST to GET (see Issue 1401). Try changing url to http://myserver/post_service/.
Have a read of the urllib Missing Manual. Pulled from there is the following simple example of a POST request.
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe', 'age' : '10'})
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
print response.read()
As suggested by #Michael Kent do consider requests, it's great.
EDIT: This said, I do not know why passing data to urlopen() does not result in a POST request; It should. I suspect your server is redirecting, or misbehaving.
The requests module may ease your pain.
url = 'http://myserver/post_service'
data = dict(name='joe', age='10')
r = requests.post(url, data=data, allow_redirects=True)
print r.content
it should be sending a POST if you provide a data parameter (like you are doing):
from the docs:
"the HTTP request will be a POST instead of a GET when the data parameter is provided"
so.. add some debug output to see what's up from the client side.
you can modify your code to this and try again:
import urllib
import urllib2
url = 'http://myserver/post_service'
opener = urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1))
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
content = opener.open(url, data=data).read()
Try this instead:
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
req = urllib2.Request(url=url,data=data)
content = urllib2.urlopen(req).read()
print content
url="https://myserver/post_service"
data["name"] = "joe"
data["age"] = "20"
data_encoded = urllib2.urlencode(data)
print urllib2.urlopen(url + "?" + data_encoded).read()
May be this can help