I'm sending a POST request with JSON body to a Django server (fairly standard). On the server I need to decode this using json.loads().
The problem is how do I get the body of the request in a string format?
I have the following code currently:
body_data = {}
if request.META.get('CONTENT_TYPE', '').lower() == 'application/json' and len(request.body) > 0:
try:
body_data = json.loads(request.body)
except Exception as e:
return HttpResponseBadRequest(json.dumps({'error': 'Invalid request: {0}'.format(str(e))}), content_type="application/json")
However, this gives an error the JSON object must be str, not 'bytes'.
How do I retrieve the body of the request as a string, with the correct encoding applied?
The request body, request.body, is a byte string. In Python 3.0 to 3.5.x, json.loads() will only accept a unicode string, so you must decode request.body before passing it to json.loads().
body_unicode = request.body.decode('utf-8')
body_data = json.loads(body_unicode)
In Python 2, json.loads will accept a unicode string or a byte sting, so the decode step is not necessary.
When decoding the string, I think you're safe to assume 'utf-8' - I can't find a definitive source for this, but see the quote below from the jQuery docs:
Note: The W3C XMLHttpRequest specification dictates that the charset is always UTF-8; specifying another charset will not force the browser to change the encoding.
In Python 3.6, json.loads() accepts bytes or bytearrays. Therefore you shouldn't need to decode request.body (assuming it's encoded in UTF-8).
I believe that the other end from where you receive this request does not convert the data to JSON before sending the request. Either you have to convert the data to JSON before you send, or just try accessing request.body in your view.
If your goal is to end up with a dictionary of the data you have just sent to the server using JSON, save yourself the trouble of decoding the body yourself and use the request.POST dictionary-like object django already provides out-of-the-box.
So suppose you POST this to the server:
{ 'foo': 'bar' }
Then the following method
def my_handler(request):
foo = request.POST['foo']
print(foo)
Would print bar to the console
Related
I'm using an API which requires to send a callback to a url. Thus I configure my url and my view :
def get_callback(request):
...
some treatment with request.body
My view always returns that request.body contains " b'' ". However, it must contain a lot of informations, encoded in JSON.
Indeed I know that theses informations are well sent to the callback url, I tried with requestbin.in (http://requestb.in/1d4dkk01?inspect#10fl7s) and the raw body is full.
What could case the body to be empty ? Could it be the nginx configuration ? or in setting.py ?
Thanks you
I think, you should return response have kind of json data for view. Like it
import json
def get_call_back(request):
# Do something to return dictionary same as {'abc': xyz}
json_data = json.dumps(data)
return HttpResponse(json_data, content_type='application/json')
I'm having an issue trying to decode a python dictionary sent to my server as json.
This is what I have in my application:
payload = {'status':[bathroomId,item,percentage,timestamp]}
r=requests.post(url,None,json.dumps(payload))
Here is what I do in my Flask server:
req = request.get_json()
print req['status']
When I try to print the content of req['status'], it seems like python won't recognize it as a dictionary and I get an internal server error.
I tried printing req, and I get None
What am I missing?
Unless you set the Content-Type header to application/json in your request, Flask will not attempt to decode any JSON found in your request body.
Instead, get_json will return None (which is what you're seeing right now).
So, you need to set the Content-Type header in your request.
Fortunately since version 2.4.2 (released a year ago), requests has a helper argument to post JSON; this will set the proper headers for you. Use:
requests.post(url, json=payload)
Alternatively (e.g. using requests < 2.4.2), you can set the header yourself:
requests.post(url, data=json.dumps(payload), headers={"Content-Type": "application/json"})
Here is the relevant code from Flask:
Flask only loads JSON if is_json is True (or if you pass force=True to get_json). Otherwise, it returns None.
is_json looks at the Content-Type header, and looks for application/json there.
I am writing a python service with webapp2 and want to get a byte-array from a client POST request and save it to a file.
Whenever I am trying to get the data field that containes the byte array from the request object I get an exception saying:
'utf8' codec can't decode byte 0xff in position 0: invalid start byte
my post() code:
def post(self):
file_data = self.request.get('file_data')
Is there another method I should use to read the field because it's not a string?
You can use self.request.body to get the raw request (a byte string)
utf-8 json string request example:
def post(self):
binary_body = self.request.body # get the binary request
utf8_json_string = binary_body.decode('utf-8')
json_object = json.loads(utf8_json_string)
More on unicode here.
I cannot for the life of me figure out how to perform an HTTP PUT request with verbatim binary data in Python 2.7 with the standard Python libraries.
I thought I could do it with urllib2, but that fails because urllib2.Request expects its data in application/x-www-form-urlencoded format. I do not want to encode the binary data, I just want to transmit it verbatim, after the headers that include
Content-Type: application/octet-stream
Content-Length: (whatever my binary data length is)
This seems so simple, but I keep going round in circles and can't seem to figure out how.
How can I do this? (aside from open up a raw binary socket and write to it)
I found out my problem. It seems there is some obscure behavior in urllib2.Request / urllib2.urlopen() (at least in Python 2.7)
The urllib2.Request(url, data, headers) constructor seems to expect the same type of string in its url and data parameters.
I was giving the data parameter raw data from a file read() call (which in Python 2.7 returns it in the form of a 'plain' string), but my url was accidentally Unicode because I concatenated a portion of the URL from the result of another function which returned Unicode strings.
Rather than trying to "downcast" url from Unicode -> plain strings, it tried to "upcast" the data parameter to Unicode, and it gave me a codec error. (oddly enough, this happens on the urllib2.urlopen() function call, not the urllib2.Request constructor)
When I changed my function call to
# headers contains `{'Content-Type': 'application/octet-stream'}`
r = urllib2.Request(url.encode('utf-8'), data, headers)
it worked fine.
You're misreading the documentation: urllib2.Request expects the data already encoded, and for POST that usually means the application/x-www-form-urlencoded format. You are free to associate any other, binary data, like this:
import urllib2
data = b'binary-data'
r = urllib2.Request('http://example.net/put', data,
{'Content-Type': 'application/octet-stream'})
r.get_method = lambda: 'PUT'
urllib2.urlopen(r)
This will produce the request you want:
PUT /put HTTP/1.1
Accept-Encoding: identity
Content-Length: 11
Host: example.net
Content-Type: application/octet-stream
Connection: close
User-Agent: Python-urllib/2.7
binary-data
Have you considered/tried using httplib?
HTTPConnection.request(method, url[, body[, headers]])
This will send a request to the server using the HTTP request method
method and the selector url. If the body argument is present, it
should be a string of data to send after the headers are finished.
Alternatively, it may be an open file object, in which case the
contents of the file is sent; this file object should support fileno()
and read() methods. The header Content-Length is automatically set to
the correct value. The headers argument should be a mapping of extra
HTTP headers to send with the request.
This snipped worked for me to PUT an image:
on HTTPS site. If you don't need HTTPS, use
httplib.HTTPConnection(URL) instead.
import httplib
import ssl
API_URL="api-mysight.com"
TOKEN="myDummyToken"
IMAGE_FILE="myimage.jpg"
imageID="myImageID"
URL_PATH_2_USE="/My/image/" + imageID +"?objectId=AAA"
headers = {"Content-Type":"application/octet-stream", "X-Access-Token": TOKEN}
imgData = open(IMAGE_FILE, "rb")
REQUEST="PUT"
conn = httplib.HTTPSConnection(API_URL, context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))
conn.request(REQUEST, URL_PATH_2_USE, imgData, headers)
response = conn.getresponse()
result = response.read()
I am using python to send a request to a server. I get a cookie from the server. I am trying to decode the encoding scheme used by the server - I suspect it's either utf-8 or base64.
So I create my header and connection objects.
resp, content = httpobj.request(server, 'POST', headers=HTTPheader, body=HTTPbody)
And then i extract the cookie from the HTTP Stream
cookie= resp['set-cookie']
I have tried str.decode() and unicode() but I am unable to get the unpacked content of the cookie.
Assume the cookie is
MjAyMTNiZWE4ZmYxYTMwOVPJ7Jh0B%2BMUcE4si5oDcH7nKo4kAI8CMYgKqn6yXpgtXOSGs8J9gm20bgSlYMUJC5rmiQ1Ch5nUUlQEQNmrsy5LDgAuuidQaZJE5z%2BFqAJPnlJaAqG2Fvvk5ishG%2FsH%2FA%3D%3D
The output I am expecting is
20213bea8ff1a309SÉì˜tLQÁ8².hÁûœª8<Æ
*©úÉzµs’Ïö¶Ñ¸•ƒ$.kš$5gQIPf®Ì¹,8�ºèA¦IœöZ€$ùå% *ao¾Nb²¶ÁöÃ
Try like this:
import urllib
import base64
cookie_val = """MjAyMTNiZWE4ZmYxYTMwOVPJ7Jh0B%2BMUcE4si5oDcH7nKo4kAI8CMYgKqn6yXpgtXOSGs8J9gm20bgSlYMUJC5rmiQ1Ch5nUUlQEQNmrsy5LDgAuuidQaZJE5z%2BFqAJPnlJaAqG2Fvvk5ishG%2FsH%2FA%3D%3D"""
res = base64.b64decode(urllib.unquote(cookie_val))
print repr(res)
Output:
"20213bea8ff1a309S\xc9\xec\x98t\x07\xe3\x14pN,\x8b\x9a\x03p~\xe7*\x8e$\x00\x8f\x021\x88\n\xaa~\xb2^\x98-\\\xe4\x86\xb3\xc2}\x82m\xb4n\x04\xa5`\xc5\t\x0b\x9a\xe6\x89\rB\x87\x99\xd4RT\x04#\xd9\xab\xb3.K\x0e\x00.\xba'Pi\x92D\xe7?\x85\xa8\x02O\x9eRZ\x02\xa1\xb6\x16\xfb\xe4\xe6+!\x1b\xfb\x07\xfc"
Of course the result here is a 8-bit string, so you have to decode it to get the the string that you want, i'm not sure which encoding to use, but there is the decoding result using the unicode-escape (unicode literal) :
>>> print unicode(res, 'unicode-escape')
20213bea8ff1a309SÉìtãpN,p~ç*$1ª~²^-\ä³Â}m´n¥`ÅBÔRT#Ù«³.K.º'PiDç?¨ORZ¡¶ûäæ+!ûü
Well Hope this can help .