I am trying to form a web payload for a particular request body but unable to get it right. What I need is to pass my body data as below
data={'file-data':{"key1": "3","key2": "6","key3": "8"}}
My complete payload request looks like this
payload={url,headers, data={'file-data':{"key1": "3","key2": "6","key3": "8"}},files=files}
However, when I pass this, python tries to parse each individual key value and assigns to the 'file-data' key like this
file-data=key1
file-data=key2
file-data=key3
and so on for as many keys I pass within the nested dictionary. The requirement however, is to pass the entire dictionary as a literal content like this(without splitting the values by each key):
file-data={"key1": "3","key2": "6","key3": "8"}
The intended HTTP trace should thus ideally look like this:
POST /sample_URL/ HTTP/1.1
Host: sample_host.com
Authorization: Basic XYZ=
Cache-Control: no-cache
Content-Type: multipart/form-data; boundary=----UVWXXXX
------WebKitFormBoundaryXYZ
Content-Disposition: form-data; name="file-data"
{"key1": "3","key2": "6","key3":"8" }
------WebKitFormBoundaryMANZXC
Content-Disposition: form-data; name="file"; filename=""
Content-Type:
------WebKitFormBoundaryBNM--
As such, I want to use this as part of a payload for a POST request(using python requests library). Any suggestions are appreciated in advance-
Edit1: To provide more clarity, the API definition is this:
Body
Type: multipart/form-data
Form Parameters
file: required (file)
The file to be uploaded
file-data: (string)
Example:
{
"key1": "3",
"key2": "6",
"key3": "8"
}
The python code snippet I used(after checking suggestions) is this:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
url = "https://sample_url/upload"
filepath='mypath'
filename='logo.png'
f=open(filepath+'\\'+filename)
filedata={'file-data':"{'key1': '3','key2': '6','key3': '8'}"}
base64string = encodestring('%s:%s' % ('user', 'password').replace('\n', '')
headers={'Content-type': 'multipart/form-data','Authorization':'Basic %s' % base64string}
r = requests.post(url=url,headers=headers,data=filedata,files={'file':f})
print r.text
The error I get now is still the same as shown below:
{"statusCode":400,"errorMessages":[{"severity":"ERROR","errorMessage":"An exception has occurred"]
It also says that some entries are either missing or incorrect. Note that I have tried passing the file parameter after opening it in binary mode as well but it throws the same error message
I got the HTTP trace printed out via python too and it looks like this:
send: 'POST sample_url HTTP/1.1
Host: abc.com
Connection: keep-alive
Accept-Encoding: gzip,deflate
Accept: */*
python-requests/2.11.1
Content-type: multipart/form-data
Authorization: Basic ABCDXXX=
Content-Length: 342
--CDXXXXYYYYY
Content-Disposition:form-data; name="file-data"
{\'key1\': \'3\',\'key2\': \'6\'
,\'key3\': \'8\'}
--88cdLMNO999999
Content-Disposition: form-data; name="file";
filename="logo.png"\x89PNG\n\r\n--cbCDEXXXNNNN--
If you want to post JSON with python requests, you should NOT use data but json:
r = requests.post('http://httpbin.org/post', json={"key": "value"})
I can only guess that you are using data because of your example
payload={url,headers, data={'file-data':{"key1": "3","key2": "6","key3": "8"}},files=files}
Whis is not valid python syntax btw.
Related
I am trying to send through an image using the following code: This is just a part of my code, I didn't include the headers here but they are set up correctly, with content-type as content-type: multipart/form-data; boundary=eBayClAsSiFiEdSpOsTiMaGe
img = "piano.jpg"
f = open(img,'rb')
out = f.read()
files = {'file':out}
p = requests.post("https://ecg-api.gumtree.com.au/api/pictures",headers=headers, data=files)
f.close()
I get a 400 error incorrect multipart/form-data format
How do I send the image properly?
Extra Details:
Network analysis shows the following request been sent:
POST https://ecg-api.gumtree.com.au/api/pictures HTTP/1.1
host: ecg-api.gumtree.com.au
content-type: multipart/form-data; boundary=eBayClAsSiFiEdSpOsTiMaGe
authorization: Basic YXV5grehg534
accept: */*
x-ecg-ver: 1.49
x-ecg-ab-test-group: gblios_9069_b;gblios-8982-b
accept-encoding: gzip
x-ecg-udid: 73453-7578p-8657
x-ecg-authorization-user: id="1635662", token="ee56hgjfjdghgjhfj"
accept-language: en-AU
content-length: 219517
user-agent: Gumtree 12.6.0 (iPhone; iOS 13.3; en_AU)
x-ecg-original-machineid: Gk435454-hhttehr
Form data:
file: ����..JFIF.....H.H..��.LExif..MM.*..................�i.........&......�..
I cut off the the formdata part for file as its too long. My headers are written as follows (I have made up the actual auth values here):
idd = "1635662"
token = "ee56hgjfjdghgjhfj"
headers = {
"authority":"ecg-api.gumtree.com.au",
"content-type":"multipart/form-data; boundary=eBayClAsSiFiEdSpOsTiMaGe",
"authorization":"Basic YXV5grehg534",
"accept":"*/*",
"x-ecg-ver":"1.49",
"x-ecg-ab-test-group":"gblios_9069_b;gblios-8982-b",
"accept-encoding":"gzip",
"x-ecg-udid":"73453-7578p-8657",
"x-ecg-authorization-user":f"id={idd}, token={token}",
"accept-language":"en-AU",
"content-length":"219517",
"user-agent":"Gumtree 12.6.0 (iPhone; iOS 13.3; en_AU)",
"x-ecg-original-machineid":"Gk435454-hhttehr"
}
Maybe its the way I have written the headers? I suspect its the way I have written the x-ecg-authorization-user part in headers? Because I realise even putting random values for the token or id gives me the same 400 error incorrect multipart/form-data format
You can try the following code. Don't set content-type in the headers.Let Pyrequests do that for you
files = {'file': (os.path.basename(filename), open(filename, 'rb'), 'application/octet-stream')}
upload_files_url = "url"
headers = {'Authorization': access_token, 'JWTAUTH': jwt}
r2 = requests.post(parcels_files_url, files=files, headers=headers)
I'm sending a get http request to a REST webservice. The webservice answers with an multipart response.
The response is split into 2 parts: The first one is a Json message and the other one is the content corresponding to a STL file. Here is an example:
--BOUNDARY
Content-Disposition: form-data
Content-Type: application/json
Content-Length: 169
{
"lightSensitivity": "Average",
"meshDensity": "Low",
"nbFaces": 25739,
"nbVertices": 13597,
"scanFormatEncoding": "RAW",
"state": "finished"
}
--BOUNDARY
Content-Disposition: form-data
Content-Type: application/octet-stream
Content-Length: 472032
�Cì��B��|��#��#à�����##��������#ٗ#��u�JcA_�#�x�#��A�|#�H�#��MA�v#��#^��zA�l#à�{�2HA�h#è���r� AON#�Tߛ�^�AZ.#ø4��?� A2���Aa9?���O�6AK+?��m�vA�&?�4W���S A�$?Ô����A��>Ð�����A*�>�����A��>�U���jAu�>�#���Z�A��#���BX1;#Aw#�N�BhbJ#~o#�cM�B��P#=]#Û��B8�V#�X#Õ |B��_#5#�8CuBPg#�;#�~nB�)o#�%#àBgB��u#]#�!�`B�r~#�#���YB�a�# �?�qBSBpO�#l�?�=�LBlc�#��?ÌYFB���#��?þ#BD{�#�?�5�9Bx��#)�?���3BP��#7�?�i-B��#;�?Â]%B4!�#\x?��{B�#Fb?�n�B��#Y?���B�D�#kB?��MB��#!?�{��Ad��#y?�+j�A,�#?�(t�A���#��>����A�#��>�6��A,��#��>äL�AH��#��>�n��AH��#Լ>��AX�#
�>Ì�vA�D�#3[>à�S#�L�#nF>�#�?�%�#�:>�#��H8�#u2>���>�|��#�.>�p��h&�# (>à����6A�>È��N�AC�=À�#�0AA�=�HA���A��=��U^�z�A��=�|�� Ai�=���G)B���#LP>�X!B�1�#�2>Ø�B좩#\*>�V�B���#��=�&��At��#v�=�C�A]�#5w=ÄYhA���#TC=��;�#,��#�=����?���#�=�^����#R��AN>����B��7#H>È��Bx�D#��=��6�B��L#��=�W��Bp�R##�=�9�{B��\#(�=��tBPNb#]�=�!�mBpRk#]�=É%gB�1r# �=��n`B�?z#��=���YBm�#�=æSB,��#h=Ò�LB$L�#�`=�oPFBΌ#�O=��#B�`�#PB=÷�9B���#�:=è~3B|[�#�.=�P[-Bć�#q=�q]%B`��#�
=ÐTB���#���܊�#��;��͘�l�#��;�����(1AĒ;�l�ԦAGz;�x�#���A�r;�(�#�ȔASm;�0J^���AAg;�H�{��AM];�xČ��
A�I;�L͛�آ
Aw%;�\4����A];�����A;Ð���A��:Ìu��AС:����:�AQ�:�0V���-%A:�z
Could you tell me how I can decode this response? Currently my code looks like:
endpoint = "http://localhost:4000/api/v1/test"
data = {"ip":"1.1.2.3"}
headers = {"Authorization":"Bearer "+ token}
response = requests.get(endpoint,data=data,headers=headers)
# request is OK, now I have to handle the content: decode JSON and save the STL content in a file
I am using anki-connect to communicate with Anki, a spaced repetition software.
In readme.md, it uses following command to get deck name.
curl localhost:8765 -X POST -d "{\"action\": \"deckNames\", \"version\": 5}"
It works right in my Windows system.
How can I use python instead of cURL?
I've tried this but get no luck.
import requests
r = requests.post("http://127.0.0.1:8765", data={'action': 'guiAddCards', 'version': 5})
print(r.text)
When creating request you should:
provide Content-Type header
provide data in format that matches Content-Type header
make sure application supports the format
Both curl and python examples you gave sends request with Content-Type: application/x-www-form-urlencoded, the default one. The difference is curl passes string and python passes an array.
Let's compare curl and requests and what is really posted:
Curl
$ curl localhost -X POST -d "{\"action\": \"deckNames\", \"version\": 5}"
Headers:
Host: localhost
User-Agent: curl/7.52.1
Accept: */*
Content-Length: 37
Content-Type: application/x-www-form-urlencoded
Posted data:
[
'{"action": "deckNames", "version": 5}'
]
Python
import requests
r = requests.post("http://127.0.0.1", data={'action': 'guiAddCards', 'version': 5})
print(r.text)
Headers:
Host: 127.0.0.1
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.10.0
Content-Length: 28
Content-Type: application/x-www-form-urlencoded
Posted data:
[
'action' -> 'guiAddCards',
'version' -> '5',
]
As you can see, incorrect post data format breaks your app.
To be sure, that posted JSON data will be properly read by application you should make requests like that:
Curl
$ curl localhost:8765 -H 'Content-Type: application/json' -d '{"action": "deckNames", "version": 5}'
Python
import requests
r = requests.post("http://127.0.0.1:8765", json={'action': 'guiAddCards', 'version': 5})
print(r.text)
I've tried following after digging and this works.
Can anybody share the reason. Thanks.
import requests
import json
#r = requests.post("http://127.0.0.1:8765", data={'action': 'guiAddCards', 'version': 5})
r = requests.post('http://localhost:8765', data=json.dumps({'action': 'guiAddCards', 'version': 5}))
print(r.text)
This is a reply to user2444791's answer. I can't reply with a comment because I don't have the reputation to comment (I'm new, please forgive a breech of etiquette!)
Without the exact error message, it's hard to be sure, but...
Looking at the Anki Connect API, it expects its POST-ed data to be a single string which contains a JSON object, not a key/value dictionary equivalent to that JSON object.
Every request consists of a JSON-encoded object containing an action, a version, and a set of contextual params.
Their example code (in Javascript): xhr.send(JSON.stringify({action, version, params}));
It might be as simple as sending your data in the wrong format. In the first example, you are sending a dictionary with the key/vale pairs already parsed. In the second example, you're sending a string for them to parse instead.
I am trying to post data to my server from my microcontroller. I need to send raw http data from my controller and this is what I am sending below:
POST /postpage HTTP/1.1
Host: https://example.com
Accept: */*
Content-Length: 18
Content-Type: application/json
{"cage":"abcdefg"}
My server requires JSON encoding and not form encoded request.
For the above request sent, I get an 400 error from the server, HTTP/1.1 400 Bad Request
However, when I try to reach the post to my server via a python script via my laptop, I am able to get a proper response.
import requests
url='https://example.com'
mycode = 'abcdefg'
def enter():
value = requests.post('url/postpage',
params={'cage': mycode})
print vars(value)
enter()
Can anyone please let me know where I could be going wrong in the raw http data I'm sending above ?
HTTP specifies the separator between headers as a single newline, and requires a double newline before the content:
POST /postpage HTTP/1.1
Host: https://example.com
Accept: */*
Content-Length: 18
Content-Type: application/json
{"cage":"abcdefg"}
If you don’t think you’ve got all of the request right, try seeing what was sent by Python:
response = ...
request = response.request # request is a PreparedRequest.
headers = request.headers
url = request.url
Read the docs for PreparedRequest for more information.
To pass a parameter, use this Python:
REQUEST = 'POST /postpage%s HTTP/1.1\r\nHost: example.com\r\nContent-Length: 0\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.4.3 CPython/2.7.9 Linux/4.4.11-v7+\r\n\r\n';
query = ''
for k, v in params.items():
query += '&' + k + '=' + v # URL-encode here if you want.
if len(query): query = '?' + query[1:]
return REQUEST % query
I'm sure I'm doing something simple wrong, but I can't for the life of me figure out how to set the "IsRead" property to true. It's the last step of my process that gets a filtered list of messagesa and stores and processes any attachments.
According to the docs "IsRead" is writable: http://msdn.microsoft.com/office%5Coffice365%5CAPi/complex-types-for-mail-contacts-calendar#ResourcesMessage
http://msdn.microsoft.com/office%5Coffice365%5CAPi/mail-rest-operations#MessageoperationsUpdatemessages
I'm using python 2.7 and the requests module:
# once file acquired mark the email as read
params = {'IsRead':'True'}
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, params, auth=(email,pwd))
log.debug( response )
The response that comes back is this:
{"error":{"code":"ErrorInvalidRequest","message":"Cannot read the request body."}}
What's the problem with my request?
At first glance it looks OK. I wonder if the Content-Type header isn't being set to "application/json" or something along those lines. Try getting a network trace and verify that the request looks something like:
PATCH https://outlook.office365.com/api/v1.0/Me/Messages('msgid') HTTP/1.1
Accept: application/json;odata.metadata=full
Authorization: Bearer <token>
Content-Type: application/json;odata.metadata=full
Host: outlook.office365.com
Content-Length: 24
Expect: 100-continue
Connection: Keep-Alive
{
"IsRead": "true"
}
Well I have an answer for myself and it is indeed a simple matter.
It was a mistake to not fully read how PATCH is different from GET or POST.
In short it's important to make sure your headers are set for the right content-type.
Here is the working code:
# once file acquired mark the email as read
changes = {u'IsRead':u'True'}
headers = {'Content-Type': 'application/json'}
json_changes = json.dumps(changes)
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, data=json_changes, auth=__AUTH, headers=headers)
log.debug( response )