We Keep Coding

Python POST multipart/form-data request different behavior from Postman

I'm attempting to use this API endpoint to upload a file:
https://h.app.wdesk.com/s/cerebral-docs/?python#uploadfileusingpost
With this python function:
def upload_file(token, filepath, table_id):
url = "https://h.app.wdesk.com/s/wdata/prep/api/v1/file"
headers = {
'Accept': 'application/json',
'Authorization': f'Bearer {token}'
}
files = {
"tableId": (None, table_id),
"file": open(filepath, "rb")
}
resp = requests.post(url, headers=headers, files=files)
print(resp.request.headers)
return resp.json()
The Content-Type and Content-Length headers are computed and added by the requests library internally as per their documentation. When assigning to the files kwarg in the post function, the library knows it's supposed to be a multipart/form-data request.
The print out of the request header is as follows, showing the Content-Type and Content-Length that the library added. I've omitted the auth token.
{'User-Agent': 'python-requests/2.24.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive',
'Authorization': 'Bearer <omitted>', 'Content-Length': '8201', 'Content-Type': 'multipart/form-data; boundary=bb582b9071574462d44c4b43ec4d7bf3'}
The json response from the API is:
{'body': ['contentType must not be null'], 'code': 400}
The odd thing is that the same request, when made through Postman, gives a different response - which is what I expected from Python as well.
{ "code": 409, "body": "duplicate file name" }
These are the Postman request headers:
POST /s/wdata/prep/api/v1/file HTTP/1.1
Authorization: Bearer <omitted>
Accept: */*
Cache-Control: no-cache
Postman-Token: 34ed08d4-4467-4168-a4e4-c83b16ce9afb
Host: h.app.wdesk.com
Content-Type: multipart/form-data; boundary=--------------------------179907322036790253179546
Content-Length: 8279
The Postman request also computes the Content-Type and Content-Length headers when the request is sent, and are not user specified.
I am quite confused as to why I'm getting two different behaviors from the API service for the same request.
There must be something I'm missing and can't figure out what it is.

Figured out what was wrong with my request, compared to NodeJS and Postman.
The contentType being referred to in the API's error message was the file parameter's content type, not the http request header Content-Type.
The upload started to work flawlessly when I updated my file parameter like so:
files = {
"tableId": (None, table_id),
"file": (Path(filepath).name, open(filepath, "rb"), "text/csv", None)
}
I learned that Python's requests library will not automatically add the file's mime type to the request body. We need to be explicit about it.
Hope this helps someone else too.

How to fix <Response 500> error in python requests?

I am using an API, which receives a pdf file and does some analysis, but I am receiving Response 500 always
Have initially tested using Postman and the request goes through, receiving response 200 with the corresponding JSON information. The SSL security should be turned off.
However, when I try to do request via Python, I always get Response 500
Python code written by me:
import requests
url = "https://{{BASE_URL}}/api/v1/documents"
fin = open('/home/train/aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf', 'rb')
files = {'file': fin}
r = requests.post(url, files=files, verify=False)
print (r)
#r.text is empty
Python code, produced by the Postman:
import requests
url = "https://{{BASE_URL}}/api/v1/documents"
payload = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"file\"; filename=\"aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf\"\r\nContent-Type: application/pdf\r\n\r\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--"
headers = {
'content-type': "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW",
'Content-Type': "application/x-www-form-urlencoded",
'cache-control': "no-cache",
'Postman-Token': "65f888e2-c1e6-4108-ad76-f698aaf2b542"
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
Have masked the API link as {{BASE_URL}} due to the confidentiality
Response by Postman:
{
"id": "5e69058e2690d5b0e519cf4006dfdbfeeb5261b935094a2173b2e79a58e80ab5",
"name": "aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf",
"fileIds": {
"original": "5e69058e2690d5b0e519cf4006dfdbfeeb5261b935094a2173b2e79a58e80ab5.pdf"
},
"creationDate": "2019-06-20T09:41:59.5930472+00:00"
}
Response by Python:
Response<500>
UPDATE:
Tried the GET request - works fine, as I receive the JSON response from it. I guess the problem is in posting pdf file. Is there any other options on how to post a file to an API?
Postman Response RAW:
POST /api/v1/documents
Content-Type: multipart/form-data; boundary=--------------------------375732980407830821611925
cache-control: no-cache
Postman-Token: 3e63d5a1-12cf-4f6b-8f16-3d41534549b9
User-Agent: PostmanRuntime/7.6.0
Accept: */*
Host: {{BASE_URL}}
cookie: c2b8faabe4d7f930c0f28c73aa7cafa9=736a1712f7a3dab03dd48a80403dd4ea
accept-encoding: gzip, deflate
content-length: 3123756
file=[object Object]
HTTP/1.1 200
status: 200
Date: Thu, 20 Jun 2019 10:59:55 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Location: /api/v1/files/95463e88527ecdc94393fde685ab1d05fa0ee0b924942f445b14b75e983c927e
api-supported-versions: 1.0
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Referrer-Policy: strict-origin
{"id":"95463e88527ecdc94393fde685ab1d05fa0ee0b924942f445b14b75e983c927e","name":"aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf","fileIds":{"original":"95463e88527ecdc94393fde685ab1d05fa0ee0b924942f445b14b75e983c927e.pdf"},"creationDate":"2019-06-20T10:59:55.7038573+00:00"}
CORRECT REQUEST
So, eventually - the correct code is the following:
import requests
files = {
'file': open('/home/train/aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf', 'rb'),
}
response = requests.post('{{BASE_URL}}/api/v1/documents', files=files, verify=False)
print (response.text)

A 500 error indicates an internal server error, not an error with your script.
If you're receiving a 500 error (as opposed to a 400 error, which indicates a bad request), then theoretically your script is fine and it's the server-side code that needs to be adjusted.
In practice, it could still be due a bad request though.
If you're the one running the API, then you can check the error logs and debug the code line-by-line to figure out why the server is throwing an error.
In this case though, it sounds like it's a third-party API, correct? If so, I recommend looking through their documentation to find a working example or contacting them if you think it's an issue on their end (which is unlikely but possible).

Why am I getting 500 Internal Server Error when calling post rest api via python request?

I'm trying to call a post rest API using python requests library. I'm able to get a proper response when I request using postman. When I tried to call it using python request library I get an internal server error.
import requests
url = "http://192.188.9.146:9886/getcontext/"
payload = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"project\"\r\n\r\ndaynight\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"file\"; filename=\"D:\\Downloads\\GTA.Imaging.Services\\GTA.Imaging.Services.Wrapper.TestApp\\PatternMatchingdata\\Go_To_Setting_Screen.jpg\"\r\nContent-Type: image/jpeg\r\n\r\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--"
headers = {
'content-type': "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW",
'cache-control': "no-cache",
'Postman-Token': "79668e4b-305b-404e-904f-92fc71a12f9f"
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
this gives me error as below
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
When I use the postman client app I get output with the expected response
Postman client
what am I doing wrong?
Any guidance will be very helpful thanks.

Remove the payload parameter and add files. Refer to requests documentation which explains this option of requests.request function.
import requests
url = "http://192.188.9.146:9886/getcontext/"
files = {'file': open('PATH_TO_FILE\Go_To_Setting_Screen.jpg','rb')}
data = {'project': 'daynight'}
response = requests.request("POST", url, files = files, data = data)
print(response.text)

Python HTTP Post with upload file and headers generated from Postman

I'm using Python 2.7.
I want to make a HTTP POST using requests, where I upload a file and a key that must go in the HTTP Headers.
For that I've used the application Postman, where it works really fine.
On Postman I've added only the necessary header, which is a Authorization with some key.
On the body, Ive choosen form-data and then the key is an input_image, and they the image itself.
Now I want to replicate this into Python2.7, so I've chose to see the Python code on Postman, which was this one:
import requests
url = "https://foo.com/bar/stuff"
payload = "------WebKitFormBoundary7MA4YDxkTrZu1gW\r\nContent-Disposition: form-data; name=\"input_image\"; filename=\"C:\\Test\\projs\\Supermarket\\doritos.jpeg\"\r\nContent-Type: image/jpeg\r\n\r\n\r\n------WebKitFormBoundary7MA4YDxkTrZu1gW--"
headers = {
'content-type': "multipart/form-data; boundary=----WebKitFormBoundary7MA4YDxkTrZu1gW",
'Authorization': "myAuthorizationKey",
'Cache-Control': "no-cache",
'Postman-Token': "0efwd6e8-051c-4ed5-8d6f-7b1bd135f4d5"
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
This simply doesn't work. It has the same behaviour as if I didn't send any image using Postman. It looks like the payload string is not being send correctly.
Question:
What is wrong with this Postman auto-generation code in order to send a HTTP POST with image upload and with header at the same time in Python?

I think Postman is doing some logic we are not really aware of. But the package requests provide a way to upload images.
files = {'media': open('my_image.jpg', 'rb')}
r = requests.post(url, files=files, headers=hearders)
According to the server you are sending the image to, the parameters name, this code might need to be slightly changed.

the only trick works here is your code should be same as you post request in postman, no extra headers need to be added , your post request should look like the same as it is in postman.
I could do this by changing my file to an image file and then posting it in my post request.
with open('grass-small.png', 'rb') as imageFile:
imageStr = base64.urlsafe_b64encode(imageFile.read())
files = {'document': ('grass-small.png', imageStr ), 'document_type':(None,'grass')}

This worked for me
import requests
url = 'http://iizuka.cs.tsukuba.ac.jp/projects/colorization/web/'
files = {'file': ("my_img_path/myImage.jpeg", open('my_img_path/myImage.jpeg', 'rb'),'image/jpg')}
r = requests.post(url, files=files)

POST multipart-form image with python through rest API call

I am relatively new to python and enjoying every day I program in it. I have been looking around for a possible solution to figure out how to post an image in a multipart-form, binary format, with a form tag. The API I am trying to call is expecting a binary image in a form.
The request payload sample I have is:
----WebkitFormBoundaryM817iTBsSwXz0iv8
Content-Disposition: form-data, name="image"; filename="123BMW.jpg"
Content-Type: image/jpeg
----WebkitFormBoundaryM817iTBsXwxz0iv8
I have tried several ideas based on some basic requests examples.
Any ideas, thoughts or pointers on where to start looking for such a solution?
def Post_Image(urlPath, filePath, fileName):
url = urlPath headers = {'content-type': 'multipart/form-data'}
files = {'file':(fileName, open(filePath,'rb'))}
payload = {"Content-Disposition": "form-data", "name":fileName}
payload = urllib.urlencode(payload)
resp = requests.post(url, data=payload, headers=headers, files= files)

The problem is you're setting both the data and files parameters, this part of the code sample here:
payload = urllib.urlencode(payload)
resp = requests.post(url, data=payload, headers=headers, files= files)
If both are present, and data value is a string, only it will be in the request. Drop it, and the files will be present.