Related
I wrote endpoint called foobar in default.py controller of web2py, it looks like
#service.json
def foobar(*args, **vars):
payload = request.vars.student
print(payload)
#Above payload prints [rollNo, firstName, lastName]
#Expected is {"rollNo": 6299857, "FirstName": Prasad, "LastName": Telkikar}
fileObj= request.vars.video
studentActivity = fileObj.file.read()
#Currently I am unable to do it from unit test, but it works from postman
print(f"Student's Roll number = {payload.rollNo} \n FirstName = {payload.firstName}, \n lastName = {payload.lastName}, fileSize = {fileObj.file.tell()}")
#storing studentActivity video to specific location
#Doing business logic
Now I am writing unit test for this endpoint, where i am trying to call this endpoint using requests,
import requests
import unittest
...
class TestStudentsData(unittest.TestCase):
def test_Individual_student(self):
payload = dict(rollNo=6299857, firstName="Prasad", lastName="Telkikar")
url = "http://127.0.0.1:8000/v1/foobar"
header = dict(Authorization="Bearer <Token>")
response = requests.post(url,files={'studentActivity':open("Activity.mp4", 'rb')}, data=payload, headers=headers)
self.assertEqual(response.status_code, 200)
if __name__ == '__main__':
unittest.main(verbosity=2)
Here I am unable to pass student payload as a json.
How can I pass json payload with studentActivity file using requests?
What I tried so far?
I tried both the approaches given in this SO answer.
I read requests documentation where it says "the json parameter is ignored if either data or files is passed" requests documentation
I resolved this problem by adding proper content-type to payload,
import os
...
filePath = os.getcwd()
files = {'studentActivity':open(filePath, "Activity.mp4", 'rb'),
'payload':(None, payload, 'application/json')}
#^^^^^^^^^^^^^^^^^ This was missing
response = requests.post(url,files={'studentActivity':open("Activity.mp4", 'rb')}, data=payload, headers=headers)
How to send a multipart/form-data with requests in python? How to send a file, I understand, but how to send the form data by this method can not understand.
Basically, if you specify a files parameter (a dictionary), then requests will send a multipart/form-data POST instead of a application/x-www-form-urlencoded POST. You are not limited to using actual files in that dictionary, however:
>>> import requests
>>> response = requests.post('http://httpbin.org/post', files=dict(foo='bar'))
>>> response.status_code
200
and httpbin.org lets you know what headers you posted with; in response.json() we have:
>>> from pprint import pprint
>>> pprint(response.json()['headers'])
{'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate',
'Connection': 'close',
'Content-Length': '141',
'Content-Type': 'multipart/form-data; '
'boundary=c7cbfdd911b4e720f1dd8f479c50bc7f',
'Host': 'httpbin.org',
'User-Agent': 'python-requests/2.21.0'}
Better still, you can further control the filename, content type and additional headers for each part by using a tuple instead of a single string or bytes object. The tuple is expected to contain between 2 and 4 elements; the filename, the content, optionally a content type, and an optional dictionary of further headers.
I'd use the tuple form with None as the filename, so that the filename="..." parameter is dropped from the request for those parts:
>>> files = {'foo': 'bar'}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--bb3f05a247b43eede27a124ef8b968c5
Content-Disposition: form-data; name="foo"; filename="foo"
bar
--bb3f05a247b43eede27a124ef8b968c5--
>>> files = {'foo': (None, 'bar')}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--d5ca8c90a869c5ae31f70fa3ddb23c76
Content-Disposition: form-data; name="foo"
bar
--d5ca8c90a869c5ae31f70fa3ddb23c76--
files can also be a list of two-value tuples, if you need ordering and/or multiple fields with the same name:
requests.post(
'http://requestb.in/xucj9exu',
files=(
('foo', (None, 'bar')),
('foo', (None, 'baz')),
('spam', (None, 'eggs')),
)
)
If you specify both files and data, then it depends on the value of data what will be used to create the POST body. If data is a string, only it willl be used; otherwise both data and files are used, with the elements in data listed first.
There is also the excellent requests-toolbelt project, which includes advanced Multipart support. It takes field definitions in the same format as the files parameter, but unlike requests, it defaults to not setting a filename parameter. In addition, it can stream the request from open file objects, where requests will first construct the request body in memory:
from requests_toolbelt.multipart.encoder import MultipartEncoder
mp_encoder = MultipartEncoder(
fields={
'foo': 'bar',
# plain file object, no filename or mime type produces a
# Content-Disposition header with just the part name
'spam': ('spam.txt', open('spam.txt', 'rb'), 'text/plain'),
}
)
r = requests.post(
'http://httpbin.org/post',
data=mp_encoder, # The MultipartEncoder is posted as data, don't use files=...!
# The MultipartEncoder provides the content-type header with the boundary:
headers={'Content-Type': mp_encoder.content_type}
)
Fields follow the same conventions; use a tuple with between 2 and 4 elements to add a filename, part mime-type or extra headers. Unlike the files parameter, no attempt is made to find a default filename value if you don't use a tuple.
Requests has changed since some of the previous answers were written. Have a look at this Issue on Github for more details and this comment for an example.
In short, the files parameter takes a dictionary with the key being the name of the form field and the value being either a string or a 2, 3 or 4-length tuple, as described in the section POST a Multipart-Encoded File in the Requests quickstart:
>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
In the above, the tuple is composed as follows:
(filename, data, content_type, headers)
If the value is just a string, the filename will be the same as the key, as in the following:
>>> files = {'obvius_session_id': '72c2b6f406cdabd578c5fd7598557c52'}
Content-Disposition: form-data; name="obvius_session_id"; filename="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52
If the value is a tuple and the first entry is None the filename property will not be included:
>>> files = {'obvius_session_id': (None, '72c2b6f406cdabd578c5fd7598557c52')}
Content-Disposition: form-data; name="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52
You need to use the files parameter to send a multipart form POST request even when you do not need to upload any files.
From the original requests source:
def request(method, url, **kwargs):
"""Constructs and sends a :class:`Request <Request>`.
...
:param files: (optional) Dictionary of ``'name': file-like-objects``
(or ``{'name': file-tuple}``) for multipart encoding upload.
``file-tuple`` can be a 2-tuple ``('filename', fileobj)``,
3-tuple ``('filename', fileobj, 'content_type')``
or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``,
where ``'content-type'`` is a string
defining the content type of the given file
and ``custom_headers`` a dict-like object
containing additional headers to add for the file.
The relevant part is: file-tuple can be a:
2-tuple (filename, fileobj)
3-tuple (filename, fileobj, content_type)
4-tuple (filename, fileobj, content_type, custom_headers).
☝ What might not be obvious is that fileobj can be either an actual file object when dealing with files, OR a string when dealing with plain text fields.
Based on the above, the simplest multipart form request that includes both files to upload and form fields will look like this:
import requests
multipart_form_data = {
'upload': ('custom_file_name.zip', open('myfile.zip', 'rb')),
'action': (None, 'store'),
'path': (None, '/path1')
}
response = requests.post('https://httpbin.org/post', files=multipart_form_data)
print(response.content)
☝ Note the None as the first argument in the tuple for plain text fields — this is a placeholder for the filename field which is only used for file uploads, but for text fields passing None as the first parameter is required in order for the data to be submitted.
Multiple fields with the same name
If you need to post multiple fields with the same name then instead of a dictionary you can define your payload as a list (or a tuple) of tuples:
multipart_form_data = (
('file2', ('custom_file_name.zip', open('myfile.zip', 'rb'))),
('action', (None, 'store')),
('path', (None, '/path1')),
('path', (None, '/path2')),
('path', (None, '/path3')),
)
Streaming requests API
If the above API is not pythonic enough for you, then consider using requests toolbelt (pip install requests_toolbelt) which is an extension of the core requests module that provides support for file upload streaming as well as the MultipartEncoder which can be used instead of files, and which also lets you define the payload as a dictionary, tuple or list.
MultipartEncoder can be used both for multipart requests with or without actual upload fields. It must be assigned to the data parameter.
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder
multipart_data = MultipartEncoder(
fields={
# a file upload field
'file': ('file.zip', open('file.zip', 'rb'), 'text/plain')
# plain text fields
'field0': 'value0',
'field1': 'value1',
}
)
response = requests.post('http://httpbin.org/post', data=multipart_data,
headers={'Content-Type': multipart_data.content_type})
If you need to send multiple fields with the same name, or if the order of form fields is important, then a tuple or a list can be used instead of a dictionary:
multipart_data = MultipartEncoder(
fields=(
('action', 'ingest'),
('item', 'spam'),
('item', 'sausage'),
('item', 'eggs'),
)
)
Here is the simple code snippet to upload a single file with additional parameters using requests:
url = 'https://<file_upload_url>'
fp = '/Users/jainik/Desktop/data.csv'
files = {'file': open(fp, 'rb')}
payload = {'file_id': '1234'}
response = requests.put(url, files=files, data=payload, verify=False)
Please note that you don't need to explicitly specify any content type.
NOTE: Wanted to comment on one of the above answers but could not because of low reputation so drafted a new response here.
By specifying a files parameter in the POST request, the Content-Type of the request is automatically set to multipart/form-data (followed by the boundary string used to separate each body part in the multipart payload), whether you send only files, or form-data and files at the same time (thus, one shouldn't attempt setting the Content-Type manually in this case). Whereas, if only form-data were sent, the Content-Type would automatically be set to application/x-www-form-urlencoded.
You can print out the Content-Type header of the request to verify the above using the example given below, which shows how to upload multiple files (or a single file) with (optionally) the same key (i.e., 'files' in the case below), as well as with optional form-data (i.e., data=data in the example below). The documentation on how to POST single and multiple files can be found here and here, respectively. In case you need to upload large files without reading them into memory, have a look at Streaming Uploads.
For the server side—in case this is needed—please have a look at this answer, from which the code snippet below has been taken, and which uses the FastAPI web framework.
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('a.txt', 'rb')), ('files', open('b.txt', 'rb'))]
#file = {'file': open('a.txt','rb')} # to send a single file
data ={"name": "foo", "point": 0.13, "is_accepted": False}
r = requests.post(url=url, data=data, files=files)
print(r.json())
print(r.request.headers['content-type'])
You need to use the name attribute of the upload file that is in the HTML of the site. Example:
autocomplete="off" name="image">
You see name="image">? You can find it in the HTML of a site for uploading the file. You need to use it to upload the file with Multipart/form-data
script:
import requests
site = 'https://prnt.sc/upload.php' # the site where you upload the file
filename = 'image.jpg' # name example
Here, in the place of image, add the name of the upload file in HTML
up = {'image':(filename, open(filename, 'rb'), "multipart/form-data")}
If the upload requires to click the button for upload, you can use like that:
data = {
"Button" : "Submit",
}
Then start the request
request = requests.post(site, files=up, data=data)
And done, file uploaded succesfully
import requests
# assume sending two files
url = "put ur url here"
f1 = open("file 1 path", 'rb')
f2 = open("file 2 path", 'rb')
response = requests.post(url,files={"file1 name": f1, "file2 name":f2})
print(response)
To clarify examples given above,
"You need to use the files parameter to send a multipart form POST request even when you do not need to upload any files."
files={}
won't work, unfortunately.
You will need to put some dummy values in, e.g.
files={"foo": "bar"}
I came up against this when trying to upload files to Bitbucket's REST API and had to write this abomination to avoid the dreaded "Unsupported Media Type" error:
url = "https://my-bitbucket.com/rest/api/latest/projects/FOO/repos/bar/browse/foobar.txt"
payload = {'branch': 'master',
'content': 'text that will appear in my file',
'message': 'uploading directly from python'}
files = {"foo": "bar"}
response = requests.put(url, data=payload, files=files)
:O=
Send multipart/form-data key and value
curl command:
curl -X PUT http://127.0.0.1:8080/api/xxx ...
-H 'content-type: multipart/form-data; boundary=----xxx' \
-F taskStatus=1
python requests - More complicated POST requests:
updateTaskUrl = "http://127.0.0.1:8080/api/xxx"
updateInfoDict = {
"taskStatus": 1,
}
resp = requests.put(updateTaskUrl, data=updateInfoDict)
Send multipart/form-data file
curl command:
curl -X POST http://127.0.0.1:8080/api/xxx ...
-H 'content-type: multipart/form-data; boundary=----xxx' \
-F file=#/Users/xxx.txt
python requests - POST a Multipart-Encoded File:
filePath = "/Users/xxx.txt"
fileFp = open(filePath, 'rb')
fileInfoDict = {
"file": fileFp,
}
resp = requests.post(uploadResultUrl, files=fileInfoDict)
that's all.
import json
import os
import requests
from requests_toolbelt import MultipartEncoder
AUTH_API_ENDPOINT = "http://localhost:3095/api/auth/login"
def file_upload(path_img, token ):
url = 'http://localhost:3095/api/shopping/product/image'
name_img = os.path.basename(path_img)
mp_encoder = MultipartEncoder(
fields={
'email': 'mcm9#gmail.com',
'source': 'tmall',
'productId': 'product_0001',
'image': (name_img, open(path_img, 'rb'), 'multipart/form-data')
#'spam': ('spam.txt', open('spam.txt', 'rb'), 'text/plain'),
}
)
head = {'Authorization': 'Bearer {}'.format(token),
'Content-Type': mp_encoder.content_type}
with requests.Session() as s:
result = s.post(url, data=mp_encoder, headers=head)
return result
def do_auth(username, password, url=AUTH_API_ENDPOINT):
data = {
"email": username,
"password": password
}
# sending post request and saving response as response object
r = requests.post(url=url, data=data)
# extracting response text
response_text = r.text
d = json.loads(response_text)
# print(d)
return d
if __name__ == '__main__':
result = do_auth('mcm4#gmail.com','123456')
token = result.get('data').get('payload').get('token')
print(token)
result = file_upload('/home/mcm/Pictures/1234.png',token)
print(result.json())
Here is the python snippet you need to upload one large single file as multipart formdata. With NodeJs Multer middleware running on the server side.
import requests
latest_file = 'path/to/file'
url = "http://httpbin.org/apiToUpload"
files = {'fieldName': open(latest_file, 'rb')}
r = requests.put(url, files=files)
For the server side please check the multer documentation at: https://github.com/expressjs/multer
here the field single('fieldName') is used to accept one single file, as in:
var upload = multer().single('fieldName');
This is one way to send file in multipart request
import requests
headers = {"Authorization": "Bearer <token>"}
myfile = 'file.txt'
myfile2 = {'file': (myfile, open(myfile, 'rb'),'application/octet-stream')}
url = 'https://example.com/path'
r = requests.post(url, files=myfile2, headers=headers,verify=False)
print(r.content)
Other approach
import requests
url = "https://example.com/path"
payload={}
files=[
('file',('file',open('/path/to/file','rb'),'application/octet-stream'))
]
headers = {
'Authorization': 'Bearer <token>'
}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)
I have tested both , both works fine.
I'm trying to send a request to URL_server with request module in python 3.
This works for me:
# -*- coding: utf-8 *-*
import json, requests
URL_SERVER_TO_POST_DATA = "URL_to_send_POST_request"
HEADERS = {"Content-Type" : "multipart/form-data;"}
def getPointsCC_Function():
file_data = {
'var1': (None, "valueOfYourVariable_1"),
'var2': (None, "valueOfYourVariable_2")
}
try:
resElastic = requests.post(URL_GET_BALANCE, files=file_data)
res = resElastic.json()
except Exception as e:
print(e)
print (json.dumps(res, indent=4, sort_keys=True))
getPointsCC_Function()
Where:
URL_SERVER_TO_POST_DATA = Server where we going to send data
HEADERS = Headers sended
file_data = Params sended
Postman generated code for file upload with additional form fields:
import http.client
import mimetypes
from codecs import encode
conn = http.client.HTTPSConnection("data.XXXX.com")
dataList = []
boundary = 'wL36Yn8afVp8Ag7AmP8qZ0SA4n1v9T'
dataList.append(encode('--' + boundary))
dataList.append(encode('Content-Disposition: form-data; name=batchSize;'))
dataList.append(encode('Content-Type: {}'.format('text/plain')))
dataList.append(encode(''))
dataList.append(encode("1"))
dataList.append(encode('--' + boundary))
dataList.append(encode('Content-Disposition: form-data; name=file; filename={0}'.format('FileName-1.json')))
fileType = mimetypes.guess_type('FileName-1.json')[0] or 'application/octet-stream'
dataList.append(encode('Content-Type: {}'.format(fileType)))
dataList.append(encode(''))
with open('FileName-1.json', 'rb') as f:
dataList.append(f.read())
dataList.append(encode('--'+boundary+'--'))
dataList.append(encode(''))
body = b'\r\n'.join(dataList)
payload = body
headers = {
'Cookie': 'XXXXXXXXXXX',
'Content-type': 'multipart/form-data; boundary={}'.format(boundary)
}
conn.request("POST", "/fileupload/uri/XXXX", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))
I have a REST PUT request to upload a file using the Django REST framework. Whenever I am uploading a file using the Postman REST client it works fine:
But when I try to do this with my code:
import requests
API_URL = "http://123.316.118.92:8888/api/"
API_TOKEN = "1682b28041de357d81ea81db6a228c823ad52967"
URL = API_URL + 'configuration/configlet/31'
#files = {
files = {'file': open('configlet.txt','rb')}
print URL
print "Update Url ==-------------------"
headers = {'Content-Type' : 'text/plain','Authorization':API_TOKEN}
resp = requests.put(URL,files=files,headers = headers)
print resp.text
print resp.status_code
I am getting an error on the server side:
MultiValueDictKeyError at /api/configuration/31/
"'file'"
I am passing file as key but still getting above error please do let me know what might I am doing wrong here.
This is how my Django server view looks
def put(self, request,id,format=None):
configlet = self.get_object(id)
configlet.config_path.delete(save=False)
file_obj = request.FILES['file']
configlet.config_path = file_obj
file_content = file_obj.read()
params = parse_file(file_content)
configlet.parameters = json.dumps(params)
logger.debug("File content: "+str(file_content))
configlet.save()
For this to work you need to send a multipart/form-data body. You should not be setting the content-type of the whole request to text/plain here; set only the mime-type of the one part:
files = {'file': ('configlet.txt', open('configlet.txt','rb'), 'text/plain')}
headers = {'Authorization': API_TOKEN}
resp = requests.put(URL, files=files, headers=headers)
This leaves setting the Content-Type header for the request as a whole to the library, and using files sets that to multipart/form-data for you.
I'm trying to send a file using Pushbullet following their API docs.
This is my function:
def push_file(AccessToken, file_name):
f = open(file_name, 'rb')
file_type = mimetypes.guess_type(file_name)[0]
print("Uploading {0}...".format(file_name))
try:
data = {
'file_name': file_name,
'file_type' : file_type
}
resp = requests.post(UPLOAD_REQUEST_URL, data=data, auth=(AccessToken, '')).json()
if resp.get('error') != None:
print("Error: {0}".format(resp.get('error')['message']))
return
file_url = resp.get('file_url')
print(file_url)
resp = requests.post(resp.get('upload_url'), data=resp.get('data'), auth=(AccessToken, ''), files={'file': f})
data = {
'type' : 'file',
'file_name' : file_name,
'file_type' : file_type,
'file_url' : file_url,
'body' : ''
}
resp = requests.post(PUSH_URL, data=data, auth=(AccessToken, '')).json()
except requests.exceptions.ConnectionError:
traceback.print_exc()
f.close()
But I keep getting:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /pushbullet-uploads (Caused by <class 'ConnectionResetError'>: [Errno 104] Connection reset by peer)
If I use another AccessToken I still get this error, even if it's the first time I post to that url.
The upload process is unfortunately not very good and will hopefully be improved soon. It is the one request that does not obey the JSON rule. There is a curl example that shows this (https://docs.pushbullet.com/#upload-request), but understanding curl syntax is basically impossible.
Here's an example that I just typed up and seems to work:
import requests
import json
ACCESS_TOKEN = '<your access token here>'
resp = requests.post('https://api.pushbullet.com/v2/upload-request', data=json.dumps({'file_name': 'image.jpg'}), headers={'Authorization': 'Bearer ' + ACCESS_TOKEN, 'Content-Type': 'application/json'})
if resp.status_code != 200:
raise Exception('failed to request upload')
r = resp.json()
resp = requests.post(r['upload_url'], data=r['data'], files={'file': open('image.jpg', 'rb')})
if resp.status_code != 204:
raise Exception('failed to upload file')
print r['file_name'], r['file_type'], r['file_url']
According to the Pusbullet API
All POST requests should be over HTTPS and use a JSON body with the
Content-Type header set to "application/json".
Try changing your requests.post calls like so:
resp = requests.post(UPLOAD_REQUEST_URL, json=data, auth=(AccessToken, '')).json()
Use json=data instead of data=data. Requests will automatically set Content-Type to application/json.
How to send a multipart/form-data with requests in python? How to send a file, I understand, but how to send the form data by this method can not understand.
Basically, if you specify a files parameter (a dictionary), then requests will send a multipart/form-data POST instead of a application/x-www-form-urlencoded POST. You are not limited to using actual files in that dictionary, however:
>>> import requests
>>> response = requests.post('http://httpbin.org/post', files=dict(foo='bar'))
>>> response.status_code
200
and httpbin.org lets you know what headers you posted with; in response.json() we have:
>>> from pprint import pprint
>>> pprint(response.json()['headers'])
{'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate',
'Connection': 'close',
'Content-Length': '141',
'Content-Type': 'multipart/form-data; '
'boundary=c7cbfdd911b4e720f1dd8f479c50bc7f',
'Host': 'httpbin.org',
'User-Agent': 'python-requests/2.21.0'}
Better still, you can further control the filename, content type and additional headers for each part by using a tuple instead of a single string or bytes object. The tuple is expected to contain between 2 and 4 elements; the filename, the content, optionally a content type, and an optional dictionary of further headers.
I'd use the tuple form with None as the filename, so that the filename="..." parameter is dropped from the request for those parts:
>>> files = {'foo': 'bar'}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--bb3f05a247b43eede27a124ef8b968c5
Content-Disposition: form-data; name="foo"; filename="foo"
bar
--bb3f05a247b43eede27a124ef8b968c5--
>>> files = {'foo': (None, 'bar')}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--d5ca8c90a869c5ae31f70fa3ddb23c76
Content-Disposition: form-data; name="foo"
bar
--d5ca8c90a869c5ae31f70fa3ddb23c76--
files can also be a list of two-value tuples, if you need ordering and/or multiple fields with the same name:
requests.post(
'http://requestb.in/xucj9exu',
files=(
('foo', (None, 'bar')),
('foo', (None, 'baz')),
('spam', (None, 'eggs')),
)
)
If you specify both files and data, then it depends on the value of data what will be used to create the POST body. If data is a string, only it willl be used; otherwise both data and files are used, with the elements in data listed first.
There is also the excellent requests-toolbelt project, which includes advanced Multipart support. It takes field definitions in the same format as the files parameter, but unlike requests, it defaults to not setting a filename parameter. In addition, it can stream the request from open file objects, where requests will first construct the request body in memory:
from requests_toolbelt.multipart.encoder import MultipartEncoder
mp_encoder = MultipartEncoder(
fields={
'foo': 'bar',
# plain file object, no filename or mime type produces a
# Content-Disposition header with just the part name
'spam': ('spam.txt', open('spam.txt', 'rb'), 'text/plain'),
}
)
r = requests.post(
'http://httpbin.org/post',
data=mp_encoder, # The MultipartEncoder is posted as data, don't use files=...!
# The MultipartEncoder provides the content-type header with the boundary:
headers={'Content-Type': mp_encoder.content_type}
)
Fields follow the same conventions; use a tuple with between 2 and 4 elements to add a filename, part mime-type or extra headers. Unlike the files parameter, no attempt is made to find a default filename value if you don't use a tuple.
Requests has changed since some of the previous answers were written. Have a look at this Issue on Github for more details and this comment for an example.
In short, the files parameter takes a dictionary with the key being the name of the form field and the value being either a string or a 2, 3 or 4-length tuple, as described in the section POST a Multipart-Encoded File in the Requests quickstart:
>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
In the above, the tuple is composed as follows:
(filename, data, content_type, headers)
If the value is just a string, the filename will be the same as the key, as in the following:
>>> files = {'obvius_session_id': '72c2b6f406cdabd578c5fd7598557c52'}
Content-Disposition: form-data; name="obvius_session_id"; filename="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52
If the value is a tuple and the first entry is None the filename property will not be included:
>>> files = {'obvius_session_id': (None, '72c2b6f406cdabd578c5fd7598557c52')}
Content-Disposition: form-data; name="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52
You need to use the files parameter to send a multipart form POST request even when you do not need to upload any files.
From the original requests source:
def request(method, url, **kwargs):
"""Constructs and sends a :class:`Request <Request>`.
...
:param files: (optional) Dictionary of ``'name': file-like-objects``
(or ``{'name': file-tuple}``) for multipart encoding upload.
``file-tuple`` can be a 2-tuple ``('filename', fileobj)``,
3-tuple ``('filename', fileobj, 'content_type')``
or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``,
where ``'content-type'`` is a string
defining the content type of the given file
and ``custom_headers`` a dict-like object
containing additional headers to add for the file.
The relevant part is: file-tuple can be a:
2-tuple (filename, fileobj)
3-tuple (filename, fileobj, content_type)
4-tuple (filename, fileobj, content_type, custom_headers).
☝ What might not be obvious is that fileobj can be either an actual file object when dealing with files, OR a string when dealing with plain text fields.
Based on the above, the simplest multipart form request that includes both files to upload and form fields will look like this:
import requests
multipart_form_data = {
'upload': ('custom_file_name.zip', open('myfile.zip', 'rb')),
'action': (None, 'store'),
'path': (None, '/path1')
}
response = requests.post('https://httpbin.org/post', files=multipart_form_data)
print(response.content)
☝ Note the None as the first argument in the tuple for plain text fields — this is a placeholder for the filename field which is only used for file uploads, but for text fields passing None as the first parameter is required in order for the data to be submitted.
Multiple fields with the same name
If you need to post multiple fields with the same name then instead of a dictionary you can define your payload as a list (or a tuple) of tuples:
multipart_form_data = (
('file2', ('custom_file_name.zip', open('myfile.zip', 'rb'))),
('action', (None, 'store')),
('path', (None, '/path1')),
('path', (None, '/path2')),
('path', (None, '/path3')),
)
Streaming requests API
If the above API is not pythonic enough for you, then consider using requests toolbelt (pip install requests_toolbelt) which is an extension of the core requests module that provides support for file upload streaming as well as the MultipartEncoder which can be used instead of files, and which also lets you define the payload as a dictionary, tuple or list.
MultipartEncoder can be used both for multipart requests with or without actual upload fields. It must be assigned to the data parameter.
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder
multipart_data = MultipartEncoder(
fields={
# a file upload field
'file': ('file.zip', open('file.zip', 'rb'), 'text/plain')
# plain text fields
'field0': 'value0',
'field1': 'value1',
}
)
response = requests.post('http://httpbin.org/post', data=multipart_data,
headers={'Content-Type': multipart_data.content_type})
If you need to send multiple fields with the same name, or if the order of form fields is important, then a tuple or a list can be used instead of a dictionary:
multipart_data = MultipartEncoder(
fields=(
('action', 'ingest'),
('item', 'spam'),
('item', 'sausage'),
('item', 'eggs'),
)
)
Here is the simple code snippet to upload a single file with additional parameters using requests:
url = 'https://<file_upload_url>'
fp = '/Users/jainik/Desktop/data.csv'
files = {'file': open(fp, 'rb')}
payload = {'file_id': '1234'}
response = requests.put(url, files=files, data=payload, verify=False)
Please note that you don't need to explicitly specify any content type.
NOTE: Wanted to comment on one of the above answers but could not because of low reputation so drafted a new response here.
By specifying a files parameter in the POST request, the Content-Type of the request is automatically set to multipart/form-data (followed by the boundary string used to separate each body part in the multipart payload), whether you send only files, or form-data and files at the same time (thus, one shouldn't attempt setting the Content-Type manually in this case). Whereas, if only form-data were sent, the Content-Type would automatically be set to application/x-www-form-urlencoded.
You can print out the Content-Type header of the request to verify the above using the example given below, which shows how to upload multiple files (or a single file) with (optionally) the same key (i.e., 'files' in the case below), as well as with optional form-data (i.e., data=data in the example below). The documentation on how to POST single and multiple files can be found here and here, respectively. In case you need to upload large files without reading them into memory, have a look at Streaming Uploads.
For the server side—in case this is needed—please have a look at this answer, from which the code snippet below has been taken, and which uses the FastAPI web framework.
Example
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('a.txt', 'rb')), ('files', open('b.txt', 'rb'))]
#file = {'file': open('a.txt','rb')} # to send a single file
data ={"name": "foo", "point": 0.13, "is_accepted": False}
r = requests.post(url=url, data=data, files=files)
print(r.json())
print(r.request.headers['content-type'])
You need to use the name attribute of the upload file that is in the HTML of the site. Example:
autocomplete="off" name="image">
You see name="image">? You can find it in the HTML of a site for uploading the file. You need to use it to upload the file with Multipart/form-data
script:
import requests
site = 'https://prnt.sc/upload.php' # the site where you upload the file
filename = 'image.jpg' # name example
Here, in the place of image, add the name of the upload file in HTML
up = {'image':(filename, open(filename, 'rb'), "multipart/form-data")}
If the upload requires to click the button for upload, you can use like that:
data = {
"Button" : "Submit",
}
Then start the request
request = requests.post(site, files=up, data=data)
And done, file uploaded succesfully
import requests
# assume sending two files
url = "put ur url here"
f1 = open("file 1 path", 'rb')
f2 = open("file 2 path", 'rb')
response = requests.post(url,files={"file1 name": f1, "file2 name":f2})
print(response)
To clarify examples given above,
"You need to use the files parameter to send a multipart form POST request even when you do not need to upload any files."
files={}
won't work, unfortunately.
You will need to put some dummy values in, e.g.
files={"foo": "bar"}
I came up against this when trying to upload files to Bitbucket's REST API and had to write this abomination to avoid the dreaded "Unsupported Media Type" error:
url = "https://my-bitbucket.com/rest/api/latest/projects/FOO/repos/bar/browse/foobar.txt"
payload = {'branch': 'master',
'content': 'text that will appear in my file',
'message': 'uploading directly from python'}
files = {"foo": "bar"}
response = requests.put(url, data=payload, files=files)
:O=
Send multipart/form-data key and value
curl command:
curl -X PUT http://127.0.0.1:8080/api/xxx ...
-H 'content-type: multipart/form-data; boundary=----xxx' \
-F taskStatus=1
python requests - More complicated POST requests:
updateTaskUrl = "http://127.0.0.1:8080/api/xxx"
updateInfoDict = {
"taskStatus": 1,
}
resp = requests.put(updateTaskUrl, data=updateInfoDict)
Send multipart/form-data file
curl command:
curl -X POST http://127.0.0.1:8080/api/xxx ...
-H 'content-type: multipart/form-data; boundary=----xxx' \
-F file=#/Users/xxx.txt
python requests - POST a Multipart-Encoded File:
filePath = "/Users/xxx.txt"
fileFp = open(filePath, 'rb')
fileInfoDict = {
"file": fileFp,
}
resp = requests.post(uploadResultUrl, files=fileInfoDict)
that's all.
import json
import os
import requests
from requests_toolbelt import MultipartEncoder
AUTH_API_ENDPOINT = "http://localhost:3095/api/auth/login"
def file_upload(path_img, token ):
url = 'http://localhost:3095/api/shopping/product/image'
name_img = os.path.basename(path_img)
mp_encoder = MultipartEncoder(
fields={
'email': 'mcm9#gmail.com',
'source': 'tmall',
'productId': 'product_0001',
'image': (name_img, open(path_img, 'rb'), 'multipart/form-data')
#'spam': ('spam.txt', open('spam.txt', 'rb'), 'text/plain'),
}
)
head = {'Authorization': 'Bearer {}'.format(token),
'Content-Type': mp_encoder.content_type}
with requests.Session() as s:
result = s.post(url, data=mp_encoder, headers=head)
return result
def do_auth(username, password, url=AUTH_API_ENDPOINT):
data = {
"email": username,
"password": password
}
# sending post request and saving response as response object
r = requests.post(url=url, data=data)
# extracting response text
response_text = r.text
d = json.loads(response_text)
# print(d)
return d
if __name__ == '__main__':
result = do_auth('mcm4#gmail.com','123456')
token = result.get('data').get('payload').get('token')
print(token)
result = file_upload('/home/mcm/Pictures/1234.png',token)
print(result.json())
Here is the python snippet you need to upload one large single file as multipart formdata. With NodeJs Multer middleware running on the server side.
import requests
latest_file = 'path/to/file'
url = "http://httpbin.org/apiToUpload"
files = {'fieldName': open(latest_file, 'rb')}
r = requests.put(url, files=files)
For the server side please check the multer documentation at: https://github.com/expressjs/multer
here the field single('fieldName') is used to accept one single file, as in:
var upload = multer().single('fieldName');
This is one way to send file in multipart request
import requests
headers = {"Authorization": "Bearer <token>"}
myfile = 'file.txt'
myfile2 = {'file': (myfile, open(myfile, 'rb'),'application/octet-stream')}
url = 'https://example.com/path'
r = requests.post(url, files=myfile2, headers=headers,verify=False)
print(r.content)
Other approach
import requests
url = "https://example.com/path"
payload={}
files=[
('file',('file',open('/path/to/file','rb'),'application/octet-stream'))
]
headers = {
'Authorization': 'Bearer <token>'
}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)
I have tested both , both works fine.
I'm trying to send a request to URL_server with request module in python 3.
This works for me:
# -*- coding: utf-8 *-*
import json, requests
URL_SERVER_TO_POST_DATA = "URL_to_send_POST_request"
HEADERS = {"Content-Type" : "multipart/form-data;"}
def getPointsCC_Function():
file_data = {
'var1': (None, "valueOfYourVariable_1"),
'var2': (None, "valueOfYourVariable_2")
}
try:
resElastic = requests.post(URL_GET_BALANCE, files=file_data)
res = resElastic.json()
except Exception as e:
print(e)
print (json.dumps(res, indent=4, sort_keys=True))
getPointsCC_Function()
Where:
URL_SERVER_TO_POST_DATA = Server where we going to send data
HEADERS = Headers sended
file_data = Params sended
Postman generated code for file upload with additional form fields:
import http.client
import mimetypes
from codecs import encode
conn = http.client.HTTPSConnection("data.XXXX.com")
dataList = []
boundary = 'wL36Yn8afVp8Ag7AmP8qZ0SA4n1v9T'
dataList.append(encode('--' + boundary))
dataList.append(encode('Content-Disposition: form-data; name=batchSize;'))
dataList.append(encode('Content-Type: {}'.format('text/plain')))
dataList.append(encode(''))
dataList.append(encode("1"))
dataList.append(encode('--' + boundary))
dataList.append(encode('Content-Disposition: form-data; name=file; filename={0}'.format('FileName-1.json')))
fileType = mimetypes.guess_type('FileName-1.json')[0] or 'application/octet-stream'
dataList.append(encode('Content-Type: {}'.format(fileType)))
dataList.append(encode(''))
with open('FileName-1.json', 'rb') as f:
dataList.append(f.read())
dataList.append(encode('--'+boundary+'--'))
dataList.append(encode(''))
body = b'\r\n'.join(dataList)
payload = body
headers = {
'Cookie': 'XXXXXXXXXXX',
'Content-type': 'multipart/form-data; boundary={}'.format(boundary)
}
conn.request("POST", "/fileupload/uri/XXXX", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))