I am trying to import a csv of responses into Qualtrics using the API shown here: https://api.qualtrics.com/docs/import-responses. But, since I'm a noob at Python and (by extension) at Requests, I'm having trouble figuring out why I keep getting a 413. I've gotten this far:
formTest = {
'surveyId': 'my_id',
'file': {
'value': open('dataFiles/myFile.csv', 'rb'),
'options': {
'contentType': 'text/csv'
}
}
}
headersTest = {
"X-API-TOKEN": "my_token",
'content-type': "multipart/form-data"
}
r = requests.request("POST", url, data=formTest, headers=headersTest)
print(r.text)
The format for the formTest variable is something I found when looking through other code bases for an angular implementation of this, which may not apply to a python version of the code. I can successfully use cUrl, but Python Requests, in my current situation is the way to go (for various reasons).
In a fit of desperation, I tried directly translating the cUrl request to python requests, but that didn't seem to help much either.
Has anyone done something like this before? I took a look at posts for importing contacts and the like, but there was no luck there either (since the data that needs to be sent is formatted differently). Is there something I am missing?
It's best not to mix post data and files but use two separate dictionaries. For the files you should use the files= parameter, because it encodes the POST data as a Multipart Form data and creates the required Content-Type headers.
import requests
url = 'Qualtrics API'
file_path = 'path/to/file'
file_name = 'file.name'
data = {'surveyId':'my_id'}
files = {'file' : (file_name, open(file_path, 'rb'), 'text/csv')}
headers = {'X-API-TOKEN': 'my_token'}
r = requests.post(url, data=data, files=files, headers=headers)
print(r.text)
The first value in files['file'] is the file name (optional), followed by the file object, followed by the file content type (optional).
You will find more info in the docs: Requests, POST a Multipart-Encoded File.
Related
I'm just working on API conection at my work. I already made some GET and PUT request, but now i have problem with POST. API documantation is here. And here is my code I test but get 400 bad request:
import requests
files = {'files': ('fv.pdf', open(r"C:\python\API\fv.pdf", 'rb'))}
data = {"order_documents":[{'file_name':"fv.pdf", 'type_code':'CUSTOMER_INVOICE' }]}
header = {
'Authorization': '###########################',
}
response = requests.post("https://######.com/api/orders/40100476277994-A/documents", headers=header, files = files, data = data)
print(response.status_code)
print(response.url)
Someone have any idea how i can handle with this?
Looks like you are missing the order_documents parameter, it needs to be an array and also needs to be called order_documents.
Try changing your data variable into:
data = {"order_documents": [ {'file_name':"fv.pdf", 'type_code':'CUSTOMER_INVOICE' } ] }
The API expects files as the parameter name and your dictionary sends file to the server. The parameter name files that you give to session.post is just for requests library and not the actual parameter sent to the server.
The API also expects multiple files in an array, so you need to change your files object.
files = [
('files', ('fv.pdf', open(r"C:\python\API\fv.pdf", 'rb')),
]
Also, I don't think you need to use requests.Session(), just use requests.post(), unless you're planning on using the session object multiple times for subsequent requests.
I am communicating with an API using HTTP.client in Python 3.6.2.
In order to upload a file it requires a three stage process.
I have managed to talk successfully using POST methods and the server returns data as I expect.
However, the stage that requires the actual file to be uploaded is a PUT method - and I cannot figure out how to syntax the code to include a pointer to the actual file on my storage - the file is an mp4 video file.
Here is a snippet of the code with my noob annotations :)
#define connection as HTTPS and define URL
uploadstep2 = http.client.HTTPSConnection("grabyo-prod.s3-accelerate.amazonaws.com")
#define headers
headers = {
'accept': "application/json",
'content-type': "application/x-www-form-urlencoded"
}
#define the structure of the request and send it.
#Here it is a PUT request to the unique URL as defined above with the correct file and headers.
uploadstep2.request("PUT", myUniqueUploadUrl, body="C:\Test.mp4", headers=headers)
#get the response from the server
uploadstep2response = uploadstep2.getresponse()
#read the data from the response and put to a usable variable
step2responsedata = uploadstep2response.read()
The response I am getting back at this stage is an
"Error 400 Bad Request - Could not obtain the file information."
I am certain this relates to the body="C:\Test.mp4" section of the code.
Can you please advise how I can correctly reference a file within the PUT method?
Thanks in advance
uploadstep2.request("PUT", myUniqueUploadUrl, body="C:\Test.mp4", headers=headers)
will put the actual string "C:\Test.mp4" in the body of your request, not the content of the file named "C:\Test.mp4" as you expect.
You need to open the file, read it's content then pass it as body. Or to stream it, but AFAIK http.client does not support that, and since your file seems to be a video, it is potentially huge and will use plenty of RAM for no good reason.
My suggestion would be to use requests, which is a way better lib to do this kind of things:
import requests
with open(r'C:\Test.mp4'), 'rb') as finput:
response = requests.put('https://grabyo-prod.s3-accelerate.amazonaws.com/youruploadpath', data=finput)
print(response.json())
I do not know if it is useful for you, but you can try to send a POST request with requests module :
import requests
url = ""
data = {'title':'metadata','timeDuration':120}
mp3_f = open('/path/your_file.mp3', 'rb')
files = {'messageFile': mp3_f}
req = requests.post(url, files=files, json=data)
print (req.status_code)
print (req.content)
Hope it helps .
I am trying to do the following with requests:
data = {'hello', 'goodbye'}
json_data = json.dumps(data)
headers = {
'Access-Key': self.api_key,
'Access-Signature': signature,
'Access-Nonce': nonce,
'Content-Type': 'application/json',
'Accept': 'text/plain'
}
r = requests.post(url, headers=headers, data=json_data,
files={'file': open('/Users/david/Desktop/a.png', 'rb')})
However, I get the following error:
ValueError: Data must not be a string.
Note that if I remove the files parameter, it works as needed. Why won't requests allow me to send a json-encoded string for data if files is included?
Note that if I change data to be just the normal python dictionary (and not a json-encoded string), the above works. So it seems that the issue is that if files is not json-encoded, then data cannot be json-encoded. However, I need to have my data encoded to match a hash signature that's being created by the API.
When you specify your body to a JSON string, you can no longer attach a file since file uploading requires the MIME type multipart/form-data.
You have two options:
Encapsulate your JSON string as part as the form data (something like json => json.dumps(data))
Encode your file in Base64 and transmit it in the JSON request body. This looks like a lot of work though.
1.Just remove the line
json_data = json.dumps(data)
and change in request as data=data.
2.Remove 'Content-Type': 'application/json' inside headers.
This worked for me.
Alternative solution to this problem is to post data as file.
You can post strings as files. Read more here:
http://docs.python-requests.org/en/latest/user/quickstart/#post-a-multipart-encoded-file
Here is explained how to post multiple files:
http://docs.python-requests.org/en/latest/user/advanced/#post-multiple-multipart-encoded-files
removing the following helped me in my case:
'Content-Type': 'application/json'
then the data should be passed as dictionary
If your files are small, you could simply convert the binary (image or anything) to base64 string and send that as JSON to the API. That is much simpler and more straight forward than the suggested solutions. The currently accepted answer claims that is a lot of work, but it's really simple.
Client:
with open('/Users/houmie/Downloads/log.zip','rb') as f:
bytes = f.read()
tb = b64encode(bytes)
tb_str = tb.decode('utf-8')
body = {'logfile': tb_str}
r = requests.post('https://prod/feedback', data=json.dumps(body), headers=headers)
API:
def create(event, context):
data = json.loads(event["body"])
if "logfile" in data:
tb_back = data["logfile"].encode('utf-8')
zip_data = base64.b64decode(tb_back)
I am trying to do the following with requests:
data = {'hello', 'goodbye'}
json_data = json.dumps(data)
headers = {
'Access-Key': self.api_key,
'Access-Signature': signature,
'Access-Nonce': nonce,
'Content-Type': 'application/json',
'Accept': 'text/plain'
}
r = requests.post(url, headers=headers, data=json_data,
files={'file': open('/Users/david/Desktop/a.png', 'rb')})
However, I get the following error:
ValueError: Data must not be a string.
Note that if I remove the files parameter, it works as needed. Why won't requests allow me to send a json-encoded string for data if files is included?
Note that if I change data to be just the normal python dictionary (and not a json-encoded string), the above works. So it seems that the issue is that if files is not json-encoded, then data cannot be json-encoded. However, I need to have my data encoded to match a hash signature that's being created by the API.
When you specify your body to a JSON string, you can no longer attach a file since file uploading requires the MIME type multipart/form-data.
You have two options:
Encapsulate your JSON string as part as the form data (something like json => json.dumps(data))
Encode your file in Base64 and transmit it in the JSON request body. This looks like a lot of work though.
1.Just remove the line
json_data = json.dumps(data)
and change in request as data=data.
2.Remove 'Content-Type': 'application/json' inside headers.
This worked for me.
Alternative solution to this problem is to post data as file.
You can post strings as files. Read more here:
http://docs.python-requests.org/en/latest/user/quickstart/#post-a-multipart-encoded-file
Here is explained how to post multiple files:
http://docs.python-requests.org/en/latest/user/advanced/#post-multiple-multipart-encoded-files
removing the following helped me in my case:
'Content-Type': 'application/json'
then the data should be passed as dictionary
If your files are small, you could simply convert the binary (image or anything) to base64 string and send that as JSON to the API. That is much simpler and more straight forward than the suggested solutions. The currently accepted answer claims that is a lot of work, but it's really simple.
Client:
with open('/Users/houmie/Downloads/log.zip','rb') as f:
bytes = f.read()
tb = b64encode(bytes)
tb_str = tb.decode('utf-8')
body = {'logfile': tb_str}
r = requests.post('https://prod/feedback', data=json.dumps(body), headers=headers)
API:
def create(event, context):
data = json.loads(event["body"])
if "logfile" in data:
tb_back = data["logfile"].encode('utf-8')
zip_data = base64.b64decode(tb_back)
I want to upload a file to an url. The file I want to upload is not on my computer, but I have the url of the file. I want to upload it using requests library. So, I want to do something like this:
url = 'http://httpbin.org/post'
files = {'file': open('report.xls', 'rb')}
r = requests.post(url, files=files)
But, only difference is, the file report.xls comes from some url which is not in my computer.
The only way to do this is to download the body of the URL so you can upload it.
The problem is that a form that takes a file is expecting the body of the file in the HTTP POST. Someone could write a form that takes a URL instead, and does the fetching on its own… but that would be a different form and request than the one that takes a file (or, maybe, the same form, with an optional file and an optional URL).
You don't have to download it and save it to a file, of course. You can just download it into memory:
urlsrc = 'http://example.com/source'
rsrc = requests.get(urlsrc)
urldst = 'http://example.com/dest'
rdst = requests.post(urldst, files={'file': rsrc.content})
Of course in some cases, you might always want to forward along the filename, or some other headers, like the Content-Type. Or, for huge files, you might want to stream from one server to the other without downloading and then uploading the whole file at once. You'll have to do any such things manually, but almost everything is easy with requests, and explained well in the docs.*
* Well, that last example isn't quite easy… you have to get the raw socket-wrappers off the requests and read and write, and make sure you don't deadlock, and so on…
There is an example in the documentation that may suit you. A file-like object can be used as a stream input for a POST request. Combine this with a stream response for your GET (passing stream=True), or one of the other options documented here.
This allows you to do a POST from another GET without buffering the entire payload locally. In the worst case, you may have to write a file-like class as "glue code", allowing you to pass your glue object to the POST that in turn reads from the GET response.
(This is similar to a documented technique using the Node.js request module.)
import requests
img_url = "http://...."
res_src = requests.get(img_url)
payload={}
files=[
('files',('image_name.jpg', res_src.content,'image/jpeg'))
]
headers = {"token":"******-*****-****-***-******"}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)
above code is working for me.