We Keep Coding

Send json from file in requests

I am trying to send a json file to a url (post). I am using data parameter to open the file and send it to the url:
r = requests.post(url_info, data=open(f, 'rb'), headers=headers, verify=False)
This works right now but I have been asked to use "json" insteand of "data" to send the file.
I have seen some examples using json created as dictionaries where this is working but I am not able to make it work from a file.
I have tried to use it directly:
json=open(f, 'rb')
In f I have the route to the json file.
Serialize the json file with json.dumps(open.... But I always get a message telling me that

Try using the json module to load JSON data from the file.
import requests
import json
# open the file and load JSON data from it
with open(f, "r") as file:
data = json.load(f) # type(data) -> <class 'dict'>
# send POST request with the loaded data
r = requests.post(url_info, data=data, headers=headers, verify=False)

Python - Outputting to .JSON with results from Microsoft's Computer Vision API

Trying to output my response from Microsoft's Computer Vision API to a .json file, it works with all of the other APIs I've been using so far. With the code below, directly from Microsoft's documentation, I get an error:
Error: the JSON object must be str, not 'bytes'
Removing the parsed = json.loads(data) and using print(json.dumps(data, sort_keys=True, indent=2)) prints out the information for the image that I want, but also says Error and is prefixed with
b
denoting it's in bytes and ending with
is not JSON serializable
I'm just trying to find out how I can get the response into a .json file like i'm able to do with other APIs and am at a loss for how I can possible convert this in a way that will work.
import http.client, urllib.request, urllib.parse, urllib.error, base64, json
API_KEY = '{API_KEY}'
uri_base = 'westus.api.cognitive.microsoft.com'
headers = {
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': API_KEY,
}
params = urllib.parse.urlencode(
{
'visualFeatures': 'Categories, Description, Color',
'language': 'en',
}
)
body = "{'url': 'http://i.imgur.com/WgPtc53.jpg'}"
try:
conn = http.client.HTTPSConnection(uri_base)
conn.request('POST', '/vision/v1.0/analyze?%s' % params, body, headers)
response = conn.getresponse()
data = response.read()
# 'data' contains the JSON data. The following formats the JSON data for display.
parsed = json.loads(data)
print ("Response:")
print (json.dumps(parsed, sort_keys=True, indent=2))
conn.close()
except Exception as e:
print('Error:')
print(e)
Shortly after posting the question, I realized I had missed looking for something: just converting bytes to a string.
found this Convert bytes to a Python string
and was able to modify my code to:
parsed = json.loads(data.decode('utf-8'))
And it seems to have resolved my issue. Now error-free and able to export to .json file like I needed.

Importing Qualtrics Responses using Python Requests library

I am trying to import a csv of responses into Qualtrics using the API shown here: https://api.qualtrics.com/docs/import-responses. But, since I'm a noob at Python and (by extension) at Requests, I'm having trouble figuring out why I keep getting a 413. I've gotten this far:
formTest = {
'surveyId': 'my_id',
'file': {
'value': open('dataFiles/myFile.csv', 'rb'),
'options': {
'contentType': 'text/csv'
}
}
}
headersTest = {
"X-API-TOKEN": "my_token",
'content-type': "multipart/form-data"
}
r = requests.request("POST", url, data=formTest, headers=headersTest)
print(r.text)
The format for the formTest variable is something I found when looking through other code bases for an angular implementation of this, which may not apply to a python version of the code. I can successfully use cUrl, but Python Requests, in my current situation is the way to go (for various reasons).
In a fit of desperation, I tried directly translating the cUrl request to python requests, but that didn't seem to help much either.
Has anyone done something like this before? I took a look at posts for importing contacts and the like, but there was no luck there either (since the data that needs to be sent is formatted differently). Is there something I am missing?

It's best not to mix post data and files but use two separate dictionaries. For the files you should use the files= parameter, because it encodes the POST data as a Multipart Form data and creates the required Content-Type headers.
import requests
url = 'Qualtrics API'
file_path = 'path/to/file'
file_name = 'file.name'
data = {'surveyId':'my_id'}
files = {'file' : (file_name, open(file_path, 'rb'), 'text/csv')}
headers = {'X-API-TOKEN': 'my_token'}
r = requests.post(url, data=data, files=files, headers=headers)
print(r.text)
The first value in files['file'] is the file name (optional), followed by the file object, followed by the file content type (optional).
You will find more info in the docs: Requests, POST a Multipart-Encoded File.

ValueError: Data must not be a string. in Python [duplicate]

I am trying to do the following with requests:
data = {'hello', 'goodbye'}
json_data = json.dumps(data)
headers = {
'Access-Key': self.api_key,
'Access-Signature': signature,
'Access-Nonce': nonce,
'Content-Type': 'application/json',
'Accept': 'text/plain'
}
r = requests.post(url, headers=headers, data=json_data,
files={'file': open('/Users/david/Desktop/a.png', 'rb')})
However, I get the following error:
ValueError: Data must not be a string.
Note that if I remove the files parameter, it works as needed. Why won't requests allow me to send a json-encoded string for data if files is included?
Note that if I change data to be just the normal python dictionary (and not a json-encoded string), the above works. So it seems that the issue is that if files is not json-encoded, then data cannot be json-encoded. However, I need to have my data encoded to match a hash signature that's being created by the API.

When you specify your body to a JSON string, you can no longer attach a file since file uploading requires the MIME type multipart/form-data.
You have two options:
Encapsulate your JSON string as part as the form data (something like json => json.dumps(data))
Encode your file in Base64 and transmit it in the JSON request body. This looks like a lot of work though.

1.Just remove the line
json_data = json.dumps(data)
and change in request as data=data.
2.Remove 'Content-Type': 'application/json' inside headers.
This worked for me.

Alternative solution to this problem is to post data as file.
You can post strings as files. Read more here:
http://docs.python-requests.org/en/latest/user/quickstart/#post-a-multipart-encoded-file
Here is explained how to post multiple files:
http://docs.python-requests.org/en/latest/user/advanced/#post-multiple-multipart-encoded-files

removing the following helped me in my case:
'Content-Type': 'application/json'
then the data should be passed as dictionary

If your files are small, you could simply convert the binary (image or anything) to base64 string and send that as JSON to the API. That is much simpler and more straight forward than the suggested solutions. The currently accepted answer claims that is a lot of work, but it's really simple.
Client:
with open('/Users/houmie/Downloads/log.zip','rb') as f:
bytes = f.read()
tb = b64encode(bytes)
tb_str = tb.decode('utf-8')
body = {'logfile': tb_str}
r = requests.post('https://prod/feedback', data=json.dumps(body), headers=headers)
API:
def create(event, context):
data = json.loads(event["body"])
if "logfile" in data:
tb_back = data["logfile"].encode('utf-8')
zip_data = base64.b64decode(tb_back)

ValueError: Data must not be a string

I am trying to do the following with requests:
data = {'hello', 'goodbye'}
json_data = json.dumps(data)
headers = {
'Access-Key': self.api_key,
'Access-Signature': signature,
'Access-Nonce': nonce,
'Content-Type': 'application/json',
'Accept': 'text/plain'
}
r = requests.post(url, headers=headers, data=json_data,
files={'file': open('/Users/david/Desktop/a.png', 'rb')})
However, I get the following error:
ValueError: Data must not be a string.
Note that if I remove the files parameter, it works as needed. Why won't requests allow me to send a json-encoded string for data if files is included?
Note that if I change data to be just the normal python dictionary (and not a json-encoded string), the above works. So it seems that the issue is that if files is not json-encoded, then data cannot be json-encoded. However, I need to have my data encoded to match a hash signature that's being created by the API.

When you specify your body to a JSON string, you can no longer attach a file since file uploading requires the MIME type multipart/form-data.
You have two options:
Encapsulate your JSON string as part as the form data (something like json => json.dumps(data))
Encode your file in Base64 and transmit it in the JSON request body. This looks like a lot of work though.

1.Just remove the line
json_data = json.dumps(data)
and change in request as data=data.
2.Remove 'Content-Type': 'application/json' inside headers.
This worked for me.

Alternative solution to this problem is to post data as file.
You can post strings as files. Read more here:
http://docs.python-requests.org/en/latest/user/quickstart/#post-a-multipart-encoded-file
Here is explained how to post multiple files:
http://docs.python-requests.org/en/latest/user/advanced/#post-multiple-multipart-encoded-files

removing the following helped me in my case:
'Content-Type': 'application/json'
then the data should be passed as dictionary

If your files are small, you could simply convert the binary (image or anything) to base64 string and send that as JSON to the API. That is much simpler and more straight forward than the suggested solutions. The currently accepted answer claims that is a lot of work, but it's really simple.
Client:
with open('/Users/houmie/Downloads/log.zip','rb') as f:
bytes = f.read()
tb = b64encode(bytes)
tb_str = tb.decode('utf-8')
body = {'logfile': tb_str}
r = requests.post('https://prod/feedback', data=json.dumps(body), headers=headers)
API:
def create(event, context):
data = json.loads(event["body"])
if "logfile" in data:
tb_back = data["logfile"].encode('utf-8')
zip_data = base64.b64decode(tb_back)