I'm trying to implement a simple Django view that accept a file attachment and some other parameters and proxy the request on a remote API call.
Please note: the proxy is not the point of my question :)
This is how I implemented the view:
def image_upload(request):
token = request.POST['token']
image_file = request.FILES.values()[0]
files = {'file': ('myupload.txt', image_file.read())}
client_id = request.POST['client_id']
folder_id = request.POST['folder_id']
advert_id = request.POST['advert_id']
image_type = request.POST['image_type']
crop_image = request.POST['crop_image']
api_base_url = settings.API_BASE_URL
file_post_data = {'client_id': client_id, 'folder_id': folder_id, 'advert_id': advert_id,
'image_type': image_type, 'crop_image': crop_image}
auth_header = {'Authorization': 'Token ' + token}
r = requests.post(api_base_url + 'assets/image/upload/',
data = json.dumps(file_post_data),
headers = auth_header,
files = files)
return r.json()
The problem is that when I test this view (I use Django Test Client to do it) I get an error on the line "files = files)" that says "ValueError: cannot encode objects that are not 2-tuples".
The complete trace log is this one:
======================================================================
ERROR: test_image_upload (fbx.tests.FbxTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/tests.py", line 18, in test_image_upload
'image_type': 'A', 'crop_image': False, 'attachment': fp})
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 449, in post
response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 262, in post
return self.request(**r)
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/views.py", line 42, in image_upload
files = files)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 324, in request
prep = req.prepare()
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 225, in prepare
p.prepare_body(self.data, self.files)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 385, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 99, in _encode_files
fields = to_key_val_list(data or {})
File "/usr/local/lib/python2.7/dist-packages/requests/utils.py", line 136, in to_key_val_list
raise ValueError('cannot encode objects that are not 2-tuples')
ValueError: cannot encode objects that are not 2-tuples
I've also tried a quick test using this to read a file: files = {'file': ('myupload.txt', open('/tmp/mytmp.txt', 'rb'))}
but it doesn't work. Do tou have any idea about how to fix this?
Thanks!
You cannot simultaneously post JSON data and multipart/form-data which is in essence what you're trying to do here. json.dumps returns a string so you're sending a string that looks like
'{"client_id": 1, "folder_id": 2, "advert_id": 3, "image_type": "jpeg", "crop_image": true}'
And then telling requests you want to use that in combination with a multipart/form-data request by sending something in via the files parameter. That is impossible and could possibly raise a better exception.
Either add the file to the JSON data after reading it into memory and send that with the appropriate Content-Type header or send the entire thing as a multipart/form-data request without using json.dumps and simply passing the dictionary you're creating to data. Use one or the other but not both.
That aside, you say that your exception comes from the line that only has files=files) on it, but it is not that line alone that causes the exception. The exception is raised by the function which happens to end on that line. The fact that your exception arises from that too is mere coincidence. This is an wart in Python that may possibly be fixed in Python 3.4. You should upgrade, because 3.4 will be awesome and newer versions of Django support Python 3.x.
Related
I have a requirement to send HTTP/2 get and post request to unit test rest api server.
I tried to use request library but it seems that it doesn't support HTTP/2. So I integrated request library along with hyper transport adapter like below:
import requests
from hyper.contrib import HTTP20Adapter
s = requests.Session()
data={
"param1": 3000,
"param2": 10,
"param3": 2,
"param4": "2200",
"param5": "800",
"param6": "2200",
"param7": "2000",
"param8": "1700",
"param9": "1200",
"param10": 60,
"param11": "23:00-22:59",
"param12": 3,
"param13": True,
"param14": 85
}
s.verify="/path to ca-cert file"
s.mount('https://', HTTP20Adapter())
r = s.post('https://localhost:1000/config', data=data)
print(r.status_code)
print(r.url)
But it is throwing error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 635, in post
return self.request("POST", url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.9/site-packages/hyper/contrib.py", line 118, in send
resp = conn.get_response()
File "/usr/local/lib/python3.9/site-packages/hyper/common/connection.py", line 136, in get_response
return self._conn.get_response(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/hyper/http20/connection.py", line 305, in get_response
return HTTP20Response(stream.getheaders(), stream)
File "/usr/local/lib/python3.9/site-packages/hyper/http20/stream.py", line 240, in getheaders
self._recv_cb(stream_id=self.stream_id)
File "/usr/local/lib/python3.9/site-packages/hyper/http20/connection.py", line 787, in _recv_cb
self._single_read()
File "/usr/local/lib/python3.9/site-packages/hyper/http20/connection.py", line 738, in _single_read
raise ConnectionError(error_string)
hyper.http20.exceptions.ConnectionError: Encountered error FRAME_SIZE_ERROR 0x6: Frame size incorrect
How to fix this issue?
I tried to explore to increase frame size which receiver receives and I got following from this link
To implement one of these objects, you will want to subclass the
BaseFlowControlManager class and implement the increase_window_size()
method. As a simple example, we can implement a very stupid flow
control manager that always resizes the window in response to incoming
data like this:
class StupidFlowControlManager(BaseFlowControlManager):
def increase_window_size(self, frame_size):
return frame_size
The class can then be plugged straight into a connection object:
HTTP20Connection('http2bin.org', window_manager=StupidFlowControlManager)
But I don't know how to make use of this class in my code.
I am new to programming and learning python, so please bear with me, I appreciate the help....
I am working on a project where I need to upload files to storage services and I am currently trying to use the box API. I am trying to work with the code on this page:
how to use python's Request library to make an API call with an attachment and a parameter
import requests
import json
#the user access token
access_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
#the name of the file as you want it to appear in box
filename = 'box_file'
#the actual file path
src_file = "C:\Python\Wildlife.wmv"
#the id of the folder you want to upload to
parent_id = '0'
headers = { 'Authorization: Bearer {0}'.format(access_token)}
url = 'https://upload.box.com/api/2.0/files/content'
files = { 'filename': (filename, open(src_file,'rb')) }
data = { "parent_id": parent_id }
response = requests.post(url, data, files, headers)
file_info = response.json()
I have tried a number of different things that really haven't gotten me any closer, so I am posting my slight adaptation of their code. Currently I am getting this error:
Traceback (most recent call last):
File "transfer2.py", line 18, in <module>
response = requests.post(url, data, files, headers)
TypeError: post() takes from 1 to 3 positional arguments but 4 were given
I have also had issues with the file_info = response.json()" in some of my other experiments. If someone could help me to get this working I would greatly appreciate it.
I am using python 3 if that helps.
edit 4/6
As requested, I changed this line:
response = requests.post(url, data=data, files=files, headers=headers)
This is the error I now get:
Traceback (most recent call last):
File "transfer2.py", line 18, in <module>
response = requests.post(url, data=data, files=files, headers=headers)
File "C:\Python34\lib\site-packages\requests\api.py", line 108, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Python34\lib\site-packages\requests\api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "C:\Python34\lib\site-packages\requests\sessions.py", line 450, in request
prep = self.prepare_request(req)
File "C:\Python34\lib\site-packages\requests\sessions.py", line 381, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "C:\Python34\lib\site-packages\requests\models.py", line 305, in prepare
self.prepare_headers(headers)
File "C:\Python34\lib\site-packages\requests\models.py", line 410, in prepare_headers
self.headers = CaseInsensitiveDict((to_native_string(name), value) for name, value in headers.items())
AttributeError: 'set' object has no attribute 'items'
In the requests library for request.post(), headers and files are both keyword arguments only, I would also make data a keyword argument, e.g.:
response = requests.post(url, data=data, files=files, headers=headers)
from boxsdk import Client, OAuth2
oauth = OAuth2( client_id="dlpjkcxxxxxxxxxxxxxxxxxxxxcom",client_secret="xxxxxxxxxxxxxxxxxxxxxxxxx", access_token="xxxxxxxxxxxxxxxxxxxxxxxxxxx", )
client = Client(oauth)
shared_folder = client.folder(folder_id='0',).create_subfolder('sxxxx')
uploaded_file = shared_folder.upload('/root/xxxxxx.txt')
I am unable to PUT Nest data like ambient_temperature_f to https://developer-api.nest.com or the redirected Firebase URL. I suspect there is something specific to the Nest that will need to be tweaked in the Firebase module I am using (https://ozgur.github.io/python-firebase/).
From firebase.py:
#http_connection(60)
def make_put_request(url, data, params, headers, connection):
"""
Helper function that makes an HTTP PUT request to the given firebase
endpoint. Timeout is 60 seconds.
`url`: The full URL of the firebase endpoint (DSN appended.)
`data`: JSON serializable dict that will be stored in the remote storage.
`params`: Python dict that is appended to the URL like a querystring.
`headers`: Python dict. HTTP request headers.
`connection`: Predefined HTTP connection instance. If not given, it
is supplied by the `decorators.http_connection` function.
The returning value is a Python dict deserialized by the JSON decoder. However,
if the status code is not 2x or 403, an requests.HTTPError is raised.
connection = connection_pool.get_available_connection()
response = make_put_request('http://firebase.localhost/users',
'{"1": "Ozgur Vatansever"}',
{'X_FIREBASE_SOMETHING': 'Hi'}, connection)
response => {'1': 'Ozgur Vatansever'} or {'error': 'Permission denied.'}
"""
timeout = getattr(connection, 'timeout')
response = connection.put(url, data=data, params=params, headers=headers, timeout=timeout)
print('[make_put_request]: [%s][%s][%s][%s]\n' %(url, data, params, headers))
if response.ok or response.status_code == 403:
return response.json() if response.content else None
else:
response.raise_for_status()
Prints out:
[make_put_request]: [https://developer-api.nest.com/devices/thermostats/DEVICE_ID/ambient_temperature_f.json?auth=VALID_AUTH_TOKEN][71][{}][{}]
Returns error:
Traceback (most recent call last):
File "C:\py\nest_auth.py", line 90, in <module>
put_result = firebase.put(data_url, field_name, 71)
File "C:\Python34\lib\site-packages\firebase\decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "C:\Python34\lib\site-packages\firebase\firebase.py", line 312, in put
connection=connection)
File "C:\Python34\lib\site-packages\firebase\decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "C:\Python34\lib\site-packages\firebase\firebase.py", line 77, in make_put_request
response.raise_for_status()
File "C:\Python34\lib\site-packages\requests\models.py", line 808, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request
This worked when using a firebaseio.com target but isn't working for Nest:
put_result = firebase.put('/devices/thermostats/DEVICE_ID/', 'ambient_temperature', 71)
According to the documentation ambient_temperature_f is a read only field that represent's the reported ambient temperature from the thermostat. It wouldn't make sense to override that since it is a sensor reading.
I think you want to write to target_temperature_f, which is the temperature the thermostat should heat or cool to.
I need to load a file using a soap endpoint url...When I use the below code to load it the files are getting loaded but they are not in a readable format...When I load using SOAPUI tool it loads properly...
import requests
xml = '''<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:v1="http://s.sa.com/services/Attachment/v1.0">
<soapenv:Header/>
<soapenv:Body>
<v1:attachment>
<filename>FUZZY.csv</filename>
<data>cid:138641430598</data>
</v1:attachment>
</soapenv:Body>
</soapenv:Envelope>'''
target_url = 'https://s.sa.com:443/soatest/FileAttachmentService'
headers = {'Content-Type': 'text/xml','charset':'utf-8'}
r = requests.post(target_url,data=xml,headers=headers,auth=('3user1',''))
print 'r.text = ', r.text
print 'r.content = ', r.content
print 'r.status_code = ', r.status_code
New changes:-
files = {'file':open('./FUZZY.csv','rb')}
print files
r = requests.post(target_url,files=files,data=xml,headers=headers,auth=('p3user1',''))
Error:
Traceback (most recent call last):
File "soapcall_python.py", line 18, in <module>
r = requests.post(target_url,files=files,data=xml,headers=headers,auth=('p3user1',''))
File "/opt/python2.7/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/opt/python2.7/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/python2.7/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/sessions.py", line 418, in request
prep = self.prepare_request(req)
File "/opt/python2.7/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/sessions.py", line 356, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/opt/python2.7/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/models.py", line 297, in prepare
self.prepare_body(data, files)
File "/opt/python2.7/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/models.py", line 432, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "/opt/python2.7/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/models.py", line 109, in _encode_files
raise ValueError("Data must not be a string.")
ValueError: Data must not be a string.
You aren't sending the contents of the file anywhere. You're just sending a reference to a file that doesn't exist anywhere that the server can see.
As the docs for SOAP references to attachments explains, the way you do this is to send a MIME-multipart message. If you're using the CID reference mechanism, that cid isn't some arbitrary string, it has to match the Content-ID header of a message in the MIME envelope.
The requests docs for POST a Multipart-Encoded File explain how to send the contents of a file as a message within a MIME request; briefly:
with open('FUZZY.csv', 'rb') as f:
files = {'file': f}
r = requests.post(target_url,
data=xml, headers=headers, auth=('3user1',''),
files=files)
However, this simple method doesn't give you access to the Content-ID that will be generated under the covers for your message. So, if you want to use the CID reference mechanism, you will need to generate the MIME envelope manually (e.g., by using email.mail.MIMEMultipart) and sending the entire thing as a data string.
here url_add is a link that contains the rdf triples that i want to store in 4store.but if i pass url_add as an argument it generates Relative URIerror .
so what is the way in which i can pass url_add as an argument only.
response = store.add_from_uri('url_add')
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/django_gstudio-0.3.dev-py2.7.egg/gstudio/testing1.py", line 152, in
response = store.add_from_uri('url_add')
File "/usr/local/lib/python2.7/dist-packages/django_gstudio-0.3.dev-py2.7.egg/gstudio/HTTP4Store/HTTP4Store.py", line 74, in add_from_uri
r_obj = self.rh.GET(uri, headers=headers)
File "/usr/local/lib/python2.7/dist-packages/django_gstudio-0.3.dev-py2.7.egg/gstudio/HTTP4Store/utils.py", line 53, in GET
return self._request("%s" % (path), method="GET", headers=headers)
File "/usr/local/lib/python2.7/dist-packages/django_gstudio-0.3.dev-py2.7.egg/gstudio/HTTP4Store/utils.py", line 92, in _request
resp, content = self.h.request(path, method, headers=headers, body=data)
File "/usr/lib/python2.7/dist-packages/httplib2/init.py", line 1297, in request
(scheme, authority, request_uri, defrag_uri) = urlnorm(uri)
File "/usr/lib/python2.7/dist-packages/httplib2/init.py", line 204, in urlnorm
raise RelativeURIError("Only absolute URIs are allowed. uri = %s" % uri)
RelativeURIError: Only absolute URIs are allowed. uri = url_add
What is the value of your URL
It is throwing an exception because you are passing a relative url instead of an absolute (you probably have something like "../../directory/filename.rdf"
If your url is an HTTP url e.g (http://host/filename.rdf) and dereferenceable then you can also use the LOAD directive as part of SPARQL update. So that simply means exectuting a SPARQL query (in the same way you execute a SPARQL query using 4store) using the following expression:
LOAD <http://host/filename.rdf>
INTO GRAPH <http://optional-name-of-graph>