I am unable to PUT Nest data like ambient_temperature_f to https://developer-api.nest.com or the redirected Firebase URL. I suspect there is something specific to the Nest that will need to be tweaked in the Firebase module I am using (https://ozgur.github.io/python-firebase/).
From firebase.py:
#http_connection(60)
def make_put_request(url, data, params, headers, connection):
"""
Helper function that makes an HTTP PUT request to the given firebase
endpoint. Timeout is 60 seconds.
`url`: The full URL of the firebase endpoint (DSN appended.)
`data`: JSON serializable dict that will be stored in the remote storage.
`params`: Python dict that is appended to the URL like a querystring.
`headers`: Python dict. HTTP request headers.
`connection`: Predefined HTTP connection instance. If not given, it
is supplied by the `decorators.http_connection` function.
The returning value is a Python dict deserialized by the JSON decoder. However,
if the status code is not 2x or 403, an requests.HTTPError is raised.
connection = connection_pool.get_available_connection()
response = make_put_request('http://firebase.localhost/users',
'{"1": "Ozgur Vatansever"}',
{'X_FIREBASE_SOMETHING': 'Hi'}, connection)
response => {'1': 'Ozgur Vatansever'} or {'error': 'Permission denied.'}
"""
timeout = getattr(connection, 'timeout')
response = connection.put(url, data=data, params=params, headers=headers, timeout=timeout)
print('[make_put_request]: [%s][%s][%s][%s]\n' %(url, data, params, headers))
if response.ok or response.status_code == 403:
return response.json() if response.content else None
else:
response.raise_for_status()
Prints out:
[make_put_request]: [https://developer-api.nest.com/devices/thermostats/DEVICE_ID/ambient_temperature_f.json?auth=VALID_AUTH_TOKEN][71][{}][{}]
Returns error:
Traceback (most recent call last):
File "C:\py\nest_auth.py", line 90, in <module>
put_result = firebase.put(data_url, field_name, 71)
File "C:\Python34\lib\site-packages\firebase\decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "C:\Python34\lib\site-packages\firebase\firebase.py", line 312, in put
connection=connection)
File "C:\Python34\lib\site-packages\firebase\decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "C:\Python34\lib\site-packages\firebase\firebase.py", line 77, in make_put_request
response.raise_for_status()
File "C:\Python34\lib\site-packages\requests\models.py", line 808, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request
This worked when using a firebaseio.com target but isn't working for Nest:
put_result = firebase.put('/devices/thermostats/DEVICE_ID/', 'ambient_temperature', 71)
According to the documentation ambient_temperature_f is a read only field that represent's the reported ambient temperature from the thermostat. It wouldn't make sense to override that since it is a sensor reading.
I think you want to write to target_temperature_f, which is the temperature the thermostat should heat or cool to.
Related
I'm running a webserver based on the Bottle framework. This framework exposes an endpoint where you can upload a file. Sometimes, if the disk is full, the webserver is supposed to return a code 429 without reading the file. For some reason, when I'm uploading a file, if the server tries to return a 429 status code the following exception is raised:
File "../hooks.py", line 325, in post_video_content
response = requests.post(url, data=fh)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 110, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 473, in send
raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))
But if try the exact same post but instead of posting a file I simply post some small amount of data, I actually get a response object back from the request and I can inspect the status code.
To reproduce:
webserver.py
from bottle import Bottle, route, run, request, response, get, post, put, abort
#post('/content')
#authenticate
def content():
response.status = 429
return "failed to write incoming movie to temp file (space might be full temporarily)"
run(host="127.0.0.1", port=8080, debug=True)
send_request.py
import requests
# this works and prints the status code
ret = requests.post("127.0.0.1:8080/content", data="sdfsdfsdfsdf")
print ret.status_code
# this part will throw an exception
try:
with open("~/some_large_video_file.mp4", 'rb') as fh:
ret = requests.post('127.0.0.1:8080/content', data=fh)
except:
print traceback.format_exc()
EDIT - I've found out that it depends on the amount of data. If you try and post something like 10KB of data we get the 429 return code. I'm trying to figure out what the limit is that causes it to start raising an exception.
EDIT2 - So it looks like the magic filesize is somewhere between 117K and 131K. If I try the former it works as intended, I get the response from the request and can access the status code. If I try the former I get the exception raised.
My questions are:
Why does this happen? Is it a bug in bottle? A bug in requests? Why would the size/type of data I'm posting change the response?
Is there any way to get around this? I'm trying to associate a ConnectionError exception with the server being down, and a 429 response code meaning the disk is full. If I'm getting the same exception for both I can't tell the difference between the situations which blocks me from implementing a back-off to wait for disk space to open up
So I cross-posted this to the requests github issue page, and it looks like the issue is in the httplib module that requests is built on top of, so this is a no-fix for now.
see: https://github.com/kennethreitz/requests/issues/4062
I'm reading some source code which raises errors for bad requests in the following way:
import requests
response = requests.get("www.google.com") # This won't work because it's missing the http://
if response.ok is False or response.json()['status'] != 'success':
raise Exception("API error: {}".format(response.json()['message']))
I was thinking that the last two lines could be replaced with
response.raise_for_status()
I actually don't see any difference in the error returned. In both cases it is
Traceback (most recent call last):
File "/home/kurt/Documents/Scratch/requests_test.py", line 3, in <module>
response = requests.get("www.google.com") # This won't work because it's missing the http://
File "/home/kurt/.local/lib/python2.7/site-packages/requests/api.py", line 69, in get
return request('get', url, params=params, **kwargs)
File "/home/kurt/.local/lib/python2.7/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/home/kurt/.local/lib/python2.7/site-packages/requests/sessions.py", line 451, in request
prep = self.prepare_request(req)
File "/home/kurt/.local/lib/python2.7/site-packages/requests/sessions.py", line 382, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/home/kurt/.local/lib/python2.7/site-packages/requests/models.py", line 304, in prepare
self.prepare_url(url, params)
File "/home/kurt/.local/lib/python2.7/site-packages/requests/models.py", line 362, in prepare_url
to_native_string(url, 'utf8')))
requests.exceptions.MissingSchema: Invalid URL 'www.google.com': No schema supplied. Perhaps you meant http://www.google.com?
It seems to me that raise_for_status() is more succinct and perhaps also doesn't lose information on the original exception (cf. Use of "except Exception" vs. "except ... raise" in Python). Would this indeed be a better approach?
response.raise_for_status() only raises an exception if the response status code is not a 200 response. The second case, where response.json()['status'] != 'success' is true, is not covered.
You have a different error, however. You never get to the if test, because the exception is raised by the requests.get() call. You failed to pass in a schema (no http:// or https:// at the start). Because the exception is raised in the requests.get() expression, the next line is simply never executed. No request is even sent, so you can't make any assertions about the response either.
The test has more problems:
requests.ok is False is not idiomatic Python; you'd use not requests.ok instead
If requests.ok is False is true, then the requests.json() call will most likely fail altogether, so the next line using response.json()['message']) will produce a different exception.
I'm trying to implement a simple Django view that accept a file attachment and some other parameters and proxy the request on a remote API call.
Please note: the proxy is not the point of my question :)
This is how I implemented the view:
def image_upload(request):
token = request.POST['token']
image_file = request.FILES.values()[0]
files = {'file': ('myupload.txt', image_file.read())}
client_id = request.POST['client_id']
folder_id = request.POST['folder_id']
advert_id = request.POST['advert_id']
image_type = request.POST['image_type']
crop_image = request.POST['crop_image']
api_base_url = settings.API_BASE_URL
file_post_data = {'client_id': client_id, 'folder_id': folder_id, 'advert_id': advert_id,
'image_type': image_type, 'crop_image': crop_image}
auth_header = {'Authorization': 'Token ' + token}
r = requests.post(api_base_url + 'assets/image/upload/',
data = json.dumps(file_post_data),
headers = auth_header,
files = files)
return r.json()
The problem is that when I test this view (I use Django Test Client to do it) I get an error on the line "files = files)" that says "ValueError: cannot encode objects that are not 2-tuples".
The complete trace log is this one:
======================================================================
ERROR: test_image_upload (fbx.tests.FbxTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/tests.py", line 18, in test_image_upload
'image_type': 'A', 'crop_image': False, 'attachment': fp})
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 449, in post
response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 262, in post
return self.request(**r)
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/views.py", line 42, in image_upload
files = files)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 324, in request
prep = req.prepare()
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 225, in prepare
p.prepare_body(self.data, self.files)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 385, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 99, in _encode_files
fields = to_key_val_list(data or {})
File "/usr/local/lib/python2.7/dist-packages/requests/utils.py", line 136, in to_key_val_list
raise ValueError('cannot encode objects that are not 2-tuples')
ValueError: cannot encode objects that are not 2-tuples
I've also tried a quick test using this to read a file: files = {'file': ('myupload.txt', open('/tmp/mytmp.txt', 'rb'))}
but it doesn't work. Do tou have any idea about how to fix this?
Thanks!
You cannot simultaneously post JSON data and multipart/form-data which is in essence what you're trying to do here. json.dumps returns a string so you're sending a string that looks like
'{"client_id": 1, "folder_id": 2, "advert_id": 3, "image_type": "jpeg", "crop_image": true}'
And then telling requests you want to use that in combination with a multipart/form-data request by sending something in via the files parameter. That is impossible and could possibly raise a better exception.
Either add the file to the JSON data after reading it into memory and send that with the appropriate Content-Type header or send the entire thing as a multipart/form-data request without using json.dumps and simply passing the dictionary you're creating to data. Use one or the other but not both.
That aside, you say that your exception comes from the line that only has files=files) on it, but it is not that line alone that causes the exception. The exception is raised by the function which happens to end on that line. The fact that your exception arises from that too is mere coincidence. This is an wart in Python that may possibly be fixed in Python 3.4. You should upgrade, because 3.4 will be awesome and newer versions of Django support Python 3.x.
The Documentation for python requests module says for hooks that "If the callback function returns a value, it is assumed that it is to replace the data that was passed in. If the function doesn’t return anything, nothing else is effected"
Now i am trying to return a value(int in my case) from my hook function and it throws an exception. This will be valid in all the cases when the return value is an object that DOESNOT have the raw() method defined for it.
Here is some code
def hook(resp,**kwargs):
print resp.url
return 1
def main()
s = requests.Session()
s.hooks = {"response":hook}
r = s.get("http://localhost/index.html")
And here is the exception:
http://localhost/index.html
Traceback (most recent call last):
File "/home/talha/ws/test.py", line 85, in <module>
main()
File "/home/talha/ws/test.py", line 72, in main
r = s.get("http://localhost/index.html")
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 347, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 446, in send
extract_cookies_to_jar(self.cookies, request, r.raw)
AttributeError: 'int' object has no attribute 'raw'
The code in sessions.py #line 446 is trying to extract cookies after the dispatch_hook..From source
# Response manipulation hooks
r = dispatch_hook('response', hooks, r, **kwargs)
# Persist cookies
extract_cookies_to_jar(self.cookies, request, r.raw)
Either the documentation needs to change or the handling needs to be re-worked. What is the best way to handle this ?
[update]
Based on the comments I tried to return the base response object. Turns out it cannot be used in that manner also since some of its fields are initialized to None.
Newer code:
def hook(resp, **kwargs):
obj = requests.Response()
return obj
Exception thrown now:
Traceback (most recent call last):
File "/home/talha/ws/test.py", line 88, in <module>
main()
File "/home/talha/ws/test.py", line 75, in main
r = s.get("http://localhost/index.html")
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 347, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 446, in send
extract_cookies_to_jar(self.cookies, request, r.raw)
File "/usr/lib/python2.7/site-packages/requests/cookies.py", line 108, in extract_cookies_to_jar
res = MockResponse(response._original_response.msg)
AttributeError: 'NoneType' object has no attribute '_original_response'
What seems is that i will have to implement a full pseudo response?
If the callback function returns a value, it is assumed that it is to replace the data that was passed in. If the function doesn’t return anything, nothing else is effected.
This means that whatever you return is expected to take the place of the response object you were passed.
Nothing in the documentation states that you can return just anything. What did you expect to happen instead?
If you wanted to return a response that has different data, return something that acts like a response still. This means that either you need to subclass the requests response object, or implement something that provides the same API:
from requests.models import Response
class MyIntResponse(Response):
def __init__(self, integer):
super(MyIntResponse, self).__init__()
self._content_consumed = True
self._content = integer
def hook(resp,**kwargs):
print resp.url
newresp = MyIntResponse(1)
newresp.raw = resp.raw # copy across original HTTP response object
You may want to copy over some of the other attributes from the original response; check the documentation on what attributes Response objects have.
I receive a CannotSendRequest in my oauth library (Django socialauth) when I try to connect to Twitter.
Traceback:
File "/Library/Python/2.6/site-packages/django/core/handlers/base.py" in get_response
100. response = callback(request, *callback_args, **callback_kwargs)
File "/Users/me/webfaction/project/socialauth/views.py" in twitter_login
94. request_token = twitter.fetch_request_token(callback=request.build_absolute_uri(reverse('socialauth_twitter_login_done')))
File "/Users/me/webfaction/project/socialauth/lib/oauthtwitter2.py" in fetch_request_token
50. return oauth.OAuthToken.from_string(oauth_response(oauth_request))
File "/Users/me/webfaction/project/socialauth/lib/oauthtwitter2.py" in oauth_response
33. connection().request(req.http_method, req.to_url())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py" in request
914. self._send_request(method, url, body, headers)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py" in _send_request
931. self.putrequest(method, url, **skips)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py" in putrequest
818. raise CannotSendRequest()
Exception Type: CannotSendRequest at /accounts/twitter_login/
Exception Value:
Here's where I'm creating the HTTP connection
def connection():
try:
return connection._connection
except AttributeError:
connection._connection = httplib.HTTPSConnection(TWITTER_URL)
return connection._connection
def oauth_response(req):
connection().request(req.http_method, req.to_url())
return connection().getresponse().read()
I've searched SO and found these links, but I'm still not sure how to implement the solution. I've tried and failed. Any help would be appreciated.
httplib CannotSendRequest error in WSGI
When I use httplib for my OAUTH in Python, I always get "CannotSendRequest" and then "
The post you linked to says this error happens when you reuse connections that have thrown an exception and didn't make it to the getresponse() stage.
Indeed, connection.request("GET", "/") x 2 throws the error.
The solution suggested is to re-create the connection every time. Is that what you want to do? Note I have no opinions on this matter, you just asked how to implement what's on those posts.
If so, get rid of your connection() function and always do
connection = httplib.HTTPSConnection(TWITTER_URL)
connection.request(req.http_method, req.to_url())
response = connection.getresponse().read()
connection.close()
return response