We Keep Coding

SSL Error in thread with simple post request in python

I'm trying to do a simple post request, I'm using a list because I want to send all my post request at the same time using thread. Here is an example of an url :
s = "https://emoncms.org/input/post.json?node="+str(test)+"&json={test_stack_overflow:0}&apikey="+str(apikey)
list.append(threading.Thread(target=requests.post, args=([s, ])))
I was using this code maybe 3 months ago and it worked perfectly.
I wanted to get back on this project this week and I realized that I got some errors, this one particularly :
Exception in thread Thread-14:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/dist-packages/requests/api.py", line 94, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/api.py", line 49, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 569, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 420, in send
raise SSLError(e, request=request)
SSLError: <unprintable SSLError object>
I got an other error, ConnectionError but I think it's due to the network or because the website can't follow it's activity or is down. I leave you the traceback if you want :
ConnectionError: ('Connection aborted.', error(101, 'Network is unreachable'))
This code is only a part of my project, the code is running every minutes and I don't know why but this issue (SSLError) comes only maybe 10 times a day. I got this script running on different Raspberry Pi and some have the same problem but not the same frequency, others don't have it at all.
Any ideas on what is going ?
Thanks in advance !

Use verify=False in the requests method like this
import requests
url="https://emoncms.org/input/post.json?node="+str(test)+"&json={test_stack_overflow:0}&apikey="+str(apikey)
requests.post(url,verify=False)
If you are using with threads then it will be like
list.append(threading.Thread(target=requests.post, args=(url,),kwargs={"verify":False})) #**kwargs should be passed seperately.
You are getting this error because python requests tries to verify certificate for https connections so you have to override it by passing verify=False or you can also provide certificate in verify like this requests.get(url,verify="/path/to/certificate.ext")
Also I doubt that this should be a get request because query parameters won't come in post request as of my knowledge. So if you use GET method same verify applies there too.

rollbar not sending exceptions once deployed in GAE

I have a project in GAE using python, in which I set up rollbar to track any errors and exceptions. Error reporting worked on localhost, but once I deployed it, my app isn't sending any error log/info that I intentionally created.
I have the rollbar set up as below:
import rollbar
rollbar.init('xxxxxxxxxxxxxxxxxxxx', 'production')
..
..
#when an error occur
try:
....
except:
rollbar.report_exc_info()
UPDATE(11/20):
I'm able to get rollbar to work now, but had to use a older version of requests library (ver 2.3.0). Newer version such as 2.7.0 or 2.8.1 would give me a connection error, anyone knows why or how to get around?
ERROR 2015-11-20 17:44:03,543 __init__.py:1158] Exception while posting item ConnectionError(ProtocolError('Connection aborted.', error(13, 'Permission denied')),)
Traceback (most recent call last):
File "/Path/to/my/project/rollbar/__init__.py", line 1156, in _send_payload
_post_api('item/', payload, access_token=access_token)
File "/Path/to/my/project/rollbar/__init__.py", line 1197, in _post_api
verify=SETTINGS.get('verify_https', True))
File "/Path/to/my/project/requests/api.py", line 109, in post
return request('post', url, data=data, json=json, **kwargs)
File "/Path/to/my/project/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/Path/to/my/project/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/Path/to/my/project/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/Path/to/my/project/requests/adapters.py", line 412, in send
raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))

I ran into the same issue with Rollbar and GAE and the underlying cause was that I was running Google App Engine locally as a normal user that didn't have permissions to open a socket. When I deployed it to Google App Engine, it ran fine.

Python requests gives SSL unknown protocol

I am trying to send a request to an API I have set up on an AWS machine.
The code I use is as follows:
import requests
import json
report_dict = {
"client_name": "Wayne Enterprises",
"client_id": 123,
"report_type": "api_testing",
"timestamp_generated": "2015-07-29T11:00:00Z",
"report_data": {"revenue": 9000.00}
}
report_json = json.dumps(report_dict)
resp = requests.post("https://my-url.com:8080/my-api/reports", data=report_json,verify=False)
Doing this, I get:
Traceback (most recent call last):
File "art2_java_test.py", line 124, in <module>
main()
File "art2_java_test.py", line 9, in main
test_post_good_data()
File "art2_java_test.py", line 29, in test_post_good_data
resp = requests.post("https://my-url.com:8080/my-api/reports", data=report_json,verify=Fal
se)
File "C:\Python27\lib\site-packages\requests-2.7.0-py2.7.egg\requests\api.py",
line 109, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Python27\lib\site-packages\requests-2.7.0-py2.7.egg\requests\api.py",
line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests-2.7.0-py2.7.egg\requests\sessions
.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests-2.7.0-py2.7.egg\requests\sessions
.py", line 573, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests-2.7.0-py2.7.egg\requests\adapters
.py", line 428, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:5
90)
But when I send the request as http instead of https, it (usually) works fine. I've found some evidence that this can have to do with proxy servers, but I am not using one. Are there any other potential reasons for this error? This is a website only available on my company's local network, if that's relevant.

.... https://my-url.com:8080/my-api/reports
...But when I send the request as http instead of https, it (usually) works fine.
My guess is that you are trying the same port 8080 for http and https. But, servers usually listen on a single port either for http or https and not both. This means that if your client is trying to start the TLS handshake needed for https against this port it will get a plain error message back. The client then tries to interpret this error message as TLS and returns some weird error messages, because the response is not TLS at all.

Python Requests PUT Timeout (10060)

I have a command-line Python script that is looping through the rows of a table, using the python-requests library to make PUT requests to an API for each row. The syntax looks like:
updateResponse = requests.put(updateRowUrl, data=json.dumps(payload), headers=headers)
The process works well for me, but I have a user getting the following error after the process makes several successful calls:
File "C:\Python27\lib\site-packages\requests\api.py", line 105, in put
return request('put', url, data=data, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 49, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 457, in reques
t
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 569, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 407, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(10060, 'A con
nection attempt failed because the connected party did not properly respond afte
r a period of time, or established connection failed because connected host has
failed to respond'))
Judging by the 10060 error message I thought that the connection was timing out, so I suggested that the user add a timeout=None argument to the put call, but this doesn't seem to have solved this issue.
Are there any suggestions on what to try next?

Elastic Search HTTPConnectionPool(host='127.0.0.1', port=9200): Max retries exceeded

I have an Elastic Search server which I am querying both from a live website and through a Django management command. The management command runs using celery beat at 3am GMT to synchronise data from an outside service. Sometimes (but not every time) this command is run elastic search appears to crash and I get the following error in my error log.
[09/Jan/2014 08:03:46] ERROR [django.request:212] Internal Server Error: /
Traceback (most recent call last):
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/django/core/handlers/base.py",
line 115, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/django/views/generic/base.py",
line 68, in view
return self.dispatch(request, *args, **kwargs)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/django/views/generic/base.py",
line 86, in dispatch
return handler(request, *args, **kwargs)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/django/views/generic/base.py",
line 153, in get
context = self.get_context_data(**kwargs)
File
"/srv/www/site.co.uk/clothes_comparison/clothes_comparison/views.py",
line 56, in get_context_data
fields=['id', 'name', 'price', 'images', 'advertiser']
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/pyelasticsearch/client.py",
line 96, in decorate
return func(*args, query_params=query_params, **kwargs)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/pyelasticsearch/client.py",
line 512, in multi_get
'GET', ['_mget'], {'docs': docs}, query_params=query_params)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/pyelasticsearch/client.py",
line 238, in send_request
**({'data': request_body} if body else {}))
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/requests/sessions.py",
line 347, in get
return self.request('GET', url, **kwargs)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/requests/sessions.py",
line 335, in request
resp = self.send(prep, **send_kwargs)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/requests/sessions.py",
line 438, in send
r = adapter.send(request, **kwargs)
File
"/srv/www/site.co.uk/env/local/lib/python2.7/site-packages/requests/adapters.py",
line 327, in send
raise ConnectionError(e)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=9200): Max
retries exceeded with url: /_mget (Caused by <class 'socket.error'>: [Errno 111]
Connection refused)
I am using pyelasticsearch to connect to Elastic Search with the following code in my settings.py file:
try:
ES_CON
except NameError:
ES_CON = None
if not ES_CON:
ES_CON = ElasticSearch(ELASTICSEARCH_URI)
Any help would be greatly appreciated.

I would suggest using the official Elasticsearch python client: elasticsearch-py which has reliable connection handling, is thread safe etc. It is also faster (according to the author, who is part of the Elasticsearch team).
You can then have your es = Elasticsearch() either at the top of your tasks.py, or in e.g. core.helpers and import es from there.