We Keep Coding

Custom exception handling when sending message to remote syslog server in Python

At the moment i am writing a syslog client that will send messages to a remote syslog server. So far this is working pretty ok but i am running into the following problem.
When the syslog server goes down for some reason i need to catch this so the program will stop sending syslog messages and we can investigate the problem.
Unfortunately, the program continues running and doesn't see that the TCP socket is closed and raise an exception.
I only receive a traceback in my terminal:
--- Logging error --- Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\logging\handlers.py", line 941, in emit
self.socket.sendall(msg) ConnectionAbortedError: [WinError 10053] Call stack:
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\ptvsd_launcher.py", line 45, in <module>
main(ptvsdArgs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\__main__.py", line 265, in main
wait=args.wait)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\__main__.py", line 258, in handle_args
debug_main(addr, name, kind, *extra, **kwargs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_local.py", line 45, in debug_main
run_file(address, name, *extra, **kwargs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_local.py", line 79, in run_file
run(argv, addr, **kwargs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_local.py", line 140, in _run
_pydevd.main()
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\pydevd.py", line 1925, in main
debugger.connect(host, port)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\pydevd.py", line 1283, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\pydevd.py", line 1290, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\_pydev_imps\_pydev_execfile.py", line 25, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "c:\Users\Administrator\OneDrive\Documents\Python Scripts\testlogger.py", line 71, in <module>
my_logger.info(i) Message: 'test4' Arguments: ()
Relevant code:
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.ERROR)
my_logger.setLevel(logging.INFO)
my_logger.setLevel(logging.DEBUG)
try:
handler = logging.handlers.SysLogHandler(('IP ADDRESS HOST', 514), socktype=socket.SOCK_STREAM)
my_logger.addHandler(handler)
except Exception as e:
print (e)
list1 = ['test','test2','test3','test4','test5','test6','test7','test8']
for i in list1:
try:
my_logger.info(i) #here i expected that an exception would be raised when the TCP socket is not alive anymore
except Exception as e:
print (e)
How can i make sure that the program stops and i can do the appropriate exception handling?
Thanks!

The default behavior of the SysLogHandler class (and all the ones who are using TCP) is to retry a connection, this is explained in the docs of the createSocket() method:
Tries to create a socket; on failure, uses an exponential back-off
algorithm. On initial failure, the handler will drop the message it
was trying to send. When subsequent messages are handled by the same
instance, it will not try connecting until some time has passed. The
default parameters are such that the initial delay is one second, and
if after that delay the connection still can’t be made, the handler
will double the delay each time up to a maximum of 30 seconds.
This behaviour is controlled by the following handler attributes:
retryStart (initial delay, defaulting to 1.0 seconds).
retryFactor (multiplier, defaulting to 2.0).
retryMax (maximum delay, defaulting to 30.0 seconds).
As it doesn't seem to have an option for the behavior of "not retrying" which you seem to want, so if you really want that you can create your own handler by subclassing it and overriding the createSocket() method with something like:
class MySysLogHandler(logging.handlers.SysLogHandler):
def createSocket(self):
try:
self.sock = self.makeSocket()
except OSError:
# do your own error handling here ...
You can dig a bit more by looking at the source code of createSocket() in CPython Github repo (beware, this is from master branch and might not be the exact version of Python you're using)

I switched to another solution and stopped using the SyslogHandler class.
I now use the following class were i wrote my own syslog sender through a socket.
class Syslog:
def __init__(self,host="localhost",port=514,facility=Facility.DAEMON):
self.host = host
self.port = port
self.facility = facility
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def connect(self):
try:
self.socket.connect((self.host, self.port))
except Exception as e:
print("failed setting up connection")
def send(self, message,level):
data = "<%d>%s" % (level + self.facility*8, message + "\n")
try:
self.socket.sendall(data.encode('utf-8'))
except Exception as e:
print("send failed")
#if __name__ == '__main__':
syslog1 = Syslog(host='HOSTIPADDRESS-NAME')
syslog1.connect()
messages = ["test1","test2","test3","test4","test5","test6","test7"]
for message in messages:
syslog1.send(message,Level.WARNING)
This is working quite well and runs into an exception when the syslog server goes down unexpectedly. The only problem now i discovered while debugging is the following:
When i shut down the syslog server it throws not immediatly an exception when i try to send a message.
Please see example below:
1.) the syslog server is started, i send the first message "test1" from the for loop, successfull.
2.) i shutdown the syslog server, now i send the second message "test2" from the for loop. Nothing happens, no exception!
3.) i send the third message "test3", now an exception is thrown.
How is this possible?

How do I troubleshoot spur failing to make an SSH connection in python?

I have two nearly identical devices. spur will connect via ssh with one, but not the other. How do I figure out why?
>>> shell1 = spur.SshShell('10.201.140.242', 'username', 'password', missing_host_key=spur.ssh.MissingHostKey.accept)
>>> results = shell1.run(['ls', '-a'])
>>> results.output
'.\n..\n.aptitude\n.bashrc\n.cache\n.config\n.profile\n'
>>> shell2 = spur.SshShell('10.201.129.56', 'username', 'password', missing_host_key=spur.ssh.MissingHostKey.accept)
>>> results = shell2.run(['ls', '-a'])
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "E:\development\virtenv\lib\site-packages\spur\ssh.py", line 166, in run
return self.spawn(*args, **kwargs).wait_for_result()
File "E:\development\virtenv\lib\site-packages\spur\ssh.py", line 178, in spawn
channel = self._get_ssh_transport().open_session()
File "E:\development\virtenv\lib\site-packages\spur\ssh.py", line 268, in _get_ssh_transport
raise self._connection_error(error)
ConnectionError: Error creating SSH connection
Original error: ('10.201.129.56', <paramiko.ecdsakey.ECDSAKey object at 0x11328070>, <paramiko.ecdsakey.ECDSAKey object at 0x1135F350>)
I'm confused by the error message. What does returning the ip, and two key objects supposed to mean? Is there helpful info here I'm supposed to glean from that?
Both devices will accept ssh connections from the command line, so that sets aside an obvious problem.
Both are running the same version of Ubuntu, with the same login credentials. Home directories are even the same (no .ssh dir). Even further, both of their sshd_config files are identical (so, also using the same version among other config options).
The problem doesn't seem to be in the ssh settings, but the error gives no indication of where the problem could be!
Any ideas?
Enabling logging doesn't add much.
shell1:
11:32:11|[ INFO] - paramiko.transport - _log - Connected (version 2.0, client OpenSSH_5.9p1)
11:32:11|[ INFO] - paramiko.transport - _log - Authentication (password) successful!
shell2:
11:32:25|[ INFO] - paramiko.transport - _log - Connected (version 2.0, client OpenSSH_5.9p1)
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "E:\development\virtenv\lib\site-packages\spur\ssh.py", line 166, in run
return self.spawn(*args, **kwargs).wait_for_result()
File "E:\development\virtenv\lib\site-packages\spur\ssh.py", line 178, in spawn
File "E:\development\virtenv\lib\site-packages\spur\ssh.py", line 268, in _get_ssh_transport
raise self._connection_error(error)
ConnectionError: Error creating SSH connection
Original error: ('10.201.129.56', <paramiko.ecdsakey.ECDSAKey object at 0x1132EF10>, <paramiko.ecdsakey.ECDSAKey object at 0x11366DF0>)
It might be telling that the SSH reports connection. The failure occurs before/during authentication. But as I said above, the passwords are the same -- both connections even use the same copy-pasted pw, which works without error on a command line connection.

Trouble getting simple Kafka producer going

I'm running Kafka locally on my Mac Pro (Sierra; 10.12.6) just to get started with development. I've started ZooKeeper and a Kafka server (0.11.0.1):
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties
I've got topics created:
bin/kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
access
my-topic
(not sure what __consumer_offsets is, I created the other two).
I've installed kafka-python (1.3.4).
My sample program is dead simple:
from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers=['localhost:9092'])
producer.send('my-topic', 'Another message')
But it croaks with the following message:
Traceback (most recent call last):
File "produce.py", line 3, in <module>
producer = KafkaProducer(bootstrap_servers=['localhost:9092'])
File "/Library/Python/2.7/site-packages/kafka/producer/kafka.py", line 347, in __init__
**self.config)
File "/Library/Python/2.7/site-packages/kafka/client_async.py", line 220, in __init__
self.config['api_version'] = self.check_version(timeout=check_timeout)
File "/Library/Python/2.7/site-packages/kafka/client_async.py", line 861, in check_version
raise Errors.NoBrokersAvailable()
kafka.errors.NoBrokersAvailable: NoBrokersAvailable
Ideas? Any assistance appreciated.

Please insure that you have the setting defined in the server.config file
advertised.listeners=PLAINTEXT://your.host.name:9092 .
It might be possible that the host name resolution is giving some other host name , by default Kafka uses java.net.InetAddress.getCanonicalHostName()

If you're using wurstmeister/kafka, please notice that in Kafka's last version many parameters have been deprecated.
Instead of using -
KAFKA_HOST:
KAFKA_PORT: 9092
KAFKA_ADVERTISED_HOST_NAME: <IP-ADDRESS>
KAFKA_ADVERTISED_PORT: 9092
you need to use -
KAFKA_LISTENERS: PLAINTEXT://:9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://<IP-ADDRESS>:9092
view this link for more details

Handling Non-SSL Traffic in Python/Tornado

I have a webservice running in python 2.7.10 / Tornado that uses SSL. This service throws an error when a non-SSL call comes through (http://...).
I don't want my service to be accessible when SSL is not used, but I'd like to handle it in a cleaner fashion.
Here is my main code that works great over SSL:
if __name__ == "__main__":
tornado.options.parse_command_line()
#does not work on 2.7.6
ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_ctx.load_cert_chain("...crt.pem","...key.pem")
ssl_ctx.load_verify_locations("...CA.crt.pem")
http_server = tornado.httpserver.HTTPServer(application, ssl_options=ssl_ctx, decompress_request=True)
http_server.listen(options.port)
mainloop = tornado.ioloop.IOLoop.instance()
print("Main Server started on port XXXX")
mainloop.start()
and here is the error when I hit that server with http://... instead of https://...:
[E 151027 20:45:57 http1connection:700] Uncaught exception
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/http1connection.py", line 691, in _server_request_loop
ret = yield conn.read_response(request_delegate)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 807, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 209, in result
raise_exc_info(self._exc_info)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 810, in run
yielded = self.gen.throw(*sys.exc_info())
File "/usr/local/lib/python2.7/dist-packages/tornado/http1connection.py", line 166, in _read_message
quiet_exceptions=iostream.StreamClosedError)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 807, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 209, in result
raise_exc_info(self._exc_info)
File "<string>", line 3, in raise_exc_info
SSLError: [SSL: HTTP_REQUEST] http request (_ssl.c:590)
Any ideas how I should handle that exception?
And what the standard-conform return value would be when I catch a non-SSL call to an SSL-only API?
UPDATE
This API runs on a specific port e.g. https://example.com:1234/. I want to inform a user who is trying to connect without SSL, e.g. http://example.com:1234/ that what they are doing is incorrect by returning an error message or status code. As it is the uncaught exception returns a 500, which they could interpret as a programming error on my part. Any ideas?

There's an excelent discussion in this Tornado issue about that, where Tornado maintainer says:
If you have both HTTP and HTTPS in the same tornado process, you must be running two separate HTTPServers (of course such a feature should not be tied to whether SSL is handled at the tornado level, since you could be terminating SSL in a proxy, but since your question stipulated that SSL was enabled in tornado let's focus on this case first). You could simply give the HTTP server a different Application, one that just does this redirect.
So, the best solution it's to HTTPServer that listens on port 80 and doesn't has the ssl_options parameter setted.
UPDATE
A request to https://example.com/some/path will go to port 443, where you must have an HTTPServer configured to handle https traffic; while a request to http://example.com/some/path will go to port 80, where you must have another instance of HTTPServer without ssl options, and this is where you must return the custom response code you want. That shouldn't raise any error.

Why is this MQTT client example failing?

I am trying out MQTT for the first time using Python and the mosquitto library. My client program is below. I'm trying to use the public demo MQTT server at http://www.mqtt-dashboard.com/subscribe. However the client code is failing, see error below. Any ideas on what's going on?
#!/usr/bin/env python
import mosquitto
client = mosquitto.Mosquitto("fredtest", clean_session=True)
client.connect("broker.mqttdashboard.com", 1883)
client.publish("fred.test", "hello world", 1)
client.loop_forever()
Error message:
C:\tmp>python mqttclient.py
Traceback (most recent call last):
File "mqttclient.py", line 6, in
client.connect("broker.mqttdashboard.com", 1883)
File "build\bdist.win-amd64\egg\mosquitto.py", line 582, in connect
File "build\bdist.win-amd64\egg\mosquitto.py", line 657, in reconnect
File "c:\python27\lib\socket.py", line 571, in create_connection
raise err
socket.error: [Errno 10060] A connection attempt failed because the connected pa
rty did not properly respond after a period of time, or established connection f
ailed because connected host has failed to respond

I'm not currently able to connect any client to broker.mqttdashboard.com:1883 - so this probably isn't an issue with your code.
To sanity check, have you tried connected to another broker, such as iot.eclipse.org:1883 ?