We Keep Coding

Airflow Can't Connect to SFTP with SSH Private Key

I am trying to use the SFTP Operator/Hook with a SSH private key instead of a password.
My connection has the following extra arguments: {"key_file": "/home/airflow/key_files/some_key", "no_host_key_check": "true"}. My SSH Private key looks like the following:
-----BEGIN OPENSSH PRIVATE KEY-----
...
-----END OPENSSH PRIVATE KEY-----
Now when I try to create a hook I get the following error:
>>> from airflow.contrib.hooks.sftp_hook import SFTPHook
>>> s = SFTPHook(ftp_conn_id='my_sftp_connection')
[2020-07-08 13:40:25,449] {base_hook.py:84} INFO - Using connection to: id: my_sftp_connection. Host: my_sftp.com, Port: 22, Schema: None, Login: user, Password: None, extra: {'key_file': '/home/airflow/key_files/some_key', 'no_host_key_check': 'true'}
[2020-07-08 13:40:25,452] {base_hook.py:84} INFO - Using connection to: id: my_sftp_connection. Host: my_sftp.com, Port: 22, Schema: None, Login: user, Password: None, extra: {'key_file': '/home/airflow/key_files/some_key', 'no_host_key_check': 'true'}
>>> s.list_directory('/')
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/pysftp/__init__.py", line 167, in _set_authentication
private_key_file, private_key_pass)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/pkey.py", line 206, in from_private_key_file
key = cls(filename=filename, password=password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/rsakey.py", line 55, in __init__
self._from_private_key_file(filename, password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/rsakey.py", line 175, in _from_private_key_file
data = self._read_private_key_file("RSA", filename, password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/pkey.py", line 279, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/pkey.py", line 289, in _read_private_key
raise SSHException("not a valid " + tag + " private key file")
paramiko.ssh_exception.SSHException: not a valid RSA private key file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/hooks/sftp_hook.py", line 151, in list_directory
conn = self.get_conn()
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/hooks/sftp_hook.py", line 114, in get_conn
self.conn = pysftp.Connection(**conn_params)
File "/home/airflow/.local/lib/python3.6/site-packages/pysftp/__init__.py", line 142, in __init__
self._set_authentication(password, private_key, private_key_pass)
File "/home/airflow/.local/lib/python3.6/site-packages/pysftp/__init__.py", line 171, in _set_authentication
private_key_file, private_key_pass)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/pkey.py", line 206, in from_private_key_file
key = cls(filename=filename, password=password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/dsskey.py", line 65, in __init__
self._from_private_key_file(filename, password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/dsskey.py", line 224, in _from_private_key_file
data = self._read_private_key_file("DSA", filename, password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/pkey.py", line 279, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "/home/airflow/.local/lib/python3.6/site-packages/paramiko/pkey.py", line 289, in _read_private_key
raise SSHException("not a valid " + tag + " private key file")
paramiko.ssh_exception.SSHException: not a valid DSA private key file
What am I doing wrong here? That same SSH key works with WINSCP and the SFTP command in bash.

Think I figured it out. I upgraded that paramiko library and now it appears to be working. Will leave this as unanswered in case there was something else I was doing wrong.

I have the exact same problem, except that I'm using Composer, which is Airflow-based managed service in Google Cloud and I do not use the SFTP contrib operator.
Instead I use the apache-airflow-backport-providers-sftp which is backported from Airflow 2.0 and is compatible with my Airflow environment, which is 1.10.14.
It works fine with a user/password, but fails with an RSA key.
The contrib operator and the provider package are equivalent, and depend on the same Python modules: paramiko, pysftp, and sshtunnel.
My pip freeze:
paramiko==2.7.2 (latest release)
pysftp==0.2.9 (latest release)
sshtunnel==0.1.5 (latest release is 0.4.0)
In my understanding, and following your description, the problem is not in the top module (contrib or provider). You said that the problem is in paramiko but it works fine for me with the SSH operator and a RSA key. So, maybe it is in pysftp...
I am lost here.

Trying to specify key for paramiko SSHClient, error: Could not deserialize key data

First off, I want to clarify that I'm pretty new to the whole python world.
I'm trying to create a python script, to ssh to multiple servers and execute bash commands.
For this, I'm using Paramiko, and this has led to some problems.
For some reason, my OpenSSH ssh key doesn't work, but this I have solved by creating a new one, with a different format.
This has led to me having to specify which key I want to use, to start with I tried to connect without a password specified, but then Paramiko couldn't decrypt the key.
password = "somepassword"
pkey_path = "/Users/user1/.ssh/id_rsa_copy"
key = RSAKey.from_private_key_file(pkey_path,password=password)
c = SSHClient()
c.set_missing_host_key_policy(AutoAddPolicy())
c.connect('somesite.com',password=password,pkey=key,look_for_keys=False)
stdin, stdout, stderr = c.exec_command('uptime')
c.close()
When executing the script, I receive the following error:
Traceback (most recent call last):
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/paramiko/rsakey.py", line 185, in _decode_key
data, password=None, backend=default_backend()
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 28, in load_der_private_key
return backend.load_der_private_key(data, password)
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1106, in load_der_private_key
password,
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1277, in _load_key
self._handle_key_loading_error()
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1335, in _handle_key_loading_error
raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/user1/projects/python/paramiko-puppet.py", line 7, in <module>
key = RSAKey.from_private_key_file(pkey_path,password=password)
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/paramiko/pkey.py", line 206, in from_private_key_file
key = cls(filename=filename, password=password)
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/paramiko/rsakey.py", line 55, in __init__
self._from_private_key_file(filename, password)
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/paramiko/rsakey.py", line 176, in _from_private_key_file
self._decode_key(data)
File "/Users/user1/projects/my_first_venv/lib/python3.7/site-packages/paramiko/rsakey.py", line 188, in _decode_key
raise SSHException(str(e))
paramiko.ssh_exception.SSHException: Could not deserialize key data.
EDIT:
As requested here is the command which I used to generate a new ssh key, and what a key looks like when generated.
First I copied my current ssh key and then "transformed" it.
ssh-keygen -p -m PEM -f ~/.ssh/id_rsa_copy
New key:
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,E48A0313BF686BDA7EB6B7F26B10BCC0
jeMD8Eh4g5cpq5YkFKMN5EbquJEVwser2NV6QI/ysO7wPs0SVy3HzoRjaOT4Iox+
gOX90VX5pml3v8S7UmYNzvI+sJMIo/pNpG3qai9OkL9/Z0DJRbTOBXY0nJhkvSVl
W41ojZOot/KWlgrHfPkYZ/Q9L4JJbmP/ChEi1SKFhSByjI9oZnmIHPTEinPib8AZ
j7Wd8S5tF6rJtuapVBPEz8NZLDOsTIME9iql3AxURRbMdQz3IfbWMuSP69ZO8ibe
PFnlHC0u6CjYpZ01axegDSw09xDFuywe0tRfCb0eK7NJQxfeLL3IoKz36+74De2Y
QA5jrJ/Yz/HIIWoOYh+M3wXBIBMSQ5I9/K9twEtyJ/zVOLaXEbJlBiWRPcXzbPXf
NwbfUUNc5ylI729FMB+cE3NuPqen3Hw+FlyKAKu2+E8I56THJIjwsZy3xEY4qkdO
pFukD7UiubB11mJUdT2RkNlI8/0zKCfTjzhRD8lEXFtMu/1OSdPdmaNu62XsXMcq
StA+TqnzMFQ+20vde6NgrZagC/jK3yMJgYSTf9uMLhone+SLVXh0AqPNmuvbHPN7
DvGcYTFlNt340of2Pi9myeOPNdZniLZtq7Oz+Z+2hs/3NxciHbO6MbMpK1Uk2pZK
wovw7+nt71IYjWrzfpVbFI2F7Rw+0rTzmIpe56gv1CtlWZ6pXYbnSakyEKXtCTh8
7CJ4ofSChp4vkCtx6wwjZK/1ysLL1i6pY9Y7TMHumKLTjN+iK0oK//h0/EJIUVVn
3HzX/qwYyTLBJrsr/wvWK6AjYRbUjncDC6fyGVSa25tlcgbo7rsTUaEsiBzhxBK/
zolpRjlBiHU4W64niXJShdU9RGANvPe9jiL1dmg58ZpaKRD4mN6qzxfG8TTBxMLS
+VmMgIFwcFVg2PtQgnoaqqcGFUlRlkoZX8rZ7xktg7ghPTRKxoxaeu0WKYrQzowi
hatyNhQbQ6h8zWb0Mt1+EVcatjAPHp8N/wosTX8QkYtyC82/uFXiz3eQr7RBZl4Z
hd2GkGZLvPPgP6dtXwHeqgGabsDCuFAKYhqnvlXeB6TURqgN9z0uBrRf/JzQV3n2
QZkHCoLKxqK86dw5/yjT+gS/kH8QBvvk/qJEh2tLWRrntD2if6KUXWWaFenq3v/U
ZNb3+ZnJKQMm8ujOJ0gzJo+SC2xDNJFOUMJbi7YVZ2JJSOcZ/Tcfbl8/VriKv+wD
kt2Ksv2vtSYXAqmItUnKcZMPVwujmmr+zpO6Nya+v4+nTfb7lgYYlVR/SxipELm4
HCgTFkQ0bQHURz+WikvSP8EvYqc/DpuvBu4m7yTcbKoJk4iozxzG6K3ligo8QT5o
uF2AFcNLOImo4E3sFUAW6FsDxUV+AuOD16XckRUiz+jYIhh6pbXNhfsYLC0bDaSf
3bC5O9IH7vVWzj8FfcuVxRCzHOoMRROrtTa983KV96YHEAzhIzb9GKEAvFkCN6wE
7a2m3MpTszPa+wnFA4CVuXmLsB5jCmzq0CcPE4SKmX03buN9v+oEOTHDUoAyMebg
RPurPnPkZfJT9RH0MNZ2HaavhJ/eXfALveYgSZBdiY6YiseQr5kKZJItKRZWBwR+
-----END RSA PRIVATE KEY-----

If I do exactly the same, it works for me.
But if I use a wrong passphrase in password=password, I get the same error as you get. So double check that you use the correct passphrase.
The password argument of from_private_key_file is your private key passphrase, not your account password.
In general you should not even need any account password, if you use private key, so I'm not sure why you pass password in the connect call – but that's not the immediate issue now.

WantReadError is raising while sending apple push notifications

I'm trying to send apple push notifications with django-ios-notifications app, but i've faced with WantReadError.
Here is a stacktrace:
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 110, in push_notification_to_devices
self._write_message(notification, devices, chunk_size)
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 132, in _write_message
self._connect()
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 100, in _connect
return super(APNService, self)._connect(self.certificate, self.private_key, self.passphrase)
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 64, in _connect
self.connection.do_handshake()
File ".../local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1076, in do_handshake
self._raise_ssl_error(self._ssl, result)
File ".../local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 847, in _raise_ssl_error
raise WantReadError()
WantReadError
I have no idea how to fix it.
I've created required APNService with correct Certificate and Private key, hostname is gateway.sandbox.push.apple.com.
Command like this
openssl s_client -CApath /etc/ssl/certs/ -connect gateway.sandbox.push.apple.com:2195 -cert my.pem works fine in Ubuntu, it returns
Start Time: 1458724825
Timeout : 300 (sec)
Verify return code: 0 (ok)
UPDATE: problem was solved.
According to https://github.com/stephenmuss/django-ios-notifications/issues/11 it's required to install https://github.com/mjs/gevent_openssl

Try/except not working with twisted starttls given cert/key mismatch

So my twisted mail receiver is working nicely. Right up until we try to handle a case where the config is fubarred, and a mismatched cert/key is passed to the certificate options object for the factory.
I have a module, custom_esmtp.py, which includes an overload of ext_STARTLS(self,rest) which I have modified as follows, to include a try/except:
elif self.ctx and self.canStartTLS:
try:
self.sendCode(220, 'Begin TLS negotiation now')
self.transport.startTLS(self.ctx)
self.startedTLS = True
except:
log.err()
self.sendCode(550, "Internal server error")
return
When I run the code, having passed a cert and key that do not match, I get the following call stack:
Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 220, in _dataReceived
rval = self.protocol.dataReceived(data)
File "/usr/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 454, in dataReceived
self.lineReceived(line)
File "/usr/local/lib/python2.7/site-packages/twisted/mail/smtp.py", line 568, in lineReceived
return getattr(self, 'state_' + self.mode)(line)
File "/usr/local/lib/python2.7/site-packages/twisted/mail/smtp.py", line 582, in state_COMMAND
method('')
--- <exception caught here> ---
File "custom_esmtp.py", line 286, in ext_STARTTLS
self.transport.startTLS(self.ctx)
File "/usr/local/lib/python2.7/site-packages/twisted/internet/_newtls.py", line 179, in startTLS
startTLS(self, ctx, normal, FileDescriptor)
File "/usr/local/lib/python2.7/site-packages/twisted/internet/_newtls.py", line 139, in startTLS
tlsFactory = TLSMemoryBIOFactory(contextFactory, client, None)
File "/usr/local/lib/python2.7/site-packages/twisted/protocols/tls.py", line 769, in __init__
contextFactory = _ContextFactoryToConnectionFactory(contextFactory)
File "/usr/local/lib/python2.7/site-packages/twisted/protocols/tls.py", line 648, in __init__
oldStyleContextFactory.getContext()
File "/usr/local/lib/python2.7/site-packages/twisted/internet/_sslverify.py", line 1429, in getContext
self._context = self._makeContext()
File "/usr/local/lib/python2.7/site-packages/twisted/internet/_sslverify.py", line 1439, in _makeContext
ctx.use_privatekey(self.privateKey)
OpenSSL.SSL.Error: [('x509 certificate routines', 'X509_check_private_key', 'key values mismatch')]
Line 286 of custom_esmtp.py is the self.transport.startTLS(self.ctx). I've looked through all the twisted modules listed in the stack, at the quoted lines, and there are no other try/except blocks.... So my understanding is that the error should be passed back up the stack, unhandled, until it reaches my handler in custom_esmtp.py? So why is it not getting handled - especially since the only except I have is a "catch all"?
Thanks in advance!

If you want this error to be caught, you can do:
from OpenSSL import SSL
# ...
try:
# ...
except SSL.Error:
# ...
Perhaps the syntax changes a bit. I can't check because I don't use this precise package, but the idea is that you have to declare the import path of the exceptions you want to catch.

IOError: unsupported XML-RPC protocol while running yum command

When I try to turn any yum command I get the follwing message. I disabled and enabled SSL before this error occurred. As the system said RHNS-CA-CERT has expired, I removed the certificate and downloaded it using wget command. Then I tried to update the certificate using the yum command and that's where the problem started.
Here's the error message:
Loaded plugins: rhnplugin
Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.AttributeError'> ignored
Traceback (most recent call last):
File "/usr/bin/yum", line 29, in <module>
yummain.user_main(sys.argv[1:], exit_code=True)
File "/usr/share/yum-cli/yummain.py", line 285, in user_main
errcode = main(args)
File "/usr/share/yum-cli/yummain.py", line 105, in main
base.getOptionsConfig(args)
File "/usr/share/yum-cli/cli.py", line 228, in getOptionsConfig
self.conf
File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 891, in <lambda>
conf = property(fget=lambda self: self._getConfig(),
File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 362, in _getConfig
self.plugins.run('init')
File "/usr/lib/python2.6/site-packages/yum/plugins.py", line 184, in run
func(conduitcls(self, self.base, conf, **kwargs))
File "/usr/share/yum-plugins/rhnplugin.py", line 118, in init_hook
login_info = up2dateAuth.getLoginInfo(timeout=timeout)
File "/usr/share/rhn/up2date_client/up2dateAuth.py", line 219, in getLoginInfo
login(timeout=timeout)
File "/usr/share/rhn/up2date_client/up2dateAuth.py", line 170, in login
server = rhnserver.RhnServer(timeout=timeout)
File "/usr/share/rhn/up2date_client/rhnserver.py", line 154, in __init__
timeout=timeout)
File "/usr/share/rhn/up2date_client/rpcServer.py", line 160, in getServer
timeout=timeout)
File "/usr/lib/python2.6/site-packages/rhn/rpclib.py", line 169, in __init__
self._reset_host_handler_and_type()
File "/usr/lib/python2.6/site-packages/rhn/rpclib.py", line 315, in _reset_host_handler_and_type
raise IOError, "unsupported XML-RPC protocol"
IOError: unsupported XML-RPC protocol

OK, my guess is you are running against RHN Classic (rhn.redhat.com). There were erratum fixing this expired certificate and here comes relevant knowleadge base article:
System connection to RHN fails with "The certificate is expired, or certificate verify failed" errors
https://access.redhat.com/solutions/353033
Traceback with IOError: unsupported XML-RPC protocol leads me to guess that you have incorrect serverURL in /etc/sysconfig/rhn/up2date. It should look like this:
serverURL=https://xmlrpc.rhn.redhat.com/XMLRPC