We Keep Coding

"No hostkey for host ***** found" when connecting to SFTP server with pysftp using private key

So I am having many issues connecting to a remote server via SFTP. I have tried the normal way like below.
sftp = pysftp.Connection(host='Host',username='username',password='passwd',private_key=".ppk")
Which did not work. I got the following error:
SSHException: No hostkey for host ***** found.
I then tried the following:
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
s = pysftp.Connection(host='host', username='user', password='password', cnopts=cnopts)
Which also did not work. I got the following error:
BadAuthenticationType: ('Bad authentication type', ['publickey']) (allowed_types=['publickey'])
Also when I run the following:
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect("host",username = "username",password = "password")
ssh_session = client.get_transport().open_session()
I get the same error:
BadAuthenticationType: ('Bad authentication type', ['publickey']) (allowed_types=['publickey'])

Your are confusing a private key used for authentication and a host key used to verify an identify of a server. Both need to be taken care of, while all your code attempts take care of one of them only. See my article on SSH key pairs to understand the difference between the two kinds of keys involved in SSH.
So this should "work":
# Accept any host key (still wrong see below)
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
# And authenticate with a private key
sftp = pysftp.Connection(
host='Host', username='user', password='passwd', private_key=".ppk",
cnopts=cnopts)
But this code will actually blindly accept any host key (cnopts.hostkeys = None), what is a security flaw. For a correct approach, see Verify host key with pysftp.

It looks like the host you are connecting is not available. This usually happens when the host-name is not accessible because of firewall rules (or typo on host).
I'd recommend first checking if you can sftp from the (unix) terminal
> sftp username#host
If you get prompted for password or get logged in, you are able to connect to that host from that machine
If not try checking if that host is available using netcat on port 22, you'd get timeout or broken pipe if host is not available
>nc -v host 22
I recommend debugging the pysftp or paramiko packages only after that.
Also, if you are authenticating using a private key, you do not need to use the password.

implement CRCCheck during SFTP file transfer [duplicate]

I use Paramiko to put a file to an SFTP server:
import paramiko
transport = paramiko.Transport((host, port))
transport.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put(local_path, remote_path)
Now, I would like to check if it worked. The idea is that I compare the checksum of the local file and the remote one (that is located on the SFTP server).
Does Paramiko functionality allows to do that? If it is the case, how exactly it works?

With the SFTP, running over an encrypted SSH session, there's no chance the file contents could get corrupted while transferring. So unless it gets corrupted when reading the local file or writing the remote file, you can be pretty sure that the file was uploaded correctly, if the .put does not throw any error.
try:
sftp.put(local_path, remote_path)
except:
# Something went wrong
If you want to test explicitly anyway:
While there's the check-file extension to the SFTP protocol to calculate a remote file checksum, it's not widely supported. Particularly it's not supported by the most widespread SFTP server implementation, the OpenSSH. See What SFTP server implementations support check-file extension.
If you are lucky to connect to another SFTP server that supports the extension, you can use the Paramiko's SFTPFile.check method.
If not, your only option is to download the file back and compare locally.
If you have a shell access to the server, you can of course try to run some shell checksum command (sha256sum) over a separate shell/SSH connection (or channel) and parse the results. But that's not an SFTP solution anymore. See Comparing MD5 of downloaded files against files on an SFTP server in Python.

if the file doesn't upload then the method will throw an error, so u can check for error
import paramiko
transport = paramiko.Transport((host, port))
transport.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(transport)
try:
sftp.put(local_path, remote_path)
except IOError:
#'Failure'
except OSError:
#'Failure'

How to configure pysftp/paramiko connection with specific HostkeyAlgorithms - python

I need to automate transferring of a file from one server to a client's SFTP server. I've done this hundreds of time using Python's pysftp package. However, on this occasion, there's a HostkeyAlgorithm that I need to set. I've read through Paramiko's doc since pysftp seems lacking of this option entirely and is built on Paramiko. But I honestly don't know what to do (I don't get to play with networking things often). I've been sending manually through bash with the following:
sftp -o HostkeyAlgorithms=+ssh-dss user#host.com
I've tried the following in Python to no success:
import paramiko
_host='somehostname.com'
_user='thisguy'
_pass='you_get_the_idea'
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.MissingHostKeyPolicy())
client.connect(_host, 22, _user, _pass)
This returns:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 424, in connect
passphrase,
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 714, in _auth
raise saved_exception
paramiko.ssh_exception.AuthenticationException: Authentication failed.
So I guess the question is where/how do I add the -o HostkeyAlgorithms=+ssh-dss when setting up my Paramiko connection?

Paramiko will use host key algorithm matching a host key that you configure for your session.
You do not specify any host key, instead you blindly accept all host keys (MissingHostKeyPolicy), what is a security flaw. You lose a protection against MITM attacks.
For a correct (and secure) approach, see:
Python - pysftp / paramiko - Verify host key using its fingerprint
Verify host key with pysftp
Though, I actually do not understand, why do you want to set "HostkeyAlgorithms", if you do not even verify the host key due to MissingHostKeyPolicy? – The "Authentication failed" error is for sure not related to host key.

Paramiko “Unknown Server”

I am just starting off with paramiko, and I'm having some issue with load_system_host_keys().
When I try:
client = SSHClient()
client.load_system_host_keys(filename='/home/barashe/.ssh/known_hosts')
client.connect(hostname='lvs.cs.bgu.ac.il')
stdin, stdout, stderr = client.exec_command('ls -l')
I get
SHException: Server 'lvs.cs.bgu.ac.il' not found in known_hosts
And it seems like the hostkeys instance is empty
list(client.get_host_keys())
[]
If I use load_host_keys() instead of load_system_host_keys() I still get the same error, but the hostkeys instance is not empty now, and it includes the server I'm trying to connect to
list(client.get_host_keys())
['lvs.cs.bgu.ac.il',
'132.72.41.50']
Which seems rather odd...
I know that by using
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
I can avoid this situation, but I prefer doing it the "right" way.
What I'm trying to understand is:
Why am I getting the same error when using load_host_keys() even though the server appears in the hostkeys?
What is the difference between load_host_keys() and load_system_host_keys() in this context?
Cheers!

If this is a private host key file in your home directory, you should not use load_system_host_keys but load_host_keys.
Just out of curiosity, where did you get your host key for that particular host if you did not use set_missing_host_key_policy? If you copied it from your .ssh directory, it is possible that the key file format is different. There are several.
You can test it by adding AutoAdd missing host key policy once and pointing to an empty private host key file. Your login should succeed now (assuming authentication succeeds). Whether it succeeds or fails, your private host key file should now contain the host key in the correct format. You can verify it works by removing the missing host key policy setting and running the script again. It should not moan about missing host keys anymore.
This works for me:
from paramiko import SSHClient
import paramiko
client = SSHClient()
client.load_host_keys(filename='/home/test/stest/kknown_hosts')
# client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(hostname='137.xx.x.x')
stdin, stdout, stderr = client.exec_command('ls -l')
Hope this helps,
Hannu

paramiko allows sftp connection without key

I was running the demo_sftp.py file from the demo folder in the paramiko github. I was stepping through it in PyDev and expected to get an error because I didn't have a key to the server I was trying to connect to but I got the print statement saying that the script couldn't open the host key file and then it went ahead and did the get and put.
Here's a code snippet.
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
except IOError:
try:
# try ~/ssh/ too, because windows can't have a folder named ~/.ssh/
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/ssh/known_hosts'))
except IOError:
print '*** Unable to open host keys file'
host_keys = {}
if host_keys.has_key(hostname):
hostkeytype = host_keys[hostname].keys()[0]
hostkey = host_keys[hostname][hostkeytype]
print 'Using host key of type %s' % hostkeytype
# now, connect and use paramiko Transport to negotiate SSH2 across the connection
try:
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password, hostkey=hostkey)
sftp = paramiko.SFTPClient.from_transport(t)
# dirlist on remote host
dirlist = sftp.listdir('.')
print "Dirlist:", dirlist
I really expected it to go to the except on the t.connect line because hostkey is NoneType.
When I open an ssh connection with
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
ssh.connect('.'.join([self.name, self.domain]),
username=self.username, password=self.password)
stdin, stdout, stderr = ssh.exec_command("ps aux | grep Xvnc | wc -l")
I have to have the AutoAddPolicy() line or it fails. So what's the difference? Obviously I'm just learning this but I thought that sftp would be just as strict as ssh.

It looks like this is an acceptable practice.
Comment from Transport.connect
'''
Negotiate an SSH2 session, and optionally verify the server's host key
and authenticate using a password or private key. This is a shortcut
for L{start_client}, L{get_remote_server_key}, and
L{Transport.auth_password} or L{Transport.auth_publickey}. Use those
methods if you want more control.
You can use this method immediately after creating a Transport to
negotiate encryption with a server. If it fails, an exception will be
thrown. On success, the method will return cleanly, and an encrypted
session exists. You may immediately call L{open_channel} or
L{open_session} to get a L{Channel} object, which is used for data
transfer.
#note: If you fail to supply a password or private key, this method may
succeed, but a subsequent L{open_channel} or L{open_session} call may
fail because you haven't authenticated yet.
'''
Comment from SSHClient.connect
'''
Connect to an SSH server and authenticate to it. The server's host key
is checked against the system host keys (see L{load_system_host_keys})
and any local host keys (L{load_host_keys}). If the server's hostname
is not found in either set of host keys, the missing host key policy
is used (see L{set_missing_host_key_policy}). The default policy is
to reject the key and raise an L{SSHException}.
'''
Maybe it is due to the fact that sftp can only transport data while ssh can run terminal commands. I do find it interesting that a man-in-the-middle attack doesn't seem to be a concern.

You can use below syntax
import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection(hostname, port=port, username=user_id, password=password,
cnopts=cnopts) as sftp:
with sftp.cd(self.directory): # temporarily chdir to public
sftp.put(filepath)