I would like to read files on a remote server using pysftp. For this I generated an ssh key on the server I want to connect to and added the public key to my code.
It worked fine before I changed my VM (debian 10). Now I have as error :
"SSHException: Bad host key from server"
If someone has an idea it would help me a lot !
Here is my code :
from base64 import decodebytes
import os
import pysftp
import paramiko
host = os.getenv('host')
username = os.getenv('username')
password = os.getenv('password')
#public key
keydata = bytes(os.getenv('key'),'UTF-8')
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add(host,'ssh-rsa', key)
with pysftp.Connection(host=host, username=username, password=password, cnopts=cnopts, port=22) as sftp:
sftp.cwd('/home/user/')
for attr in sftp.listdir():
print(attr)
You can debug this in following ways:
Check if your key have any white spaces at the beginning or at the end.
Then ssh to that server with your key directly from
terminal make sure you use -v option to verbose logs.
Check /etc/ssh/sshd_config if sftp is allowed or not (good approach is to have diffrent sftp user for secure communication).
Thanks
Related
Everyday I am uploading files to SFTP server using WinSCP.
I decided to use Python to automate this process.
This SFTP has host, username, port, and .ppk file
I found a code, but it didn't work. It says
SSHException: not a valid DSA private key file
import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
HOST = 'HOST'
USERNAME = 'USERNAME'
PRIVATE_KEY = 'file.ppk'
PORT = 1111
srv = pysftp.Connection(host = HOST,
username = USERNAME,
private_key = PRIVATE_KEY,
port = PORT
cnopts = cnopts)
data = srv.listdir()
srv.close()
for i in data:
print(i)
What should I do? I know how to upload files etc I just would like to know how to connect the SFTP.
Here is how .ppk file looks like.
PuTTY-User-Key-File-3: ssh-rsa
Encryption: none
Comment: rsa-key-20220408
Public-Lines: 6
AAAAB3NzaC1yc2EAAAADAQABAAABAQCKXdZXgtuaGfRayz7cz6bFkXWhVmbmGebJ
XFND5bIz73/cemi8TPiU7yLt7HiZNh189kOCRYrz51zX3JDi7s8tQp4pa2LsJ2jn
YeTTAXS2HzxL0aDWzom22A7FaHVAHTO48o9MzjdEek+0Mi2taZMy7hKQqCJUAl9U
5zQrbRvczyCRlp0N824lUfkCXvOs4ib92YzEhOgfnG7aH9rLdft5T2CyBMmT2c4P
bybKp2m8tl17ClAzQUQC6JfxTO3Kd0Zvw1AQmD8pqkr6pbp3a+pWOd/PLG+8NniJ
0ipbFhaq0ptn3iCaMbUySF1R1J3CIlrTM51fhH/knTwKV9A3zscD
Private-Lines: 14
...
Private-MAC: 16d619f3cca3b69ea882fc95b7f5b3a153aeb455af4e9cfc56a5a85bf6fce2b3
For your literal problem, see:
How to ssh connect through Python Paramiko with ppk public key
It's about Paramiko. But pysftp is just a thin wrapper around Paramiko. And you should not use pysftp anyway, it's dead. See pysftp vs. Paramiko.
Though note, that as you have WinSCP working, you can automate your task using WinSCP scripting.
WinSCP GUI can even generate a working script template for you.
So I am having many issues connecting to a remote server via SFTP. I have tried the normal way like below.
sftp = pysftp.Connection(host='Host',username='username',password='passwd',private_key=".ppk")
Which did not work. I got the following error:
SSHException: No hostkey for host ***** found.
I then tried the following:
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
s = pysftp.Connection(host='host', username='user', password='password', cnopts=cnopts)
Which also did not work. I got the following error:
BadAuthenticationType: ('Bad authentication type', ['publickey']) (allowed_types=['publickey'])
Also when I run the following:
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect("host",username = "username",password = "password")
ssh_session = client.get_transport().open_session()
I get the same error:
BadAuthenticationType: ('Bad authentication type', ['publickey']) (allowed_types=['publickey'])
Your are confusing a private key used for authentication and a host key used to verify an identify of a server. Both need to be taken care of, while all your code attempts take care of one of them only. See my article on SSH key pairs to understand the difference between the two kinds of keys involved in SSH.
So this should "work":
# Accept any host key (still wrong see below)
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
# And authenticate with a private key
sftp = pysftp.Connection(
host='Host', username='user', password='passwd', private_key=".ppk",
cnopts=cnopts)
But this code will actually blindly accept any host key (cnopts.hostkeys = None), what is a security flaw. For a correct approach, see Verify host key with pysftp.
It looks like the host you are connecting is not available. This usually happens when the host-name is not accessible because of firewall rules (or typo on host).
I'd recommend first checking if you can sftp from the (unix) terminal
> sftp username#host
If you get prompted for password or get logged in, you are able to connect to that host from that machine
If not try checking if that host is available using netcat on port 22, you'd get timeout or broken pipe if host is not available
>nc -v host 22
I recommend debugging the pysftp or paramiko packages only after that.
Also, if you are authenticating using a private key, you do not need to use the password.
I found a lot of similar questions here in SO, but any of them helped me, even this problem seems to be simple.
I'm trying to connect to a remote Google Compute Engine instance through SSH (I want to use SFTP) using paramiko. Below is my code:
client = paramiko.SSHClient()
client.load_system_host_keys()
client.connect('External_IP_Get_In_GCE_panel', username='myuser')
stdin, stdout, stderr = client.exec_command('ls')
for line in stdout:
print('... ' + line.strip('\n'))
client.close()
With this code, I have the error
PasswordRequiredException: Private key file is encrypted
When I tried client.connect('External_IP_Get_In_GCE_panel', username='myuser', password='') the error is:
BadAuthenticationType: ('Bad authentication type', [u'publickey']) (allowed_types=[u'publickey'])
My SSH key to access Google Compute engine has no password. I can use the gcloud compute ssh instance-name and also access SFTP throug Filezilla without problems.
As I said, I tried a lot of alternatives found here in SO, but any of them helped me. Below are 3 other versions of the code:
Using key
key = paramiko.RSAKey(data=base64.b64decode(b"""AAAAB3Nza..."""))
client = paramiko.SSHClient()
client.get_host_keys().add('External_IP_Get_In_GCE_panel', 'ssh-rsa', key)
client.connect('External_IP_Get_In_GCE_panel', username='myuser', password='')
# I got the key using ssh-keyscan `External_IP_Get_In_GCE_panel`in my local machine
Using another lib
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add('External_IP_Get_In_GCE_panel', 'ssh-rsa', key)
with pysftp.Connection('External_IP_Get_In_GCE_panel', username='myuser', cnopts=cnopts) as sftp:
with sftp.cd('../directory'):
sftp.get('remote_file')'''
Using the ssh key file
self.client = paramiko.SSHClient()
self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.client.connect(hostname="External_IP_Get_In_GCE_panel", username="myuser", key_filename=os.path.expanduser('~/.ssh/ssh_key_file'), password='') # also tried ssh_key_file.pub
In all this versions (and some others) I tried to use password='', password=None and not send password argument. The results is always the same erros above.
Any tip about what I'm doing wrong?
The key is encrypted, you need a password (and presumably non-empty so) to decrypt the private key, i.e.
key = paramiko.RSAKey(data=base64.b64decode(b"""AAAAB3Nza..."""), password='my key password')
The server there only allows public key authentication, so giving password to client.connect does not make any sense.
I was running the demo_sftp.py file from the demo folder in the paramiko github. I was stepping through it in PyDev and expected to get an error because I didn't have a key to the server I was trying to connect to but I got the print statement saying that the script couldn't open the host key file and then it went ahead and did the get and put.
Here's a code snippet.
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
except IOError:
try:
# try ~/ssh/ too, because windows can't have a folder named ~/.ssh/
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/ssh/known_hosts'))
except IOError:
print '*** Unable to open host keys file'
host_keys = {}
if host_keys.has_key(hostname):
hostkeytype = host_keys[hostname].keys()[0]
hostkey = host_keys[hostname][hostkeytype]
print 'Using host key of type %s' % hostkeytype
# now, connect and use paramiko Transport to negotiate SSH2 across the connection
try:
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password, hostkey=hostkey)
sftp = paramiko.SFTPClient.from_transport(t)
# dirlist on remote host
dirlist = sftp.listdir('.')
print "Dirlist:", dirlist
I really expected it to go to the except on the t.connect line because hostkey is NoneType.
When I open an ssh connection with
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
ssh.connect('.'.join([self.name, self.domain]),
username=self.username, password=self.password)
stdin, stdout, stderr = ssh.exec_command("ps aux | grep Xvnc | wc -l")
I have to have the AutoAddPolicy() line or it fails. So what's the difference? Obviously I'm just learning this but I thought that sftp would be just as strict as ssh.
It looks like this is an acceptable practice.
Comment from Transport.connect
'''
Negotiate an SSH2 session, and optionally verify the server's host key
and authenticate using a password or private key. This is a shortcut
for L{start_client}, L{get_remote_server_key}, and
L{Transport.auth_password} or L{Transport.auth_publickey}. Use those
methods if you want more control.
You can use this method immediately after creating a Transport to
negotiate encryption with a server. If it fails, an exception will be
thrown. On success, the method will return cleanly, and an encrypted
session exists. You may immediately call L{open_channel} or
L{open_session} to get a L{Channel} object, which is used for data
transfer.
#note: If you fail to supply a password or private key, this method may
succeed, but a subsequent L{open_channel} or L{open_session} call may
fail because you haven't authenticated yet.
'''
Comment from SSHClient.connect
'''
Connect to an SSH server and authenticate to it. The server's host key
is checked against the system host keys (see L{load_system_host_keys})
and any local host keys (L{load_host_keys}). If the server's hostname
is not found in either set of host keys, the missing host key policy
is used (see L{set_missing_host_key_policy}). The default policy is
to reject the key and raise an L{SSHException}.
'''
Maybe it is due to the fact that sftp can only transport data while ssh can run terminal commands. I do find it interesting that a man-in-the-middle attack doesn't seem to be a concern.
You can use below syntax
import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection(hostname, port=port, username=user_id, password=password,
cnopts=cnopts) as sftp:
with sftp.cd(self.directory): # temporarily chdir to public
sftp.put(filepath)
How I can make SFTP transport through SSHClient on the remote server? I have a local host and two remote hosts. Remote hosts are backup server and web server. I need to find on backup server necessary backup file and put it on web server over SFTP. How can I make Paramiko's SFTP transport work with Paramiko's SSHClient?
paramiko.SFTPClient
Sample Usage:
import paramiko
paramiko.util.log_to_file("paramiko.log")
# Open a transport
host,port = "example.com",22
transport = paramiko.Transport((host,port))
# Auth
username,password = "bar","foo"
transport.connect(None,username,password)
# Go!
sftp = paramiko.SFTPClient.from_transport(transport)
# Download
filepath = "/etc/passwd"
localpath = "/home/remotepasswd"
sftp.get(filepath,localpath)
# Upload
filepath = "/home/foo.jpg"
localpath = "/home/pony.jpg"
sftp.put(localpath,filepath)
# Close
if sftp: sftp.close()
if transport: transport.close()
The accepted answer "works". But with its use of the low-level Transport class, it bypasses a host key verification, what is a security flaw, as it makes the code susceptible to Man-in-the-middle attacks.
Better is to use the right Paramiko SSH API, the SSHClient, which does verify the host key:
import paramiko
paramiko.util.log_to_file("paramiko.log")
ssh = paramiko.SSHClient()
ssh.connect(host, username='user', password='password')
# or
# key = paramiko.RSAKey.from_private_key_file('id_rsa')
# ssh.connect(host, username='user', pkey=key)
sftp = ssh.open_sftp()
sftp.get(remotepath, localpath)
# or
sftp.put(localpath, remotepath)
For details about verifying the host key, see:
Paramiko "Unknown Server"
If you have a SSHClient, you can also use open_sftp():
import paramiko
# lets say you have SSH client...
client = paramiko.SSHClient()
sftp = client.open_sftp()
# then you can use upload & download as shown above
...
In addition to the first answer which is great but depends on username/password, the following shows how to use an ssh key:
from paramiko import Transport, SFTPClient, RSAKey
key = RSAKey(filename='path_to_my_rsakey')
con = Transport('remote_host_name_or_ip', 22)
con.connect(None,username='my_username', pkey=key)
sftp = SFTPClient.from_transport(con)
sftp.listdir(path='.')
For those anyone need to integrate with an ssh/sftp server that requires a private key and want to perform host key verification for the known host by using a specific public key, here is a snippet code with paramiko:
import paramiko
sftp_hostname = "target.hostname.com"
sftp_username = "tartgetHostUsername"
sftp_private_key = "/path/to/private_key_file.pvt"
sftp_private_key_password = "private_key_file_passphrase_if_it_encrypted"
sftp_public_key = "/path/to/public_certified_file.pub"
sftp_port = 22
remote_path = "."
target_local_path = "/path/to/target/folder"
ssh = paramiko.SSHClient()
# Load target host public cert for host key verification
ssh.load_host_keys(sftp_public_key)
# Load encrypted private key and ssh connect
key = paramiko.RSAKey.from_private_key_file(sftp_private_key, sftp_private_key_password)
ssh.connect(host=sftp_hostname, port=sftp_port, username=sftp_username, pkey=key)
# Get the sftp connection
sftp_connection = ssh.open_sftp()
directory_list = sftp_connection.listdir(remote_path)
# ...
if sftp_connection: sftp_connection.close()
if ssh: ssh.close()
Notice that only certificates in classic Openssh format are supported, otherwise needs to be converted with the following commands (also for the latest Openssh formats):
$chmod 400 /path/to/private_key_file.pvt
$ssh-keygen -p -f /path/to/private_key_file.pvt -m pem -P <currentPassphrase> -N <newPassphrase>
In order to avoid man in the middle attack, it is important to do not use paramiko.AutoAddPolicy() and load the public host key programmatically as above or load it from ~/.ssh/known_hosts
The file must be in the format "<host_name> ssh-rsa AAAAB3NzaC1yc2EAAAA..."
In case you don't have the public key and you trust the target host (take care to mitm), you can download it using $ssh-keyscan target.hostname.com command.
The above code is the only way I found to avoid the following error during connection:
paramiko.ssh_exception.SSHException: Server 'x.y.z' not found in known_hosts
This error was prompted also with the following way to load the public certificates:
key = paramiko.RSAKey(data=decodebytes(sftp_public_key))
ssh_client.get_host_keys().add(sftp_hostname, 'ssh-rsa', key)
Also the following code was not able for me to load the certificate (tried also by encoding the certificate in base64):
ssh_client=paramiko.SSHClient()
rsa_key = paramiko.RSAKey.from_private_key_file(sftp_private_key, sftp_private_key_password)
rsa_key.load_certificate(sftp_public_key)
It always ends with:
File "/usr/local/lib/python3.9/site-packages/paramiko/pkey.py", line 720, in from_string
key_blob = decodebytes(b(fields[1]))
File "/usr/lib64/python3.9/base64.py", line 538, in decodebytes
return binascii.a2b_base64(s)
binascii.Error: Incorrect padding
The above code above worked for the SFTP integration with GoAnywhere.
I hope this is helpful, I've not found any working example and spent many hours in searches and tests.
The implementations using pysftp wrapper it is now to be considered as discontinued from 2016.