We Keep Coding

Parakimo asking password when try to send a scp file [duplicate]

What's the most pythonic way to scp a file in Python? The only route I'm aware of is
os.system('scp "%s" "%s:%s"' % (localfile, remotehost, remotefile) )
which is a hack, and which doesn't work outside Linux-like systems, and which needs help from the Pexpect module to avoid password prompts unless you already have passwordless SSH set up to the remote host.
I'm aware of Twisted's conch, but I'd prefer to avoid implementing scp myself via low-level ssh modules.
I'm aware of paramiko, a Python module that supports SSH and SFTP; but it doesn't support SCP.
Background: I'm connecting to a router which doesn't support SFTP but does support SSH/SCP, so SFTP isn't an option.
EDIT:
This is a duplicate of How to copy a file to a remote server in Python using SCP or SSH?. However, that question doesn't give an scp-specific answer that deals with keys from within Python. I'm hoping for a way to run code kind of like
import scp
client = scp.Client(host=host, user=user, keyfile=keyfile)
# or
client = scp.Client(host=host, user=user)
client.use_system_keys()
# or
client = scp.Client(host=host, user=user, password=password)
# and then
client.transfer('/etc/local/filename', '/etc/remote/filename')

Try the Python scp module for Paramiko. It's very easy to use. See the following example:
import paramiko
from scp import SCPClient
def createSSHClient(server, port, user, password):
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(server, port, user, password)
return client
ssh = createSSHClient(server, port, user, password)
scp = SCPClient(ssh.get_transport())
Then call scp.get() or scp.put() to do SCP operations.
(SCPClient code)

You might be interested in trying Pexpect (source code). This would allow you to deal with interactive prompts for your password.
Here's a snip of example usage (for ftp) from the main website:
# This connects to the openbsd ftp site and
# downloads the recursive directory listing.
import pexpect
child = pexpect.spawn ('ftp ftp.openbsd.org')
child.expect ('Name .*: ')
child.sendline ('anonymous')
child.expect ('Password:')
child.sendline ('noah#example.com')
child.expect ('ftp> ')
child.sendline ('cd pub')
child.expect('ftp> ')
child.sendline ('get ls-lR.gz')
child.expect('ftp> ')
child.sendline ('bye')

Couldn't find a straight answer, and this "scp.Client" module doesn't exist.
Instead, this suits me:
from paramiko import SSHClient
from scp import SCPClient
ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('example.com')
with SCPClient(ssh.get_transport()) as scp:
scp.put('test.txt', 'test2.txt')
scp.get('test2.txt')

You could also check out paramiko. There's no scp module (yet), but it fully supports sftp.
[EDIT]
Sorry, missed the line where you mentioned paramiko.
The following module is simply an implementation of the scp protocol for paramiko.
If you don't want to use paramiko or conch (the only ssh implementations I know of for python), you could rework this to run over a regular ssh session using pipes.
scp.py for paramiko

import paramiko
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect('<IP Address>', username='<User Name>',password='' ,key_filename='<.PEM File path')
#Setup sftp connection and transmit this script
print ("copying")
sftp = client.open_sftp()
sftp.put(<Source>, <Destination>)
sftp.close()

if you install putty on win32 you get an pscp (putty scp).
so you can use the os.system hack on win32 too.
(and you can use the putty-agent for key-managment)
sorry it is only a hack
(but you can wrap it in a python class)

As of today, the best solution is probably AsyncSSH
https://asyncssh.readthedocs.io/en/latest/#scp-client
async with asyncssh.connect('host.tld') as conn:
await asyncssh.scp((conn, 'example.txt'), '.', recurse=True)

You can use the package subprocess and the command call to use the scp command from the shell.
from subprocess import call
cmd = "scp user1#host1:files user2#host2:files"
call(cmd.split(" "))

Have a look at fabric.transfer.
from fabric import Connection
with Connection(host="hostname",
user="admin",
connect_kwargs={"key_filename": "/home/myuser/.ssh/private.key"}
) as c:
c.get('/foo/bar/file.txt', '/tmp/')

It has been quite a while since this question was asked, and in the meantime, another library that can handle this has cropped up:
You can use the copy function included in the Plumbum library:
import plumbum
r = plumbum.machines.SshMachine("example.net")
# this will use your ssh config as `ssh` from shell
# depending on your config, you might also need additional
# params, eg: `user="username", keyfile=".ssh/some_key"`
fro = plumbum.local.path("some_file")
to = r.path("/path/to/destination/")
plumbum.path.utils.copy(fro, to)

If you are on *nix you can use sshpass
sshpass -p password scp -o User=username -o StrictHostKeyChecking=no src dst:/path

Hmmm, perhaps another option would be to use something like sshfs (there an sshfs for Mac too). Once your router is mounted you can just copy the files outright. I'm not sure if that works for your particular application but it's a nice solution to keep handy.

I while ago I put together a python SCP copy script that depends on paramiko. It includes code to handle connections with a private key or SSH key agent with a fallback to password authentication.
http://code.activestate.com/recipes/576810-copy-files-over-ssh-using-paramiko/

Download file from SFTP server in Python Paramiko

I am using Python Paramiko to retrieve/search file from an SFTP server. I get all file in the directory.
What I need is specific file from that directory. How do I get it?

Use Paramiko SFTPClient.get to download a single file:
with paramiko.SSHClient() as ssh:
ssh.connect(host, username=username, password=password)
with ssh.open_sftp() as sftp:\
sftp.get("/remote/path/file.txt", "/local/path/file.txt")
You will also have to deal with the server's host key verification.

What you need to do is create an ssh client, then execute an ls with piped grep to find your file. such as ls /srv/ftp | grep '^FTP_' to find files in the /srv/ftp directory and start with FTP. Then open an sftp connection and execute the get command to bring the files over.
EDIT: Martin below mentioned that there is a better way to get the directory contents using SFTPClient.listdir() - I have revised to that method. More info in the docs: https://docs.paramiko.org/en/stable/api/sftp.html
Putting all that together looks like
import paramiko
host = ''
port = 22
username = ''
password = ''
with paramiko.SSHClient() as client:
client.connect(host, port, username, password)
with client.open_sftp() as sftp:
files = sftp.listdir('/srv/ftp')
for i, file in enumerate(files):
if file and file.startswith('FTP'):
sftp.get(f'/srv/ftp/{file}', f'~/ftp/{file}')
print(f'Moved {file}')
This code is untested but should work. Hope that's clear.

Here is an answer in case you need a kind of find using a SFTP connection, not knowing the exact path and name of the file. If it is not what you were looking for, I am sorry.
I made a library named sftputil, based on paramiko, which implements advanced functionalities such as glob. To find a specific file and download it, you can do it this way :
from sftputil import SFTP
sftp = SFTP("hostname", "username", password="password")
# Here we look for a file with a name starting with `foo`
found_files = sftp.glob("directory/foo*")
# Here we look for the file `bar` in any subdirectory
found_files = sftp.glob("directory/*/bar")
# But you can use other patterns of course.
# And now the files can be downloaded
for f in found_files:
sftp.get(f, "my/local/path")
If you don’t know glob, you should read the python documentation, as this implementation works the same way.

I recently solved a similar issue of connecting to SFTP and downloading files via paramiko.
The code below assumes that you know the name of the file you are looking for (file_name). If there are multiple files you need, just add them to file_list.
import paramiko
host = 'host.name.com'
usr = 'username'
pwd = 'password'
remote_path = '/path/on/sftp/server/'
local_path = '/path/on/local/machine/'
file_name = 'specific_file_you_need.csv'
transport = paramiko.Transport((host,22))
transport.connect(None,usr,pwd)
sftp = paramiko.SFTPClient.from_transport(transport)
file_list = [file_name]
for file in file_list:
sftp.get(remote_path+'{0}'.format(file),
local_path+'{0}'.format(file))

Upload file via SFTP with Python

I wrote a simple code to upload a file to a SFTP server in Python. I am using Python 2.7.
import pysftp
srv = pysftp.Connection(host="www.destination.com", username="root",
password="password",log="./temp/pysftp.log")
srv.cd('public') #chdir to public
srv.put('C:\Users\XXX\Dropbox\test.txt') #upload file to nodejs/
# Closes the connection
srv.close()
The file did not appear on the server. However, no error message appeared. What is wrong with the code?
I have enabled logging. I discovered that the file is uploaded to the root folder and not under public folder. Seems like srv.cd('public') did not work.

I found the answer to my own question.
import pysftp
srv = pysftp.Connection(host="www.destination.com", username="root",
password="password",log="./temp/pysftp.log")
with srv.cd('public'): #chdir to public
srv.put('C:\Users\XXX\Dropbox\test.txt') #upload file to nodejs/
# Closes the connection
srv.close()
Put the srv.put inside with srv.cd

Do not use pysftp it's dead. Use Paramiko directly. See also pysftp vs. Paramiko.
The code with Paramiko will be pretty much the same, except for the initialization part.
import paramiko
with paramiko.SSHClient() as ssh:
ssh.load_system_host_keys()
ssh.connect(host, username=username, password=password)
sftp = ssh.open_sftp()
sftp.chdir('public')
sftp.put('C:\Users\XXX\Dropbox\test.txt', 'test.txt')
To answer the literal OP's question: the key point here is that pysftp Connection.cd works as a context manager (so its effect is discarded without with statement), while Paramiko SFTPClient.chdir does not.

import pysftp
with pysftp.Connection(host="www.destination.com", username="root",
password="password",log="./temp/pysftp.log") as sftp:
sftp.cwd('/root/public') # The full path
sftp.put('C:\Users\XXX\Dropbox\test.txt') # Upload the file
No sftp.close() is needed, because the connection is closed automatically at the end of the with-block
I did a minor change with cd to cwd
Syntax -
# sftp.put('/my/local/filename') # upload file to public/ on remote
# sftp.get('remote_file') # get a remote file

How to Transfer Files from Client to Server Computer by using python script?

I am writing a python script to copy python(say ABC.py) files from one directory to another
directory with the same folder name(say ABC) as script name excluding .py.
In the local system it works fine and copying the files from one directory to others by
creating the same name folder.
But actually I want copy these files from my local system (windows XP) to the remote
system(Linux) located in other country on which I execute my script. But I am getting
the error as "Destination Path not found" means I am not able to connect to remote
that's why.
I use SSH Secure client.
I use an IP Address and Port number to connect to the remote server.
Then it asks for user id and password.
But I am not able to connect to the remote server by my python script.
Can Any one help me out how can I do this??

paramiko provides a SFTPClient that can be used to do this.
import paramiko
source = r'C:\Somedir\somefile.txt'
dest = r'/home/user/file.txt'
hostname = 'linux.server.com'
port = 22 # default port for SSH
username = 'user'
password = 'secret'
try:
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(t)
sftp.put(source, dest)
finally:
t.close()

I used the same script, but my host failed to respond. My host is in different network.
WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

Paramiko's SSHClient with SFTP

How I can make SFTP transport through SSHClient on the remote server? I have a local host and two remote hosts. Remote hosts are backup server and web server. I need to find on backup server necessary backup file and put it on web server over SFTP. How can I make Paramiko's SFTP transport work with Paramiko's SSHClient?

paramiko.SFTPClient
Sample Usage:
import paramiko
paramiko.util.log_to_file("paramiko.log")
# Open a transport
host,port = "example.com",22
transport = paramiko.Transport((host,port))
# Auth
username,password = "bar","foo"
transport.connect(None,username,password)
# Go!
sftp = paramiko.SFTPClient.from_transport(transport)
# Download
filepath = "/etc/passwd"
localpath = "/home/remotepasswd"
sftp.get(filepath,localpath)
# Upload
filepath = "/home/foo.jpg"
localpath = "/home/pony.jpg"
sftp.put(localpath,filepath)
# Close
if sftp: sftp.close()
if transport: transport.close()

The accepted answer "works". But with its use of the low-level Transport class, it bypasses a host key verification, what is a security flaw, as it makes the code susceptible to Man-in-the-middle attacks.
Better is to use the right Paramiko SSH API, the SSHClient, which does verify the host key:
import paramiko
paramiko.util.log_to_file("paramiko.log")
ssh = paramiko.SSHClient()
ssh.connect(host, username='user', password='password')
# or
# key = paramiko.RSAKey.from_private_key_file('id_rsa')
# ssh.connect(host, username='user', pkey=key)
sftp = ssh.open_sftp()
sftp.get(remotepath, localpath)
# or
sftp.put(localpath, remotepath)
For details about verifying the host key, see:
Paramiko "Unknown Server"

If you have a SSHClient, you can also use open_sftp():
import paramiko
# lets say you have SSH client...
client = paramiko.SSHClient()
sftp = client.open_sftp()
# then you can use upload & download as shown above
...

In addition to the first answer which is great but depends on username/password, the following shows how to use an ssh key:
from paramiko import Transport, SFTPClient, RSAKey
key = RSAKey(filename='path_to_my_rsakey')
con = Transport('remote_host_name_or_ip', 22)
con.connect(None,username='my_username', pkey=key)
sftp = SFTPClient.from_transport(con)
sftp.listdir(path='.')

For those anyone need to integrate with an ssh/sftp server that requires a private key and want to perform host key verification for the known host by using a specific public key, here is a snippet code with paramiko:
import paramiko
sftp_hostname = "target.hostname.com"
sftp_username = "tartgetHostUsername"
sftp_private_key = "/path/to/private_key_file.pvt"
sftp_private_key_password = "private_key_file_passphrase_if_it_encrypted"
sftp_public_key = "/path/to/public_certified_file.pub"
sftp_port = 22
remote_path = "."
target_local_path = "/path/to/target/folder"
ssh = paramiko.SSHClient()
# Load target host public cert for host key verification
ssh.load_host_keys(sftp_public_key)
# Load encrypted private key and ssh connect
key = paramiko.RSAKey.from_private_key_file(sftp_private_key, sftp_private_key_password)
ssh.connect(host=sftp_hostname, port=sftp_port, username=sftp_username, pkey=key)
# Get the sftp connection
sftp_connection = ssh.open_sftp()
directory_list = sftp_connection.listdir(remote_path)
# ...
if sftp_connection: sftp_connection.close()
if ssh: ssh.close()
Notice that only certificates in classic Openssh format are supported, otherwise needs to be converted with the following commands (also for the latest Openssh formats):
$chmod 400 /path/to/private_key_file.pvt
$ssh-keygen -p -f /path/to/private_key_file.pvt -m pem -P <currentPassphrase> -N <newPassphrase>
In order to avoid man in the middle attack, it is important to do not use paramiko.AutoAddPolicy() and load the public host key programmatically as above or load it from ~/.ssh/known_hosts
The file must be in the format "<host_name> ssh-rsa AAAAB3NzaC1yc2EAAAA..."
In case you don't have the public key and you trust the target host (take care to mitm), you can download it using $ssh-keyscan target.hostname.com command.
The above code is the only way I found to avoid the following error during connection:
paramiko.ssh_exception.SSHException: Server 'x.y.z' not found in known_hosts
This error was prompted also with the following way to load the public certificates:
key = paramiko.RSAKey(data=decodebytes(sftp_public_key))
ssh_client.get_host_keys().add(sftp_hostname, 'ssh-rsa', key)
Also the following code was not able for me to load the certificate (tried also by encoding the certificate in base64):
ssh_client=paramiko.SSHClient()
rsa_key = paramiko.RSAKey.from_private_key_file(sftp_private_key, sftp_private_key_password)
rsa_key.load_certificate(sftp_public_key)
It always ends with:
File "/usr/local/lib/python3.9/site-packages/paramiko/pkey.py", line 720, in from_string
key_blob = decodebytes(b(fields[1]))
File "/usr/lib64/python3.9/base64.py", line 538, in decodebytes
return binascii.a2b_base64(s)
binascii.Error: Incorrect padding
The above code above worked for the SFTP integration with GoAnywhere.
I hope this is helpful, I've not found any working example and spent many hours in searches and tests.
The implementations using pysftp wrapper it is now to be considered as discontinued from 2016.