I am using Python Paramiko to retrieve/search file from an SFTP server. I get all file in the directory.
What I need is specific file from that directory. How do I get it?
Use Paramiko SFTPClient.get to download a single file:
with paramiko.SSHClient() as ssh:
ssh.connect(host, username=username, password=password)
with ssh.open_sftp() as sftp:\
sftp.get("/remote/path/file.txt", "/local/path/file.txt")
You will also have to deal with the server's host key verification.
What you need to do is create an ssh client, then execute an ls with piped grep to find your file. such as ls /srv/ftp | grep '^FTP_' to find files in the /srv/ftp directory and start with FTP. Then open an sftp connection and execute the get command to bring the files over.
EDIT: Martin below mentioned that there is a better way to get the directory contents using SFTPClient.listdir() - I have revised to that method. More info in the docs: https://docs.paramiko.org/en/stable/api/sftp.html
Putting all that together looks like
import paramiko
host = ''
port = 22
username = ''
password = ''
with paramiko.SSHClient() as client:
client.connect(host, port, username, password)
with client.open_sftp() as sftp:
files = sftp.listdir('/srv/ftp')
for i, file in enumerate(files):
if file and file.startswith('FTP'):
sftp.get(f'/srv/ftp/{file}', f'~/ftp/{file}')
print(f'Moved {file}')
This code is untested but should work. Hope that's clear.
Here is an answer in case you need a kind of find using a SFTP connection, not knowing the exact path and name of the file. If it is not what you were looking for, I am sorry.
I made a library named sftputil, based on paramiko, which implements advanced functionalities such as glob. To find a specific file and download it, you can do it this way :
from sftputil import SFTP
sftp = SFTP("hostname", "username", password="password")
# Here we look for a file with a name starting with `foo`
found_files = sftp.glob("directory/foo*")
# Here we look for the file `bar` in any subdirectory
found_files = sftp.glob("directory/*/bar")
# But you can use other patterns of course.
# And now the files can be downloaded
for f in found_files:
sftp.get(f, "my/local/path")
If you don’t know glob, you should read the python documentation, as this implementation works the same way.
I recently solved a similar issue of connecting to SFTP and downloading files via paramiko.
The code below assumes that you know the name of the file you are looking for (file_name). If there are multiple files you need, just add them to file_list.
import paramiko
host = 'host.name.com'
usr = 'username'
pwd = 'password'
remote_path = '/path/on/sftp/server/'
local_path = '/path/on/local/machine/'
file_name = 'specific_file_you_need.csv'
transport = paramiko.Transport((host,22))
transport.connect(None,usr,pwd)
sftp = paramiko.SFTPClient.from_transport(transport)
file_list = [file_name]
for file in file_list:
sftp.get(remote_path+'{0}'.format(file),
local_path+'{0}'.format(file))
Related
I am trying to move the contents from source directory C:\report to the remote directory remote_server_path/Test. Below is the code that I am trying with. Instead of just moving the contents of C:\report, it moves along the folder as such to the remote location. Any recommendations on how this could not be done?
import paramiko
from scp import SCPClient
import os
# create variables
host = "host"
username = "uname"
password = "password"
# Move files from network drive to marketing server
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname=host, username=username, password=password)
scp = SCPClient(ssh.get_transport())
scp.put('C:\\report', recursive=True, remote_path='remote_server_path/Test')
Use C:\report\* to select the files within the folder.
scp.put('C:\\report\\*', recursive=True, remote_path='remote_server_path/Test/')
Obligatory warning: Do not use AutoAddPolicy – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".
I am using the below Python code to upload a file via SFTP using Paramiko. The connection "seems" to be fine, the code executes to the end, just the file isn't reaching the destination when I check in FileZilla.
I have checked and set permissions on the file to 777 (just to be sure). I have also checked my file path string in a separate terminal and the path is valid.
import paramiko
.
.
transport = paramiko.Transport((host, port))
transport.connect(username = username, password = password)
sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put(filePath, "/") # Upload file to root FTP folder
sftp.close()
transport.close()
What can I do to debug this? Anything I can print out, check connection succeeded etc?
The second argument of SFTPClient.put (remotepath) is path to a file, not a folder:
the destination path on the SFTP server. Note that the filename should be included. Only specifying a directory may result in an error.
Try this:
sftp.put(filePath, "/filename")
import paramiko
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect('hostname', username='test1234', password='test')
path = ['/home/test/*.txt', '/home/test1/*.file', '/home/check/*.xml']
for i in path:
for j in glob.glob(i):
print j
client.close()
I am trying to list the wildcard files on remote server by using glob.glob. But glob.glob() is not working.
Using Python 2.6.
Remote server contains these files: /home/test1/check.file, /home/test1/validate.file, /home/test1/vali.file
Can anyone please help on this issue.
The glob will not magically start working with a remote server, just because you have instantiated SSHClient before.
You have to use Paramiko API to list the files, like SFTPClient.listdir:
import fnmatch
sftp = client.open_sftp()
for filename in sftp.listdir('/home/test'):
if fnmatch.fnmatch(filename, "*.txt"):
print filename
You can also use a regular expression for the matching, if it suits your needs better. See Using wildcard in remote path using Paramiko's SFTPClient.
Side note: Do not use AutoAddPolicy. You
lose security by doing so. See Paramiko "Unknown Server".
Or use pysftp which is paramiko wrapper and write something like this:
import pysftp
def store_files_name(fname):
pass
def store_dir_name(dir_name):
pass
def store_other_file_type(other_file):
pass
with pysftp.Connection('server', username='user', password='pass') as sftp:
sftp.walktree('.', store_files_name, store_dir_name, store_other_file_type)
I wrote a simple code to upload a file to a SFTP server in Python. I am using Python 2.7.
import pysftp
srv = pysftp.Connection(host="www.destination.com", username="root",
password="password",log="./temp/pysftp.log")
srv.cd('public') #chdir to public
srv.put('C:\Users\XXX\Dropbox\test.txt') #upload file to nodejs/
# Closes the connection
srv.close()
The file did not appear on the server. However, no error message appeared. What is wrong with the code?
I have enabled logging. I discovered that the file is uploaded to the root folder and not under public folder. Seems like srv.cd('public') did not work.
I found the answer to my own question.
import pysftp
srv = pysftp.Connection(host="www.destination.com", username="root",
password="password",log="./temp/pysftp.log")
with srv.cd('public'): #chdir to public
srv.put('C:\Users\XXX\Dropbox\test.txt') #upload file to nodejs/
# Closes the connection
srv.close()
Put the srv.put inside with srv.cd
Do not use pysftp it's dead. Use Paramiko directly. See also pysftp vs. Paramiko.
The code with Paramiko will be pretty much the same, except for the initialization part.
import paramiko
with paramiko.SSHClient() as ssh:
ssh.load_system_host_keys()
ssh.connect(host, username=username, password=password)
sftp = ssh.open_sftp()
sftp.chdir('public')
sftp.put('C:\Users\XXX\Dropbox\test.txt', 'test.txt')
To answer the literal OP's question: the key point here is that pysftp Connection.cd works as a context manager (so its effect is discarded without with statement), while Paramiko SFTPClient.chdir does not.
import pysftp
with pysftp.Connection(host="www.destination.com", username="root",
password="password",log="./temp/pysftp.log") as sftp:
sftp.cwd('/root/public') # The full path
sftp.put('C:\Users\XXX\Dropbox\test.txt') # Upload the file
No sftp.close() is needed, because the connection is closed automatically at the end of the with-block
I did a minor change with cd to cwd
Syntax -
# sftp.put('/my/local/filename') # upload file to public/ on remote
# sftp.get('remote_file') # get a remote file
Suppose I made a file using Paramiko module in CentOS and accessing it from a Python script in Windows:
stdin,stdout,stderr=ssh.exec_command("touch Hello")
Then how can I write something and save the file?
print "hello,world"
Please help me I am stuck here.
Use the SFTP to upload the file contents. Do not try to hack this with shell commands.
transport = paramiko.Transport(("host", 22))
transport.connect(username = "username", password = "password")
sftp = paramiko.SFTPClient.from_transport(transport)
f = sftp.open("/path/to/remote/file", "wb")
f.write("hello,world")
f.close()