I can't find anything about this exception. I am trying to rename a remote file on a local (Windows) SFTP server with fsspec. Paramiko behind the scenes is doing a posix_rename(). What does the error mean?
fs.rename(old_file_path, new_file_path)
Paths look like /folder/file.ext.
I can rename files with other FTP clients on that same server.
Indeed, fsspec SFTPFileSystem.mv calls Paramiko SFTPClient.posix_rename. That's imo a bad choice. The SFTPClient.posix_rename internally uses a proprietary OpenSSH posix-rename#openssh.com extension, which is naturally not supported by most other SFTP servers (such as yours).
I do not know what is the best solution/workaround. You can probably add your own "file system" implementation based on SFTPFileSystem, reimplementing SFTPFileSystem.mv to call standard Paramiko SFTPClient.rename (which uses standard SFTP rename request).
Actually, I just found that the SFTPClient is exposed through the SFTPFileSystem and I can call rename() on it directly, which worked!
fs.ftp.rename("testfile.txt", "x")
Related
I have an SFTP server. I can take data by transferring/downloading files. Is there a way that I can do without downloading files?
My code is as below:
# Connection to the SFTP server
with pysftp.Connection(hostname, username, passowrd, port) as sftp:
with sftp.cd('directory'):
sftp.get('filename.txt')
This code downloads file to my local machine.
Yes and no. You can use the data from the remote (SFTP) server without storing the files to a local disk.
But you cannot use data locally without downloading them. That's impossible. You have to transfer the data to use them – at least to a memory of the local machine.
See A way to load big data on Python from SFTP server, not using my hard disk.
My answer there talks about Paramiko. But pysftp is a just a thin wrapper around Paramiko. Its Connection.open is directly mapped to underlying Paramiko's SFTPClient.open. So you can keep using pysftp:
with sftp.open('filename.txt', bufsize=32768) as f:
# use f as if you have opened a local file with open()
Though I'd recommend you not to: pysftp vs. Paramiko.
So i am trying to send a certain file from my local server to another server. I am able to send the file if i know the file with exact name.
Bur what i actually want to do is pick up a file having a matching name and send the same file over.
For example my filename is filename_: test_file_20190918
i want to pick up all the file matching test_file_*
Here's what i am trying to do, but it doesn't seem to be working
import paramiko
ssh=paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='host_name',username='username',password='pwd',port=22)
sftp_client=ssh.open_sftp()
sftp_client.put("/home/mylocation/test_file_*",'/incoming/test_file_send*')
sftp_client.close()
ssh.close()
Paramiko does not support wildcards.
You have to find the right file yourself, before calling SFTPClient.put.
See Get a filtered list of files in a directory.
Obligatory warning: Do not use AutoAddPolicy, unless you do not care about security. You are losing a protection against MITM attacks this way.
For a correct solution, see Paramiko "Unknown Server".
I need to connect to a server with SSH to download files. I have Ubuntu and I've set up SSH in the standard way: I have a ssh_config file in .ssh which defines a host entry (say host_key) for the server address (Hostname.com) and username, and I've set up an RSA key. So when I try to log into SSH from the command line or bash, I just need to use ssh host_key
I would like to do this in Python. The standard solutions seems to be to use Paramiko to set up the connection. I tried this:
from paramiko import SSHClient
from scp import SCPClient
ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('host_key')
scp = SCPClient(ssh.get_transport())
# etc...
However, it always seems to hang and time out on ssh.connect('host_key'). Even when I try to include the username and password: ssh.connect('host_key', username='usrnm', password='pswd').
Are my host keys not loading properly? And would this take care of the RSA keys as well?
It only works if I use the whole Hostname.com with username and typed-out password. Which is maybe a bit insecure.
Since paramiko has a SSHConfig class, you can use it for your ~/.ssh/config.
However, it is slightly messy, I recommend you to use fabric instead of that.
Here is the code example:
from fabric.api import put
put('local path', 'remote path')
I do not think that it is common to use ssh_config file with Paramiko (or any other code/language). ssh_config is a configuration file for OpenSSH tools, not for SSH in general.
Usually, you specify your private key directly in your code as an argument of SSHClient.connect method:
How to access to a remote server using Paramiko with a public key-file
If you want to keep using ssh_config, Paramiko can parse it. Check parse_ssh_config and lookup_ssh_host_config functions. But I believe, you still have to look up the key file from the config and pass it explicitly to SSHClient.connect method.
I'm trying to send over multiple files from one server to another using Python. I've found a few ssh2 libraries, but either I can't find documentation on them (e.g. ssh), or they don't seem to support mput.
Anyone know of any sftp libraries which support mput?
Paramiko is a library that handles SSH and related things, such as SFTP, but it only supports a regular put, no mput that I can see.
What exactly does mput do? My sftp client doesn't have that command...
Guessing from the name, I'm thinking "multiple puts" or something like that, to send multiple files in one go? If that is the case, I suggest just looping over your list of files and using put.
I was faced with a similar problem a long time ago and could not get a satisfactory mput-method to run in Python. So the paramiko library seemed to make the most sense to me.
To enable progress output or other actions in your Python application, it is advantageous to send the files individually in a for-loop. However, the overhead increases minimally with this variant.
A small Paramiko-example code :
import pysftp
import os
list_files_to_transfer = []
# Check if List is empty
if list_files_to_transfer:
# Advanced connection options beyond authentication
cnopts = pysftp.CnOpts()
# Compression for lower Transfer-Load
cnopts.compression = True
cnopts.hostkeys = None
# Establish a connection with the SFTP server
with pysftp.Connection(host=host, username=username, port=port,
private_key=os.path.abspath(path_private_key),
cnopts=cnopts) as sftp:
# Change to the specified remote directory
with sftp.cd(self.path_remote_sink_folder):
# browse the list of files
for file in list_files_to_transfer:
# Upload the file to the server
sftp.put(file)
I need a method of paramiko based file transfer with a lightweight SSH2 server (dropbear) which has no support for SCP or SFTP. Is there a way of achieving a cat and redirect style file transfer, such as:
ssh server "cat remote_file" > local_file
with paramiko channels?
Can paramiko.Transport.open_channel() or Message() do the job? I am unsure of how to proceed.
If the limitation, as you say, is only in your client, you can easily implement a SFTP client directly with paramiko -- e.g., look at this example code.
pyfilesystem implements an sftp filesystem on top of paramiko.