I'm trying to connect to SFTP server that is hosted locally (that's a POC ...). I'm using python 3.9 and the library paramiko.
The code I use is the following :
import paramiko
import sys
def main() -> None:
hostkeys = paramiko.hostkeys.HostKeys("known_hosts.txt")
ip = "127.0.0.1"
username = "my_username"
password = "my_password"
# As per the paramiko doc. The host fingerprint is stored using the ed25519 algorithm
hostFingerprint = hostkeys.lookup(ip)["ssh-ed25519"]
try:
tp = paramiko.Transport(ip, 22)
tp.connect(username=username, password=password, hostkey=hostFingerprint)
# Upload operations happening here...
tp.close()
except Exception as e:
print(e)
finally:
return None
The file known_hsts.txt has been generated via Powershell cmd : ssh-keyscan 127.0.0.1 > C:\Users\my_username\PycharmProjects\POC\known_hosts.txt.
When i run the script i don't understand why hostkeys.keys() == []
When i look at the file known_hosts.txt I see the different keys : ssh-rsa, ssh-ed25519, etc.
Related
I am trying to replicate: ssh -i [KEY] -L [FROM_PORT]:localhost:[TO_PORT] ubuntu#[REMOTE_SERVER_IP] in python and decided to use sshtunnel for it. The command given above works and I can connect to the remote Theia IDE, but I can't figure out how I need to configure SSHtunnelForwarder to achieve the same from within python. I am probably misunderstanding its documentation: https://github.com/pahaz/sshtunnel
EDIT: This code is faulty. Look for an answer below.
# I replaced the remote-ip with `123.45.67.89`.
# I replaced my ip with `987.65.43.21`.
with SSHTunnelForwarder(
ssh_address_or_host="123.45.67.89", # ip of remote server
ssh_pkey="PRIVATE_KEY", # I am unsure which one is needed tried both
ssh_private_key="PRIVATE_KEY", # I am unsure which one is needed tried both
ssh_username="ubuntu", # username of remote server
ssh_bind_address= ("127.0.0.1", 9191), # where to listen locally
remote_bind_address= ("127.0.0.1", 8181) # where to send info coming to 22
# ssh-port 22 (default port)
# localhost:9191 -> 123.45.67.89:22 -> localhost:8181
# which is basically 987.65.43.21:9191 -> 123.45.67.89:22 -> 123.45.67.89:8181
) as server:
server.start()
But when I try to connect to http://localhost:9191/ I receive unable to connect; so the tunneling is probably broken. I added plenty of comments to the code to make it easier to spot my misunderstanding.
When I run your code then I see error
ValueError: You can't use both 'ssh_private_key' and 'ssh_pkey'. Please only use one of them.
You have to use one of them and example in your link show what it should be.
ssh_pkey="/var/ssh/rsa_key",
or
ssh_private_key_password="secret",
It needs to generate file rsa_key using program ssh-keygen which should be installed with ssh
(on Linux I would keep this file in user's folder ~/.ssh)
Next I see error
ValueError: Unknown arguments: {'ssh_bind_address': ('127.0.0.1', 9191)}
it has to be local_bind_address instead of ssh_bind_address.
Even example in your link uses local_bind_address.
You can see all arguments in commends in source code
or using help(SSHTunnelForwarder) in code.
This code works for me:
I can use pair username, password
ssh_username = "pi",
ssh_password = "raspberry",
or pair username and standard file id_rsa with my keys
ssh_username = "pi",
ssh_pkey='/home/furas/.ssh/id_rsa',
Code:
from sshtunnel import SSHTunnelForwarder
import time
#import sshtunnel
#print('author :', sshtunnel.__author__)
#print('version:', sshtunnel.__version__)
#help(SSHTunnelForwarder)
with SSHTunnelForwarder(
ssh_address_or_host = "192.168.1.29", # Raspberry Pi in my network
ssh_username = "pi",
ssh_password = "raspberry",
#ssh_username = "pi",
#ssh_pkey='/home/furas/.ssh/id_rsa',
local_bind_address = ("127.0.0.1", 9191),
remote_bind_address = ("127.0.0.1", 8181)
) as server:
print('Starting ...')
server.start()
print('Keep running ...')
while True:
time.sleep(1)
#print('Stopping ...')
#server.stop()
Is it Possible if the Connection to a Server cuts off for some reason that I get a Text or something, so I know that the Error has something to do with the Server?
import paramiko
import time
def connSFTP(ssh_key_filepath,host,user):
#Fehleroptionen 1. Pfad zum SSH-KeyError
#-SSH-Key vorhanden?
#2.Host und/oder Username ist falsch
k = paramiko.RSAKey.from_private_key_file(ssh_key_filepath) #SSH KEY
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
c.connect( hostname = host, username = user , pkey = k )
sftp = c.open_sftp()
return sftp
def main():
ssh_key_filepath = '/home/dtv/.ssh/id_rsa'
hostname = 'HostAdress'
username = 'user'
sftp = connSFTP(ssh_key_filepath,hostname,username)
print('Connected')
time.sleep(30)
#sftp.close()
if __name__ == '__main__':
main()
I built in a sleep, so I can kill the procedure for the Connection and Simulate the Connection lost in time.
You should use exception handling with the following:
Raises:
BadHostKeyException – if the server’s host key could not be verified
Raises:
AuthenticationException – if authentication failed
Raises:
SSHException – if there was any other error connecting or establishing an SSH session
Raises:
socket.error – if a socket error occurred while connecting
Paramiko api documentation link: http://docs.paramiko.org/en/1.15/api/client.html#paramiko.client.SSHClient.connect
I have a directory containing some files and sub-directories on my local machine that is generated by a daily Python script running. And then I want to copy all those generated files in a directory to the server and then run some series of commands on it by ssh using python package paramiko.
I want to add little code to securely copy that entire directory and files and sub-directories inside it to to my server over SSH/SCP using the paramiko package of python. I can send the individual files using the same but unable to send the entire directory and its content using paramiko. It gives me IOError saying it's a directory.
IOError: [Errno 21] Is a directory: 'temp'
Here is the my code:
from paramiko import SSHClient, AutoAddPolicy
from scp import SCPClient
class SSh(object):
def __init__(self, address, username, password, port=22):
print "Connecting to server."
self._address = address
self._username = username
self._password = password
self._port = port
self.sshObj = None
self.connect()
self.scp = SCPClient(self.sshObj.get_transport())
def sendCommand(self, command):
if(self.sshObj):
stdin, stdout, stderr = self.sshObj.exec_command(command)
while not stdout.channel.exit_status_ready():
# Print data when available
if stdout.channel.recv_ready():
alldata = stdout.channel.recv(1024)
prevdata = b"1"
while prevdata:
prevdata = stdout.channel.recv(1024)
alldata += prevdata
print str(alldata)
else:
print "Connection not opened."
def connect(self):
try:
ssh = SSHClient()
ssh.set_missing_host_key_policy(AutoAddPolicy())
ssh.connect(self._address, port=self._port, username=self._username, password=self._password, timeout=20, look_for_keys=False)
print 'Connected to {} over SSh'.format(self._address)
return True
except Exception as e:
ssh = None
print "Unable to connect to {} over ssh: {}".format(self._address, e)
return False
finally:
self.sshObj = ssh
if __name__ == "__main__":
# Parse and check the arguments
ssh = SSh('10.24.143.190', 'username', 'password')
ssh.scp.put("Valigrind_BB.py") # This works perfectly fine
ssh.scp.put("temp") # IOError over here Is a directory
ssh.sendCommand('ls')
Thank you in advance.
Looking at the source code for SCP, it appears you can pass the parameter "recursive" as a bool to the put() method to specify transferring the contents of a directory recursively. Here is a link to the source code I am talking about. Try changing last part of your code to the following:
if __name__ == "__main__":
# Parse and check the arguments
ssh = SSh('10.24.143.190', 'username', 'password')
ssh.scp.put("Valigrind_BB.py") # This works perfectly fine
ssh.scp.put("temp", recursive=True) # IOError over here Is a directory
ssh.sendCommand('ls')
Also, if you just want to transfer files, you can try rsync as an alternative. The modification to your code above should work though. I hope this helps.
I am having a very specific problem. I use Putty for database management for my small business. We recently had an update and our normal command path to update our records is no longer in use.
We run Putty on all computers in store. Putty is used on a virtual machine with oracle. We have a main computer for the system in which the update occurred.
We normally input ~/Desktop/getdata.sh into putty using root user and it complies an updated list, creates a text file that we use. Unfortunately, the person who created this script no longer works with us.
I am trying to find a way to re execute this file again.
After the update, when we type in ~/Desktop/getdata.sh (after logging in as root) into Putty we get 'directory cannot be found.' I've searched everyday to find this file. However, I did find a getdata.py file and a getdata.bat files.
I can show both scripts if needed, I can update the question.
When I tried to run getdata.py I get
[root#RT_Store-01 /]# ~/Desktop/getdata.py
import: unable to open X server `'.
import: unable to open X server `'.
import: unable to open X server `'.
import: unable to open X server `'.
: command not foundta.py: line 5:
/root/Desktop/getdata.py: line 6: from: command not found
/root/Desktop/getdata.py: line 7: from: command not found
: command not foundta.py: line 8:
/root/Desktop/getdata.py: line 9: syntax error near unexpected token `('
/root/Desktop/getdata.py: line 9: `dir_path = os.path.dirname(os.path.realpath(_'file__))
Do I need to convert my files to .sh? How would I do that? Is this a bigger problem?
The script for getdata.py is
import os
import tempfile
import paramiko
import time
from pywinauto import Application
from subprocess import Popen, PIPE, STDOUT
dir_path = os.path.dirname(os.path.realpath(__file__))
class Connection(object):
"""Connects and logs into the specified hostname.
Arguments that are not given are guessed from the environment."""
def __init__(self,
host,
username=None,
private_key=None,
password=None,
port=22,
):
self._sftp_live = False
self._sftp = None
if not username:
username = os.environ['LOGNAME']
# Log to a temporary file.
templog = tempfile.mkstemp('.txt', 'ssh-')[1]
paramiko.util.log_to_file(templog)
# Begin the SSH transport.
self._transport = paramiko.Transport((host, port))
self._tranport_live = True
# Authenticate the transport.
if password:
# Using Password.
self._transport.connect(username=username, password=password)
else:
# Use Private Key.
if not private_key:
# Try to use default key.
if os.path.exists(os.path.expanduser('~/.ssh/id_rsa')):
private_key = '~/.ssh/id_rsa'
elif os.path.exists(os.path.expanduser('~/.ssh/id_dsa')):
private_key = '~/.ssh/id_dsa'
else:
raise TypeError(
"You have not specified a password or key.")
private_key_file = os.path.expanduser(private_key)
rsa_key = paramiko.RSAKey.from_private_key_file(private_key_file)
self._transport.connect(username=username, pkey=rsa_key)
def _sftp_connect(self):
"""Establish the SFTP connection."""
if not self._sftp_live:
self._sftp = paramiko.SFTPClient.from_transport(self._transport)
self._sftp_live = True
def get(self, remotepath, localpath=None):
"""Copies a file between the remote host and the local host."""
if not localpath:
localpath = os.path.split(remotepath)[1]
self._sftp_connect()
self._sftp.get(remotepath, localpath)
def put(self, localpath, remotepath=None):
"""Copies a file between the local host and the remote host."""
if not remotepath:
remotepath = os.path.split(localpath)[1]
self._sftp_connect()
self._sftp.put(localpath, remotepath)
def execute(self, command):
"""Execute the given commands on a remote machine."""
channel = self._transport.open_session()
channel.exec_command(command)
output = channel.makefile('rb', -1).readlines()
if output:
return output
else:
return channel.makefile_stderr('rb', -1).readlines()
def update(self):
"""Execute the given commands on a remote machine."""
channel = self._transport.invoke_shell(term='xterm')
channel.exec_command('~/Desktop/update.sh')
output = channel.makefile('rb', -1).readlines()
if output:
return output
else:
return channel.makefile_stderr('rb', -1).readlines()
def close(self):
"""Closes the connection and cleans up."""
# Close SFTP Connection.
if self._sftp_live:
self._sftp.close()
self._sftp_live = False
# Close the SSH Transport.
if self._tranport_live:
self._transport.close()
self._tranport_live = False
def __del__(self):
"""Attempt to clean up if not explicitly closed."""
self.close()
def getData():
"""Create, get, and put delim file when called directly."""
app = Application().start(r"c:\putty.exe trak#10.1.10.70 -pw trak")
app.window_(
title_re=".*trak.*").TypeKeys("/home/trak/Desktop/getdata.sh && exit{ENTER}")
app.window_(title_re=".*trak.*").WaitNot("exists", timeout=120)
trakfile = dir_path + '/storage/trakdelim.txt'
shell = Connection('10.1.10.70', "trak", password="trak")
shell.get('/trak/data/trakdelim.txt', trakfile)
shell.close()
if __name__ == "__main__":
getData()
I appreciate anyone who can help and I can clarify when needed!
A bit of googling showed me what this is:
Someone copied this code and wrote a simple script with it that takes a file (/trak/data/trakdelim.txt) on a remote computer with ip address 10.1.10.70 username trak and password trak and copies it to the storage/trakdelim.txt file. if this is not working for you now then take a tool that allows you to do this manually with such as winSCP and use that instead.
Good luck.
You need to execute it as a python script.
python ~/Desktop/getdata.py
So I've heard that it's an incredibly bad idea to host media on the same server as your Django/Apache web framework. So to solve this problem with my iOS app and greatly improve server side performance in the long run would be to upload the images to the Django server and then right after that, transfer the newly uploaded image to a separate server dedicated to hosting user profile images.
So using Paramiko I can send files directly to the directory of my choosing, but this does not seem to be working in my Django view.
Here's an incredible simple python script using Paramiko that allows you to upload files to a remote server via SMTP:
import base64
import getpass
import os
import socket
import sys
import traceback
import paramiko
#from paramiko.py3compat import input
# setup logging
#paramiko.util.log_to_file('demo_sftp.log')
port = 22
hostname = '198.199.101.115'
password = 'XXXXXXXXX'
username = 'root'
hostkeytype = 'ecdsa-sha2-nistp256'
hostkey = 'XXXXXXXXX'
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
except IOError:
try:
# try ~/ssh/ too, because windows can't have a folder named ~/.ssh/
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/ssh/known_hosts'))
except IOError:
print('*** Unable to open host keys file')
host_keys = {}
if hostname in host_keys:
hostkeytype = host_keys[hostname].keys()[0]
hostkey = host_keys[hostname][hostkeytype]
print('Using host key of type %s' % hostkeytype)
# now, connect and use paramiko Transport to negotiate SSH2 across the connection
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password, hostkey=hostkey)
sftp = paramiko.SFTPClient.from_transport(t)
# dirlist on remote host
dirlist = sftp.listdir('.')
print("Dirlist: %s" % dirlist)
# copy this demo onto the server
target_directory = '3COOL'
sftp.put('test_image.jpg', target_directory+'/test_image.jpg')
t.close()
So this script works perfectly fine when I run the script on my remote server which successfully transfers the jpeg image to another remote server I own. So I'd like to just cut and paste this script into my Django view which will transfer all freshly uploaded images to a different server, here's how this looks:
def profile_picture(request):
if request.POST:
form = UserProfileForm(request.POST, request.FILES)
obj = form.save(commit=False)
obj.user_id = request.user.id
obj.profile_picture = obj.profile_picture
check = UserProfile.objects.filter(user_id=request.user.id)
if check:
oldup = UserProfile.objects.get(user_id=request.user.id)
oldup.delete()
obj.save()
formNew = UserProfileForm()
args = {}
args.update(csrf(request))
args['uid'] = request.user.id
args['form'] = formNew
# CONVERT THE IMAGE TO A SMALLER SIZE
basewidth = 256
img = Image.open('var/www/bitcraft/static/'+str(obj.profile_picture))
wpercent = (basewidth/float(img.size[0]))
hsize = int((float(img.size[1])*float(wpercent)))
img = img.resize((basewidth,hsize), PIL.Image.ANTIALIAS)
img.save('var/www/bitcraft/static/'+str(obj.profile_picture))
import getpass
import os
import socket
import sys
import traceback
import paramiko
from paramiko.py3compat import input
# TRANSFER THIS IMAGE TO MEDIA HOSTING SERVER
port = 22
hostname = '198.199.101.115'
password = 'pcorysatqwrw'
username = 'root'
hostkeytype = 'ecdsa-sha2-nistp256'
hostkey = 'pcorysatqwrw'
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
except IOError:
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/ssh/known_hosts'))
except IOError:
print('*** Unable to open host keys file')
host_keys = {}
if hostname in host_keys:
hostkeytype = host_keys[hostname].keys()[0]
hostkey = host_keys[hostname][hostkeytype]
print('Using host key of type %s' % hostkeytype)
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password, hostkey=hostkey)
sftp = paramiko.SFTPClient.from_transport(t)
dirlist = sftp.listdir('.')
print("Dirlist: %s" % dirlist)
target_directory = '3COOL'
sftp.put('var/www/bitcraft/static/'+str(obj.profile_picture), target_directory+str(obj.profile_picture))
t.close()
return render_to_response('profile.html', args, RequestContext(request))
else:
formNew = UserProfileForm()
args = {}
args.update(csrf(request))
args['uid'] = request.user.id
args['form'] = formNew
return render_to_response('profile.html', args, RequestContext(request))
Ultimately though, adding the simple Python script has not worked due to a strange error:
AttributeError at /upload_profile/ 'str' object has no attribute 'get_name'
Exception Location: /usr/local/lib/python2.7/dist-packages/paramiko-1.14.0-py2.7.egg/paramiko/transport.py in connect, line 873
/srv/www/django/chatfeed/views.py in profile_picture
t.connect(username=username, password=password, hostkey=hostkey)
I don't understand what's causing this... is this related to directory permission denied??
The problem is that you're passing a string for hostkey. As the docs show, that's supposed to be a PKey—that is, an object that wraps a private key.
And if you click on PKey, you'll see that a PKey has a get_name method. A string obviously doesn't. Hence the error.
You have some code that's supposed to use Paramiko to load a hostkey out of ~/.ssh/known_hosts at the top of your script. But if that hostname in host_keys is false, it won't do anything, and you'll end up with the default values you stuck at the top of the script:
hostkeytype = 'ecdsa-sha2-nistp256'
hostkey = 'XXXXXXXXX'
I don't know what you have in place of that 'XXXXXXXXX' in your real code, but presumably it's a string, and therefore not a valid hostkey.
I don't understand what's causing this... is this related to directory permission denied??
You'd have to tell us what you're talking about. Exactly what error or warning or whatever are you getting, and where? If the server is running as a user who doesn't have access to its own ~/.ssh, so you're seeing a "directory permission denied" warning of some kind and your host keys aren't getting loaded, then yes, this problem is indirectly related to that one—as in, if the host actually was in known_keys, and you solved the permissions problem, that would mask the error in your code, so you wouldn't notice it. But there are plenty of other things you could mean where the answer would be no, that has nothing to do with anything.
key = paramiko.RSAKey.from_private_key_file(filename='openssh key f',password='passphrase')
The keys file does not support .ppk format.