We Keep Coding

Python 3.9 SFTP connection

I'm trying to connect to SFTP server that is hosted locally (that's a POC ...). I'm using python 3.9 and the library paramiko.
The code I use is the following :
import paramiko
import sys
def main() -> None:
hostkeys = paramiko.hostkeys.HostKeys("known_hosts.txt")
ip = "127.0.0.1"
username = "my_username"
password = "my_password"
# As per the paramiko doc. The host fingerprint is stored using the ed25519 algorithm
hostFingerprint = hostkeys.lookup(ip)["ssh-ed25519"]
try:
tp = paramiko.Transport(ip, 22)
tp.connect(username=username, password=password, hostkey=hostFingerprint)
# Upload operations happening here...
tp.close()
except Exception as e:
print(e)
finally:
return None
The file known_hsts.txt has been generated via Powershell cmd : ssh-keyscan 127.0.0.1 > C:\Users\my_username\PycharmProjects\POC\known_hosts.txt.
When i run the script i don't understand why hostkeys.keys() == []
When i look at the file known_hosts.txt I see the different keys : ssh-rsa, ssh-ed25519, etc.

Open SSH Tunnel with private key stored in S3

If I run the following code, my SSH Tunnel works perfectly.
from sshtunnel import SSHTunnelForwarder
tunnel = SSHTunnelForwarder(
ssh_host=(SSH_JUMPHOST, SSH_PORT),
ssh_username=SSH_USERNAME,
ssh_pkey="/path/to/key/in/my/machine",
remote_bind_address=(
REMOTE_HOST,
REMOTE_PORT,
),
local_bind_address=("127.0.0.1", 12345),
ssh_private_key_password=SSH_PKEY_PASSWORD,
)
tunnel.start()
# Things happen in the tunnel...
However, I want to read a .pem key that is stored in an S3 bucket. How can I read and pass the key to the SSHTunnelForwarder constructor?
from sshtunnel import SSHTunnelForwarder
S3_BUCKET = "the_bucket"
S3_KEY_PATH = "the_key.pem"
tunnel = SSHTunnelForwarder(
ssh_host=(SSH_JUMPHOST, SSH_PORT),
ssh_username=SSH_USERNAME,
ssh_pkey=??????, ################ What should I include here?
remote_bind_address=(
REMOTE_HOST,
REMOTE_PORT,
),
local_bind_address=("127.0.0.1", 12345),
ssh_private_key_password=SSH_PKEY_PASSWORD,
)
tunnel.start()
# Things happen in the tunnel...

In the end, I surrendered to Furas suggestion since I couldn't find an alternative way to get it done.
The idea is to download the key file and point to the downloaded copy. With the following code, it can be structured to leave the file available for the shortest amount of time possible and ensure to best ability that it gets deleted after the tunnel has been opened.
from sshtunnel import SSHTunnelForwarder
S3_BUCKET = "the_bucket"
S3_KEY_PATH = "the_key.pem"
try:
s3.download_file(S3_BUCKET_NAME, S3_KEY_PATH , "temp")
tunnel = SSHTunnelForwarder(
ssh_host=(SSH_JUMPHOST, SSH_PORT),
ssh_username=SSH_USERNAME,
ssh_pkey="temp",
remote_bind_address=(
DW_HOST,
DW_PORT,
),
local_bind_address=("127.0.0.1", DW_PORT),
ssh_private_key_password=SSH_PKEY_PASSWORD,
)
except Exception as e:
raise e
finally:
# No matter what happens above, we always delete the temp copy of the key
os.remove("temp")
tunnel.start()
# Things happen in the tunnel...

How to keep ssh connection open and doing multiple requests and outputs within python

Because this question seems to aim somewhere else I am going to point my problem here:
In my python script I am using multiple requests to a remote server using ssh:
def ssh(command):
command = 'ssh SERVER "command"'
output = subprocess.check_output(
command,
stderr=subprocess.STDOUT,
shell=True,
universal_newlines=True
)
return output
here I will get the content of file1 as output.
I have now multiple methods which use this function:
def show_one():
ssh('cat file1')
def show_two():
ssh('cat file2')
def run():
one = show_one()
print(one)
two = show_two()
print(two)
Executing run() will open and close the ssh connection for each show_* method which makes it pretty slow.
Solutions:
I can put:
Host SERVER
ControlMaster auto
ControlPersist yes
ControlPath ~/.ssh/socket-%r#%h:%p
into my .ssh/config but I would like to solve this within python.
There is the ssh flag -T to keep a connection open, and in the before mentioned Question one answer was to use this with Popen() and p.communicate() but it is not possible to get the output between the communicates because it throws an error ValueError: Cannot send input after starting communication
I could somehow change my functions to execute a single ssh command like echo "--show1--"; cat file1; echo "--show2--"; cat file2 but this looks hacky to me and I hope there is a better method to just keep the ssh connection open and use it like normal.
What I would like to have: For example a pythonic/bashic to do the same as I can configure in the .ssh/config (see 1.) to declare a specific socket for the connection and explicitly open, use, close it

Try to create ssh object from class and pass it to the functions:
import paramiko
from pythonping import ping
from scp import SCPClient
class SSH():
def __init__(self, ip='192.168.1.1', username='user', password='pass',connect=True,Timeout=10):
self.ip = ip
self.username = username
self.password = password
self.Timeout=Timeout
self.ssh = paramiko.SSHClient()
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
if connect:
self.OpenConnection()
self.scp = SCPClient(self.ssh.get_transport())
def OpenConnection(self):
try:
skip_ping = False
ping_res=False
log.info('Sending ping to host (timeout=3,count=3) :'+self.ip)
try:
PingRes = ping(target=self.ip,timeout=3,count=3, verbose=True)
log.info('Ping to host result :' + str(PingRes.success()))
ping_res=PingRes.success()
except:
skip_ping=True
if ping_res or skip_ping:
log.info('Starting to open connection....')
self.ssh.connect(hostname=self.ip, username=self.username, password=self.password, timeout=self.Timeout, auth_timeout=self.Timeout,banner_timeout=self.Timeout)
self.scp = SCPClient(self.ssh.get_transport())
log.info('Connection open')
return True
else:
log.error('ssh OpenConnection failed: No Ping to host')
return False
myssh = SSH(ip='192.168.1.1',password='mypass',username='myusername')
the ping result is wrapped in try catch because sometimes my machine return an error you can remove it and just verify a ping to the host.
The self.scp is for file transfer.

Paramiko server: Redirect the output of a SUBPROCESS to a channel

I am trying to implement a (local fake) server with paramiko that responds to custom commands just like the original for testing purposes. Mainly copying from the supplied demo server, I managed to come up with this custom method to handle exec_requests for the server implemented via paramiko.ServerInterface:
def check_channel_exec_request(self,channel,command):
print("User wants to execute '%s'" %(command))
comm_main, comm_add = command.decode().split(sep=" ",maxsplit=1)
if comm_main in self.allowed_commands:
chout = channel.makefile("w")
subprocess.Popen(command,stdout=chout)
return True
else:
return False
After the server is running via:
PORT = 50007 # The same port as used by the server
HOST = 'localhost'
host_key = paramiko.RSAKey(filename = <FILEPATH>)
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((HOST,PORT))
sock.listen(1)
conn, addr = sock.accept() # Connected!
with conn:
trans = paramiko.Transport(conn)
trans.add_server_key(host_key)
trans.set_subsystem_handler("job",JobHandler)
server = FakeCluster() # Custom class sublassing paramiko.ServerInterface
trans.start_server(server=server)
chan = trans.accept() # Accept authentication from client
while trans.is_active(): # Do not close until inactive
time.sleep(1)
chan.close()
the client would try to execute echo 123 in the following manner:
PORT = 50007 # The same port as used by the server
HOST = 'localhost'
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind((HOST,PORT))
ssh = paramiko.client.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(HOST, PORT,username=<USERNAME>,password=<PASSWORD>)
ssh.exec_command("echo 123")
Right now, I am getting the error trying to execute the subprocess that 'ChannelFile' has no attribute 'fileno'. Furthermore I am wondering how to later execute a python script as a custom command called by exec_command. (Maybe by calling a batchfile that calls the python script?)

your question is really very confusing , here is what i did for communicating with remote server .
local machine : window
Remote Machine :ubuntu
Command on remote machine :'who'
import paramiko
ssh=paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('192.268.21.26',port=22,username='root',password='default')
stdin,stdout,stderr=ssh.exec_command('who')
output=stdout.readlines()
print '\n'.join(output)
#output
#root tty7 2017-11-28 14:13 (:0)
#if you wish to execute a python file there , this should work
stdin,stdout,stderr=ssh.exec_command('python file.py')

Paramiko - ssh to console server, having to hit return for script to continue

Just using my first paramiko script, we have an opengear console server, so I'm trying to automate setup of any device we plug into it.
The open gear listens for ssh connections on ports, for example a device in port 1 would be 3001. I am connecting to a device on port 8, which works and my script runs, but for some reason, after I get the "Interactive SSH session established" message, I need to hit return on the session to make it run (so I have a ssh session and the script does too, its shared).
It just waits there until I hit return, I've tried sending returns as you can see but they don't work, only a manual return works, which is odd because technically they are the same thing?
import paramiko
import time
def disable_paging(remote_conn):
'''Disable paging on a Cisco router'''
remote_conn.send("terminal length 0\n")
time.sleep(1)
# Clear the buffer on the screen
output = remote_conn.recv(1000)
return output
if __name__ == '__main__':
# VARIABLES THAT NEED CHANGED
ip = '192.168.1.10'
username = 'root'
password = 'XXXXXX'
port = 3008
# Create instance of SSHClient object
remote_conn_pre = paramiko.SSHClient()
# Automatically add untrusted hosts (make sure okay for security policy in your environment)
remote_conn_pre.set_missing_host_key_policy(
paramiko.AutoAddPolicy())
# initiate SSH connection
remote_conn_pre.connect(ip, username=username, password=password,port=port, look_for_keys=False, allow_agent=False)
print "SSH connection established to %s" % ip
# Use invoke_shell to establish an 'interactive session'
remote_conn = remote_conn_pre.invoke_shell()
print "Interactive SSH session established"
time.sleep(1)
remote_conn.send("\n")
# Strip the initial router prompt
#output = remote_conn.recv(1000)
# See what we have
#print output
# Turn off paging
#disable_paging(remote_conn)
# clear any config sessions
is_global = remote_conn.recv(1024)
if ")#" in is_global:
remote_conn.send("end\n")
time.sleep(2)
# if not in enable mode go to enable mode
is_enable = remote_conn.recv(1024)
if ">" in is_enable:
remote_conn.send("enable\n")
time.sleep(1)
remote_conn.send("conf t\n")
remote_conn.send("int g0/0/1\n")
remote_conn.send("ip address 192.168.1.21 255.255.255.0\n")
remote_conn.send("no shut\n")
remote_conn.send("end\n")
# Wait for the command to complete
time.sleep(2)
remote_conn.send("ping 192.168.1.1\n")
time.sleep(1)
output = remote_conn.recv(5000)
print output

I tried this and saw that
is_global = remote_conn.recv(1024)
hangs,
Are you sure '192.168.1.10' sends somthing to be received ?
Try setting a timeout
remote_conn.settimeout(3)
3 seconds for example, do it after this line:
remote_conn = remote_conn_pre.invoke_shell()
this way the recv func does not hang and continues when timeout expires
works for me

first send some command "ls -ltr\n" and then call sleep
remote_conn.send("ls -ltr\n")
time.sleep(1)

Try running your command in a debugger and find out what line is waiting for input. You might also try sending \r or \r\n instead if just \n. Remember the enter key is really ^M
You might also try turning on detailed logging.
import logging
# ...
logging.getLogger("paramiko").setLevel(logging.DEBUG)

ive found another module (netmiko) which does exactly what i want and does all these checks. ive since abandoned trying to do it myself when someone else has already done it better.
use Netmiko! :)