This is my first post in StackOverflow, so I hope to do it the right way! :)
I have this task to do for my new job that needs to connect to several servers and execute a python script in all of them. I'm not very familiar with servers (and just started using paramiko), so I apologize for any big mistakes!
The script I want to run on them modifies the authorized_keys file but to start, I'm trying it with only one server and not yet using the aforementioned script (I don't want to make a mistake and block the server in my first task!).
I'm just trying to list the directory in the remote machine with a very simple function called getDir(). So far, I've been able to connect to the server with paramiko using the basics (I'm using pdb to debug the script by the way):
try_paramiko.py
#!/usr/bin/python
import paramiko
from getDir import get_dir
import pdb
def try_this(server):
pdb.set_trace()
ssh = paramiko.SSHClient()
ssh.load_host_keys("pth/to/known_hosts")
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
my_key = paramiko.RSAKey.from_private_key_file("pth/to/id_rsa")
ssh.connect(server, username = "root", pkey = my_key)
i, o, e = ssh.exec_command(getDir())
This is the function to get the directory list:
getDir.py
#!/usr/bin/python
import os
import pdb
def get_dir():
pdb.set_trace()
print "Current dir list is:"
for item in os.listdir(os.getcwd()):
print item
While debugging I got the directory list of my local machine instead of the one from the remote machine... is there a way to pass a python function as a parameter through paramiko? I would like to just have the script locally and run it remotely like when you do it with a bash file from ssh with:
ssh -i pth/to/key username#domain.com 'bash -s' < script.sh
so to actually avoid to copy the python script to every machine and then run it from them (I guess with the above command the script would also be copied to the remote machine and then deleted, right?) Is there a way to do that with paramiko.sshClient()?
I have also tried to modify the code and use the standard output of the channel that creates exec_command to list the directory leaving the scripts like:
try_paramiko.py
#!/usr/bin/python
import paramiko
from getDir import get_dir
import pdb
def try_this(server):
pdb.set_trace()
ssh = paramiko.SSHClient()
ssh.load_host_keys("pth/to/known_hosts")
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
my_key = paramiko.RSAKey.from_private_key_file("pth/to/id_rsa")
ssh.connect(server, username = "root", pkey = my_key)
i, o, e = ssh.exec_command(getDir())
for line in o.readlines():
print line
for line in e.readlines():
print line
getDir.py
def get_dir():
return ', '.join(os.listdir(os.getcwd()))
But with this, it actually tries to run the local directory list as commands (which actually makes sense they way I have it). I had to convert the list to a string because I was having a TypeError saying that it expects a string or a read-only character buffer, not a list... I know this was a desperate attempt to pass the function... Does anyone know how I could do such thing (pass a local function through paramiko to execute it on a remote machine)?
If you have any corrections or tips on the code, they are very much welcome (actually, any kind of help would be very much appreciated!).
Thanks a lot in advance! :)
You cannot just execute python function through ssh. ssh is just a tunnel with your code on one side (client) and shell on another (server). You should execute shell commands on remote side.
If using raw ssh code is not critical, i suggest fabric as library for writing administration tools. It contains tools for easy ssh handling, file transferring, sudo, parallel execution and other.
I think you might want change the paramaters you're passing into ssh.exec_command Here's an idea:
Instead of doing:
def get_dir():
return ', '.join(os.listdir(os.getcwd()))
i, o, e = ssh.exec_command(getDir())
You might want to try:
i, o, e = ssh.exec_command('pwd')
o.printlines()
And other things to explore:
Writing a bash script or a Python that lives on your servers. You can use Paramiko to log onto the server and executing the script with ssh.exec_command(some_script.sh) or ssh.exec_command(some_script.py)
Paramiko has some FTP/SFTP utilities so you can actually use it to put the script on the server and then execute it.
It is possible to do this by using a here document to feed a module into the remote server's python interpreter.
remotepypath = "/usr/bin/"
# open the module as a text file
with open("getDir.py", "r") as f:
mymodule = f.read()
# setup from OP code
ssh = paramiko.SSHClient()
ssh.load_host_keys("pth/to/known_hosts")
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
my_key = paramiko.RSAKey.from_private_key_file("pth/to/id_rsa")
ssh.connect(server, username = "root", pkey = my_key)
# use here document to feed module into python interpreter
stdin, stdout, stderr = ssh.exec_command("{p}python - <<EOF\n{s}\nEOF".format(p=remotepypath, s=mymodule))
print("stderr: ", stderr.readlines())
print("stdout: ", stdout.readlines())
Related
I need to establish an ssh connection and execute commands remotely, and the flow is: connect by ssh, open a new shell by command line and finally execute the commands.
use Net::SSH::Perl;
my $ssh = Net::SSH::Perl->new('test.com');
$ssh->login('user', 'pass');
my($stdout, $stderr, $exit) = $ssh->cmd('shell');
my($stdout, $stderr, $exit) = $ssh->cmd('ls -l');
print($stdout);
but when you run $ssh->cmd('shell'); it waits, I assume that this opens the shell and since it is a "command" that does not end, it does not pass to the next command which would be to make an ls.
I need to execute commands in the new shell opened by ssh. It is not possible to modify absolutely anything on the remote machine.
Any solution in perl or python will help :)
You don't need to start a new shell explicitly. That's how ssh runs a given command, by passing it as a command line to a new shell.
use Net::SSH::Perl;
my $ssh = Net::SSH::Perl->new('test.com');
$ssh->login('user', 'pass');
my($stdout, $stderr, $exit) = $ssh->cmd('ls -l');
print($stdout);
Perl CPAN recommends to use Net::OpenSSH here instead of Net::SSH::Perl.
Here is the snippet how it should look like:
#!/usr/bin/perl
use strict;
use warnings;
use Net::OpenSSH;
#Function declaration by passing host, user and password as an arguments
my $ssh = ConnectToRemoteMachine ("host_ip_address", "user", "password");
my $result_ls = $ssh->capture("ls -l");
my $result_shell = $ssh->capture("shell");
print "Result of ls:\n";
print $result_ls."\n";
print "Shell: $result_shell\n"
undef $ssh;
.
.
#Function definition
sub ConnectToRemoteMachine {
my ( $host, $user, $passwd ) = #_;
my $ssh = Net::OpenSSH->new($host,
user => $user,
password => $passwd);
$ssh->error and die "Couldn't establish SSH connection: ". $ssh->error;
return $ssh;
}
Does anyone know how to make environment variables registered for
exec_command calls when using SSHClient?
I'm using a basic script that instantiates the SSHClient class, connects to another computer using the connect method, then sends out commands using the exec_command method. However, none of the environment variables seem to be registered when I try to issue commands. I can do basic things like 'ls' and see the stdout, but when trying to run installed programs, the fact that the environment variables are missing makes it impossible to run them. Using ssh in the command line to do the same thing works, as the environment variables for the user are set.
#!/usr/bin/python
import paramiko
ssh.connect('mymachine',username='myname',password='pass')
stdin,stdout,stderr=ssh.exec_command('cd /myfolder/path')
stdin,stdout,stderr=ssh.exec_command('ls')
....
....
ssh.close()
Note: I can't change my directory in paramiko. I appended the cd command in the followup command in a single ssh.exec_command('cd /dddd/ddd;ls'). I have given ls as an example but my actual followup command is different.
Since release 2.1.0 2016-12-09 , you can add an environment variable dictionary to the exec_command:
import paramiko
paramiko.util.log_to_file("paramiko.log")
ssh = paramiko.SSHClient()
k = paramiko.RSAKey.from_private_key_file("<private_key_file>")
ssh.connect(<hostname>,username=<username>,pkey=k)
env_dict={"LC_TELEPHONE":"ET_HOME","LC_MEASUREMENT":"MILES_APART"}
stdin , stdout, stderr = ssh.exec_command('echo $LC_TELEPHONE; echo "..."; echo $LC_MEASUREMENT',environment=env_dict)
print stdout.read()
output:
ET_HOME
...
MILES_APART
But why did I choose LC_TELEPHONE and LC_MEASUREMENT? Because those are two of the few environments that the target host's ssh config allows me to set:
grep AcceptEnv /etc/ssh/sshd_config
output:
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL
In other words, this doesn't work:
env_dict={"HELLO":"WORLD","GOODBYE":"CRUEL_WORLD"}
stdin , stdout, stderr = ssh.exec_command("echo $HELLO; echo '...'; echo $GOODBYE")
print stdout.read()
output:
...
As the documentation warns, the environment variables are silently rejected
http://docs.paramiko.org/en/2.1/api/client.html
http://docs.paramiko.org/en/2.1/api/channel.html#paramiko.channel.Channel.set_environment_variable
If you cannot control the target server's sshd config, putting the environment variables into a file and sourcing it works:
stdin , stdout, stderr = ssh.exec_command("cat .set_env;source .set_env; echo $HELLO; echo '...'; echo $GOODBYE")
print stdout.read()
output:
# begin .set_env
HELLO="WORLD"
GOODBYE="CRUEL_WORLD"
# end .set_env
WORLD
...
CRUEL_WORLD
#!/usr/bin/python
import paramiko
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy)
client.connect(myhostname, theport, myuser, thepass)
stdin,stdout,stderr = client.exec_command('cd /tmp;pwd;ls -al')
#returns your output
print stdout.read()
which all works fine for me. If you have special environment Variables you might
have to set them on the remote command prompt. Maybe it helps if you write the
variables into a myENV file and then call
stdin,stdout,stderr = client.exec_command('source ./myEnv')
Did you tried something like that?
You can do: client.exec_command(..., get_pty=True).
This will make paramiko allocate a pseudo terminal, similar to ssh.
I found this problem too. And besides the above approaches, I also fixed the problem by using the following approach:
e.g.,
...
bin_paths = '/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin'
path_prefix = 'PATH=$PATH:%s && ' % bin_paths
command = path_prefix + command
ssh.exec_command(command=command)
I'm trying to write a python script that can ssh into remote server and can execute simple commands like ls,cd from the python client. However, I'm not able to read the output from the pseudo-terminal after successfully ssh'ing into the server. Could anyone please help me here so that I could execute some commands on the server.
Here is the sample code:
#!/usr/bin/python2.6
import os,sys,time,thread
pid,fd = os.forkpty()
if pid == 0:
os.execv('/usr/bin/ssh',['/usr/bin/ssh','user#host',])
sys.exit(0)
else:
output = os.read(fd,1024)
print output
data = output
os.write(fd,'password\n')
time.sleep(1)
output = os.read(fd,1024)
print output
os.write(fd,'ls\n')
output = os.read(fd,1024)
print output
Sample output:
user#host's password:
Last login: Wed Aug 24 03:16:57 2011 from 1x.x.x.xxxx
-bash: ulimit: open files: cannot modify limit: Operation not permitted
host: /home/user>ls
I'd suggest trying the module pexpect, which is built exactly for this sort of thing (interfacing with other applications via pseudo-TTYs), or Fabric, which is built for this sort of thing more abstractly (automating system administration tasks on remote servers using SSH).
pexpect: http://pypi.python.org/pypi/pexpect/
Fabric: http://docs.fabfile.org/en/1.11/
As already stated, better use public keys. As I use them normally, I have changed your program so that it works here.
#!/usr/bin/python2.6
import os,sys,time,thread
pid,fd = os.forkpty()
if pid == 0:
os.execv('/usr/bin/ssh',['/usr/bin/ssh','localhost',])
sys.exit(0)
else:
output = os.read(fd,1024)
print output
os.write(fd,'ls\n')
time.sleep(1) # this is new!
output = os.read(fd,1024)
print output
With the added sleep(1), I give the remote host (or, in my case, not-so-remote host) time to process the ls command and produce its output.
If you send ls and read immediately, you only read what is currently present. Maybe you should read in a loop or so.
Or you just should do it this way:
import subprocess
sp = subprocess.Popen(("ssh", "localhost", "ls"), stdout=subprocess.PIPE)
print sp.stdout.read()
I'm running a script from crontab that will just ssh and run a command and store the results in a file.
The function that seems to be failing is subprocess.popen.
Here is the python function:
def _executeSSHCommand(sshcommand,user,node):
'''
Simple function to execute an ssh command on a remote node.
'''
sshunixcmd = '/usr/bin/ssh %s#%s \'%s\'' % (user,node,sshcommand)
process = subprocess.Popen([sshunixcmd],
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
process.wait()
result = process.stdout.readlines()
return result
When it's run from the command line, it executes correctly, from cron it seems to fail with the error message below.
Here are the crontab entries:
02 * * * * /home/matt/scripts/check-diskspace.py >> /home/matt/logs/disklog.log
Here are the errors:
Sep 23 17:02:01 timmy CRON[13387]: (matt) CMD (/home/matt/scripts/check-diskspace.py >> /home/matt/logs/disklog.log)
Sep 23 17:02:01 timmy CRON[13386]: (CRON) error (grandchild #13387 failed with exit status 2)
I'm going blind trying to find exactly where I have gone so wrong. Any ideas?
The cron PATH is very limited. You should either set absolute path to your ssh /usr/bin/ssh or set the PATH as a first line in your crontab.
You probably need to pass ssh the -i argument to tell ssh to use a specific key file. The problem is that your environment is not set up to tell ssh which key to use.
The fact that you're using python here is a bit of a red herring.
For everything ssh-related in python, you might consider using paramiko. Using it, the following code should do what you want.
import paramiko
client = paramiko.SSHClient()
client.load_system_host_keys()
client.connect(node, username=user)
stdout = client.exec_command(ssh_command)[0]
return stdout.readlines()
When running python scripts from cron, the environment PATH can be a hangup, as user1652558 points out.
To expand on this answer with example code to add custom PATH values to the environment for a subprocess call:
import os
import subprocess
#whatever user PATH values you need
my_path = "/some/custom/path1:/some/custom/path2"
#append the custom values to the current PATH settings
my_env = os.environ.copy()
my_env["PATH"] = my_path + ":" + my_env["PATH"]
#subprocess call
resp = subprocess.check_output([cmd], env=my_env, shell=True)
I've just become the system admin for my research group's cluster and, in this respect, am a novice. I'm trying to make a few tools to monitor the network and need help getting started implementing them with python (my native tongue).
For example, I would like to view who is logged onto remote machines. By hand, I'd ssh and who, but how would I get this info into a script for manipulation? Something like,
import remote_info as ri
ri.open("foo05.bar.edu")
ri.who()
Out[1]:
hutchinson tty7 2009-08-19 13:32 (:0)
hutchinson pts/1 2009-08-19 13:33 (:0.0)
Similarly for things like cat /proc/cpuinfo to get the processor information of a node. A starting point would be really great. Thanks.
Here's a simple, cheap solution to get you started
from subprocess import *
p = Popen('ssh servername who', shell=True, stdout=PIPE)
p.wait()
print p.stdout.readlines()
returns (eg)
['usr pts/0 2009-08-19 16:03 (kakapo)\n',
'usr pts/1 2009-08-17 15:51 (kakapo)\n',
'usr pts/5 2009-08-17 17:00 (kakapo)\n']
and for cpuinfo:
p = Popen('ssh servername cat /proc/cpuinfo', shell=True, stdout=PIPE)
I've been using Pexpect, which let's you ssh into machines, send commands, read the output, and react to it, with success. I even started an open-source project around it, Proxpect - which haven't been updated in ages, but I digress...
The pexpect module can help you interface with ssh. More or less, here is what your example would look like.
child = pexpect.spawn('ssh servername')
child.expect('Password:')
child.sendline('ABCDEF')
(output,status) = child.sendline('who')
If your needs overgrow simple "ssh remote-host.example.org who" then there is an awesome python library, called RPyC. It has so called "classic" mode which allows to almost transparently execute Python code over the network with several lines of code. Very useful tool for trusted environments.
Here's an example from Wikipedia:
import rpyc
# assuming a classic server is running on 'hostname'
conn = rpyc.classic.connect("hostname")
# runs os.listdir() and os.stat() remotely, printing results locally
def remote_ls(path):
ros = conn.modules.os
for filename in ros.listdir(path):
stats = ros.stat(ros.path.join(path, filename))
print "%d\t%d\t%s" % (stats.st_size, stats.st_uid, filename)
remote_ls("/usr/bin")
If you're interested, there's a good tutorial on their wiki.
But, of course, if you're perfectly fine with ssh calls using Popen or just don't want to run separate "RPyC" daemon, then this is definitely an overkill.
This covers the bases. Notice the use of sudo for things that needed more privileges. We configured sudo to allow those commands for that user without needing a password typed.
Also, keep in mind that you should run ssh-agent to make this "make sense". But all in all, it works really well. Running deploy-control httpd configtest will check the apache configuration on all the remote servers.
#!/usr/local/bin/python
import subprocess
import sys
# The user#host: for the SourceURLs (NO TRAILING SLASH)
RemoteUsers = [
"deploy#host1.example.com",
"deploy#host2.appcove.net",
]
###################################################################################################
# Global Variables
Arg = None
# Implicitly verified below in if/else
Command = tuple(sys.argv[1:])
ResultList = []
###################################################################################################
for UH in RemoteUsers:
print "-"*80
print "Running %s command on: %s" % (Command, UH)
#----------------------------------------------------------------------------------------------
if Command == ('httpd', 'configtest'):
CommandResult = subprocess.call(('ssh', UH, 'sudo /sbin/service httpd configtest'))
#----------------------------------------------------------------------------------------------
elif Command == ('httpd', 'graceful'):
CommandResult = subprocess.call(('ssh', UH, 'sudo /sbin/service httpd graceful'))
#----------------------------------------------------------------------------------------------
elif Command == ('httpd', 'status'):
CommandResult = subprocess.call(('ssh', UH, 'sudo /sbin/service httpd status'))
#----------------------------------------------------------------------------------------------
elif Command == ('disk', 'usage'):
CommandResult = subprocess.call(('ssh', UH, 'df -h'))
#----------------------------------------------------------------------------------------------
elif Command == ('uptime',):
CommandResult = subprocess.call(('ssh', UH, 'uptime'))
#----------------------------------------------------------------------------------------------
else:
print
print "#"*80
print
print "Error: invalid command"
print
HelpAndExit()
#----------------------------------------------------------------------------------------------
ResultList.append(CommandResult)
print
###################################################################################################
if any(ResultList):
print "#"*80
print "#"*80
print "#"*80
print
print "ERRORS FOUND. SEE ABOVE"
print
sys.exit(0)
else:
print "-"*80
print
print "Looks OK!"
print
sys.exit(1)
Fabric is a simple way to automate some simple tasks like this, the version I'm currently using allows you to wrap up commands like so:
run('whoami', fail='ignore')
you can specify config options (config.fab_user, config.fab_password) for each machine you need (if you want to automate username password handling).
More info on Fabric here:
http://www.nongnu.org/fab/
There is a new version which is more Pythonic - I'm not sure whether that is going to be better for you int his case... works fine for me at present...