I have used the subprocess module in Python 2.7.6 to establish an SSH. I realise that this is not recommended, but I am unable to install other Python SSH libraries such as paramiko and fabric.
I was just wondering if someone wouldn't mind just telling me how I'd now go about
sshProcess = subprocess.call(['ssh', '-t', '<REMOTE>', 'ssh', '<REMOTE>'])
I want to carry out commands in REMOTE with the subprocess approach. Is there any way to do this? Unfortunately, REMOTE is protected by a password which the user manually enters. If it helps, I'm running the Windows 10 Bash shell.
Any help is appreciated.
Running a remote command is as simple as putting it on the command line. (This is distinguishable to the SSH server at a protocol level from feeding it on stdin, but the protocol in question is built for programmatic use, vs built for human use -- as the latter was the design intent behind the interactive-shell model).
By the way, if you want to run multiple commands via distinct SSH invocations over a single connection after authenticating only once, I'd strongly suggest using Paramiko for this, but you can do it with OpenSSH command-line tools by using SSH multiplexing support.
Let's say you have an array representing your remote command:
myCommand = [ 'ls', '-l', '/tmp/my directory name with spaces' ]
To get that into a string (in a way that honors the spaces and can't let a maliciously-selected name run arbitrary commands on the remote server), you'd use:
myCommandStr = ' '.join(pipes.quote(n) for n in myCommand)
Now, you have something you can pass as a command line argument to ssh:
subprocess.call(['ssh', '-t', hostname, myCommandStr])
However, let's say you want to nest this. You can just repeat the process:
myCommand = [ 'ssh', '-t', hostname1, myCommandStr ]
myCommandStr = ' '.join(pipes.quote(n) for n in myCommand)
subprocess.call(['ssh', '-t', hostname2, myCommandStr])
Because we aren't redirecting stdin or stdout, they should still be pointed at the terminal from which your Python program was started, so SSH should be able to execute its password prompts directly.
That said, specifically for ssh'ing through an interim system, you don't need to go through this much trouble: You can tell ssh to do that work for you with the ProxyJump option:
myCommand = [ 'ls', '-l', '/tmp/my directory name with spaces' ]
myCommandStr = ' '.join(pipes.quote(n) for n in myCommand)
subprocess.call(['ssh', '-o', 'ProxyJump=%s' % hostname1, hostname2, myCommandStr])
From your comment, you say you can connect. So after that, to interact over ssh using subprocess you will need something like:
ssh = subprocess.Popen(['ssh', <remote client>],
stdin=subprocess.PIPE,
stdout = subprocess.PIPE)
back = ssh.stdout.readlines()
if result == []:
error = ssh.stderr.readlines()
print error
else:
print back
then to send commands, like list directory, something like:
ssh.stdin.write("ls\n")
Related
I am trying to execute remote command using subprocess:
import subprocess
x=subprocess.Popen(['ssh','15.24.13.14', ' ps -ef | grep -i upgrade | wc -l'],stdout=subprocess.PIPE)
y=x.stdout.read()
print y
print '\n'
z=int(y)
print z
I need to get number of processes runing with 'upgrade' in their name. But for some reason, script is not executed well. I get message:
"Warning: Permanently added '15.24.13.14' (RSA) to the list of known hosts."
And then nothing happens.
Where is the problem?
The problem is that if you are connecting for the first time via ssh to the given host, it asks you to add this host to the known hosts list and user has to confirm this by pressing 'y'. Since you didn't, than it hangs and does nothing.
You should either:
turn off host verification: ssh -o "StrictHostKeyChecking no" user#host
send 'y' to the ssh input, or
add this host manually to the known hosts, or
change the method of performing remote calls.
Because you didn't specify any stderr to the subprocess.Popen, the standard error will be directly print to your display. This is why you will always have the Warning: Permanently added '<hostname>' (ECDSA) to the list of known hosts. message until you clearly redirect stderr to a subprocess.PIPE (or /dev/null)
Also, to avoid hosts file issues, here is a little trick (be careful with it, it's kind of dangerous) :
from subprocess import Popen, PIPE
p = Popen(['ssh', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', hostname, 'ps aux | grep -i upgrade | wc -l'], stdout=PIPE, stderr=PIPE)
result = int(p.communicate()[0][:-1]) # don't forget there's the \n at the end.
Why is it dangerous ? Because in case of MITM attack, you don't have any knowledge base of the remote, so you considere the attacker as your remote destination. Be careful about over-using this feature.
Python subprocess.popen is easy as pie on local machine, but is it possible to call it over a network?
Example, say I have 3 PCs, one is called workstation-pc, the others are called node1-pc and node2-pc...
Is it possible to call a process on, say, node1-pc, from workstation-pc, preferably without having to run special server software on node1-pc?
In any case, many thanks for any response!
Gilles
EDIT
Forgot to mention that I am using Python 3
I would recommend the use of either:
execnet
fabric
Example with Fabric:
from fabric.api import env, run
env.hosts = ['host1', 'host2']
def mytask():
run('ls /var/www')
Fabric is probably the way to go, however, there is nothing to stop you executing remote commands via ssh in *nix environments.
import subprocess
p = subprocess.Popen(['ssh', 'node1-pc', 'ls', '-ltr', '/etc'], stdout=subprocess.PIPE)
out, err = p.communicate()
out will contain the stdout of the remote process.
I've searched around for quite a bit, finding pieces of what I wish to achieve but not fully. I'm making a sync-script to synchronize files between two machines. The script itself is somewhat more advanced than this question (it provides possibility for both sides to request for file deletion and so on, no "master side").
First question
The following bash-command works for me:
rsync -rlvptghe 'sshpass -p <password> ssh -p <port>' <source> <destination>
how can I translate it into a python command to be used with the subprocess object?
I've managed to get the following python to work:
pw = getpass.getpass("Password for remote host: ")
command = ['sshpass', '-p', pw, 'rsync', '-rlvptgh', source, destination]
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while p.poll() is None:
out = p.stdout.read(1)
sys.stdout.write(out)
sys.stdout.flush()
but it doesn't specify port (it uses standard 22, I want another one). To clarify, I wish to use similar code as this one but with the support for a specific port as well.
I have already tried to change the command to:
command = ['sshpass', '-p', pw, 'rsync', '-rlvptghe', 'ssh', '-p', '2222', source, destination]
which gives the following error:
ssh: illegal option -- r
and also many other variations such as for instance:
command = ['rsync', '-rlvptghe', 'sshpass', '-p', pw, 'ssh', '-p', '2222', source, destination]
Which gives the following error (where <source> is the remote host source host to sync from, ie variable source above command declaration):
Unexpected remote arg: <source>
How should I specify this command to nest them according to my first bash command?
Second question
When I've done all my searching I've found lots of frowning upon using a command containing the password for scp/rsync (ie ssh), which I use in my script. My reasoning is that I want to be prompted for a password when I do the synchronization. It is done manually since it gives feedback on filesystem modifications and other things. However, since I do 2 scp and 2 rsync calls I don't want to type the same password 4 times. That is why I use this approach and let python (the getpass module) collect the password one time and then use it for all the 4 logins.
If the script was planned for an automated setup I would of course use certificates instead, I would not save the password in clear text in a file.
Am I still reasoning the wrong way about this? Are there things I could do to strengthen the integrity of the password used? I've already realized that I should suppress errors coming from the subprocess module since it might display the command with the password.
Any light on the problem is highly appreciated!
EDIT:
I have updated question 1 with some more information as to what I'm after. I also corrected a minor copy + paste error in the python code.
Edit 2 explained further that I do have tried the exact same order as the first bash command. That was the first I tried. It doesn't work. The reason for changing the order was because it worked with another order (sshpass first) without specifying port.
I have found one way to solve this for my own needs. It includes invoking a shell to handle the command, which I avoided in the first place. It works for me though, but might not be satisfactory to others. It depends on the environment you want to run the command in. For me this is more or less an extension of the bash shell, I want to do some other things that are easier in python and at the same time run some bash commands (scp and rsync).
I'll wait for a while and if there's no better solution than this I will mark my answer as the answer.
A basic function for running rsync via python with password and port could be:
def syncFiles(pw, source, destination, port, excludeFile=None, dryRun=False, showProgress=False):
command = 'rsync -rlvptghe \'sshpass -p ' + pw + ' ssh -p ' + port + '\' ' + source + ' ' + destination
if excludeFile != None:
command += ' --exclude-from='+excludeFile
if dryRun:
command += ' --dry-run'
if showProgress:
command += ' --progress'
p = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
while p.poll() is None:
out = p.stdout.read(1)
sys.stdout.write(out)
sys.stdout.flush()
The reason this works is as I wrote because the invoked bash shell handles the command instead. This way I can write the command exactly as I would directly in a shell. I still don't know how to do this without shell=True.
Note that the password is collected from the user with the getpass module:
pw = getpass.getpass("Password for current user on remote host: ")
It is not recommended to store your password in the python file or any other file. If you are looking for an automated solution it is better to use private keys. Answers for such solutions can be found by searching.
To call the scp-command with password the following python should do:
subprocess.check_output(['sshpass', '-p', pw, 'scp', '-P', port, source, destination])
I hope this can be useful to someone who wants to achieve what I am doing.
I am creating a movie controller (Pause/Stop...) using python where I ssh into a remote computer, and issue commands into a named pipe like so
echo -n q > ~/pipes/pipename
I know this works if I ssh via the terminal and do it myself, so there is no problem with the setup of the named pipe redirection. My problem is that setting up an ssh session takes time (1-3 seconds), whereas I want the pause command to be instantaneous. Therefore, I thought of setting up a persistent pipe like so:
controller = subprocess.Popen ( "ssh -T -x <hostname>", shell = True, close_fds = True, stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE )
Then issue commands to it like so
controller.stdin.write ( 'echo -n q > ~/pipes/pipename' )
I think the problem is that ssh is interactive so it expects a carriage return. This is where my problems begin, as nearly everyone who has asked this question has been told to use an existing module:
Vivek's answer
Chakib's Answer
shx2's Answer
Crafty Thumber's Answer
Artyom's Answer
Jon W's Answer
Which is fine, but I am so close. I just need to know how to include the carriage return, otherwise, I have to go learn all these other modules, which mind you is not trivial (for example, right now I can't figure out how pexpect uses either my /etc/hosts file or my ssh keyless authentications).
To add a newline to the command, you will need to add a newline to the string:
controller.stdin.write('\n')
You may also need to flush the pipe:
controller.stdin.flush()
And of course the controller has to be ready to receive new data, or you could block forever trying to send it data. (And if the reason it's not ready is that it's blocking forever waiting for you to read from its stdout, which is possible on some platforms, you're deadlocked unrecoverably.)
I'm not sure why it's not working the way you have it set up, but I'll take a stab at this. I think what I would do is change the Popen call to:
controller = subprocess.Popen("ssh -T -x <hostname> \"sh -c 'cat > ~/pipes/pipename'\"", ...
And then simply controller.stdin.write('q').
So I want to use python to run a command on a server, but to do that I have to go through another box. I connect to the first box by running ssh through subprocess. However, Im unsure how to then get into the second box and run commands through that subprocess object.
My subprocess statement:
command = "ssh servername"
sshConnection = subprocess.Popen(command.split(),stderr=subprocess.PIPE
,stdout=subprocess.PIPE,stdin=subprocess.PIPE)
The only methods I can think of:
Somehow connect the pipes of two subprocess commands?
Establish a SSH tunnel first then run commands through that?
Any other ideas? Am I approaching this wrong?
If, by "go through another box", you mean "initiate an ssh connection from the jumpbox to the server", try:
command = "ssh jumpbox ssh servername /bin/ls -l"
sshConnection = subprocess.Popen(command.split(),stderr=subprocess.PIPE
,stdout=subprocess.PIPE,stdin=subprocess.PIPE)
How about:
cmd = "mycmd myarg1 myarg2"
ssh_command = "ssh servername %s" % cmd
sshConnection = subprocess.Popen(ssh_command.split(),stderr=subprocess.PIPE
,stdout=subprocess.PIPE,stdin=subprocess.PIPE)
Of course you need to also shell-escape the command, if it contains special characters.
A slicker alternative, I personally like, is using plumbum. You can nest remote (ssh) commands.
As you are doing a lot of ssh with subprocess I suggest you should use Fabric