How can I execute a command on a remote server in Python and pipe the stdout to a local command? To do ssh host 'echo test' | cat in Python, I have tried
import paramiko
import subprocess
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('host', username='user')
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command('echo test')
proc = subprocess.Popen(['cat'], stdin=ssh_stdout)
outs, errs = proc.communicate()
print(outs)
but I get the exception 'ChannelFile' object has no attribute 'fileno'. It seems that Paramiko's ssh_stdout can't be used as stdin with subprocess.Popen.
Yes, subprocess cannot redirect output on a "fake" file. It needs fileno which is defined only with "real" files (io.BytesIO() doesn't have it either).
I would do it manually like the following code demonstrates:
proc = subprocess.Popen(['cat'], stdin=subprocess.PIPE)
proc.stdin.write(ssh_stdout.read())
proc.stdin.close()
so you're telling Popen that the input is a pipe, and then you write ssh output data in the pipe (and close it so cat knows when it must end)
According to the docs, the ChannelFile object does not directly wrap an actual file descriptor (because of the decryption and demuxing and so on that occurs within SSH), so it can't directly be used as a file descriptor for Popen.
Something like
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command('echo test')
proc = subprocess.Popen(['cat'], stdin=subprocess.PIPE)
while proc.poll() is not None: # (fixed)
buf = ssh_stdout.read(4096)
if not buf:
break
proc.stdin.write(buf)
might work; i.e. you read the SSH stdout stream manually, up to 4096 bytes at a time, and write them to the subprocess's stdin pipe.
I'm not sure how the code above will behave when the remote command exits, though, so YMMV.
Related
would like to open an ssh session, run commands and get the output real-time as the process runs (this base will involve running additional commands on the remote server)
from subprocess import Popen, PIPE
with Popen(['ssh <server-domain-name>',
],shell=True,
stdin=PIPE, stdout=PIPE, stderr=PIPE,
universal_newlines=True) as ssh:
output1 = ssh.stdin.write('ls -l')
output2 = ssh.stdin.write('mkdir test')
status = ssh.poll()
print(output1)
print(output2)
so far this is what I have, using ssh.communicate[<command>] gives the right output but closes the subproceess after the first command, any thoughts?
worked for me
from fabric2 import Connection
with Connection('<host>') as c:
print(CGREEN +'connected succsfully!' + CEND)
#gather user info
user = io.StringIO
user = c.run("whoami", hide=True)
print(f'user found:{user.stdout} ')
#fetching files
c.run(<command>, pty=True)
I start a script with:
t = subprocess.Popen('rosrun ros_pkg ros_node', shell=True,
stdout = subprocess.PIPE,
stdin = subprocess.PIPE, universal_newlines=True)
I then want to communictate with that process like this:
stdout = t.communicate('new command')[0]
print(stdout)
if stdout == []:
logic
stdout = t.communicate('new command')[0]
....
The problem is that after t.commincate the subprocess closes
There are solutions for similar problems but nothing worked for me yet please help
Using t.communicate() will close the input pipe after sending the data, meaning it can only be called once in order to send something to the subprocess.
However you can use t.stdin.write() to do sequential writes without closing the pipe and then use t.stdin.readline() to get the output. These work the same way the handler returned by open() does.
import subprocess
t = subprocess.Popen("cat", stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True, shell=True)
#These calls will have their results written to the output pipe
t.stdin.write('hi there ')
t.stdin.write('hi again\n')
#However the input data is buffered first, so call flush() before reading
t.stdin.flush()
#Read from the pipe
a = t.stdout.readline()
print(a)
t.stdin.write('hi back at you\n')
t.stdin.flush()
a = t.stdout.readline()
print(a)
I now switched from using subprocess to using pexpect.
My syntax is now as follows:
child = pexpect.spawn('rosrun ros_pkg ros_node')
command = child.sendline('new command')
output = child.read_nonblocking(10000, timeout=1)
....
logic
....
command = child.sendline('new command')
output = child.read_nonblocking(10000, timeout=1)
Many thanks to novel_yet_trivial on reddit: https://www.reddit.com/r/learnpython/comments/2o2viz/subprocess_popen_multiple_times/
I am trying to write to a custom program's stdin with paramiko. Here is a minimal (non-)working example:
~/stdin_to_file.py:
#! /usr/bin/python
import time, sys
f = open('/home/me/LOG','w')
while True:
sys.stdin.flush()
data = sys.stdin.read()
f.write(data+'\n\n')
f.flush()
time.sleep(0.01)
Then I do these commands in IPython:
import paramiko
s = paramiko.client.SSHClient
s.load_system_host_keys()
s.connect('myserver')
stdin, stdout, stderr = s.exec_command('/home/me/stdin_to_file.py')
stdin.write('Hello!')
stdin.flush()
Unfortunately, nothing then appears in ~/LOG. However, if I do
$ ~/stdin_to_file.py < some_other_file
The contents of some_other_file appear in ~/LOG.
Can anyone suggest where I've gone wrong? It seems like I'm doing the logical thing. None of these work either:
stdin.channel.send('hi')
using the get_pty parameter
sending the output of cat - to stdin_to_file.py
sys.stdin.read() will keep reading until EOF so in your paramiko script you need to close the stdin (returned from exec_command()). But how?
1. stdin.close() would not work.
According to Paramiko's doc (v1.16):
Warning: To correctly emulate the file object created from a socket’s makefile() method, a Channel and its ChannelFile should be able to be closed or garbage-collected independently. Currently, closing the ChannelFile does nothing but flush the buffer.
2. stdin.channel.close() also has problem.
Since stdin, stdout and stderr all share one single channel, stdin.channel.close() will also close stdout and stderr which is not expected.
3. stdin.channel.shutdown_write()
The correct solution is to use stdin.channel.shutdown_write() which disallows writing to the channel but still allows reading from the channel so stdout.read() and stderr.read() would still work.
See following example to see the difference between stdin.channel.close() and stdin.channel.shutdown_write().
[STEP 101] # cat foo.py
import paramiko, sys, time
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy() )
ssh.connect(hostname='127.0.0.1', username='root', password='password')
cmd = "sh -c 'read v; sleep 1; echo $v'"
stdin, stdout, stderr = ssh.exec_command(cmd)
if sys.argv[1] == 'close':
stdin.write('hello world\n')
stdin.flush()
stdin.channel.close()
elif sys.argv[1] == 'shutdown_write':
stdin.channel.send('hello world\n')
stdin.channel.shutdown_write()
else:
raise Exception()
sys.stdout.write(stdout.read() )
[STEP 102] # python foo.py close # It outputs nothing.
[STEP 103] # python foo.py shutdown_write # This works fine.
hello world
[STEP 104] #
I have a custom input method and I have a python module to communicate with it. I'm trying to control the shell with it so everything from local stdout is printed on the remote device and everything sent from the remote device goes into local stdin, so that remote device can control the input given to the program, like if there was an input function inside the program the remote device can answer to that too (like in ssh).
I used python subprocess to control the stdin and stdout:
#! /usr/bin/python
from subprocess import Popen, PIPE
import thread
from mymodule import remote_read, remote_write
def talk2proc(dap):
while True:
try:
remote_write(dap.stdout.read())
incmd = remote_read()
dap.stdin.write(incmd)
except Exception as e:
print (e)
break
while True:
cmd = remote_read()
if cmd != 'quit':
p = Popen(['bash', '-c', '"%s"'%cmd], stdout=PIPE, stdin=PIPE, stderr=PIPE)
thread.start_new_thread(talk2proc, (p,))
p.wait()
else:
break
But it doesn't work, what should I do?
p.s.
is there a difference for windows?
I had this problem, I used this for STDIN
from subprocess import call
call(['some_app', 'param'], STDIN=open("a.txt", "rb"))
a.txt
:q
This I used for a git wrapper, this will enter the data line wise whenever there is an interrupt in some_app that is expecting and user input
There is a difference for Windows. This line won't work in Windows:
p = Popen(['bash', '-c', '"%s"'%cmd], stdout=PIPE, stdin=PIPE, stderr=PIPE)
because the equivalent of 'bash' is 'cmd.exe'.
When I use subprocess I can normally capture the stdout and display it however I like. E.g,
import subprocess
proc = subprocess.Popen(['./foo.py'], stdin=subprocess.Pipe, stdout=subprocess.Pipe)
# the standard out is not displayed unless I do something with the stdout var
stdout, stderr = proc.communicate()
However, if I use subprocess to call the Linux passwd utility, the standard out is displayed as soon as proc.communicate() is called:
import subprocess
proc = subprocess.Popen(['passwd', 'foo'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
# standard out is displayed immediately
stdout, stderr = proc.communicate('password\npassword\n')
BAD PASSWORD: it is based on a dictionary word
Retype new password:
How come this happens only with passwd? For example, it doesn't happen with ls. Is there anything I can do to prevent the standard out from being printed when calling passwd from subprocess?
Note that I want to actually capture the standard out and do something with it later, so I would not want to set stdout to a devnull pipe.
It only happens with passwd because passwd directly communicates with the TTY, not via stdin or stdout. This is a security measure, and accepted best practice for prompting for a password directly from a user.
If you really must bypass this security measure, consider using the unbuffer utility (shipped with expect) to create a fake TTY:
p = subprocess.Popen(['unbuffer', 'passwd', 'foo'],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(stdout, stderr) = p.communicate('password\npassword\n')