I tried to search for an answer for a while, but I did not find anything so far for my specific case. I want to run command in python:
ssh -o ConnectTimeout=3 -o ProxyCommand="ssh -q -W %h:%p bastion.host.com" host.com "screen -dmS TEST /bin/bash --login -c 'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log'"
this is my code:
import subprocess
server_command = "screen -dmS TEST /bin/bash --login -c 'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log'"
command = ['ssh', '-o', 'ConnectTimeout=3', 'ProxyCommand="ssh -q -W %h:%p bastion.host.com"', 'host.com', server_command]
p = subprocess.Popen(command, stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
stdout, stderr = p.communicate(input=None)
Everything was working (screen was spawned with script running) until I added option with spaces: ProxyCommand="ssh -q -W %h:%p bastion.host.com".
After that I get error:
>>> print(stderr)
b'ssh: Could not resolve hostname ProxyCommand="ssh -q -W %h:%p bastion.host.com": Name or service not known\r\n'
How can I please pass this option to my command?
Your SSH command contains invalid arguments: ProxyCommand is an option, so it needs to be preceded by -o, same as ConnectTimeout (and, as noted by Charles Duffy, the redundant quotes inside that option string need to be removed, since the command is not passed to the shell):
server_command = 'screen -dmS TEST /bin/bash --login -c \'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log\''
command = ['ssh', '-o', 'ConnectTimeout=3', '-o', 'ProxyCommand=ssh -q -W %h:%p bastion.host.com', 'host.com', server_command]
In general when your command line contains spaces and/or quotes and is passed to another command, it may be necessary to shell-quote it. The Python function shlex.quote automates this. In your case it’s not necessary because you (correctly) manually quoted the command you’re passing to screen inside server_command. Alternatively you could have written the following:
script_command = 'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log'
server_command = f'screen -dmS TEST /bin/bash --login -c {shlex.quote(script_command)}'
— Note the absence of manual quotes inside the shell command line. The advantage over manual quoting is that this will also work with nested levels of shell quoting, e.g. when nesting command invocations.
Related
I have one issue with subprocess.run.
This command in a Bash shell works without any problem:
tar -C '/home/' --exclude={'/home/user1/.cache','/home/user1/.config'} -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1
But if I execute it through Python:
cmd = "tar -C '/home/' --exclude={'/home/user1/.cache','/home/user1/.config'} -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1"
subprocess.run(cmd, shell=True, stdout=subprocess.PIPE)
The execution works without errors but the --exclude clause is not considered.
Why?
Whether or not curly brace expansion is handled correctly depends on what the standard system shell is. By default, subprocess.run() invokes /bin/sh. On systems like Linux, /bin/sh is bash. On others, such as FreeBSD, it's a different shell that doesn't support brace expansion.
To ensure the subprocess runs with a shell that can handle braces properly, you can tell subprocess.run() what shell to use with the executable argument:
subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, executable='/bin/bash')
As a simple example of this, here's a system where /bin/sh is bash:
>>> subprocess.run("echo foo={a,b}", shell=True)
foo=a foo=b
and one where it's not:
>>> subprocess.run("echo foo={a,b}", shell=True)
foo={a,b}
but specifying another shell works:
>>> subprocess.run("echo foo={a,b}", shell=True, executable='/usr/pkg/bin/bash')
foo=a foo=b
Bash curly expansion doesn't work inside Python and will be sent by subprocess as they are - they will not be expanded, regardless of the arguments you use on run().
Edit: unless of course the argument executable='/bin/bash' as stated on the other answer which seems to work after all
In a bash shell,
--exclude {'/home/user1/.cache','/home/user1/.config'}
becomes:
--exclude=/home/user1/.cache --exclude=/home/user1/.config
So to achieve the same result, in Python it must be expressed like this (one of the possible ways) before sending the command string to subprocess.run:
' '.join(["--exclude=" + path for path in ['/home/user1/.cache','/home/user1/.config']])
cmd = "tar -C '/home/' " + ' '.join(["--exclude=" + path for path in ['/home/user1/.cache','/home/user1/.config']]) + " -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1"
print(cmd) # output: "tar -C '/home/' --exclude=/home/user1/.cache --exclude=/home/user1/.config -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1"
subprocess.run(cmd, shell=True, stdout=subprocess.PIPE)
I have a bash script that takes two parameters. Inside that script, I need to call ssh using a heredoc and call a method that expects the two arguments. For example:
ssh -o "IdentitiesOnly=yes" -t -i $key -l user localhost << 'ENDSSH'
/my_python_app.py -u -t tar -p $1 -f $2
ENDSSH
key is set by my script, I know that part is good.
However, my_python_app prints out args and it doesn't show any arguments for -p and -f
I would call my script like
my_script /tmp filename
I use argparse in my python app, but I am also printing out sys.argv and it gives me:
['my_python_app.py', '-u', '-t', 'tar', '-p', '-f']
Note there are no values received for -p and -f. (-u is a flag, and that is set correctly).
How do I pass $1 and $2 to my_python_app as the -p and -f values?
Remove the quotes around the here-document delimiter (i.e. use << ENDSSH instead of << 'ENDSSH'). The quotes tell the shell not to expand variable references (and some other things) in the here-document, so $1 and $2 are passed through to the remote shell... which doesn't have any parameters so it replaces them with nothing.
BTW, removing the single-quotes may not fully work, since if either argument contains whitespace or shell metacharacters, the remote end will parse those in a way you probably don't intend. As long as neither argument can contain a single-quote, you can use this:
ssh -o "IdentitiesOnly=yes" -t -i $key -l user localhost << ENDSSH
/my_python_app.py -u -t tar -p '$1' -f '$2'
ENDSSH
If either might contain single-quotes, it gets a little more complicated.
The more paranoid way to do this would be:
# store these in an array to reduce the incidental complexity below
ssh_args=( -o "IdentitiesOnly=yes" -t -i "$key" -l user )
posixQuote() {
python -c 'import sys, pipes; sys.stdout.write(pipes.quote(sys.argv[1])+"\n")' "$#"
}
ssh "${ssh_args[#]}" localhost "bash -s $(posixQuote "$1") $(posixQuote "$2")" << 'ENDSSH'
/path/to/my_python_app.py -u -t tar -p "$1" -f "$2"
ENDSSH
If you know with certainty that the destination account's shell matches the local one (bash if the local shell is bash, ksh if the local shell is ksh), consider the following instead:
printf -v remoteCmd '%q ' /path/to/my_python_app.py -u -t tar -p "$1" -f "$2"
ssh "${ssh_args[#]}" localhost "$remoteCmd"
I need to integrate " echo /bin/meteo | at 23:00 today " in to a python script.
In the python script the command "at 23:00 today" should call the bash script /bin/meteo
I did install plumbum and intergrated this in my python scrip.
from plumbum.cmd import echo, grep
Unfortunately I have no clue how to proceed from here.
I tryed:
#!/usr/bin/python2.7
if pfd.input_pins[0].value ==0:
cmd = "echo /bin/meteo | at 06:36 today"
subprocess.Popen(cmd, shell=True)
but the lights in /bin/meteo are randomly swiching on and off (not blinking as they should)
They do it from 06:36 until 06:37 and not only 5 times.
/bin/meteo:
#!/bin/bash -x
for i in {1..5}; do #blink 5x
echo -n -e "\x37\x00\x55" | nc -u -q 1 192.168.0.6 8899 #Zone 3 on
sleep 0.1
echo -n -e "\x3A\x00\x55" | nc -u -q 1 192.168.0.6 8899 #Zone 3 off
done
sleep 0.1
exit
subprocess.Popen will run the command:
import subprocess
cmd = "echo /bin/meteo | at 23:00 today "
subprocess.Popen(cmd, shell=True)
Execute a child program in a new process. On Unix, the class uses os.execvp()-like behavior to execute the child program. On Windows, the class uses the Windows CreateProcess() function. The arguments to Popen are as follows.
args should be a sequence of program arguments or else a single string. By default, the program to execute is the first item in args if args is a sequence. If args is a string, the interpretation is platform-dependent and described below. See the shell and executable arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass args as a sequence.
It is not totally clear what you want but you can run any commands like:
In [9]: cmd = "date"
In [10]: subprocess.call(cmd, shell=True)
Sun Jul 6 22:30:47 IST 2014
Or using sudo:
import subprocess
cmd = "sudo which python"
my_pass="xxxx"
subprocess.call('echo {} | sudo -S {}'.format(my_pass,cmd), shell=True)
In [29]: subprocess.call('echo {} | sudo -S {}'.format(my_pass,cmd), shell=True)
/usr/local/bin/python
Out[29]: 0
With Python 3.4, it's easy to call a command and exchange input/output in a bulk:
subprocess.check_output(["at", "23:00", "today"], input="/bin/meteo")
Therefore in this very case, shell=True shouldn't be needed as we just call the at command with arguments and give it the script on input.
With older versions of python, this needs to be rewritten as:
process = subprocess.Popen(["at", "23:00", "today"])
process.communicate(input="/bin/meteo")
With the plumbum module, you could instead use:
from plumbum.cmd import at, echo
(echo["/bin/meteo"] | at["23:30", "today"])()
But I don't believe that it's very useful.
I have a very long string ssh_cmd, I get it from
cmd = """kill -9 `ps -ef|grep "udp_receiver"|grep -v "grep"|awk '{print $2}'`"""
HostName="133.33.22.1"
ssh_cmd = """ssh -t inria_spoofing#{0} 'sudo nohup bash -c "{1} > /nohup.out 2>&1 &"'""".format(HostName, cmd)
the resulted ssh_cmd is:
ssh -t kitty#133.33.22.1 'sudo nohup bash -c "kill -9 `ps -ef|grep "udp_receiver"|grep -v "grep"|awk '{print $2}'` > /nohup.out 2>&1 &"'
however, I'm afraid when I run
child = pexpect.spawn(ssh_cmd)
there is problem，
so how to organize the string?
thanks!
To answer the question, here's the proper ssh_cmd: ssh -t kitty#133.33.22.1 "sudo nohup bash -c \"kill -9 \\\`ps -ef | grep 'udp_receiver' | grep -v 'grep' | awk '{print \\\$2}'\\\` > /nohup.out 2>&1 &\""
Basically, you need to escape double quotes, backticks and backslashes in a command each time you embed this command in another one. I did not use single quotes except at the lower level because you cannot use escaped single quotes inside single quotes.
You do need to escape the $ too when it is just a character inside a string quoted with double quotes, even if the string does also contain single quotes.
Hi I have to execute a shell command :diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)
I tried
cmd="diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)"
args = shlex.split(cmd)
output,error = subprocess.Popen(args,stdout = subprocess.PIPE, stderr= subprocess.PIPE).communicate()
However I am getting an error diff: extra operand cat
I am pretty new to python. Any help would be appreciated
You are using <(...) (process substitution) syntax, which is interpreted by the shell. Provide shell=True to Popen to get it to use a shell:
cmd = "diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)"
output,error = subprocess.Popen(cmd, shell=True, executable="/bin/bash", stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
Since you don't want the Bourne shell (/bin/sh), use the executable argument to determine the shell to use.
You are using a special syntax called process substitiution in your command line. This is supported by most modern shells (bash, zsh), but not by /bin/sh. Therefore, the method suggested by Ned might not work. (It could, if another shell provides /bin/sh and does not "correctly emulate" sh's behaviour, but it is not guaranteed to).
try this instead:
cmd = "diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)"
output,error = subprocess.Popen(['/bin/bash', '-c', cmd], stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
This is basically what the shell=True parameter does, but with /bin/bash instead of /bin/sh (as described in the subprocess docs).