If I am running the command that is specified in the args on the terminal then it goes successfully on terminal but doing the same in python program is not working; I am seeing junk characters in the screen to the size of the input tar file and lot of xterm words too;
I feel the problem is handling the ' ' letters in the args;
import subprocess
try:
args = "cat parsing.tgz <(echo -n ''| gzip)> new-file.tgz".split()
subprocess.check_call(args)
except subprocess.CalledProcessError as e:
print e
I am not specialist, but this i found - this commands not working in sh, but working in bash:
$ sh -c "cat parsing.tgz <(echo -n ''| gzip)> new-file.tgz"
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `cat parsing.tgz <(echo -n ''| gzip)> new-file.tgz'
$
$ bash -c "cat parsing.tgz <(echo -n ''| gzip)> new-file.tgz"
$
Thats a reason why it not work in subprocess directly. This code looks work fine:
import subprocess
command = "cat parsing.tgz <(echo -n ''| gzip)> new-file.tgz"
subprocess.Popen(command, shell=True, executable='/bin/bash')
I tried a number of alternatives, and none of them were satisfactory. The best I found was switching over to Popen.
# this should have the a similar signature to check_call
def run_in_shell(*args):
# unfortunately, `args` won't be escaped as it is actually a string argument to bash.
proc = subprocess.Popen(['/bin/bash', '-c', ' '.join(args)])
# This will also work, though I have found users who had problems with it.
# proc = subprocess.Popen(' '.join(args), shell=True, executable='/bin/bash')
stat = proc.wait()
if stat != 0:
subprocess.CalledProcessError(returncode=stat, cmd=command)
return stat
run_in_shell("cat parsing.tgz <(echo -n ''| gzip)> new-file.tgz")
As a note: /bin/sh has problems with the unescaped parentheses. If you don't want to specify '/bin/bash' above, then you will need to escape the paren:
args = 'cat parsing.tgz <\\(echo -n ''| gzip\\)> new-file.tgz'
Related
I have one issue with subprocess.run.
This command in a Bash shell works without any problem:
tar -C '/home/' --exclude={'/home/user1/.cache','/home/user1/.config'} -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1
But if I execute it through Python:
cmd = "tar -C '/home/' --exclude={'/home/user1/.cache','/home/user1/.config'} -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1"
subprocess.run(cmd, shell=True, stdout=subprocess.PIPE)
The execution works without errors but the --exclude clause is not considered.
Why?
Whether or not curly brace expansion is handled correctly depends on what the standard system shell is. By default, subprocess.run() invokes /bin/sh. On systems like Linux, /bin/sh is bash. On others, such as FreeBSD, it's a different shell that doesn't support brace expansion.
To ensure the subprocess runs with a shell that can handle braces properly, you can tell subprocess.run() what shell to use with the executable argument:
subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, executable='/bin/bash')
As a simple example of this, here's a system where /bin/sh is bash:
>>> subprocess.run("echo foo={a,b}", shell=True)
foo=a foo=b
and one where it's not:
>>> subprocess.run("echo foo={a,b}", shell=True)
foo={a,b}
but specifying another shell works:
>>> subprocess.run("echo foo={a,b}", shell=True, executable='/usr/pkg/bin/bash')
foo=a foo=b
Bash curly expansion doesn't work inside Python and will be sent by subprocess as they are - they will not be expanded, regardless of the arguments you use on run().
Edit: unless of course the argument executable='/bin/bash' as stated on the other answer which seems to work after all
In a bash shell,
--exclude {'/home/user1/.cache','/home/user1/.config'}
becomes:
--exclude=/home/user1/.cache --exclude=/home/user1/.config
So to achieve the same result, in Python it must be expressed like this (one of the possible ways) before sending the command string to subprocess.run:
' '.join(["--exclude=" + path for path in ['/home/user1/.cache','/home/user1/.config']])
cmd = "tar -C '/home/' " + ' '.join(["--exclude=" + path for path in ['/home/user1/.cache','/home/user1/.config']]) + " -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1"
print(cmd) # output: "tar -C '/home/' --exclude=/home/user1/.cache --exclude=/home/user1/.config -caf '/transito/user1.tar' '/home/user1' > /dev/null 2>&1"
subprocess.run(cmd, shell=True, stdout=subprocess.PIPE)
I tried to search for an answer for a while, but I did not find anything so far for my specific case. I want to run command in python:
ssh -o ConnectTimeout=3 -o ProxyCommand="ssh -q -W %h:%p bastion.host.com" host.com "screen -dmS TEST /bin/bash --login -c 'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log'"
this is my code:
import subprocess
server_command = "screen -dmS TEST /bin/bash --login -c 'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log'"
command = ['ssh', '-o', 'ConnectTimeout=3', 'ProxyCommand="ssh -q -W %h:%p bastion.host.com"', 'host.com', server_command]
p = subprocess.Popen(command, stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
stdout, stderr = p.communicate(input=None)
Everything was working (screen was spawned with script running) until I added option with spaces: ProxyCommand="ssh -q -W %h:%p bastion.host.com".
After that I get error:
>>> print(stderr)
b'ssh: Could not resolve hostname ProxyCommand="ssh -q -W %h:%p bastion.host.com": Name or service not known\r\n'
How can I please pass this option to my command?
Your SSH command contains invalid arguments: ProxyCommand is an option, so it needs to be preceded by -o, same as ConnectTimeout (and, as noted by Charles Duffy, the redundant quotes inside that option string need to be removed, since the command is not passed to the shell):
server_command = 'screen -dmS TEST /bin/bash --login -c \'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log\''
command = ['ssh', '-o', 'ConnectTimeout=3', '-o', 'ProxyCommand=ssh -q -W %h:%p bastion.host.com', 'host.com', server_command]
In general when your command line contains spaces and/or quotes and is passed to another command, it may be necessary to shell-quote it. The Python function shlex.quote automates this. In your case it’s not necessary because you (correctly) manually quoted the command you’re passing to screen inside server_command. Alternatively you could have written the following:
script_command = 'yes | script.sh --option-1 value1 -option2 value2 2>&1 | tee output.log'
server_command = f'screen -dmS TEST /bin/bash --login -c {shlex.quote(script_command)}'
— Note the absence of manual quotes inside the shell command line. The advantage over manual quoting is that this will also work with nested levels of shell quoting, e.g. when nesting command invocations.
How can I pass a python variable into the gnome-terminal command option ? I want to open multiple terminal with a specific command.
Here is the code (an example) of what I want to do :
cmd = "echo OK"
os.system("gnome-terminal -e 'bash -c \"'cmd' ; exec bash\"'")
But it's not working because the shell tries to interpret the command "cmd" (bash : cmd: command not found)
Can you help me please ?
Thank you guys
You could use format to replace your variable into the string :
cmd = "echo OK"
os.system("gnome-terminal -e 'bash -c \"'{}' ; exec bash\"'".format(cmd))
I think you're passing the string "cmd" not the variable cmd = "echo OK". Try out this.
cmd = "echo OK"
os.system("gnome-terminal -e 'bash -c " + cmd + " ; exec bash'")
EDIT>
Maybe the module subprocess can help you. Try this snippet.
import subprocess
cmd_line = "echo Hello!"
p = subprocess.Popen(cmd_line, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
out = p.communicate()[0]
print out
subprocess.Popen( ["-c", "kill -SIGUSR2 %s" % master],
stdout=subprocess.PIPE, shell=True).wait()
I'm getting
kill: 1: Illegal option -S
which refers to -SIGUSR2.
What would I do for that parameter to be passed completely as -SIGUSR2?
PS:
If I use -s SIGUSR2 I'm getting kill: 1: invalid signal number or name: SIGUSR2
PPS:
If I use ["-c", "kill", "-SIGUSR2", master] I'm getting
kill: 1: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
kill -l [exitstatus]
SIG is implied.
You want -USR2 instead:
subprocess.Popen( ["-c", "kill -USR2 %s" % master],
stdout=subprocess.PIPE, shell=True).wait()
GNU coreutils'kill can give you this list itself.
$ /bin/kill --list|grep USR2
USR2
EDIT: My mistake. Your example is using the shell's kill and not GNU's. You didn't say which shell, but based on the results, it's likely dash and not bash. bash would allow you to use either -SIGUSR2 or -USR2, but dash does not.
$ dash -c 'kill -l' |grep USR
USR1
USR2
$ bash -c 'kill -l' |grep USR
6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1
11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM
$ bash -c 'kill -SIGUSR2 99999'
bash: line 0: kill: (99999) - No such process
$ bash -c 'kill -USR2 99999'
bash: line 0: kill: (99999) - No such process
$ bash -c 'kill -NOTASIGSPEC 99999'
bash: line 0: kill: NOTASIGSPEC: invalid signal specification
$ dash -c 'kill -SIGUSR2 99999'
dash: 1: kill: Illegal option -S
$ dash -c 'kill -USR2 99999'
dash: 1: kill: No such process
I think you want something like this:
subprocess.Popen(["-c", "kill -USR2 %s" % master],
stdout=subprocess.PIPE, shell=True).wait()
However, SIGUSR2 is not a valid sigspec. Perhaps Brian Cain is correct, but I see examples including the SIG.
For me this works fine:
subprocess.Popen( ["-c", "kill -USR2 %s" % master],
stdout=subprocess.PIPE,shell=True).wait()
Hi I have to execute a shell command :diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)
I tried
cmd="diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)"
args = shlex.split(cmd)
output,error = subprocess.Popen(args,stdout = subprocess.PIPE, stderr= subprocess.PIPE).communicate()
However I am getting an error diff: extra operand cat
I am pretty new to python. Any help would be appreciated
You are using <(...) (process substitution) syntax, which is interpreted by the shell. Provide shell=True to Popen to get it to use a shell:
cmd = "diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)"
output,error = subprocess.Popen(cmd, shell=True, executable="/bin/bash", stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
Since you don't want the Bourne shell (/bin/sh), use the executable argument to determine the shell to use.
You are using a special syntax called process substitiution in your command line. This is supported by most modern shells (bash, zsh), but not by /bin/sh. Therefore, the method suggested by Ned might not work. (It could, if another shell provides /bin/sh and does not "correctly emulate" sh's behaviour, but it is not guaranteed to).
try this instead:
cmd = "diff <(ssh -n root#10.22.254.34 cat /vms/cloudburst.qcow2.*) <(ssh -n root#10.22.254.101 cat /vms/cloudburst.qcow2)"
output,error = subprocess.Popen(['/bin/bash', '-c', cmd], stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
This is basically what the shell=True parameter does, but with /bin/bash instead of /bin/sh (as described in the subprocess docs).