My goal is to execute the following bash command in Python and store its output:
echo 'sudo ./run_script.sh -dates \\{\\'2017-11-16\\',\\'2017-11-29\\'\\}'|sed 's;\\\\;\\;'
When I run this command in bash, the output is: sudo ./run_script.sh -dates \{\'2019-10-05\',\'2019-10-04\'\}
My initial idea was to replace the double backslash by a single backslash in Python. As ridiculous as it seems, I couldn't do it in Python (only when using print() the output is as I would like but I can't store the output of print() and str() doesn't convert \ to . So I decided to do it in bash.
import subprocess
t= 'some \\ here'
cmd = "echo \'"+ t+"\'|sed 's;\\\\;\\;'"
ps = subprocess.run(cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
ps.stdout
Out[6]: b"sed: -e expression #1, char 7: unterminated `s' command\n"
Running Python 3.6.8 on Ubuntu 18
Try using subprocess.check_output instead. You're also forgetting an extra backslash for every backslash in your command.
import subprocess
command = "echo 'some \\\\here'|sed 's;\\\\\\\\;\\\\;'"
output = subprocess.check_output(command, shell=True).decode()
print(output) # prints your expect result "some \here"
After re-reading your question I kinda understood what you wanted.
a = r'some \here'
print(a) #some \here
Again, raw string literals...
Related
I have a function that takes in a command as a string, and runs it on an AWS container using the aws binary.
The command adds a few extra symbols before and after the command the user requested to run - I won't get into why.
def run_command_on_aws_container(command: str, aws_info):
full_command = f"bash -c 'echo -n \"|\"; {command}; echo -n \"|\"'"
subprocess.run(["aws", ..., "--command", full_command], ...)
...
command_to_run_on_aws_machine = 'python -c "for i in range(10): print(i)"'
run_command_on_aws_container(command_to_run_on_aws_machine, aws_info)
This works, but only if my command_to_run_on_aws_machine doesn't contain single-quotes. If my command_to_run_on_aws_machine was this for example:
command_to_run_on_aws_machine = "python -c 'for i in range(10): print(i)'"
which is exactly the same command, but using single quotes instead of double quotes, the whole thing crashes. Or at least it doesn't do what you'd expect.
Is there a way to make my run_command_on_aws_container function work with both strings, so that as long as the command arg is a proper bash command, it will run? Ideally not just blindly converting all single quotes to double quotes in a string, but in a way where if the command includes properly escaped quotes, it will still work?
Note: All commands being sent to run_command_on_aws_container as the command arg are hard-coded into the program. There's no security issue here of executing arbitrary commands on a remote system. This is only for convenience, so that code being written outside of the function doesn't need to worry about how to properly use a string.
shlex.quote() is designed specifically for this:
full_command = "bash -c " + shlex.quote(f'echo -n "|"; {command}; echo -n "|"')
I am running the sed command inside python using os.system. Below is the code.
os.system("sed -i /solid/s/Visualization Toolkit generated SLA File/chestwall/g mesh1.stl")
The name to be changed has spaces in it. Also, in the end part i.e. mesh1.stl, the 1 need to be variable. How to do it?
Firstly, for this code, I am getting error as:
sed: -e expression #1, char 22: unterminated s command
I tried putting / at the end.
Second, I need the mesh1 to be a variable from previous line. Say, mesh1 as a and everytime, a changes. How to write like that?
Make sure that the sed statement/command is in either double or single quotes and then use "+" to concatenate strings before passing them to os.system
import os
var=1
os.system("sed -i 's/solid/s/Visualization Toolkit generated SLA File/chestwall/g' mesh" + var + ".stl")
The function os.system() is now considered to be superseded by
subprocess.call().
Would you please try the following:
import subprocess
a = 'mesh1'
cmd = ['sed', '-i', '/solid/s/Visualization Toolkit generated SLA File/chestwall/g', '{0}.stl'.format(a)]
subprocess.call(cmd)
You can pass the command as a list, not a string, and you can explicitly divide the arguments.
seems like I can't use apostrophe, the command fails with no informative error.
I'm trying to execute the following:
secretLsCmd = subprocess.Popen(('docker', 'secret', 'ls') , stdout=subprocess.PIPE)
oneWhitespaceCmd = subprocess.Popen(('tr', '-s','" "') , stdout=subprocess.PIPE, stdin=secretLsCmd.stdout)
onlySecretsCmd = subprocess.check_output(('cut', "-d' '", '-f2') , stdin=oneWhitespaceCmd.stdout)
in a normal Linux terminal, it would execute the following command:
docker secret ls | tr -s " " | cut -d' ' -f2
Running this command in CLI works fine, but once I put it in python it isn't working. The 2 first commands in the pipe are working fine (i have checked), the last command is not working, exiting with error code 1... I'm almost 100% sure it is the -d' ' , but how can I fix that? any idea?
This line:
oneWhitespaceCmd = subprocess.Popen(('tr', '-s','" "'), ...)
is actually running:
tr -s '" "'
so you want to lose the extra double quotes there: Python will quote any arguments that it need to quote for the shell.
This command:
onlySecretsCmd = subprocess.check_output(('cut', "-d' '", '-f2'), ...)
is equivalent to the shell command:
cut '-d'"'"' '"'"'' -f2
so again, you probably just want to lose the quotes round the whitespace:
onlySecretsCmd = subprocess.check_output(('cut', "-d ", '-f2'), ...)
and leave it to Python to insert quotes where required. This will actually run (what should be equivalent to what you want though not identical):
cut '-d ' -f2
I used shlex.quote() to create a shell equivalent to the commands you are running, though in practice unless you tell subprocess to invoke a shell it will just be executing the equivalent of the command bash would execute after parsing all the escape quote marks. So internally the escaping isn't happening but the quotes to distinguish the arguments aren't needed either.
I need to write a python script where I need to call a few awk commands inside of it.
#!/usr/bin/python
import os, sys
input_dir = '/home/abc/data'
os.chdir(input_dir)
#wd=os.getcwd()
#print wd
os.system ("tail -n+2 ./*/*.tsv|cat|awk 'BEGIN{FS="\t"};{split($10,arr,"-")}{print arr[1]}'|sort|uniq -c")
It gives an error in line 8: SyntaxError: unexpected character after line continuation character
Is there a way I can get the awk command get to work within the python script?
Thanks
You have both types of quotes in that string, so use triple quotes around the whole thing
>>> x = '''tail -n+2 ./*/*.tsv|cat|awk 'BEGIN{FS="\t"};{split($10,arr,"-")}{print arr[1]}'|sort|uniq -c'''
>>> x
'tail -n+2 ./*/*.tsv|cat|awk \'BEGIN{FS="\t"};{split($10,arr,"-")}{print arr[1]}\'|sort|uniq -c'
You should use subprocess instead of os.system:
import subprocess
COMMAND = "tail -n+2 ./*/*.tsv|cat|awk 'BEGIN{FS=\"\t\"};{split($10,arr,\"-\")}{print arr[1]}'|sort|uniq -c"
subprocess.call(COMMAND, shell=True)
As TehTris has pointed out, the arrangement of quotes in the question breaks the command string into multiple strings. Pre-formatting the command and escaping the double-quotes fixes this.
I am trying to run a python script, and I need to Rsh a command from the script, the command I want to run is : df -Pk|grep "sd\|md"|gawk '{print $2}'
and I do it as -
cmd2='df -Pk|grep \\\"sd\|md\\\"|gawk \'{print $2}\''
process = subprocess.Popen(['rsh',ip,cmd2],stdout=subprocess.PIPE)
output = process.communicate()[0]
However when I do run the script,I get nothing in output.
I am new to python and as far as I know, its a problem with the escape characters.
Any help would be great.
Note:
I have to use Rsh only and cannot use ssh
Thanks
Enclose the command, with proper shell quoting, in triple quote marks:
"""df -Pk|grep "sd\|md"|gawk '{print $2}'"""
See also the Python tutorial's bit on strings.