I'm executing a shell command in python but when it comes to execute this one in python (which deletes from a text file this symbol ' sigle quote) it doesn't work at all. Please could yout help me with it.
Command:
commands.getoutput('tr -d "'" < /tmp/file_1.txt > /tmp/file_2.txt')
P.S.
The shell command executed in a terminal does work.
Thanks
A syntax highlighter should already show you the problem. You need to escape the single quote inside the command string:
commands.getoutput('tr -d "\'" < /tmp/file_1.txt > /tmp/file_2.txt')
Related
I have a function that takes in a command as a string, and runs it on an AWS container using the aws binary.
The command adds a few extra symbols before and after the command the user requested to run - I won't get into why.
def run_command_on_aws_container(command: str, aws_info):
full_command = f"bash -c 'echo -n \"|\"; {command}; echo -n \"|\"'"
subprocess.run(["aws", ..., "--command", full_command], ...)
...
command_to_run_on_aws_machine = 'python -c "for i in range(10): print(i)"'
run_command_on_aws_container(command_to_run_on_aws_machine, aws_info)
This works, but only if my command_to_run_on_aws_machine doesn't contain single-quotes. If my command_to_run_on_aws_machine was this for example:
command_to_run_on_aws_machine = "python -c 'for i in range(10): print(i)'"
which is exactly the same command, but using single quotes instead of double quotes, the whole thing crashes. Or at least it doesn't do what you'd expect.
Is there a way to make my run_command_on_aws_container function work with both strings, so that as long as the command arg is a proper bash command, it will run? Ideally not just blindly converting all single quotes to double quotes in a string, but in a way where if the command includes properly escaped quotes, it will still work?
Note: All commands being sent to run_command_on_aws_container as the command arg are hard-coded into the program. There's no security issue here of executing arbitrary commands on a remote system. This is only for convenience, so that code being written outside of the function doesn't need to worry about how to properly use a string.
shlex.quote() is designed specifically for this:
full_command = "bash -c " + shlex.quote(f'echo -n "|"; {command}; echo -n "|"')
I am trying to integrate a Python script into a bash script. However when I use the input() function, I am getting an EOFError. How can I fix this problem?
#!/bin/bash
python3 <<END
print(input(">>> "))
END
You cannot source both the script and the user input through the program's standard input. (That's in effect what you're trying to do. << redirects the standard input.)
Ideally, you would provide the script as command line argument instead of stdin using -c SCRIPT instead of <<EOF heredoc EOF:
#!/bin/bash
python3 -c 'print(input(">>> "))'
Note that you may need to mind your quoting and escaping in case you have a more complicated Python script with nested quotes.
You can still let the script run over multiple lines, if you need to:
#!/bin/bash
python3 -c '
import os.path
path_name = input("enter a path name >>> ")
file_exists = os.path.exists(path_name)
print("file " + path_name + " " +
("exists" if file_exists else "does not exist"))
'
Note that you will get into trouble when you want to use single quotes in your Python script, as happens when you want to print doesn't instead of does not.
You can work around that using several approaches. The one I consider most flexible (apart from putting you into quoting hell) is surrounding the Python script with double quotes instead and properly escape all inner double quotes and other characters that the shell interprets:
#!/bin/bash
python3 -c "
print(\"It doesn't slice your bread.\")
print('But it can', 'unsliced'[2:7], 'your strings.')
print(\"It's only about \$0. Neat, right?\")
"
Note that I also escaped $, as the shell would otherwise interpret it inside the surrounding double quotes and the result may not be what you wanted.
seems like I can't use apostrophe, the command fails with no informative error.
I'm trying to execute the following:
secretLsCmd = subprocess.Popen(('docker', 'secret', 'ls') , stdout=subprocess.PIPE)
oneWhitespaceCmd = subprocess.Popen(('tr', '-s','" "') , stdout=subprocess.PIPE, stdin=secretLsCmd.stdout)
onlySecretsCmd = subprocess.check_output(('cut', "-d' '", '-f2') , stdin=oneWhitespaceCmd.stdout)
in a normal Linux terminal, it would execute the following command:
docker secret ls | tr -s " " | cut -d' ' -f2
Running this command in CLI works fine, but once I put it in python it isn't working. The 2 first commands in the pipe are working fine (i have checked), the last command is not working, exiting with error code 1... I'm almost 100% sure it is the -d' ' , but how can I fix that? any idea?
This line:
oneWhitespaceCmd = subprocess.Popen(('tr', '-s','" "'), ...)
is actually running:
tr -s '" "'
so you want to lose the extra double quotes there: Python will quote any arguments that it need to quote for the shell.
This command:
onlySecretsCmd = subprocess.check_output(('cut', "-d' '", '-f2'), ...)
is equivalent to the shell command:
cut '-d'"'"' '"'"'' -f2
so again, you probably just want to lose the quotes round the whitespace:
onlySecretsCmd = subprocess.check_output(('cut', "-d ", '-f2'), ...)
and leave it to Python to insert quotes where required. This will actually run (what should be equivalent to what you want though not identical):
cut '-d ' -f2
I used shlex.quote() to create a shell equivalent to the commands you are running, though in practice unless you tell subprocess to invoke a shell it will just be executing the equivalent of the command bash would execute after parsing all the escape quote marks. So internally the escaping isn't happening but the quotes to distinguish the arguments aren't needed either.
Quote of the script:
!/bin/sh
## repo default configuration
##
REPO_URL='https://android.googlesource.com/tools/repo'
REPO_REV='stable'
...
magic='--calling-python-from-/bin/sh--'
"""exec" python -E "$0" "$#" """#$magic"
if __name__ == '__main__':
import sys
if sys.argv[-1] == '#%s' % magic:
del sys.argv[-1]
del magic
..all python from here on..
How can they put bash and python in a single script and make it run?
This is how I understand it:
The script is first invoked as a shell script and then calls python on line 23, so if you invoked the sync command on repo it would do the following:
"""exec" python -E "$0" "$#" """#$magic"
which I believe turns into:
exec "python -E "repo" "sync" "#--calling-python-from-/bin/sh--"
This then calls the script repo as a python script. You will notice that all of the syntax is legal in bash and python down to the exec line. The triple quotes on line 23 are really sweet, they work in the shell script and then work as a doc string in python. How awesome is that!
In python """ begin and end a docstring which can span multiple lines. Very roughly, it's a very special kind of comment that the python help system can read. In bash, I believe that the double quote is matched to the next double quote the interpreter encounters. So the first two quotes, quote the null string and it disappears from the interpreter. In our example, the next quotes quote the exec keyword. The last set of triple quotes, """#$magic", first quote the null string and then quote the #$magic argument. Since double quotes allow the variable $magic to be expanded, it becomes, #--calling-python-from-/bin/sh-- and passed as an argument.
If you look at the script with a text editor with syntax highlighting as a shell script and then as a python script, it's even easier to see. This is an extremely clever, way of starting the shell script and then giving control to python. Hope this helps!
I don't know if this is a problem with python or with the shell (zsh on linux), I've an argument like this: "#xyz" that starts with a "#"
python the_script.py first_argument #second_argument third_arg
I tried to escape # with \ or \\, or use "" but the program doesn't start. If I leave the # from #second_arguments everything's ok.
Perhaps the "#" is a glob character in zsh, expanding to all symbolic links in the current directory. Try escaping it with "##"?
Try running the argument list with echo, i.e:
echo the_script.py first_argument #second_argument third_arg
That way, you can figure out if it was expanded or passed as-is to the script.