When using bash shell commands it would sometimes be usefull to pipe in python and write a short program and then maybe pipe that into something else. Im not finding a lot of documentation about writing python programs like this although it looks like the "-c" option is the option to use..but when writing even the simplest python program the compiler or should i say interpreter complains. See example below:
$ python -c "
import os
if os.path.isfile("test"):
print "test is a file"
else:
print "test is not a file"
"
When entering the last " the interpretor complains. This runs fine if i put it in a file but if i type it like that on the command line i get errors.
$ python -c "
import os
if os.path.isfile("test"):
print "test is a file"
else:
print "test is not a file"
"
Traceback (most recent call last):
File "<string>", line 4, in <module>
NameError: name 'test' is not defined
I have no idea why the interpretor is complaining here. Does someone know why this isnt working ?
What im really after is something like this:
$ cat somefile | python -c "
import re
check = re.search(pattern, <file input>)
"
I dont know how to access the output of cat in this situation so i just wrote it literally.
You are using double quotes inside double quotes which is ending the quoted string you are passing to python, in a place where you don't expect. Try replacing the outer quotes with single quotes, like I did here:
python -c '
import os
if os.path.isfile("test"):
print "test is a file"
else:
print "test is not a file"
'
If you are using single quotes to terminate the string you are passing to python, make sure to only use double quotes in your code. Additionally, if you can guarantee the availability of Bash as your shell, you can gain added awesome points by using heredoc format instead:
$ python <<EOF
> print "I can put python code here"
> EOF
I can put python code here
Another solution is to escape your inner double quotes so bash doesn't parse them. Like this:
$ python -c "
import os
if os.path.isfile(\"test\"):
print \"test is a file\"
else:
print \"test is not a file\"
"
Either use single quotes to enclose your short program or, if you want to use double quotes to enclose it, escape the quotes with \.
Examples:
1. Escaping quotes
$ python -c "
print \"hello\"
for i in (1,2,3):
print i
"
Output:
hello
1
2
3
2. With single quotes
$ python -c '
print "hello"
for i in (1,2,3):
print i
'
Of course, if you use single quotes to enclose your program and you want to use single quotes inside your python code, you'll have to escape them with \ ;-).
The output is the same.
You can use what is commonly called a "here document" (as in "use the document that is right here"). This avoids all quoting problems when using python -c "..." or python -c '...'
For example:
#!/bin/sh
python <<EOF
print "hello"
for i in (1,2,3):
print i
EOF
The "here document" takes an arbitrary marker ("EOF" is a common choice, but it can be any string you know doesn't occur anywhere else in the data), and accepts all data up unto it finds a line that contains that marker.
Related
I'd like to add some words in a file using python script. Here is my script(for which name is test.py) and how I excute the script.
import os
some_python_words = "sys.path.insert(0, os.path.abspath('..'))\n"
scripts = "echo '%s' >> 'conf.py' " % (some_python_words)
os.system(scripts)
As I am execute in the terminal by python test.py. What writes in my file is
sys.path.insert(0, os.path.abspath(..))
The single quotation '' of .. is gone. What should I do?
This is because of the handling of single- and double-qoutes in the shell. A single-quoted string can't have interior single-quotes, but a double-quote can. See Difference between single and double quotes in Bash for a great writeup.
Just change your quoting when creating the shell string
scripts = """echo "%s" >> 'conf.py' """ % (some_python_words)
I am trying to integrate a Python script into a bash script. However when I use the input() function, I am getting an EOFError. How can I fix this problem?
#!/bin/bash
python3 <<END
print(input(">>> "))
END
You cannot source both the script and the user input through the program's standard input. (That's in effect what you're trying to do. << redirects the standard input.)
Ideally, you would provide the script as command line argument instead of stdin using -c SCRIPT instead of <<EOF heredoc EOF:
#!/bin/bash
python3 -c 'print(input(">>> "))'
Note that you may need to mind your quoting and escaping in case you have a more complicated Python script with nested quotes.
You can still let the script run over multiple lines, if you need to:
#!/bin/bash
python3 -c '
import os.path
path_name = input("enter a path name >>> ")
file_exists = os.path.exists(path_name)
print("file " + path_name + " " +
("exists" if file_exists else "does not exist"))
'
Note that you will get into trouble when you want to use single quotes in your Python script, as happens when you want to print doesn't instead of does not.
You can work around that using several approaches. The one I consider most flexible (apart from putting you into quoting hell) is surrounding the Python script with double quotes instead and properly escape all inner double quotes and other characters that the shell interprets:
#!/bin/bash
python3 -c "
print(\"It doesn't slice your bread.\")
print('But it can', 'unsliced'[2:7], 'your strings.')
print(\"It's only about \$0. Neat, right?\")
"
Note that I also escaped $, as the shell would otherwise interpret it inside the surrounding double quotes and the result may not be what you wanted.
I am generating a command that comprises multi nested quotes and then execute:
cmd = "python datax.py -p \"\"\" -Dpsql=\"DELETE FROM xxx WHERE date(yyy) = \'2019-02-02\'\" \"\"\" "
os.system(cmd)
The cmd can be generated as expected:
python datax.py -p """ -Dpsql="DELETE FROM xxx WHERE date(yyy) = '2019-02-02'" """
But it seems this does not work well. System still returns exceptions like '(' unexpected... etc.
I also tried '''...''' and doesn't work either. How to address this issue and is there any other way to deal with this case? Any help is appreciated.
You need to add an escaped backslash \\ for each nesting level, because your shell most likely does not support triple quotes. Therefore it parses The first two off your triple quotes and " -Dpsql=" as quoted strings and
DELETE FROM xxx WHERE date(yyy) = '2019-02-02' as shell code. But date(yyy) is not valid shell syntax
An example to reproduce this without having your datax script:
cmd = "python -c \"print(\"Hello (world!)\")\""
os.system(cmd)
fails with
syntax error near unexpected token `('
But
cmd = "python -c \"print(\\\"Hello (world!)\\\")\""
os.system(cmd)
outputs
Hello (world!)
I suggest to change your code to:
cmd = "python datax.py -p \" -Dpsql=\\\"DELETE FROM xxx WHERE date(yyy) = \'2019-02-02\'\\\"\" "
Now your command looks like:
python datax.py -p " -Dpsql=\"DELETE FROM xxx WHERE date(yyy) = '2019-02-02'\""
You see, there is an outer quotation level for your -p argument and an inner quotation level for the sql code.
If you print cmd you get:
python datax.py -p """ -Dpsql="DELETE FROM xxx WHERE date(yyy) = '2019-02-02'" """
What os.system does is just open a shell with a basic .sh interpreter and runs that. Stuff like """ are big no-nos for sh (mainly because you're not sure what it does - something like an empty string with another start of one) and of course, the parenthesis are also bad for sh,
You have to escape or make sure they are in a string. You are going to have to escape everything or make sure a proper sh string is passed. This is one way:
cmd = r"""python datax.py -p ' -Dpsql=\" DELETE FROM xxx WHERE date(yyy) = \\'2019-02-02\\' \" ' """
r"" conserves backslashes as is. Printing this results in:
python datax.py -p ' -Dpsql=\" DELETE FROM xxx WHERE date(yyy) = 2019-02-02 \" '
A better option I think would be to just use:
cmd = r"""python datax.py -p -Dpsql=" DELETE FROM xxx WHERE date(yyy) = '2019-02-02' " """
So I dropped one level of '. Now just fix datax.py to parse argv to better suite your needs (take everything after the -p here). This would make it easier to call it directly as well. When printed:
python datax.py -p -Dpsql=" DELETE FROM xxx WHERE date(yyy) = '2019-02-02' "
argparse handles these things well.
I'm trying to pass file list to my python script via argument:
python script.py -o aaa -s bbb "filename.txt" "filename2.txt" "file name3.txt"
Unfortunately ArgumentParser is ignoring quotes and instead of giving list of 3 files it gives me list of 4 elements as followed:
1) "filename.txt"
2) "filename2.txt"
3) "file
4) name3.txt"
It completely ignores quotes. How to make it work with quotes?
Hard without seeing what you're using or any code.
Your shell may be interfering, you may need to escape the spaces with \.
Example:
python script.py -o a -f "file1.txt" "file\ 2.csv"
Is hard without code but considering you are using sys.argv[] you can easily pass the file arguments with quotes like when you need a file or argv with blank spaces: python script.py "myFile.txt" "otherFile.jpeg"
Try this simple code to understand:
import sys
for n, p in enumerate(sys.argv):
print("Parameter: %d = %s") % (n, p))`
You can see that first argv is the file name you are running.
It looks like that this is not python's fault. I'm calling python script from inside of bash script and this make mess with quotes as parameters.
Quote of the script:
!/bin/sh
## repo default configuration
##
REPO_URL='https://android.googlesource.com/tools/repo'
REPO_REV='stable'
...
magic='--calling-python-from-/bin/sh--'
"""exec" python -E "$0" "$#" """#$magic"
if __name__ == '__main__':
import sys
if sys.argv[-1] == '#%s' % magic:
del sys.argv[-1]
del magic
..all python from here on..
How can they put bash and python in a single script and make it run?
This is how I understand it:
The script is first invoked as a shell script and then calls python on line 23, so if you invoked the sync command on repo it would do the following:
"""exec" python -E "$0" "$#" """#$magic"
which I believe turns into:
exec "python -E "repo" "sync" "#--calling-python-from-/bin/sh--"
This then calls the script repo as a python script. You will notice that all of the syntax is legal in bash and python down to the exec line. The triple quotes on line 23 are really sweet, they work in the shell script and then work as a doc string in python. How awesome is that!
In python """ begin and end a docstring which can span multiple lines. Very roughly, it's a very special kind of comment that the python help system can read. In bash, I believe that the double quote is matched to the next double quote the interpreter encounters. So the first two quotes, quote the null string and it disappears from the interpreter. In our example, the next quotes quote the exec keyword. The last set of triple quotes, """#$magic", first quote the null string and then quote the #$magic argument. Since double quotes allow the variable $magic to be expanded, it becomes, #--calling-python-from-/bin/sh-- and passed as an argument.
If you look at the script with a text editor with syntax highlighting as a shell script and then as a python script, it's even easier to see. This is an extremely clever, way of starting the shell script and then giving control to python. Hope this helps!