I am trying to integrate a Python script into a bash script. However when I use the input() function, I am getting an EOFError. How can I fix this problem?
#!/bin/bash
python3 <<END
print(input(">>> "))
END
You cannot source both the script and the user input through the program's standard input. (That's in effect what you're trying to do. << redirects the standard input.)
Ideally, you would provide the script as command line argument instead of stdin using -c SCRIPT instead of <<EOF heredoc EOF:
#!/bin/bash
python3 -c 'print(input(">>> "))'
Note that you may need to mind your quoting and escaping in case you have a more complicated Python script with nested quotes.
You can still let the script run over multiple lines, if you need to:
#!/bin/bash
python3 -c '
import os.path
path_name = input("enter a path name >>> ")
file_exists = os.path.exists(path_name)
print("file " + path_name + " " +
("exists" if file_exists else "does not exist"))
'
Note that you will get into trouble when you want to use single quotes in your Python script, as happens when you want to print doesn't instead of does not.
You can work around that using several approaches. The one I consider most flexible (apart from putting you into quoting hell) is surrounding the Python script with double quotes instead and properly escape all inner double quotes and other characters that the shell interprets:
#!/bin/bash
python3 -c "
print(\"It doesn't slice your bread.\")
print('But it can', 'unsliced'[2:7], 'your strings.')
print(\"It's only about \$0. Neat, right?\")
"
Note that I also escaped $, as the shell would otherwise interpret it inside the surrounding double quotes and the result may not be what you wanted.
Related
I'd like to add some words in a file using python script. Here is my script(for which name is test.py) and how I excute the script.
import os
some_python_words = "sys.path.insert(0, os.path.abspath('..'))\n"
scripts = "echo '%s' >> 'conf.py' " % (some_python_words)
os.system(scripts)
As I am execute in the terminal by python test.py. What writes in my file is
sys.path.insert(0, os.path.abspath(..))
The single quotation '' of .. is gone. What should I do?
This is because of the handling of single- and double-qoutes in the shell. A single-quoted string can't have interior single-quotes, but a double-quote can. See Difference between single and double quotes in Bash for a great writeup.
Just change your quoting when creating the shell string
scripts = """echo "%s" >> 'conf.py' """ % (some_python_words)
I'd like to execute the following UNIX command in Python:
cd 2017-02-10; pwd; echo missing > 123.txt
The date directory DATE = 2017-02-10 and OUT = 123.txt are already variables in Python so I have tried variations of
call("cd", DATE, "; pwd; echo missing > ", OUT)
using the subprocess.call function, but I’m struggling to find documentation for multiple UNIX commands at once, which are normally separated by ; or piping with >
Doing the commands on separate lines in Python doesn’t work either because it “forgets” what was executed on the previous line and essentiality resets.
You can pass a shell script as a single argument, with strings to be substituted as out-of-band arguments, as follows:
date='2017-02-10'
out='123.txt'
subprocess.call(
['cd "$1"; pwd; echo missing >"$2"', # shell script to run
'_', # $0 for that script
date, # $1 for that script
out, # $2 for that script
], shell=True)
This is much more secure than substituting your date and out values into a string which is evaluated by the shell as code, because these values are treated as literals: A date of $(rm -rf ~) will not in fact try to delete your home directory. :)
Doing the commands on separate lines in Python doesn’t work either
because it “forgets” what was executed on the previous line and
essentiality resets.
This is because if you have separate calls to subprocess.call it will run each command in its own shell, and the cd call has no effect on the later shells.
One way around that would be to change the directory in the Python script itself before doing the rest. Whether or not this is a good idea depends on what the rest of the script does. Do you really need to change directory? Why not just write "missing" to 2017-02-10/123.txt from Python directly? Why do you need the pwd call?
Assuming you're looping through a list of directories and want to output the full path of each and also create files with "missing" in them, you could perhaps do this instead:
import os
base = "/path/to/parent"
for DATE, OUT in [["2017-02-10", "123.txt"], ["2017-02-11", "456.txt"]]:
date_dir = os.path.join(base, DATE)
print(date_dir)
out_path = os.path.join(date_dir, OUT)
out = open(out_path, "w")
out.write("missing\n")
out.flush()
out.close()
The above could use some error handling in case you don't have permission to write to the file or the directory doesn't exist, but your shell commands don't have any error handling either.
>>> date = "2017-02-10"
>>> command = "cd " + date + "; pwd; echo missing > 123.txt"
>>> import os
>>> os.system(command)
When using bash shell commands it would sometimes be usefull to pipe in python and write a short program and then maybe pipe that into something else. Im not finding a lot of documentation about writing python programs like this although it looks like the "-c" option is the option to use..but when writing even the simplest python program the compiler or should i say interpreter complains. See example below:
$ python -c "
import os
if os.path.isfile("test"):
print "test is a file"
else:
print "test is not a file"
"
When entering the last " the interpretor complains. This runs fine if i put it in a file but if i type it like that on the command line i get errors.
$ python -c "
import os
if os.path.isfile("test"):
print "test is a file"
else:
print "test is not a file"
"
Traceback (most recent call last):
File "<string>", line 4, in <module>
NameError: name 'test' is not defined
I have no idea why the interpretor is complaining here. Does someone know why this isnt working ?
What im really after is something like this:
$ cat somefile | python -c "
import re
check = re.search(pattern, <file input>)
"
I dont know how to access the output of cat in this situation so i just wrote it literally.
You are using double quotes inside double quotes which is ending the quoted string you are passing to python, in a place where you don't expect. Try replacing the outer quotes with single quotes, like I did here:
python -c '
import os
if os.path.isfile("test"):
print "test is a file"
else:
print "test is not a file"
'
If you are using single quotes to terminate the string you are passing to python, make sure to only use double quotes in your code. Additionally, if you can guarantee the availability of Bash as your shell, you can gain added awesome points by using heredoc format instead:
$ python <<EOF
> print "I can put python code here"
> EOF
I can put python code here
Another solution is to escape your inner double quotes so bash doesn't parse them. Like this:
$ python -c "
import os
if os.path.isfile(\"test\"):
print \"test is a file\"
else:
print \"test is not a file\"
"
Either use single quotes to enclose your short program or, if you want to use double quotes to enclose it, escape the quotes with \.
Examples:
1. Escaping quotes
$ python -c "
print \"hello\"
for i in (1,2,3):
print i
"
Output:
hello
1
2
3
2. With single quotes
$ python -c '
print "hello"
for i in (1,2,3):
print i
'
Of course, if you use single quotes to enclose your program and you want to use single quotes inside your python code, you'll have to escape them with \ ;-).
The output is the same.
You can use what is commonly called a "here document" (as in "use the document that is right here"). This avoids all quoting problems when using python -c "..." or python -c '...'
For example:
#!/bin/sh
python <<EOF
print "hello"
for i in (1,2,3):
print i
EOF
The "here document" takes an arbitrary marker ("EOF" is a common choice, but it can be any string you know doesn't occur anywhere else in the data), and accepts all data up unto it finds a line that contains that marker.
I'm trying to call 'sed' from Python and having troubles passing the command line via either subprocess.check_call() or os.system().
I'm on Windows 7, but using the 'sed' from Cygwin (it's in the path).
If I do this from the Cygwin shell, it works fine:
$ sed 's/ /\ /g' <"C:foobar" >"C:foobar.temp"
In Python, I've got the full pathname I'm working with in "name". I tried:
command = r"sed 's/ /\ /g' " + "<" '\"' + name + '\" >' '\"' + name + '.temp' + '\"'
subprocess.check_call(command, shell=True)
All the concatenation is there to make sure I have double quotes around the input and output filenames (in case there are blank spaces in the Windows file path).
I also tried it replacing the last line with:
os.system(command)
Either way, I get this error:
sed: -e expression #1, char 2: unterminated `s' command
'amp' is not recognized as an internal or external command,
operable program or batch file.
'nbsp' is not recognized as an internal or external command,
operable program or batch file.
Yet, as I said, it works OK from the console. What am I doing wrong?
The shell used by subprocess is probably not the shell you want. You can specify the shell with executable='path/to/executable'. Different shells have different quoting rules.
Even better might be to skip subprocess altogether, and write this as pure Python:
with open("c:foobar") as f_in:
with open("c:foobar.temp", "w") as f_out:
for line in f_in:
f_out.write(line.replace(' ', ' '))
I agree with Ned Batchelder's assessment, but think what you might want to consider using the following code because it likely does what you ultimately want to accomplish which can be done easily with the help of Python's fileinput module:
import fileinput
f = fileinput.input('C:foobar', inplace=1)
for line in f:
line = line.replace(' ', ' ')
print line,
f.close()
print 'done'
This will effectively update the given file in place as use of the keyword suggests. There's also an optional backup= keyword -- not used above -- which will save a copy of the original file if desired.
BTW, a word of caution about using something like C:foobar to specify the file name because on Windows it means a file of that name in whatever the current directory is on drive C:, which might not be what you want.
I think you'll find that, in Windows Python, it's not actually using the CygWin shell to run your command, it's instead using cmd.exe.
And, cmd doesn't play well with single quotes the way bash does.
You only have to do the following to confirm that:
c:\pax> echo hello >hello.txt
c:\pax> type "hello.txt"
hello
c:\pax> type 'hello.txt'
The system cannot find the file specified.
I think the best idea would be to use Python itself to process the file. The Python language is a cross-platform one which is meant to remove all those platform-specific inconsistencies, such as the one you've just found.
I don't know if this is a problem with python or with the shell (zsh on linux), I've an argument like this: "#xyz" that starts with a "#"
python the_script.py first_argument #second_argument third_arg
I tried to escape # with \ or \\, or use "" but the program doesn't start. If I leave the # from #second_arguments everything's ok.
Perhaps the "#" is a glob character in zsh, expanding to all symbolic links in the current directory. Try escaping it with "##"?
Try running the argument list with echo, i.e:
echo the_script.py first_argument #second_argument third_arg
That way, you can figure out if it was expanded or passed as-is to the script.