I have the following code in bash script
python -u - << EOF
from my.package import script
script.run()
EOF
>> /path/to/log/file
In script.py there are a number of print statements and I would like to redirect that output from the console to a file. The file gets created successfully, but it's empty.
How can I go about this? What am I missing here?
The idea is right, but the way you re-direct the output to a file from here-document is wrong, Heredocs themselves re-directs just like any other commands, just do
python -u - << EOF >> /path/to/log/file
from my.package import script
script.run()
EOF
try this,
import sys
f = open("log_file.txt", 'w')
sys.stdout = f
print "My print statements"
f.close()
Right syntax is:
python -u - >> /path/to/log/file << EOF
from my.package import script
script.run()
EOF
The reason is that if you write in bash something like that:
util_1 | util_2 | util_3 >> some_file < another_file
or
util_1 | util_2 | util_3 >> some_file << EOF
...
EOF
another_file or here-document goes to the standard input of the first utility in pipeline (in this case to util_1).
Related
I am trying to pipe into python using bash:
#!/bin/sh
echo "from rest_framework_api_key.models import APIKey\n_, key = APIKey.objects.create_key(name='test_key')\nprint(key)" | python manage.py shell
This gives the following error:
from rest_framework_api_key.models import APIKey\n_, key = APIKey.objects.create_key(name='test_key')\nprint(key)
^
SyntaxError: unexpected character after line continuation character
Strangely, this works:
#!/bin/sh
echo "from users.models import CustomUser\nCustomUser.objects.create_superuser('foo', '$1')" | python manage.py shell
And I can't see the difference in terms of new lines / escaping / quoting. If I remove the pipe to python (i.e. it runs only the echo command), it does not error.
bash's echo doesn't expand \n to a newline by default. Use printf:
printf "from rest_framework_api_key.models import APIKey\n_, key = APIKey.objects.create_key(name='test_key')\nprint(key)" | python manage.py shell
or more simply, a here document:
python manage.py shell << EOF
from rest_framework_api_key.models import APIKey
_, key = APIKey.objects.create_key(name='test_key')
print(key)
EOF
You are missing the -e flag in the echo.
It should be :
echo -e "from rest_framework_api_key.models import APIKey\n_, key = APIKey.objects.create_key(name='test_key')\nprint(key)"
Let me know if it helps!
Note : I only now noticed the shebang in your code. Unless you have a very specific reason not to do so, you should change your #!/bin/sh for #!/bin/bash.
I wrote a script and I want it to be pipeable in bash. Something like:
echo "1stArg" | myscript.py
Is it possible? How?
See this simple echo.py:
import sys
if __name__ == "__main__":
for line in sys.stdin:
sys.stderr.write("DEBUG: got line: " + line)
sys.stdout.write(line)
running:
ls | python echo.py 2>debug_output.txt | sort
output:
echo.py
test.py
test.sh
debug_output.txt content:
DEBUG: got line: echo.py
DEBUG: got line: test.py
DEBUG: got line: test.sh
I'll complement the other answers with a grep example that uses fileinput to implement the typical behaviour of UNIX tools: 1) if no arguments are specified, it reads data from stdin; 2) many files can be specified as arguments; 3) a single argument of - means stdin.
import fileinput
import re
import sys
def grep(lines, regexp):
return (line for line in lines if regexp.search(line))
def main(args):
if len(args) < 1:
print("Usage: grep.py PATTERN [FILE...]", file=sys.stderr)
return 2
regexp = re.compile(args[0])
input_lines = fileinput.input(args[1:])
for output_line in grep(input_lines, regexp):
sys.stdout.write(output_line)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Example:
$ seq 1 20 | python grep.py "4"
4
14
In your Python script you simply read from stdin.
Everything that reads from stdin is "pipeable". Pipe simply redirects stdout of former program to the latter.
Why it does not give output while doing from bash >>, so that it can be saved to a file.
$ cat > /var/tmp/runme.sh << \EOF
#!/bin/bash
export DISPLAY=:0.0
python /var/tmp/t.py >> /var/tmp/log.log &
sleep 3
ps aux | grep "t.py" | grep -v "grep" | awk '{print $2}' | xargs kill -9;
EOF
$ cat > /var/tmp/t.py << \EOF
import sys
print "[RAN]: OK"
sys.stdout.write("[RAN]: OK")
sys.stdout.flush()
EOF
$ chmod +x /var/tmp/runme.sh ; /var/tmp/runme.sh &
$ cat /var/tmp/log.log
$ tail -f /var/tmp/log.log
^
Showing nothing.
How can i get the outputs to log.log using Bash and Python combination?
you could have python do:
var = "hello world"
f = open('log.log', 'r+')
print(var)
f.write(var)
Within var/tmp/t.py, do something similar to the following:
var = "[RAN]: OK"
f = open("log.log", "a+")
f.write(var)
print var
f.close()
Opening "log.log" with the "a+" argument will allow you to append the output of t.py to the file so that you can keep track of multiple runs of t.py. If you just want the output for the most recent run you could just use "w+" which will create a new file if one does not exist and rewrite the file if it does.
You could also put a time stamp on each log using datetime like so:
import datetime
now = datetime.datetime.now()
var = "Date: " + now.strftime("%Y-%m-%d %H:%M") + "\n" + "[RAN]: OK"
f = open("log.log", "a+")
f.write(var)
print var
f.close()
I don't know if this script is something you will run periodically in which knowing the date the log was created would be useful or whether it is a one and done script but I thought you may find date logging useful.
I am using zbarimg to scan bar codes, I want to redirect the output to a python script. How can I redirect the output of the following command:
zbarimg code.png
to a python script, and what should be the script like?
I tried the following script:
#!/usr/local/bin/python
s = raw_input()
print s
I made it an executable by issuing the following:
chmod +x in.py
Than I ran the following :
zbarimg code.png | in.py
I know it's wrong but I can't figure out anything else!
Use sys.stdin to read from stdin in your python script. For example:
import sys
data = sys.stdin.readlines()
Using the pipe operator | from the command is correct, actually. Did it not work?
You might need to explicitly specify the path for the python script as in
zbarimg code.png | ./in.py
and as #dogbane says, reading from stdin like sys.stdin.readlines() is better than using raw_input
I had to invoke the python program command as
somecommand | python mypythonscript.py instead of somecommand | ./mypythonscript.py. This worked for me. The latter produced errors.
My purpose: Sum up the durations of all mp3 files by piping output of soxi -D *mp3
into python: soxi -D *mp3 | python sum_durations.py
Details:
soxi -D *mp3produces:
122.473016
139.533016
128.456009
307.802993
...
sum_durations.py script:
import sys
import math
data = sys.stdin.readlines()
#print(data)
sum = 0.0
for line in data:
#print(line)
sum += float(line)
mins = math.floor(sum / 60)
secs = math.floor(sum) % 60
print("total duration: " + str(mins) + ":" + str(secs))
I wrote a script and I want it to be pipeable in bash. Something like:
echo "1stArg" | myscript.py
Is it possible? How?
See this simple echo.py:
import sys
if __name__ == "__main__":
for line in sys.stdin:
sys.stderr.write("DEBUG: got line: " + line)
sys.stdout.write(line)
running:
ls | python echo.py 2>debug_output.txt | sort
output:
echo.py
test.py
test.sh
debug_output.txt content:
DEBUG: got line: echo.py
DEBUG: got line: test.py
DEBUG: got line: test.sh
I'll complement the other answers with a grep example that uses fileinput to implement the typical behaviour of UNIX tools: 1) if no arguments are specified, it reads data from stdin; 2) many files can be specified as arguments; 3) a single argument of - means stdin.
import fileinput
import re
import sys
def grep(lines, regexp):
return (line for line in lines if regexp.search(line))
def main(args):
if len(args) < 1:
print("Usage: grep.py PATTERN [FILE...]", file=sys.stderr)
return 2
regexp = re.compile(args[0])
input_lines = fileinput.input(args[1:])
for output_line in grep(input_lines, regexp):
sys.stdout.write(output_line)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Example:
$ seq 1 20 | python grep.py "4"
4
14
In your Python script you simply read from stdin.
Everything that reads from stdin is "pipeable". Pipe simply redirects stdout of former program to the latter.