Basically I want to write a python script that does several things and one of them will be to run a checkout on a repository using subversion (SVN) and maybe preform a couple more of svn commands. What's the best way to do this ? This will be running as a crond script.
Would this work?
p = subprocess.Popen("svn info svn://xx.xx.xx.xx/project/trunk | grep \"Revision\" | awk '{print $2}'", stdout=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
print "Revision is", output
Try pysvn
Gives you great access as far as i've tested it.
Here's some examples: http://pysvn.tigris.org/docs/pysvn_prog_guide.html
The reason for why i'm saying as far as i've tested it is because i've moved over to Git.. but if i recall pysvn is (the only and) the best library for svn.
Take a look into the python xonsh module: http://xon.sh/tutorial.html
It can call shell commands plus piping and output redirection with close touch to the python native code (nested) without need to play with python communicate bullshet and escape characters around.
Examples:
env | uniq | sort | grep PATH
COMMAND1 e>o < input.txt | COMMAND2 > output.txt e>> errors.txt
echo "my home is $HOME"
echo #(7+3)
Related
I am working with shell scripting and trying to learn Python scripting, any suggestion is welcome.
I want to achieve something like below:
Usage1:
ps_result=`ps -eaf|grep -v "grep" | grep "programmanager"`
then can we straight away use ps_result variable in python code; if yes, how?
Usage2:
matched_line=`cat file_name |grep "some string"`
can we use matched_line variable in python code as a list, if yes how?
PS: If possible assume I am writing the bash and python code in the one file, if not possible request you to please suggest a way. TIA
Yes, you can do it via environment variables.
First, define the environment variable for the shell using export:
$ export ps_result=`ps -eaf|grep -v "grep" | grep "programmanager"`
Then, import the os module in Python and read the environment variable:
$ python -c 'import os; ps_result=os.environ.get("ps_result"); print(ps_result)'
Second question first, if you run python -, python will run the script piped on stdin. There are several functions in python subprocess that let you run other programs. So you could write
test.sh
#!/bin/sh
python - << endofprogram
import subprocess as subp
result = subp.run('ps -eaf | grep -v "grep" | grep "python"',
shell=True, capture_output=True)
if result.returncode == 0:
matched_lines = result.stdout.decode().split("\n")
print(matched_lines)
endofprogram
In this example we pipe via the shell but python can chain stdout/stdin also, albeit more verbosely.
I have to write a python script to find available space in /var/tmp folder in linux. So I am using awk command to filter out only available space.
import subprocess
subprocess.call("$(awk 'NR==2 {print $4}' file1.txt)")
The output should be 4.7G but it comes
/bin/sh: 1: 4.7G: not found
and the return value is 127 and not 0.
Apart from the fact that python should be sufficient to do what awk does here, what you are doing with
$(awk 'NR==2 {print $4}' file1.txt)
is to run the awk command and expand its output as part of the command line by $(...). As this is the only part of the command line, the shell tries to execute awk's output as a command.
If you really want to run awk from your python script, remove $( and ) from your command.
I'm working on a small project where I need to control a console player via python. This example command works perfectly on the Linux terminal:
mplayer -loop 0 -playlist <(find "/mnt/music/soundtrack" -type f | egrep -i '(\.mp3|\.wav|\.flac|\.ogg|\.avi|\.flv|\.mpeg|\.mpg)'| sort)
In Python I'm doing the following:
command = """mplayer -loop 0 -playlist <(find "/mnt/music/soundtrack" -type f | egrep -i '(\.mp3|\.wav|\.flac|\.ogg|\.avi|\.flv|\.mpeg|\.mpg)'| sort)"""
os.system(command)
The problem is when I try it using Python it gives me an error when I run it:
sh: 1: Syntax error: "(" unexpected
I'm really confused here because it is the exact same string. Why doesn't the second method work?
Thanks.
Your default user shell is probably bash. Python's os.system command calls sh by default in linux.
A workaround is to use subprocess.check_call() and pass shell=True as an argument to tell subprocess to execute using your default user shell.
import subprocess
command = """mplayer -loop 0 -playlist <(find "/mnt/music/soundtrack" -type f | egrep -i '(\.mp3|\.wav|\.flac|\.ogg|\.avi|\.flv|\.mpeg|\.mpg)'| sort)"""
subprocess.check_call(command, shell=True)
Your python call 'os.system' is probably just using a different shell than the one you're using on the terminal:
os.system() execute command under which linux shell?
The shell you've spawned with os.system may not support parentheses for substitution.
import subprocess
COMND=subprocess.Popen('command',shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
COMND_bytes = COMND.stdout.read() + COMND.stderr.read()
print(COMND_bytes)
I want to run a command like this
grep -w 1 pattern <(tail -f mylogfile.log)
basically from a python script i want to monitor a log file for a specific string and continue with the python script as soon as i found that.
I am using os.system(), but that is hanging. The same command in bash works good.
I have a very old version of python (v2.3) and so don't have sub-process module.
do we have a way to acheive this
In Python 2.3, you need to use subprocess from SVN
import subprocess
import shlex
subprocess.call(shlex.split("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'"))
To be explicit, you need to install it from the SVN link above.
You need to call this with /bin/bash -c due to the shell redirection you're using
EDIT
If you want to solve this with os.system(), just wrap the command in /bin/bash -c since you're using shell redirection...
os.system("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'")
First of all, the command i think you should be using is grep -w -m 1 'pattern' <(tail -f in)
For executing commands in python, use the Popen constructor from the subprocess module. Read more at
http://docs.python.org/library/subprocess.html
If I understand correctly, you want to send the output to python like this -
tail -f mylogfile.log | grep -w 1 pattern | python yourscript.py
i.e., read all updates to the log file, and send matching lines to your script.
To read from standard input, you can use the file-like object: sys.stdin.
So your script would look like
import sys
for line in sys.stdin.readlines():
#process each line here.
I'm working in Linux and am wondering how to have python tell whether it is being run directly from a terminal or via a GUI (like alt-F2) where output will need to be sent to a window rather than stdout which will appear in a terminal.
In bash, this done by:
if [ -t 0 ] ; then
echo "I'm in a terminal"
else
zenity --info --title "Hello" --text "I'm being run without a terminal"
fi
How can this be accomplished in python? In other words, the equivalent of [ -t 0 ])?
$ echo ciao | python -c 'import sys; print sys.stdin.isatty()'
False
Of course, your GUI-based IDE might choose to "fool" you by opening a pseudo-terminal instead (you can do it yourself to other programs with pexpect, and, what's sauce for the goose...!-), in which case isatty or any other within-Python approach cannot tell the difference. But the same trick would also "fool" your example bash program (in exactly the same way) so I guess you're aware of that. OTOH, this will make it impossible for the program to accept input via a normal Unix "pipe"!
A more reliable approach might therefore be to explicitly tell the program whether it must output to stdout or where else, e.g. with a command-line flag.
I scoured SE for an answer to this but everywhere indicated the use of sys.stdout.isatty() or os.isatty(sys.stdout.fileno()). Neither of these dependably caught my GUI test cases.
Testing standard input was the only thing that worked for me:
sys.stdin.isatty()
I had the same issue, and I did as follow:
import sys
mode = 1
try:
if sys.stdin.isatty():
mode = 0
except AttributeError: # stdin is NoneType if not in terminal mode
pass
if mode == 0:
# code if terminal mode ...
else:
# code if gui mode ...
There are several examples of this on PLEAC which counts for a third case: running at an interactive Python prompt.
In bash I use this script:
$ cat ~/bin/test-term.sh
#!/bin/bash
#See if $TERM has been set when called from Desktop shortcut
echo TERM environment variable: $TERM > ~/Downloads/test-term.txt
echo "Using env | grep TERM output below:" >> ~/Downloads/test-term.txt
env | grep TERM >> ~/Downloads/test-term.txt
exit 0
When you create a desktop shortcut to call the script the output is:
$ cat ~/Downloads/test-term.txt
TERM environment variable: dumb
Using env | grep TERM output below:
Notice grepping env command returns nothing?
Now call the script from the command line:
$ cat ~/Downloads/test-term.txt
TERM environment variable: xterm-256color
Using env | grep TERM output below:
TERM=xterm-256color
This time the TERM variable from env command returns xterm-256color
In Python you can use:
#import os
result = os.popen("echo $TERM")
result2 = os.popen("env | grep TERM")
Then check the results. I haven't done this in python yet but will probably need to soon for my current project. I came here looking for a ready made solution but noone has posted one like this yet.