I'd like Popen to execute:
grep -i --line-buffered "grave" data/*.txt
When run from the shell, this gives me the wanted result. If I start, in the very same directory where I test grep, a python repl and follow the instruction from the docs, I obtain what should be the proper argument list to feed Popen with:
['grep', '-i', '--line-buffered', 'grave', 'data/*.txt']
The result of p = subprocess.Popen(args) is
grep: data/*.txt: No such file or directory
and if I try p = subprocess.Popen(args, shell=True), I get:
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
Any help on how to perform the wanted process? I'm on MacOS Lion.
If you type * in bash the shell expands it to the files in the given directory before executing the command. Python's Popen does no such thing, so what you're doing when you call Popen like that is telling grep there is a file called *.txt in the data directory, instead of all the .txt files in the data directory. That file doesn't exist and you get the expected error.
To solve this you can tell python to run the command through the shell by passing shell=True to Popen:
subprocess.Popen('grep -i --line-buffered grave data/*.txt', shell=True)
Which gets translated to:
subprocess.Popen(['/bin/sh', '-c', 'grep -i --line-buffered "grave" data/*.txt'])
As explained in the documentation of Popen.
You have to use a string instead of a list here, because you want to execute /bin/sh -c "grep -i --line-buffered "grave" data/*.txt" (N.B. quotes around the command, making it a single argument to sh). If you use a list this command is run: /bin/sh -c grep -i --line-buffered "grave" data/*.txt, which gives you the output of simply running grep.
The problem is that shell makes file globbing for you: data/*.txt
You will need to do it youself, for example, by using glob module.
import glob
cmd_line = ['grep', '-i', '--line-buffered', 'grave'] + glob.glob('data/*.txt')
Related
I would like to know how I can use command prompt in python. Here is the thing, I need to run a program which is python based,and I used to do it in command prompt. However, I need to run this program multiple times and, thus, would like to automate it. The program need to run with files in a specific folder, and it uses a config file located in the same specific folder. Finally, I also need it to give a log file once it finishes each process. I used to do all this in command prompt:
C:\Users\Gabriel\Documents\vina_tutorial>"\Program Files (x86)\The Scripps Research Institute\Vina\vina.exe" --config conf.txt --log log.txt
I tried using python:
import subprocess
subprocess.Popen('C:\\Program Files (x86)\\The Scripps Research Institute\\Vina\\vina.exe -config ' + 'conf.txt', cwd='C:\\Users\\Gabriel\\Documents\\vina_tutorial')
However, it didn't seem to work. (I did omit the log file thing in this first step)
Any tips on how to proceed or where I can learn something about it?
You need to split up your shell command into separate arguments passed into Popen. Read the documentation
>>> import shlex, subprocess
>>> command_line = input()
/bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
>>> args = shlex.split(command_line)
>>> print(args)
['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
>>> p = subprocess.Popen(args) # Success!
Also, you may need to escape the backslashes in the Windows filepath. You may also need to enclose quotes IE '"C:\\Program Files (x86)\\etc..\\foo.exe"'
I've tried what's told in How to force /bin/bash interpreter for oneliners
By doing
os.system('GREPDB="my command"')
os.system('/bin/bash -c \'$GREPDB\'')
However no luck, unfortunately I need to run this command with bash and subp isn't an option in this environment, I'm limited to python 2.4. Any suggestions to get me in the right direction?
Both commands are executed in different subshells.
Setting variables in the first system call does not affect the second system call.
You need to put two command in one string (combining them with ;).
>>> import os
>>> os.system('GREPDB="echo 123"; /bin/bash -c "$GREPDB"')
123
0
NOTE You need to use "$GREPDB" instead of '$GREPDBS'. Otherwise it is interpreted literally instead of being expanded.
If you can use subprocess:
>>> import subprocess
>>> subprocess.call('/bin/bash -c "$GREPDB"', shell=True,
... env={'GREPDB': 'echo 123'})
123
0
The solution below still initially invokes a shell, but it switches to bash for the command you are trying to execute:
os.system('/bin/bash -c "echo hello world"')
I use this:
subprocess.call(["bash","-c",cmd])
//OK, ignore this because I have not notice subprocess not considered.
subprocess.Popen(cmd, shell=True, executable='/bin/bash')
I searched this command for some days and found really working code:
import subprocess
def bash_command(cmd):
subprocess.Popen(['/bin/bash', '-c', cmd])
code="abcde"
// you can use echo options such as -e
bash_command('echo -ne "'+code+'"')
Output:
abcde
I'm working on a small project where I need to control a console player via python. This example command works perfectly on the Linux terminal:
mplayer -loop 0 -playlist <(find "/mnt/music/soundtrack" -type f | egrep -i '(\.mp3|\.wav|\.flac|\.ogg|\.avi|\.flv|\.mpeg|\.mpg)'| sort)
In Python I'm doing the following:
command = """mplayer -loop 0 -playlist <(find "/mnt/music/soundtrack" -type f | egrep -i '(\.mp3|\.wav|\.flac|\.ogg|\.avi|\.flv|\.mpeg|\.mpg)'| sort)"""
os.system(command)
The problem is when I try it using Python it gives me an error when I run it:
sh: 1: Syntax error: "(" unexpected
I'm really confused here because it is the exact same string. Why doesn't the second method work?
Thanks.
Your default user shell is probably bash. Python's os.system command calls sh by default in linux.
A workaround is to use subprocess.check_call() and pass shell=True as an argument to tell subprocess to execute using your default user shell.
import subprocess
command = """mplayer -loop 0 -playlist <(find "/mnt/music/soundtrack" -type f | egrep -i '(\.mp3|\.wav|\.flac|\.ogg|\.avi|\.flv|\.mpeg|\.mpg)'| sort)"""
subprocess.check_call(command, shell=True)
Your python call 'os.system' is probably just using a different shell than the one you're using on the terminal:
os.system() execute command under which linux shell?
The shell you've spawned with os.system may not support parentheses for substitution.
import subprocess
COMND=subprocess.Popen('command',shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
COMND_bytes = COMND.stdout.read() + COMND.stderr.read()
print(COMND_bytes)
I am using pexpect.run to execute a command. See below:
cmd = "grep -L killed /dir/dumps/*MAC-66.log"
output = pexpect.run(cmd)
When I run this, output equals to:
grep: /dir/dumps/*MAC-66.log: No such file or directory
But when I run the same command in my shell, it works, everytime. I don't see the problem. Any help is appreciated! Does pexpect.run require the command to be split in some fancy way?
Your shell is interpreting the glob, pexpect is not. You could either use python's glob.glob() function to evaluate the glob yourself, or run it through your shell, for example:
cmd = "bash -c 'grep -L killed /dir/dumps/*MAC-66.log'"
Also, if all you're after is output of this command, you ought to check out the subprocess module.
I want to run a command like this
grep -w 1 pattern <(tail -f mylogfile.log)
basically from a python script i want to monitor a log file for a specific string and continue with the python script as soon as i found that.
I am using os.system(), but that is hanging. The same command in bash works good.
I have a very old version of python (v2.3) and so don't have sub-process module.
do we have a way to acheive this
In Python 2.3, you need to use subprocess from SVN
import subprocess
import shlex
subprocess.call(shlex.split("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'"))
To be explicit, you need to install it from the SVN link above.
You need to call this with /bin/bash -c due to the shell redirection you're using
EDIT
If you want to solve this with os.system(), just wrap the command in /bin/bash -c since you're using shell redirection...
os.system("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'")
First of all, the command i think you should be using is grep -w -m 1 'pattern' <(tail -f in)
For executing commands in python, use the Popen constructor from the subprocess module. Read more at
http://docs.python.org/library/subprocess.html
If I understand correctly, you want to send the output to python like this -
tail -f mylogfile.log | grep -w 1 pattern | python yourscript.py
i.e., read all updates to the log file, and send matching lines to your script.
To read from standard input, you can use the file-like object: sys.stdin.
So your script would look like
import sys
for line in sys.stdin.readlines():
#process each line here.