I wrote a script to run mafft module from the terminal:
import subprocess
def linsi_MSA(sequnces_file_path):
cmd = ' mafft --maxiterate 1000 --localpair {seqs} > {out}'.format(seqs=sequnces_file_path, out=sequnces_file_path)
subprocess.call(cmd.split(), shell=True)
if __name__ == '__main__':
import logging
logger = logging.getLogger('main')
from sys import argv
if len(argv) < 2:
logger.error('Usage: MSA <sequnces_file_path> ')
exit()
else:
linsi_MSA(*argv[1:])
for some reason when trying to run the script from the terminal using:
python ./MSA.py ./sample.fa
I get the mafft interactive version opening directly in the trminal (asking for input ..output etc..)
when i'm trying to write the cmd directly in the terminal using:
mafft --maxiterate 1000 --localpair sample.fa > sample.fa
its working as expected and perfoming the command line version as without opening the interactive version.
I want my script to be able to perform the cmd line version on the terminal. what seems to be the problem?
thanks!
If you use shell=True you should pass one string as argument, not a list, e.g.:
subprocess.call("ls > outfile", shell=True)
It's not explained in the docs, but I suspect it has to do with what low-level library function is ultimately called:
call(["ls", "-l"]) --> execlp("ls", "-l")
^^^^^^^^^^ ^^^^^^^^^^
call("ls -l", shell=True) --> execlp("sh", "-c", "ls -l")
^^^^^^^ ^^^^^^^
call(["ls", "-l"], shell=True) --> execlp("sh", "-c", "ls", "-l")
# which can be tried from command line:
sh -c ls -l
# result is a list of files without details, -l was ignored.
# see sh(1) man page for -c string syntax and what happens to further arguments.
Related
I am trying to run python subprocess.run function to execute following command:
pdftoppm -jpeg -f 1 -scale-to 200 data/andromeda.pdf and-page
pdftoppm - is part of poppler utility and it generates images from pdf files.
File data/andromeda.pdf exists. Folder data is on same level with python script and/or where I run command from.
Command basically will generate a jpeg file, from page 1 (-f 1) 200px wide (-scale-to) from given file of and-page-1.jpeg format (so called ppmtroot).
Long story short: from command line it works as expected i.e. if I call the above command either from zsh or bash shell, manually - it generates thumbnail as expected. However if I run it from python subprocess module - it fails it returns 99 error code!
Following is python code (file name is sc_02_thumbnails.py):
import subprocess
import sys
def main(filename, ppmroot):
cmd = [
'pdftoppm',
'-f 1',
'-scale-to 200',
'-jpeg',
filename,
ppmroot
]
result = subprocess.run(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
if result.returncode:
print("Failed to generate thumbnail. Return code: {}. stderr: {}".format(
result.returncode,
result.stderr
))
print("Used cmd: {}".format(' '.join(cmd)))
sys.exit(1)
else:
print("Success!")
if __name__ == "__main__":
if len(sys.argv) > 2:
filename = sys.argv[1]
ppmroot = sys.argv[2]
else:
print("Usage: {} <pdffile> <ppmroot>".format(sys.argv[0]))
sys.exit(1)
main(filename, ppmroot)
And here is repo which includes data/andromeda.pdf file as well.
I call my script with as (from zsh):
$ chmod +x ./sc_02_thumbnauils.py
$ ./sc_02_thumbnails.py data/andromeda.pdf and-page
and ... thumbnail generating fails!
I have tried executing python script from both, from zsh and bash shells :(
What I am doing wrong?
The quoting is wrong, you should have '-f', '1', etc
I have a python script that calls a shell scrips, that in turn calls a .exe called iv4_console. I need to print the stdout of iv4_console for debugging purposes. I used this:
Python:
import sys
import subprocess
var="rW015005000000"
proc = subprocess.Popen(["c.sh", var], shell=True, stdout=subprocess.PIPE)
output = ''
for line in iter(proc.stdout.readline, ""):
print line
output += line
Shell:
start_dir=$PWD
release=$1
echo Release inside shell: $release
echo Directory: $start_dir
cd $start_dir
cd ../../iv_system4/ports/visualC12/Debug
echo Debug dir: $PWD
./iv4_console.exe ../embedded/LUA/analysis/verbose-udp-toxml.lua ../../../../../logs/$release/VASP_DUN722_20160307_Krk_Krk_113048_092_1_$release.dvl &>../../../../FCW/ObjectDetectionTest/VASP_DUN722_20160307_Krk_Krk_113048_092_1_$release.xml
./iv4_console.exe ../embedded/LUA/analysis/verbose-udp-toxml.lua ../../../../../logs/$release/VASP_FL140_20170104_C60_Checkout_afterIC_162557_001_$release.dvl &>../../../../FCW/ObjectDetectionTest/VASP_FL140_20170104_C60_Checkout_afterIC_162557_001_$release.xml
exit
But this didn't work, it prints nothing. What do you think?
See my comment, best approach (i.m.o) would be to just use python only.
However, in answer of your question, try:
import sys
import subprocess
var="rW015005000000"
proc = subprocess.Popen(["/bin/bash", "/full/path/to/c.sh"], stdout=subprocess.PIPE)
# Best to always avoid shell=True because of security vulnerabilities.
proc.wait() # To make sure the shell script does not continue running indefinitely in the background
output, errors = proc.communicate()
print(output.decode())
# Since subprocess.communicate() returns a bytes-string, you can use .decode() to print the actual output as a string.
You can use
import subprocess
subprocess.call(['./c.sh'])
to call the shell script in python file
or
import subprocess
import shlex
subprocess.call(shlex.split('./c.sh var'))
I am invoking shell script using os.execvp() in python. my shell script has some echo statements whcih I want to redirect in file.
Here is what I am trying:
cmd = "/opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log"
cmdline = ["/bin/sh", cmd]
os.execvp(cmdline[0], cmdline)
Below is the error I am getting:
Error: /bin/sh: /opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log: No such file or directory
Can any one help?
This is happening because you are passing this entire string as if it were the program name to execute:
"/opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log"
The easy way to fix this is:
cmdline = ["/bin/sh", "/opt/rpm/rpm_upgrade.sh",
">&", "/opt/rpm/upgrader.log"]
os.execvp(cmdline[0], cmdline)
Now sh will receive three arguments rather than one.
Or you can switch to the more full-featured subprocess module, which lets you redirect output in Python:
import subprocess
with open("/opt/rpm/upgrader.log", "wb") as outfile:
subprocess.check_call(["/opt/rpm/rpm_upgrade.sh"], shell=True,
stdout=outfile, stderr=subprocess.STDOUT)
I have an .R file saved locally at the following path:
Rfilepath = "C:\\python\\buyback_parse_guide.r"
The command for RScript.exe is:
RScriptCmd = "C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe --vanilla"
I tried running:
subprocess.call([RScriptCmd,Rfilepath],shell=True)
But it returns 1 -- and the .R script did not run successfully. What am I doing wrong? I'm new to Python so this is probably a simple syntax error... I also tried these, but they all return 1:
subprocess.call('"C:\Program Files\R\R-2.15.2\bin\Rscript.exe"',shell=True)
subprocess.call('"C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe"',shell=True)
subprocess.call('C:\Program Files\R\R-2.15.2\bin\Rscript.exe',shell=True)
subprocess.call('C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe',shell=True)
Thanks!
The RScriptCmd needs to be just the executable, no command line arguments. So:
RScriptCmd = "\"C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe\""
Then the Rfilepath can actually be all of the arguments - and renamed:
RArguments = "--vanilla \"C:\\python\\buyback_parse_guide.r\""
It looks like you have a similar problem to mine. I had to reinstall RScript to a path which has no spaces.
See: Running Rscript via Python using os.system() or subprocess()
This is how I worked out the communication between Python and Rscript:
part in Python:
from subprocess import PIPE,Popen,call
p = subprocess.Popen([ path/to/RScript.exe, path/to/Script.R, Arg1], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
out = p.communicate()
outValue = out[0]
outValue contains the output-Value after executing the Script.R
part in the R-Script:
args <- commandArgs(TRUE)
argument1 <- as.character(args[1])
...
write(output, stdout())
output is the variable to send to Python
This question already has answers here:
Running Bash commands in Python
(11 answers)
Closed 9 months ago.
I read this somewhere a while ago but cant seem to find it. I am trying to find a command that will execute commands in the terminal and then output the result.
For example: the script will be:
command 'ls -l'
It will out the result of running that command in the terminal
There are several ways to do this:
A simple way is using the os module:
import os
os.system("ls -l")
More complex things can be achieved with the subprocess module:
for example:
import subprocess
test = subprocess.Popen(["ping","-W","2","-c", "1", "192.168.1.70"], stdout=subprocess.PIPE)
output = test.communicate()[0]
I prefer usage of subprocess module:
from subprocess import call
call(["ls", "-l"])
Reason is that if you want to pass some variable in the script this gives very easy way for example take the following part of the code
abc = a.c
call(["vim", abc])
import os
os.system("echo 'hello world'")
This should work. I do not know how to print the output into the python Shell.
Custom standard input for python subprocess
In fact any question on subprocess will be a good read
https://stackoverflow.com/questions/tagged/subprocess
for python3 use subprocess
import subprocess
s = subprocess.getstatusoutput(f'ps -ef | grep python3')
print(s)
You can also check for errors:
import subprocess
s = subprocess.getstatusoutput('ls')
if s[0] == 0:
print(s[1])
else:
print('Custom Error {}'.format(s[1]))
# >>> Applications
# >>> Desktop
# >>> Documents
# >>> Downloads
# >>> Library
# >>> Movies
# >>> Music
# >>> Pictures
import subprocess
s = subprocess.getstatusoutput('lr')
if s[0] == 0:
print(s[1])
else:
print('Custom Error: {}'.format(s[1]))
# >>> Custom Error: /bin/sh: lr: command not found
You should also look into commands.getstatusoutput
This returns a tuple of length 2..
The first is the return integer (0 - when the commands is successful)
second is the whole output as will be shown in the terminal.
For ls
import commands
s = commands.getstatusoutput('ls')
print s
>> (0, 'file_1\nfile_2\nfile_3')
s[1].split("\n")
>> ['file_1', 'file_2', 'file_3']
In python3 the standard way is to use subprocess.run
res = subprocess.run(['ls', '-l'], capture_output=True)
print(res.stdout)
The os.popen() is pretty simply to use, but it has been deprecated since Python 2.6.
You should use the subprocess module instead.
Read here: reading a os.popen(command) into a string
Jupyter
In a jupyter notebook you can use the magic function !
!echo "execute a command"
files = !ls -a /data/dir/ #get the output into a variable
ipython
To execute this as a .py script you would need to use ipython
files = get_ipython().getoutput('ls -a /data/dir/')
execute script
$ ipython my_script.py
You could import the 'os' module and use it like this :
import os
os.system('#DesiredAction')
Running: subprocess.run
Output: subprocess.PIPE
Error: raise RuntimeError
#! /usr/bin/env python3
import subprocess
def runCommand (command):
output=subprocess.run(
command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if output.returncode != 0:
raise RuntimeError(
output.stderr.decode("utf-8"))
return output
output = runCommand ([command, arguments])
print (output.stdout.decode("utf-8"))