I'm working with python in a jupyter notebook.
I want to execute the following command :
$ gdalbuildvrt tmp_merge files
with tmp_merge being an output file of the function and set to : /home/prambaud/gfc_results/test/tmp_tile.vrt
and files being all the tiles to merge in the vrt file set to :/home/prambaud/gfc_results/test/tile_*.tif
This function authorize the use of the wildcard.
To run it in my Jupyter notebook I use the subprocess module:
command = [
'gdalbuildvrt',
'/home/prambaud/gfc_results/test/tmp_tile.vrt',
'/home/prambaud/gfc_results/test/tile_*.tif'
]
process = subprocess.run(
command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True,
#cwd=os.path.expanduser('~')
)
print(process. stdout)
as a result I obtain the following :
0...10...20...30...40...50...60...70...80...90...100 - done.
with no error messages. BUT the output file is not created. Does anyone know what could prevent the subprocess.run function to create and write in a file ?
PS:
I've also tried to run the command from the jupyter notebook with ! and the same parameters and the tmp file have of course bee created...
My command can only be executed from shell so using subprocess I need to add the shell keyword as :
process = subprocess.run(
command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True,
shell=True
)
Related
Hey i'm trying to run a shell Script with python using the Following lines:
import subprocess
shellscript = subprocess.Popen(["displaySoftware.sh"], stdin=subprocess.PIPE)
shellscript.stdin.write("yes\n")
shellscript.stdin.close()
returncode = shellscript.wait()
But when I run the Program it says that it can't find the .sh file.
Your command is missing "sh", you have to pass "shell=True" and "yes\n" has to be encoded.
Your sample code should look like this:
import subprocess
shellscript = subprocess.Popen(["sh displaySoftware.sh"], shell=True, stdin=subprocess.PIPE )
shellscript.stdin.write('yes\n'.encode("utf-8"))
shellscript.stdin.close()
returncode = shellscript.wait()
This method might be better:
import subprocess
shellscript = subprocess.Popen(["displaySoftware.sh"], shell=True, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
returncode = shellscript.communicate(input='yes\n'.encode())[0]
print(returncode)
When running this on my machine the "displaySoftware.sh" script, that is in the same directory as the python script, is successfully executed.
I cant figure out how to start the server using a python command.
s = subprocess.Popen('"D:\MC SERVER 2k19\server.jar" -jar server.jar java', stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True)
This code runs without error but doesn't start the server in cmd.
Thanks.
It's got to do with how you're passing your arguments.
subprocess.Popen(['java', '-jar', 'server.jar'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True, cwd='D:\MC SERVER 2k19')
If you need to start a Java program using CMD process from Python and show the window, you can use subprocess to call open another CMD terminal and run the command.
In Windows you will need to CMD-escape spaces in the path you passing to the secondary CMD process. This is done with the carrot ^
proc = subprocess.Popen(
['start', 'cmd', '/k', "D:\\MC^ SERVER^ 2k19\\server.jar",
'-jar', 'server.jar', 'java'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True
)
Keep in mind you will NOT be able to retrieve any output from the secondary CMD process from Python.
I.e. the process will return nothing.
proc.communicate()
# returns:
(b'', b'')
I am invoking shell script using os.execvp() in python. my shell script has some echo statements whcih I want to redirect in file.
Here is what I am trying:
cmd = "/opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log"
cmdline = ["/bin/sh", cmd]
os.execvp(cmdline[0], cmdline)
Below is the error I am getting:
Error: /bin/sh: /opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log: No such file or directory
Can any one help?
This is happening because you are passing this entire string as if it were the program name to execute:
"/opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log"
The easy way to fix this is:
cmdline = ["/bin/sh", "/opt/rpm/rpm_upgrade.sh",
">&", "/opt/rpm/upgrader.log"]
os.execvp(cmdline[0], cmdline)
Now sh will receive three arguments rather than one.
Or you can switch to the more full-featured subprocess module, which lets you redirect output in Python:
import subprocess
with open("/opt/rpm/upgrader.log", "wb") as outfile:
subprocess.check_call(["/opt/rpm/rpm_upgrade.sh"], shell=True,
stdout=outfile, stderr=subprocess.STDOUT)
I have an .R file saved locally at the following path:
Rfilepath = "C:\\python\\buyback_parse_guide.r"
The command for RScript.exe is:
RScriptCmd = "C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe --vanilla"
I tried running:
subprocess.call([RScriptCmd,Rfilepath],shell=True)
But it returns 1 -- and the .R script did not run successfully. What am I doing wrong? I'm new to Python so this is probably a simple syntax error... I also tried these, but they all return 1:
subprocess.call('"C:\Program Files\R\R-2.15.2\bin\Rscript.exe"',shell=True)
subprocess.call('"C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe"',shell=True)
subprocess.call('C:\Program Files\R\R-2.15.2\bin\Rscript.exe',shell=True)
subprocess.call('C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe',shell=True)
Thanks!
The RScriptCmd needs to be just the executable, no command line arguments. So:
RScriptCmd = "\"C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe\""
Then the Rfilepath can actually be all of the arguments - and renamed:
RArguments = "--vanilla \"C:\\python\\buyback_parse_guide.r\""
It looks like you have a similar problem to mine. I had to reinstall RScript to a path which has no spaces.
See: Running Rscript via Python using os.system() or subprocess()
This is how I worked out the communication between Python and Rscript:
part in Python:
from subprocess import PIPE,Popen,call
p = subprocess.Popen([ path/to/RScript.exe, path/to/Script.R, Arg1], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
out = p.communicate()
outValue = out[0]
outValue contains the output-Value after executing the Script.R
part in the R-Script:
args <- commandArgs(TRUE)
argument1 <- as.character(args[1])
...
write(output, stdout())
output is the variable to send to Python
There's a file named startup.cmd that sets some environment variables, runs some preparation commands, then does:
start "startup" cmd /k
Which opens a command shell named startup. The manual process I'm trying to automate is to then enter the following command into this shell: get startup.xml. I thought the correct way to do this in Python would be something like this:
import subprocess
p = subprocess.Popen('startup.cmd', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
getcommand = 'get startup.xml'
servercommand = 'startserver'
p.stdin.write(getcommand)
p.stdin.write(startserver)
(stdoutdata, stderrdata) = p.communicate()
print stdoutdata
print stderrdata
But those commands don't seem to be executing in the shell. What am I missing? Also, the command shell appears regardless of whether shell is set to True or False.
I found this warning in subprocess's document,
Warning Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
So my suggestion is to use communicate to send your command.
import subprocess
p = subprocess.Popen('startup.cmd', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
command = 'get startup.xml\n'
command += 'startserver\n'
(stdoutdata, stderrdata) = p.communicate(command)
print stdoutdata
print stderrdata
This is a new process, so one cannot communicate directly with Popen.