I am trying to write a python script that will execute a bash script I have on my Windows machine. Up until now I have been using the Cygwin terminal so executing the bash script RunModels.scr has been as easy as ./RunModels.scr. Now I want to be able to utilize subprocess of Python, but because Windows doesn't have the built in functionality to handle bash I'm not sure what to do.
I am trying to emulate ./RunModels.scr < validationInput > validationOutput
I originally wrote this:
os.chdir(atm)
vin = open("validationInput", 'r')
vout = open("validationOutput", 'w')
subprocess.call(['./RunModels.scr'], stdin=vin, stdout=vout, shell=True)
vin.close()
vout.close()
os.chdir(home)
But after spending a while trying to figure out why my access was denied, I realized my issue wasn't the file permissions but the fact that I was trying to execute a bash file on Windows in general. Can someone please explain how to execute a bash script with directed input/output on windows using a python script?
Edit (Follow up Question):
Thanks for the responses, I needed the full path to my bash.exe as the first param. Now however, command line calls from within RunModels.scr come back in the python output as command not found. For example, ls, cp, make. Any suggestions for this?
Follow up #2:
I updated my call to this:
subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'], stdin=vin, stdout=vout, cwd='C:\\path\\dir_where_RunModels\\')
The error I now get is /usr/bin/bash: RunModels.scr: No such file or directory.
Using cwd does not seem to have any effect on this error, either way the subprocess is looking in /usr/bin/bash for RunModels.scr.
SELF-ANSWERED
I needed to specify the path to RunModels.scr in the call as well as using cwd.
subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'C:\\path\\dir_where_RunModels\\RunModels.scr'], stdin=vin, stdout=vout, cwd='C:\\path\\dir_where_RunModels\\')
But another problem...
Regardless of specifying cwd, the commands executed by RunModels.scr are throwing errors as if RunModels.scr is in the wrong directory. The script executes, but cp and cd throw the error no such file or directory. If I navigate to where RunModels.scr is through the command line and execute it the old fashioned way I don't get these errors.
Python 3.4 and below
Just put bash.exe in first place in your list of subprocess.call arguments. You can remove shell=True, that's not necessary in this case.
subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'],
stdin=vin, stdout=vout,
cwd='C:\\path\\dir_where_RunModels\\')
Depending on how bash is installed (is it in the PATH or not), you might have to use the full path to the bash executable.
Python 3.5 and above
subprocess.call() has been effectively replaced by subprocess.run().
subprocess.run(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'],
stdin=vin, stdout=vout,
cwd='C:\\path\\dir_where_RunModels\\')
Edit:
With regard to the second question, you might need to add the -l option to the shell invocation to make sure it reads all the restart command files like /etc/profile. I presume these files contain settings for the $PATH in bash.
Edit 2:
Add something like pwd to the beginning of RunModels.scr so you can verify that you are really in the right directory. Check that there is no cd command in the rc-files!
Edit 3:
The error /usr/bin/bash: RunModels.scr: No such file or directory can also be generated if bash cannot find one of the commands that are called in the script. Try adding the -v option to bash to see if that gives more info.
A better solution than the accepted answer is to use the executable keyword argument to specify the path to your shell. Behind the curtain, Python does something like
exec([executable, '-c`, subprocess_arg_string])
So, concretely, in this case,
subprocess.call(
'./RunModels.scr',
stdin=vin, stdout=vout,
shell=True,
executable="C:/cygwin64/bin/bash.exe")
(Windows thankfully lets you use forward slashes instead of backslashes, so you can avoid the requirement to double the backslashes or use a raw string.)
Related
I guess I have to use either use os.system or subprocess.call, but I can't figure out how to use it.
I don't have the permission to edit the original folder.
subprocess.Popen('file.py', cwd=dirName) gives me 'The system cannot find the file specified' even though the file clearly exists
If I was typing in cmd,
cd directory
file.py -arg
Edit: Just to be clear I want to run another script using a python script
As you have tagged the question with cmd, I assume that you use Windows. Windows is kind enough to automatically use the appropriate command when you type a document name in cmd, but Python subprocess is not. So you have 2 possible ways here
use shell=True to ask a cmd interpretor to execute the command:
subprocess.Popen('file.py', cwd=dirName, shell=True)
pass explicitely the path of the Python interpretor (or the name if it is in the path)
subprocess.Popen([python_path, 'file.py'], cwd=dirName, shell=True)
At first you need to add the python in windows environment.
Then you can add the file path (the file that you want to run it on command line).
Then you can go to command line page and type the file name and use it like the line below:
FileName commands
for example :
pip install datetime
Its clear you are probably using python 3 rather than python 2. Otherwise you might use os.system as already suggested or commands. Commands is now obsolete as of python 3. To get this working I would use instead:
statusAndOutputText = subprocess.getstatusoutput( os.path.join( dirName, 'file.py' ) )
This will definitely work. I've used it many times in python 3 and it will give you the status in statusAndOutputText[0] and output buffer in statusAndOutputText[1] which are both very useful to have.
So, I am trying to run a Shell script from Python and I double checked that the location of the script.sh is all correct (because when I run it from sublime, the script.sh opens). What I have to call script.sh is:
subprocess.call("script.sh", shell=True)
When I run that, the function returns 0. However, the script is supposed to create a file in my folder and write into it, which it is not doing. It does work when I run the script from cygwin command prompt.
What could be wrong?
Please ensure you have added:
!/bin/bash
as the first line and also make sure that the file script.sh has executable permission.
chmod u+x script.sh
then try specifying the complete path:
subprocess.call("/complete/path/script.sh", shell=True)
At the top of your shell script somewhere, put the line:
pwd >/tmp/mytempfile
and then run the Python script and go look into that file.
This will let you find out the working directory of your scripts which, in the majority of cases where a file doesn't appear to be created, is different to what you think it should be.
You may also want to check the shell script to ensure it's not changing the working directory before creating the file but that would be unlikely given you've stated it works okay from the command line.
If you add the line to create the temporary file and it doesn't actually get created, then your script is not executing.
In that case, you could try a few things. The first is to fully specify the script on the off chance that Python isn't in the correct directory. In other words, something like:
subprocess.call("/actual/path/to/script.sh", shell=True)
Or you could try to run the actual bash executable with the script as an argument:
subprocess.call("bash -c script.sh", shell=True)
Yes, I know I can do
python2 cal.py
What I am asking for is a way to execute it on the command line such as:
calpy
and then the command afterwards. I put in in a path and when I write cal.py in the command line:
/usr/bin/cal.py: line 5: print: command not found
I don't want to issue cal.py to run my script, I want it to be issued with calpy
I'm running Arch Linux if that helps, thanks. Sorry for my English.
In order for bash to know to run your script via the Python interpreter, you need to put an appropriate shebang at the start. For example:
#!/usr/bin/python
tells bash to run /usr/bin/python with your script as the first argument. I personally prefer
#!/usr/bin/env python
which is compatible with virtualenv. You also need to ensure that the permissions on your script allow it to be executed:
~$ chmod +x path/to/cal.py
Finally, in order to call cal rather than path/to/cal.py, you need to remove the .py extension and make sure that the directory containing cal is in your command search path. I prefer to add ~/bin to the search path by modifying the $PATH environment variable in ~/.bashrc:
export PATH=$HOME/bin:$PATH
then put my own executables in ~/bin. You could also copy (or symlink) cal to one of the system-wide binary directories (/bin or /usr/bin), but I consider it bad practice to mess with system-wide directories unnecessarily.
Ok, you need a couple of things for achive what you want.
First you have to tell your script "How" is going to execute/interpret it. You can do this writting
#/usr/bin/env python
at the very beggining of the file.
The problem you have is the system is trying to execute the script using bash. And in bash there is no print command.
Second you need give execution privileges to your script. And of course if you want to call your script through the command "calcpy", the script has to be called like that.
Put this (exactly this) as the first line of your script:
#!/usr/bin/env python
I am looking to open a Python script in edit mode in IDLE by passing in a directory from another Python script. I understand that I can use os.system to execute idle.py but I then don't know how to pass the appropriate -e parameter and get it to use a specific directory i.e. open something.py in diredtory C:\Python all from the original Python script.
Thanks for your help,
Ben
You can use subprocess.call(). By setting shell = True, the function treats the string as a literal shell command. Modify the paths as you see fit.
import subprocess
subprocess.call(r'C:\Python27\Lib\idlelib\idle.py -e C:\Python27\something.py',
shell=True)
I need to execute the simple command below in windows 7 command prompt using Python26.
cd C:\Python26\main project files\Process
C:\Aster\runtime\waster Analysis.comm
It runs a FEM simulation and I tried it manually and it worked well. Now, I want to automate the write procedure using Python26.
I studied the other questions and found that the os.system works but it didn't. Also I saw subprocess module but it didn't work.
The current directory is a process property: Every single process has its own current directory. A line like
os.system("cd xyz")
starts a command interpreter (cmd.exe on Windows 7) and execute the cd command in this subprocess, not affecting the calling process in any way. To change the directory of the calling process, you can use os.chdir() or the cwd keyword parameter to subprocess.Popen().
Example code:
p = subproces.Popen(["C:/Aster/runtime/waster", "Analysis.comm"],
cwd="C:/Python26/main project files/Process")
p.wait()
(Side notes: Use forward slashes in path names in Python files. You should avoid os.system() and passing shell=True to the function in the subprocess module unless really necessary.)