We Keep Coding

Run terminal script in python?

I am using python and I am trying to run a shell script that is located in another folder I am trying
subprocess.call(['source','../Apps/appName/run'])
Where 'run' is a shell script I wrote and made an executable. But it keeps giving errors such as
No such file or directory or **No such file or directory: "source"
I have also tried the following
subprocess.call(['source','../Apps/appName/run'])
subprocess.call(['source ../Apps/appName/run'])
subprocess.call(['.','../Apps/appName/run'])
I am trying to run the script and leave it alone (as in I do not want any return value or to see what the output of the shell script is.
Thank you in advance

source is a shell builtin that reads a file and interprets the commands in the current shell instance. It's similar to C #include or Python import. You should not be using it to run shell scripts.
The correct way to run a script is to add a shebang like #!/bin/bash to the first line, and chmod +x yourscriptfile. The OS will then consider it a real executable, which can be executed robustly from any context. In Python, you would do:
subprocess.call(['../Apps/appName/run'])
If for whichever reason this is not an option, you can instead explicitly invoke bash on the file, since this is similar to what would happen if you're in bash and type source:
subprocess.call(['bash', '../Apps/appName/run'])

Run a script from different folder with arguments in python (like in cmd)

I guess I have to use either use os.system or subprocess.call, but I can't figure out how to use it.
I don't have the permission to edit the original folder.
subprocess.Popen('file.py', cwd=dirName) gives me 'The system cannot find the file specified' even though the file clearly exists
If I was typing in cmd,
cd directory
file.py -arg
Edit: Just to be clear I want to run another script using a python script

As you have tagged the question with cmd, I assume that you use Windows. Windows is kind enough to automatically use the appropriate command when you type a document name in cmd, but Python subprocess is not. So you have 2 possible ways here
use shell=True to ask a cmd interpretor to execute the command:
subprocess.Popen('file.py', cwd=dirName, shell=True)
pass explicitely the path of the Python interpretor (or the name if it is in the path)
subprocess.Popen([python_path, 'file.py'], cwd=dirName, shell=True)

At first you need to add the python in windows environment.
Then you can add the file path (the file that you want to run it on command line).
Then you can go to command line page and type the file name and use it like the line below:
FileName commands
for example :
pip install datetime

Its clear you are probably using python 3 rather than python 2. Otherwise you might use os.system as already suggested or commands. Commands is now obsolete as of python 3. To get this working I would use instead:
statusAndOutputText = subprocess.getstatusoutput( os.path.join( dirName, 'file.py' ) )
This will definitely work. I've used it many times in python 3 and it will give you the status in statusAndOutputText[0] and output buffer in statusAndOutputText[1] which are both very useful to have.

Execute bash script from Python on Windows

I am trying to write a python script that will execute a bash script I have on my Windows machine. Up until now I have been using the Cygwin terminal so executing the bash script RunModels.scr has been as easy as ./RunModels.scr. Now I want to be able to utilize subprocess of Python, but because Windows doesn't have the built in functionality to handle bash I'm not sure what to do.
I am trying to emulate ./RunModels.scr < validationInput > validationOutput
I originally wrote this:
os.chdir(atm)
vin = open("validationInput", 'r')
vout = open("validationOutput", 'w')
subprocess.call(['./RunModels.scr'], stdin=vin, stdout=vout, shell=True)
vin.close()
vout.close()
os.chdir(home)
But after spending a while trying to figure out why my access was denied, I realized my issue wasn't the file permissions but the fact that I was trying to execute a bash file on Windows in general. Can someone please explain how to execute a bash script with directed input/output on windows using a python script?
Edit (Follow up Question):
Thanks for the responses, I needed the full path to my bash.exe as the first param. Now however, command line calls from within RunModels.scr come back in the python output as command not found. For example, ls, cp, make. Any suggestions for this?
Follow up #2:
I updated my call to this:
subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'], stdin=vin, stdout=vout, cwd='C:\\path\\dir_where_RunModels\\')
The error I now get is /usr/bin/bash: RunModels.scr: No such file or directory.
Using cwd does not seem to have any effect on this error, either way the subprocess is looking in /usr/bin/bash for RunModels.scr.
SELF-ANSWERED
I needed to specify the path to RunModels.scr in the call as well as using cwd.
subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'C:\\path\\dir_where_RunModels\\RunModels.scr'], stdin=vin, stdout=vout, cwd='C:\\path\\dir_where_RunModels\\')
But another problem...
Regardless of specifying cwd, the commands executed by RunModels.scr are throwing errors as if RunModels.scr is in the wrong directory. The script executes, but cp and cd throw the error no such file or directory. If I navigate to where RunModels.scr is through the command line and execute it the old fashioned way I don't get these errors.

Python 3.4 and below
Just put bash.exe in first place in your list of subprocess.call arguments. You can remove shell=True, that's not necessary in this case.
subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'],
stdin=vin, stdout=vout,
cwd='C:\\path\\dir_where_RunModels\\')
Depending on how bash is installed (is it in the PATH or not), you might have to use the full path to the bash executable.
Python 3.5 and above
subprocess.call() has been effectively replaced by subprocess.run().
subprocess.run(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'],
stdin=vin, stdout=vout,
cwd='C:\\path\\dir_where_RunModels\\')
Edit:
With regard to the second question, you might need to add the -l option to the shell invocation to make sure it reads all the restart command files like /etc/profile. I presume these files contain settings for the $PATH in bash.
Edit 2:
Add something like pwd to the beginning of RunModels.scr so you can verify that you are really in the right directory. Check that there is no cd command in the rc-files!
Edit 3:
The error /usr/bin/bash: RunModels.scr: No such file or directory can also be generated if bash cannot find one of the commands that are called in the script. Try adding the -v option to bash to see if that gives more info.

A better solution than the accepted answer is to use the executable keyword argument to specify the path to your shell. Behind the curtain, Python does something like
exec([executable, '-c`, subprocess_arg_string])
So, concretely, in this case,
subprocess.call(
'./RunModels.scr',
stdin=vin, stdout=vout,
shell=True,
executable="C:/cygwin64/bin/bash.exe")
(Windows thankfully lets you use forward slashes instead of backslashes, so you can avoid the requirement to double the backslashes or use a raw string.)

Why is Python Popen using a different executable

I have a program with two different versions on this computer. I can't get rid of the older one because I don't have root access, but I put the newer one first in 'bin' in my home directory (which is the first thing in my $PATH)
I tried calling it with Python's Popen
Popen(['clingo'...]...)
and it worked fine
But then I needed to set an environment variable before calling it so I renamed 'clingo' to 'run_clingo' in the bin directory and replaced it with a script:
# File: ~/bin/clingo
export LD_LIBRARY_PATH=/usr/local/gcc4.8.1/lib64:$LD_LIBRARY_PATH
run_clingo $#
When I run 'clingo' from terminal, it works fine, but Python Popen calls the older version (which is later on the PATH).
How can I get Popen to call the right one? Why does changing an executable to a script cause Popen to search elsewhere?

Add shebang #!/bin/sh at the very top of ~/bin/clingo and run:
$ chmod +x ~/bin/clingo
to make the file executable.
Popen() calls os.execvp() to execute a program. Its behaviour differs from your shell that emulates execvp(): if there is no the shebang (!#.. at the top) in the script then the shell reruns it as a shell script but Python's os.execvp() runs the next file in PATH instead.

Wild guess is that Popen does not use the same PATH as your terminal. I would check the shebang line (#!) sometimes people put env -i python there, which explicitly asks fro empty, i.e. default environment.
Note that subprocess.call is the recommended way, not using the Popen directly.

Calling multiple commands using os.system in Python

I would like to invoke multiple commands from my python script.
I tried using the os.system(), however, I'm running into issues when the current directory is changed.
example:
os.system("ls -l")
os.system("<some command>") # This will change the present working directory
os.system("launchMyApp") # Some application invocation I need to do.
Now, the third call to launch doesn't work.

os.system is a wrapper for the C standard library function system(). Its argument can be any valid shell command as long as it fits into the memory reserved for environment and argument lists of a process.
So, delimit each command with a semicolon or a newline and they will be executed one after another in the same environment.
os.system(" ls -l; <some command>; launchMyApp")
os.system('''
ls -l
<some command>
launchMyApp
''')

Try this
import os
os.system("ls -l")
os.chdir('path') # This will change the present working directory
os.system("launchMyApp") # Some application invocation I need to do.

Each process has its own current working directory. Normally, child processes can't change parent's directory that is why cd is a builtin shell command: it runs in the same (shell) process.
Each os.system() call creates a new shell process. Changing the directory inside these processes has no effect on the parent python process and therefore on the subsequent shell processes.
To run multiple commands in the same shell instance, you could use subprocess module:
#!/usr/bin/env python
from subprocess import check_call
check_call(r"""set -e
ls -l
<some command> # This will change the present working directory
launchMyApp""", shell=True)
If you know the destination directory; use cwd parameter suggested by #Puffin GDI instead.

It’s simple, really.
For Windows separate your commands with &, for Linux, separate them with ;.
str.replace is a very good way to approach the problem, used in the example below:
import os
os.system('''cd /
mkdir somedir'''.replace('\n', ';')) # or use & for Windows

When you call os.system(), every time you create a subshell - that closes immediately when os.system returns (subprocess is the recommended library to invoke OS commands). If you need to invoke a set of commands - invoke them in one call.
BTW, you may change working director from Python - os.chdir

Try to use subprocess.Popen and cwd
example:
subprocess.Popen('launchMyApp', cwd=r'/working_directory/')

os.system("ls -l && <some command>")

You can change back to the directory you need to be in with os.chdir()

Just use
os.system("first command\nsecond command\nthird command")
I think you have got the idea what to do
Note: This is not a very reliable approach if you are doing a complex
job using CLI tools. Popen and subprocess methods are better there.
Although small task link copy, move, list will work fine
.