I'm trying to run one python program from another using subprocess. Here's the function I've got so far:
def runProcess(exe):
p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while(True):
retcode = p.poll() #returns None while subprocess is running
line = p.stdout.readline()
yield line
if(retcode is not None):
break
then i run:
for line in runProcess('python myotherprogram.py'): print line
but I get an OS error: no such file, but it doesn't tell me what file doesn't exist. It's baffling. Any suggestions? I can use the runProcess function for normal terminal commands, such as ls.
What doesn't exist is a single executable named python myotherprogram.py. To specify arguments, you need to provide a list consisting of the command and its argument, such as with runProcess(["python", "myotherprogram.py"]), or specify shell=True to the Popen constructor.
The relevant quote from the documentation:
args should be a sequence of program arguments or else a single
string. By default, the program to execute is the first item in args
if args is a sequence. If args is a string, the interpretation is
platform-dependent and described below. See the shell and executable
arguments for additional differences from the default behavior. Unless
otherwise stated, it is recommended to pass args as a sequence.
On Unix, if args is a string, the string is interpreted as the name or
path of the program to execute. However, this can only be done if not
passing arguments to the program.
Related
This question already has answers here:
How to set environment variables in Python?
(19 answers)
Closed 6 years ago.
The following code allows me to dynamically identify and load the custom module if it is not located in any of the directory of sys.path variable
import sys
sys.path.append("/lib")
But, this gives me OSError
import subprocess
x = subprocess.Popen(["export", "PYTHONPATH=/lib"], stdout=subprocess.PIPE)
Not just this, even simple Linux/Unix variable declaration setting fails in subprocess.Popen()
import subprocess
x = subprocess.Popen("x=y", stdout=subprocess.PIPE)
I wanted to check subprocess as I tried setting PYTHONPATH via os.system(), os.popen() etc., and the variable did not set (may be it is set in the child process shell)
Try this:
>>> subprocess.call(["export foo=bar && echo foo=$foo"], shell=True)
foo=bar
0
>>>
There are several things that are going on here and are probably confusing you a little. One thing is, that whatever instructions given to Popen will be executed in the child process and will not affect your main process. You can merely Pipe or retrieve results from it.
First to comment on your second use case, where you use string as an argument. From the docs you can read:
class subprocess.Popen(args, bufsize=-1, executable=None, stdin=None,
stdout=None, stderr=None, preexec_fn=None, close_fds=True,
shell=False, cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0, restore_signals=True,
start_new_session=False, pass_fds=())
...
args should be a sequence of program arguments or else a single
string. By default, the program to execute is the first item in args
if args is a sequence. If args is a string, the interpretation is
platform-dependent and described below. See the shell and executable
arguments for additional differences from the default behavior. Unless
otherwise stated, it is recommended to pass args as a sequence.
On POSIX, if args is a string, the string is interpreted as the name
or path of the program to execute. However, this can only be done if
not passing arguments to the program.
So in your second case, you are trying to execute a file or program x=y which doesn't go.
Even if you use list, like in your first use case, you must be aware, that this isn't equivalent to passing code to the bash shell. If you want this, you can use shell=True as an keyword argument, but this has other issues as indicated by the docs. But your code will execute with shell=True.
If your sole purpose is to set environmental variable, then you should consider the option to use os.environ variable that maps your environmental variables to values (as indicated by #cdarke first).
I want to use a python script to show up all local administrators in our domain.
My Code :
for line in open(anfangsrechner,"r"):
zeile = line.strip()
command ='\\\\' +zeile+ ' -i' ' net' ' localgroup' ' Administratoren'
abfrage = subprocess.Popen(['PsExec.exe ',command,],stdin=subprocess.PIPE,
stdout=subprocess.PIPE, )
# print (abfrage)
while True:
line = abfrage.communicate()
if not line:
break
print (line)
But I only get this from the psexec command:
PsExec v2.1 - Execute processes remotely Copyright (C) 2001-2013 Mark
Russinovich Sysinternals - www.sysinternals.com
Process finished with exit code 0
I don't get the whole output. Does someone know how I can fix it?
You are passing the arguments as a long string, rather than a list.
The quick fix would be using shell=True:
abfrage = subprocess.Popen('PsExec.exe '+command,
stdout=subprocess.PIPE,
shell=True)
The right way to do this would be creating a list of arguments and passing it.
Quoting the documentation:
args is required for all calls and should be a string, or a sequence
of program arguments. Providing a sequence of arguments is generally
preferred, as it allows the module to take care of any required
escaping and quoting of arguments (e.g. to permit spaces in file
names). If passing a single string, either shell must be True (see
below) or else the string must simply name the program to be executed
without specifying any arguments.
I am trying to use subprocess.call() to execute a command-line program in python3. I can get it to work fine, the following example executes with no problems:
subprocess.call(['add_phenotype.py', '-t', threads, '-s'])
However, I want to parse a file, and then based on what I find, run the command with different flags. I can't figure out how to do this.
For example:
if zeroed_out_file:
args = '-z'
else:
args = ''
subprocess.call(['add_phenotype.py', '-t', threads, '-s', args])
fails if zeroed_out_file is FALSE. The add_phenotype.py script exits immediately claiming that it doesn't recognize the arguments.
The first argument takes a list; just build that list dynamically:
args = ['add_phenotype.py', '-t', threads, '-s']
if zeroed_out_file:
args.append('-z')
subprocess.call(args)
Appending additional command line switches is just a question of appending more values to args.
I tried executing a server daemon with gnu screen from subprocess call but it didn't even start
subprocess.call(["screen", "-dmS test ./server"])
I was told that running screen requires terminal, hence the reason why I can't simply execute it with call. Can you show me some piece of codes to do this?
Try
subprocess.call( ["screen", "-d", "-m", "-S", "test", "./server"] )
You need to break the argument string into separate arguments, one per string.
Here's the relevant quote from the subprocess docs:
On UNIX, with shell=False (default): In this case, the Popen class
uses os.execvp() to execute the child program. args should normally
be a sequence. A string will be treated as a sequence with the string
as the only item (the program to execute).
On UNIX, with shell=True: If args is a string, it specifies the
command string to execute through the shell. If args is a sequence,
the first item specifies the command string, and any additional items
will be treated as additional shell arguments.
So by default, the arguments are used exactly as you give them; it doesn't try to parse a string into multiple arguments. If you set shell to true, you could try the following:
subprocess.call("screen -dmS test ./server", shell=True)
and the string would be parsed exactly like a command line.
I'm using a crude IDE (Microchip MPLAB) with C30 toolchain on Windows XP.
The C compiler has a very noisy output that I'm unable to control, and it's very hard to spot actual warnings and errors in output window.
I want to write a python script that would receive arguments for compiler, call the compiler with same arguments, filter results and output them to stdout. Then I can replace the compiler executable with my script in toolchain settings. The IDE calls my script and receives filtered compiler output.
My code for executing the compiler looks like this:
arguments = ' '.join(sys.argv[1:])
cmd = '%s %s' % (compiler_path, arguments)
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
The problem is that quotes from arguments are consumed on script execution, so if IDE calls my script with following arguments:
main.c -o"main.o"
the value of arguments is
main.c -omain.o
The most obvious solution is to put whole argument list in quotes, but this would require modification in compiler calling code in IDE. I also tried using batch file, but it can only accept nine parameters (%1 to %9), and compiler is called with 15+ parameters.
Is there a way to forward exactly the same arguments to a process from script?
Give the command arguments to Popen as a list:
arguments = sys.argv[1:]
cmd = [compiler_path] + arguments
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
As ChristopheD said the shell removes the quotes.
But you don't need to create the string yourself when using Popen: it can handle that for you automatically. You can do this instead:
import sys, subprocess
process = subprocess.Popen(sys.argv[1:], executable=compiler_path, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
The subprocess module hopefully will pass the arguments correctly for you.
Your shell is eating the quotes (the python script never even receives them) so I suppose it's not very easy to get them 'unaltered'.