This is how my code looks and I get an error, while using Popen
test.py
import subprocess
import sys
def test(jobname):
print jobname
p=subprocess.Popen([sys.executable,jobname,parm1='test',parm2='test1'])
if __name__ == "__main__":
test(r'C:\Python27\test1.py')
test1.py
def test1(parm1,parm2):
print 'test1',parm1
if __name__ = '__main__':
test1(parm1='',parm2='')
Error
Syntax error
In test1.py:
You need two equal signs in :
if __name__ = '__main__':
Use instead
if __name__ == '__main__':
since you want to compare the value of __name__ with the string '__main__', not assign a value to __name__.
In test.py:
parm1='test' is a SyntaxError. You can not to assign a value to a variable in the middle of a list:
p=subprocess.Popen([sys.executable,jobname,parm1='test',parm2='test1'])
It appears you want to feed different values for parm1 and parm2 into the function test1.test1. You can not do that by calling python test1.py since there parm1='' and parm2='' are hard-coded there.
When you want to run a non-Python script from Python, use subprocess. But when you want to run Python functions in a subprocess, use multiprocessing:
import multiprocessing as mp
import test1
def test(function, *args, **kwargs):
print(function.__name__)
proc = mp.Process(target = function, args = args, kwargs = kwargs)
proc.start()
proc.join() # wait for proc to end
if __name__ == '__main__':
test(test1.test1, parm1 = 'test', parm2 = 'test1')
Related
I have a script test.py and I want it to execute another script this_other_script.py which will return a list object. test.py looks like:
if __name__ == '__main__':
someValue = this_other_script
print(len(someValue))
this_other_script.py looks like:
if __name__ == '__main__':
data = [a,b,c,d]
return(data)
When I run test.py I receive an error of SyntaxError: 'return' outside function.
If this is due to program scope I would have thought it would be OK for the calling program to be given a return value from a program it is calling. I wouldn't expect for this_other_script to access a value of a variable not given to it by test.py so I'm not sure why this error is displayed.
in test.py:
if __name__ == '__main__':
import this_other_script
someValue = this_other_script.get_data()
print(len(someValue))
in this_other_script.py:
def get_data():
data = [1,2,3,4]
return(data)
alternative answer:
in test.py
if __name__ == '__main__':
import this_other_script
someValue = this_other_script.get_data()
print(len(someValue))
in this_other_script.py:
def get_data():
data = [1,2,3,4]
return(data)
if __name__ == '__main__':
get_data()
So I am trying to execute a file and get the returned value back using the python builtin methods available in the subprocess library.
For example, lets say I want to execute this hello_world python file:
def main():
print("in main")
return("hello world!")
if __name__ == '__main__':
main()
I do not care about getting back the statement in main. What I want to get back is the return value hello world!.
I tried numerous things but non of them worked.
Here's a list of what I tried and their outputs:
args is common for all trials:
args = ['python',hello_cmd]
First trial:
p1 = subprocess.Popen(args, stdout=subprocess.PIPE)
print(p1.communicate())
print("returncode is:")
print(p1.returncode)
output is:
(b'in main\n', None)
returncode is:
0
second trial:
p2 = subprocess.check_output(args,stderr=subprocess.STDOUT)
print(p2)
output is:
b'in main\n'
third trial:
output, result = subprocess.Popen(args, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell=False).communicate()
print(output)
print(result)
output is:
b'in main\n'
b''
fourth trial:
p4 = subprocess.run(args, stdout=PIPE, stderr=PIPE)
print(p4)
output is:
CompletedProcess(args=['python', '/path/to/file/hello.py'], returncode=0, stdout=b'in main\n', stderr=b'')
fifth trial:
p5 =subprocess.getstatusoutput(args)
print(p5)
output is:
(0, '')
Can anyone help?
The return value of the main function is not the return code that is passed to the OS. To pass a return code to the OS use sys.exit(), which expects an integer. You can pass it a string, but if you do, Python will pass 1 to the OS.
You cannot return strings as return codes it must be an integer. If you want to act differently depending on the process. Try to map your return code to some function in your main program. For example
def execute_sub_program(): ...
# somewhere else:
return_code = execute_sub_program()
if return_code == 0:
# do something ...
elif ...
You can try with subprocess.run().returncode, it gives 0 if successful execution and 1 if failed execution.
driver.py
import subprocess
args = ['python', './hello_cmd.py']
status_code = subprocess.run(args).returncode
print(["Successful execution", "Failed execution"][status_code])
For happy flow (hello_cmd.py):
def main():
print("in main")
return("hello world!")
if __name__ == '__main__':
main()
For failed flow (hello_cmd.py):
def main():
print("in main")
raise ValueError('Failed')
if __name__ == '__main__':
main()
This is my main python script:
import time
import subprocess
def main():
while(True):
a=input("Please enter parameter to pass to subprocess:")
subprocess.Popen(args="python child.py")
print(f"{a} was started")
time.sleep(5)
if __name__ == '__main__':
main()
This is python child script named child.py:
def main(a):
while(True):
print(a)
if __name__ == '__main__':
main(a)
How to pass value to argument a which is in the child subprocess?
You need to use command line arguments, like this;
import time
import subprocess
def main():
while(True):
a=input("Please enter parameter to pass to subprocess:")
subprocess.Popen(["python", "child.py", a])
print(f"{a} was started")
time.sleep(5)
if __name__ == '__main__':
main()
child.py:
import sys
def main(a):
while(True):
print(a)
if __name__ == '__main__':
a = sys.argv[1]
main(a)
You may use subprocess.PIPE to pass data between your main process and spawned subprocess.
Main script:
import subprocess
def main():
for idx in range(3):
a = input(
'Please enter parameter to pass to subprocess ({}/{}): '
.format(idx + 1, 3))
print('Child in progress...')
pipe = subprocess.Popen(
args='python child.py',
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
pipe.stdin.write(str(a).encode('UTF-8'))
pipe.stdin.close()
print('Child output is:')
print(pipe.stdout.read().decode('UTF-8'))
if __name__ == '__main__':
main()
Child script:
import sys
import time
def main(a):
for dummy in range(3):
time.sleep(.1)
print(a)
if __name__ == '__main__':
a = sys.stdin.read()
main(a)
Output:
>>> python main.py
Please enter parameter to pass to subprocess (1/3): qwe
Child in progress...
Child output is:
qwe
qwe
qwe
Please enter parameter to pass to subprocess (2/3): qweqwe
Child in progress...
Child output is:
qweqwe
qweqwe
qweqwe
Please enter parameter to pass to subprocess (3/3): 123
Child in progress...
Child output is:
123
123
123
The easiest way to pass arguments to a child process is to use command line parameters.
The first step is to rewrite child.py so that it accepts command line arguments. There is detailed information about parsing command line arguments in this question: How to read/process command line arguments? For this simple example though, we will simply access the command line arguments through sys.argv.
import sys
def main(a):
while(True):
print(a)
if __name__ == '__main__':
# the first element in the sys.argv list is the name of our program ("child.py"), so
# the argument the parent process sends to the child process will be the 2nd element
a = sys.argv[1]
main(a)
Now child.py can be started with an argument like python child.py foobar and the text "foobar" will be used as the value for the a variable.
With that out of the way, all that's left is to rewrite parent.py and make it pass an argument to child.py. The recommended way to pass arguments with subprocess.Popen is as a list of strings, so we'll do just that:
import time
import subprocess
def main():
while(True):
a = input("Please enter parameter to pass to subprocess:")
subprocess.Popen(["python", "child.py", a]) # pass a as an argument
print(f"{a} was started")
time.sleep(5)
if __name__ == '__main__':
main()
Lets assume that i am starting a process in python with the following code:
from multiprocessing import Process
import time
def f(name):
print ('hello ', name)
if __name__ == '__main__':
p = Process(target=f,name = "Process-1", args=('bob',))
p.start()
Now,i want to terminate the process.I can simply do:
p.terminate()
However, i would like to terminate the process by its name.Is that possible?
To do that, you need to store a map between your process objects and their names. Using an helper function it makes your code even easier to read (IMO):
def terminate(procname):
return pmap[procname].terminate()
if __name__ == '__main__':
pmap = {}
pname = "process-1"
p = Process(target=f,name = pname, args=('bob',))
pmap[pname] = p
p.start()
Then to terminate:
terminate(pname)
I am running Windows 7.
In my main module, I call a function, A, which is in module a.
import a
a.A(listOfInputTuplesForB)
A calls multiple instances of the function B:
import multiprocessing as mp
def A(listOfInputTuplesForB):
if __name__ == '__main__':
pool = mp.Pool(processes = mp.cpu_count())
pool.map(poolWrapperForB, listOfInputTuplesForB)
pool.close()
pool.join()
def poolWrapperForB(inputTuple):
return B(*inputTuple)
def B(arg1, arg2, arg3):
print "I did nothing with my arguments!"
Now, obviously, when I run my main module, nothing happens, as the conditional if __name__ == '__main__' fails, since __name__ == 'a'.
Where should if __name__ == '__main__' go in this program?
Remove main from def A(listOfInputTuplesForB): and put it in the other file.
import a
if __name__ == "__main__":
a.A(listOfInputTuplesForB)