I have 2 programs, one is calling the other through subprocess. Running this in pyCharm. My issue is that the call to the second program doesn't print out the desired string (see programs). What am I doing wrong, or is my understanding of the subprocess wrong?
this is something.py:
import subprocess
def func():
print("this is something")
sb = subprocess.call("diff.py", shell=True)
return sb
if __name__=="__main__":
func()
this is diff.py:
print("this is diff running")
def caller():
print("this is diff running called from name main")
if __name__=="__main__":
caller()
I decided to try subprocessing instead of importing for the purpose of running the calls concurrently in diff threads in the future. For now I just wanted to make sure I grasp subprocessing but I'm stuck at this basic level with this issue and get figure it out.
You must use python to run python files.
import subprocess
def func():
print("this is something")
sb = subprocess.call("python diff.py", shell=True)
# It is also important to keep returns in functions
return sb
if __name__=="__main__":
func()
I would be careful to understand the layout of how pycharm saves files. Maybe consider trying to run a program that already exists for the Windows command line if you are just trying to learn about the subprocess module.
import subprocess
print("this is where command prompt is located")
sb = subprocess.call("where cmd.exe", shell=True)
returns
this is where command prompt is located
C:\Windows\System32\cmd.exe
Thank you.
subprocess.call("python something.py", shell=True) now works as intended but for some reason the very same call from pyCharm does not return the second string from diff.py I assume the issue is with pyCharm then
To run diff.py script from the current directory using the same Python interpreter that runs the parent script:
#!/usr/bin/env python
import sys
from subprocess import check_call
check_call([sys.executable, 'diff.py'])
do not use shell=True unless it is necessary e.g., unless you need to run an internal command such as dir, you don't need shell=True in most cases
use check_call() instead of call() to raise an exception if the child script returns with non-zero exit code
Why[When] I try python something.py pyCharm fires up to interpret it.
You should associate .py extension with py (Python launcher). Though if running the command:
T:\> python something.py
literally brings up PyCharm with the file something.py opened instead of running the script using a Python interpreter then something is really broken. Find out what program is run then you type python (without arguments).
Make sure you understand the difference between:
subprocess.Popen(['python', 'diff.py'])
subprocess.Popen('diff.py')
subprocess.Popen('diff.py', shell=True)
os.startfile('diff.py')
os.startfile('diff.py', 'edit')
Try to run it from the command-line (cmd.exe) and from IDLE (python3 -m idlelib) and see what happens in each case.
You should prefer to import the Python module and use multiprocessing, threading modules if necessary to run the corresponding functions instead of running the Python script as a child process via subprocess module.
Related
I have a Python script that I want to be able to debug from the command line. Let's say it's called simple.py, and it contains the following code:
var1 = "Hello"
var2 = "World"
print(var1, var2)
I want to debug this. Of course I can use the debugger from the command line:
python -m pdb simple.py
But I want to debug this specifically from within another python file. I've tried using subprocess to do this, by putting this code into another script, debug.py:
import subprocess
subprocess.Popen(["bash", "-ic", "python -m pdb simple.py"])
When I run it in a Bash shell, this happens:
$ python debug.py
$ > /[path]/simple.py(1)<module>()
-> var1 = "Hello"
(Pdb) n
bash: n: command not found
$
So I've typed "n", expecting the debugger to move to the next line. Instead, it seems to just go straight back to Bash, and the debugger doesn't do anything.
Any idea why this happens? Is there a way to spawn a Bash shell containing a Python debugger from within a Python script, which I can actually use to debug?
Your debug.py finish its run and you are back to the shell (try typing ls instead, and see what happens)
What you are looking for is a way to interact with the other process, you need to get the input from your stdin and pass it to the other process stdin. It can looks something like:
import subprocess
p = subprocess.Popen(["python3", "-m", "pdb", "test.py"])
while True:
cmd = input()
p.stdin.write(cmd.encode())
Not sure if this is possible. I have a set of python scripts and have modified the linux PATH in ~/.bashrc so that whenever I open a terminal, the python scripts are available to run as a command.
export PATH=$PATH:/home/user/pythonlib/
my_command.py resides in the above path.
I can run my_command.py (args) from anywhere in terminal and it will run the python scripts.
I'd like to control this functionality from a different python script as this will be the quickest solution to automating my processing routines. So I need it to open a terminal and run my_command.py (args) from within the python script I'm working on.
I have tried subprocess:
import subprocess
test = subprocess.Popen(["my_command.py"], stdout=subprocess.PIPE)
output = test.communicate()[0]
While my_command.py is typically available in any terminal I launch, here I have no access to it, returns file not found.
I can start a new terminal using os then type in my_command.py, and it works
os.system("x-terminal-emulator -e /bin/bash")
So, is there a way to get the second method to accept a script you want to run from python with args?
Ubuntu 16
Thanks :)
Popen does not load the system PATH for the session you create in a python script. You have to modify the PATH in the session to include the directory to your project like so:
someterminalcommand = "my_command.py (args)"
my_env = os.environ.copy()
my_env["PATH"] = "/home/usr/mypythonlib/:" + my_env["PATH"]
combine = subprocess.Popen(shlex.split(someterminalcommand), env=my_env)
combine.wait()
This allows me to run my "my_command.py" file from a different python session just like I had a terminal window open.
If you're using Gnome, the gnome-terminal command is rather useful in this situation.
As an example of very basic usage, the following code will spawn a terminal, and run a Python REPL in it:
import subprocess
subprocess.Popen(["gnome-terminal", "-e", "python"])
Now, if you want to run a specific script, you will need to concatenate its path with python, for the last element of that list it the line that will be executed in the new terminal.
For instance:
subprocess.Popen(["gnome-terminal", "-e", "python my_script.py"])
If your script is executable, you can omit python:
subprocess.Popen(["gnome-terminal", "-e", "my_script.py"])
If you want to pass parameters to your script, simply add them to the python command:
subprocess.Popen(["gnome-terminal", "-e", "python my_script.py var1 var2"])
Note that if you want to run your script with a particular version of Python, you should specify it, by explicitly calling "python2" or "python3".
A small example:
# my_script.py
import sys
print(sys.argv)
input()
# main.py
import subprocess
subprocess.Popen(["gnome-terminal", "-e", "python3 my_script.py hello world"])
Running python3 main.py will spawn a new terminal, with ['my_script.py', 'hello', 'world'] printed, and waited for an input.
I executed this code in python: (test.py)
from subprocess import Popen
p = Popen("file.bat").wait()
Here is file.bat:
#echo off
start c:\python27\python.exe C:\Test\p1.py %*
start c:\python27\python.exe C:\Test\p2.py %*
pause
Here is p1.py:
This line is error
print "Hello word"
p2.py is not interesting
I want to know the exception(not only compiling error) in p1.py by running test.py?
How can I do this?
Thanks!
Here's how I got it working:
test.py
from subprocess import Popen
p = Popen(["./file.sh"]).wait()
Make sure to add the [] around file, as well as the ./. You can also add arguments, like so:
["./file.sh", "someArg"]
Note that I am not on Windows, but this fixed it on Ubuntu. Please comment if you are still having issues
EDIT:
I think the real solution is: Using subprocess to run Python script on Windows
This way you can run a python script from python, while still using Popen
The python script I would use (source code here) would parse some arguments when called from the command line. However, I have no access to the Windows command prompt (cmd.exe) in my environment. Can I call the same script from within a Python console? I would rather not rewrite the script itself.
%run is a magic in IPython that runs a named file inside IPython as a program almost exactly like running that file from the shell. Quoting from %run? referring to %run file args:
This is similar to running at a system prompt python file args,
but with the advantage of giving you IPython's tracebacks, and of
loading all variables into your interactive namespace for further use
(unless -p is used, see below). (end quote)
The only downside is that the file to be run must be in the current working directory or somewhere along the PYTHONPATH. %run won't search $PATH.
%run takes several options which you can learn about from %run?. For instance: -p to run under the profiler.
If you can make system calls, you can use:
import os
os.system("importer.py arguments_go_here")
You want to spawn a new subprocess.
There's a module for that: subprocess
Examples:
Basic:
import sys
from subprocess import Popen
p = Popen(sys.executable, "C:\test.py")
Getting the subprocess's output:
import sys
from subprocess import Popen, PIPE
p = Popen(sys.executable, "C:\test.py", stdout=PIPE)
stdout = p.stdout
print stdout.read()
See the subprocess API Documentation for more details.
So, I created a simple python module, test.py
import commands
def main():
cmd = 'ls -l'
(status, output) = commands.getstatusoutput(cmd)
print status, output
if __name__ == '__main__':
main()
When I ran it using "Python test.py", I got the result that I expected. But when I ran it as an executable (yes, it has the 'x' permission), the program didn't respond at all and I had to Ctrl+C to quit it. Why is that? Shouldn't both ways give the same result?
Add a hash-bang line to the top:
#!/usr/bin/env python
import commands
...
This tells your system what interpreter to use to execute the script. Without it it doesn't know if it's a shell script, Perl script, Python script, what.
You need the hashbang to be the first line of your script, referencing the path of the Python interpreter. Otherwise, all the OS knows is that you're trying to execute a script, and it has no idea how to go about doing that.