How do i do sub processes properly with Python - python

I am new to python so I'm in the early stages of learning it. I was wondering if anyone knows how to run a system command after another. It's hard to explain:
subprocess.call('dir',shell=True)
subprocess.call('cd ..',shell=True)
subprocess.call('dir',shell=True)
When I run the command I expect to see the directory which the file is run. Which was fine.
Then the second process I expect to go up a directory.
Then the third command I expected to see the higher directory. Which I didn't I just saw the first directory.
Could some one explain why it isn't working as I expected and what I should do to correct it.


The general rule is that children cannot affect the parent's environment.
subprocess.call creates a child process. The child process can do many things. But, any changes it makes to the current working directory or to environment variables only last for the duration of the subprocess call. After the call completes and control returns to the parent, the parent's environment is restored unchanged.
If you want the cd to affect the next dir command, you need to have both in the same child. For example:
subprocess.call('cd ..; dir', shell=True)
You probably asked this question for more general purposes. But, for the specific examples that you provided, note that those actions might be better performed with the os module, rather than the subprocess module: listing files in the current directory can be done with os.listdir and changing the current working directory can be done with os.chdir


If you are trying to change the working directory in python that can be accomplished simply by os module. You can find that documentation here. I would suggest only using subprocess.call to call a script or another program that isn't trying to modify stuff based on the current environment.


When you run a subprocess with shell=True, python starts up a new shell to run the command in. It is basically the same as if python start up a new command prompt, entered in the command, and then closed the command prompt.
The consequence is that any action which only affects the shell is lost when the shell is closed. So you can create files and you'll see that because the hard drive is changed. But if you change the current directory of the shell that change will be lost.
You might wonder about the output of the program. Basically, the default is for the output of the program to be copied to the output of the calling program. (You can override this.)
If you want to change the current directory you want os.chdir. In general, you should avoid calling subprocesses and prefer python's tools. For example, instead of dir use os.listdir.

Related

Python execute code in parent shell upon exit

I have a search program that helps users find files on their system. I would like to have it perform tasks, such as opening the file within editor or changing the parent shell directory to the parent folder of the file exiting my python program.
Right now I achieve this by running a bash wrapper that executes the commands the python program writes to the stdout. I was wondering if there was a way to do this without the wrapper.
Note:
subprocess and os commands create a subshell and do not alter the parent shell. This is an acceptable answer for opening a file in the editor, but not for moving the current working directory of the parent shell to the desired location on exit.
An acceptable alternative might be to open a subshell in a desired directory
example
#this opens a bash shell, but I can't send it to the right directory
subprocess.run("bash")

This, if doable, will require quite a hack. Because the PWD is passed from the shell into the subprocess - in this case, the Python process, as a subprocess owned variable, and changing it won't modify what is in the super program.
On Unix, maybe it is achievable by opening a detachable sub-process that will pipe keyboard strokes into the TTY after the main program exits - I find this the most likely to succeed than any other thing.

python shell script execution that passes environment to the parent

So I know how subprocess works and use it a lot, but I've run into a strange issue. I need to execute an export of some environment variables. The reason is that some program (black-box) executes a program that seems like it runs in a subshell, so it doesn't have access to the environment variables but it has access to all my files.
I can't hard code the environment variables so I want to source or . the file that has the export commands in it. However, if I source or . that file in a subprocess, it won't make any difference to its parent process. In which case I either need some function besides subprocess that can execute shell commands without creating a subprocess, if that exists. Another issue is that a subprocess doesn't have the proper permissions to read the file.
And copying the environment variables via os isn't really possible either.
Does anything besides subprocess exist? Or is there some other kind of workaround?

IMHO the simplest solution consists in creating a new shell script (let's call it run_black_box.sh) which sources the setup script (let's assume it is named setup.sh) to initialize the environment and then calls the black_box program.
Here is a possible content of run_black_box.sh:
#/bin/bash
source setup.sh
black_box
The you can pass run_black_box.sh to subprocess for execution.

Writing a line to CMD in python

I am very new to Python and I have been trying to find a way to write in cmd with python.
I tried os.system and subprocess too. But I am not sure how to use subprocess.
While using os.system(), I got an error saying that the file specified cannot be found.
This is what I am trying to write in cmd os.system('cd '+path+'tesseract '+'a.png out')
I have tried searching Google but still I don't understand how to use subprocess.
EDIT:
It's not a problem with python anymore, I have figured out. Here is my code now.
os.system("cd C:\\Users\\User\\Desktop\\Folder\\data\\")
os.system("tesseract a.png out")
Now it says the file cannot be open. But if I open the cmd separately and write the above code, it successfully creates a file in the folder\data.

Each call to os.system is a separate instance of the shell. The cd you issued only had effect in the first instance of the shell. The second call to os.system was a new shell instance that started in the Python program's current working directory, which was not affected by the first cd invocation.
Some ways to do what you want:
1 -- put all the relevant commands in a single bash file and execute that via os.system
2 -- skip the cd call; just invoke your tesseract command using a full path to the file
3 -- change the directory for the Python program as a whole using os.chdir but this is probably not the right way -- your Python program as a whole (especially if running in a web app framework like Django or web2py) may have strong feelings about the current working directory.
The main takeaway is, os.system calls don't change the execution environment of the current Python program. It's equivalent to what would happen if you created a sub-shell at the command line, issued one command then exited. Some commands (like creating files or directories) have permanent effect. Others (like changing directories or setting environment variables) don't.

How to use cmd from python

Im trying to use python to run cmd.exe and thereby running commands like cd C:\name..... and executing other programs from the cmd what I have so far is.
os.system("cmd.exe").
os.system("cd C:\name\first\second").
When I try to run three other commands a new cmd window replaces the old one and the commands dont work since they need to be consecutively after each other.I already tried the above code and need help running the next three. Also can you explain what suproccess are.

See my answer to this recent question for why os.system("cd WHEREVER") does not do what you expect.
Briefly, when you run os.system('cd WHEREVER') you are creating a new command shell which has its own idea of the current directory. This change in the current directory will be entirely "forgotten" on subsequent calls to os.system(). You need to change the current directory in the parent process (the script) with os.chdir('WHEREVER') in order to retain the change for subsequent os.system() calls.

Changing prompt working directory via Python script

Is it possible to change the Windows command prompt working directory via Python script?
e.g.
>> cd
>> c:\windows\system32
>> make_decision_change_dir.py
>> cd
>> c:\windows
I have tried a few things which don't work:
import os
os.chdir(path)
import os, subprocess
subprocess.Popen("chdir /D \"%s\"" %path, shell=True)
import os, subprocess
subprocess.Popen("cd \"%s\"" %path, shell=True)
import os, subprocess
subprocess.Popen("CD=\"%s\"" %path, shell=True)
As I understand it and observe these operations change the current processes working directory - which is the Python process and not the prompt its executing from.
Thanks.
UPDATE
The path I would want to change to is dynamic (based on what project I am working on, the full path to a build location changes) hence I wanted to code a solution in Python rather than hack around with a Windows batch file.
UPDATE
I ended up hacking a batch file together to do this ;(
Thanks everyone.

I'm not clear what you want to do here. Do you want a python script which you can run from a Windows command prompt which will change the working directory of the Windows command session?
If so, I'm 99.9% sure that's impossible. As you said yourself the python.exe process is a separate process from the Windows cmd.exe and anything you do in Python won't affect the Command prompt.
There may be something you can do via the Windows API by sending keystrokes to the Windows or something but it would be pretty brittle.
The only two practical options I can think of involve wrapping your Python script in a Batch file:
Output your desired directory from the Python script, read the output in your Batch file and CD to it.
Start your Python script from a batch file, allow your Python script to start a new cmd.exe Window and get the Batch file to close the original Command window.

I have a Python script to make moving around a file tree easier: xdir.py
Briefly, I have an xdir.py file, which writes Windows commands to stdout:
# Obviously, this should be more interesting..
import sys
print "cd", sys.argv[1]
Then an xdir.cmd file:
#echo off
python xdir.py %* >%TEMP%\__xdir.cmd
call %TEMP%\__xdir.cmd
Then I create a doskey alias:
doskey x=xdir.cmd $*
The end result is that I can type
$ x subdir
and change into subdir.
The script I linked to above does much more, including remembering history, maintaining a stack of directories, accepting shorthand for directories, and so on.

One common solution is a two-part script.
Part 1 is Python, which creates a temporary .BAT file that contains the appropriate CD command.
Part 2 is the temporary .BAT file.
fancycd.bat
python figurethepath.py >temp.bat
temp.bat

As people mentioned, child processes (i.e. your program) can't change the current working directory of a parent process (i.e. the terminal). This is why you need the two steps that everybody is describing. In most shells there's a way to make a macro or function to perform this two-step functionality.
For example, in bash, you can make a single alias to compute the path and change the current working directory, similar to what #S.Lott describes for Windows:
alias my_cd='TMP=`compute_path.py`; cd $TMP;'
Note that the cd command is still being interpreted in the parent process (the terminal), which has the ability to change its own current working directory.

The subprocess.Popen() doc page says a child process will be created for the sub-process, so any working directory changes will be local to that subprocess.
If cwd is not None, the child’s current directory will be changed to cwd before it is executed. Note that this directory is not considered when searching the executable, so you can’t specify the program’s path relative to cwd.
This will be the same for any changes done explicitly inside the subproceess, similar to the commands that appear in the question.

imoprt os
os.system("start cmd.exe /k \"cd /d c:\\windows\\system32 & python make_decision_change_dir.py\"")

Categories