We Keep Coding

Subprocesss.py issue when running "dir" in Windows [duplicate]

This question already has answers here:
Windows can't find the file on subprocess.call()
(7 answers)
Closed 4 months ago.
I'm playing around with subprocess.Popen for shell commands as it seems it has more flexbility with regards to piping compared to subprocess.run
I'm starting off with some simple examples but I'm getting FileNotFoundError:
I was told that shell = True is not necessary if I make the arguments as proper lists. However it doesn't seem to be working.
Here are my attempts:
import subprocess
p1 =subprocess.Popen(['dir'], stdout =subprocess.PIPE)
output = p1.communicate[0]
p = subprocess.Popen([ "dir", "c:\\Users"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
outputs = p.communicate()
Both are leading to FileNotFoundError

As dir is simply a command understood by cmd.exe (or powershell.exe) then you could:
subprocess.Popen(["cmd", "/c", "dir", "c:\\Users"], stdout=subprocess.PIPE)
which corresponds to doing the following in a shell
C:\>cmd /c dir c:\Users
You may find you have to fully path cmd, as c:\\Windows\\System32\\cmd.exe

Your problem is that "dir" is an internal Windows command and your "popen" is looking for the name of an executable. You could try setting up a "dir.bat" file that runs the "dir" command to see if this works or simply try any of the commands in \Windows\system32 instead.

Try this (on windows):
import subprocess
file_name = "test.txt"
sp = subprocess.Popen(["cmd", "/c", 'dir', '/s', '/p', file_name], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = sp.communicate()
output = output[1].decode()
if file_name in output:
print("yes")
else:
print("no")
On Linux, replace the call to subprocess like this:
sp = subprocess.Popen(['find', '-name', file_name, '/'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
I htink the key is in: stdout=subprocess.PIPE, stderr=subprocess.PIPE

How to execute a file without the ".exe" extension

I have a file: "abc", it's a executable file.
I want to execute it by Phthon or windows CMD.
If I write code:
subprocess.Popen('-a -b -c', creationflags=0x08, shell=True, executable="C:\\abc")
Then, abc executed, but params(-a -b -c) been ignored
So...How Can I reslove it?

Why not change your code to the following version
args = ['C:\\abc', '-a', '-b', '-c']
p = subprocess.Popen(' '.join(args), shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
or without shell=True you can further reduce it to
p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
or you can use another method to get output from your exe and pass arguments to it
Output = subprocess.check_output(' '.join(args), shell=True).decode()

Finally, I found the answer myself, which is to use win32process.CreateProcess
Thanks to other authors, but their answers are wrong.
The correct answer is:
win32process.CreateProcess(r"C:\abc", "-a -b -c", None, None, 0, 0, None, None, win32process.STARTUPINFO())
The following answer does not work:
subprocess.Popen('-a -b -c', creationflags=0x08, shell=True, executable="C:\\abc")
#or
p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
Maybe I really don’t know how to use subprocess, but Windows is really special. Executable files without a suffix cannot be executed directly on the command line.

In PowerShell
& "C:\abc.exe"

Python program can't find Shellscript File

Hey i'm trying to run a shell Script with python using the Following lines:
import subprocess
shellscript = subprocess.Popen(["displaySoftware.sh"], stdin=subprocess.PIPE)
shellscript.stdin.write("yes\n")
shellscript.stdin.close()
returncode = shellscript.wait()
But when I run the Program it says that it can't find the .sh file.

Your command is missing "sh", you have to pass "shell=True" and "yes\n" has to be encoded.
Your sample code should look like this:
import subprocess
shellscript = subprocess.Popen(["sh displaySoftware.sh"], shell=True, stdin=subprocess.PIPE )
shellscript.stdin.write('yes\n'.encode("utf-8"))
shellscript.stdin.close()
returncode = shellscript.wait()
This method might be better:
import subprocess
shellscript = subprocess.Popen(["displaySoftware.sh"], shell=True, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
returncode = shellscript.communicate(input='yes\n'.encode())[0]
print(returncode)
When running this on my machine the "displaySoftware.sh" script, that is in the same directory as the python script, is successfully executed.

python subprocess.call output is not interleaved

I have a python (v3.3) script that runs other shell scripts. My python script also prints message like "About to run script X" and "Done running script X".
When I run my script I'm getting all the output of the shell scripts separate from my print statements. I see something like this:
All of script X's output
All of script Y's output
All of script Z's output
About to run script X
Done running script X
About to run script Y
Done running script Y
About to run script Z
Done running script Z
My code that runs the shell scripts looks like this:
print( "running command: " + cmnd )
ret_code = subprocess.call( cmnd, shell=True )
print( "done running command")
I wrote a basic test script and do *not* see this behaviour. This code does what I would expect:
print("calling")
ret_code = subprocess.call("/bin/ls -la", shell=True )
print("back")
Any idea on why the output is not interleaved?

Thanks. This works but has one limitation - you can't see any output until after the command completes. I found an answer from another question (here) that uses popen but also lets me see the output in real time. Here's what I ended up with this:
import subprocess
import sys
cmd = ['/media/sf_git/test-automation/src/SalesVision/mswm/shell_test.sh', '4', '2']
print('running command: "{0}"'.format(cmd)) # output the command.
# Here, we join the STDERR of the application with the STDOUT of the application.
process = subprocess.Popen(cmd, bufsize=1, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(process.stdout.readline, ''):
line = line.replace('\n', '')
print(line)
sys.stdout.flush()
process.wait() # Wait for the underlying process to complete.
errcode = process.returncode # Harvest its returncode, if needed.
print( 'Script ended with return code of: ' + str(errcode) )
This uses Popen and allows me to see the progress of the called script.

It has to do with STDOUT and STDERR buffering. You should be using subprocess.Popen to redirect STDOUT and STDERR from your child process into your application. Then, as needed, output them. Example:
import subprocess
cmd = ['ls', '-la']
print('running command: "{0}"'.format(cmd)) # output the command.
# Here, we join the STDERR of the application with the STDOUT of the application.
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
process.wait() # Wait for the underlying process to complete.
out, err = process.communicate() # Capture what it outputted on STDOUT and STDERR
errcode = process.returncode # Harvest its returncode, if needed.
print(out)
print('done running command')
Additionally, I wouldn't use shell = True unless it's really required. It forces subprocess to fire up a whole shell environment just to run a command. It's usually better to inject directly into the env parameter of Popen.

How to control a command window opened from a .cmd file using Python

There's a file named startup.cmd that sets some environment variables, runs some preparation commands, then does:
start "startup" cmd /k
Which opens a command shell named startup. The manual process I'm trying to automate is to then enter the following command into this shell: get startup.xml. I thought the correct way to do this in Python would be something like this:
import subprocess
p = subprocess.Popen('startup.cmd', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
getcommand = 'get startup.xml'
servercommand = 'startserver'
p.stdin.write(getcommand)
p.stdin.write(startserver)
(stdoutdata, stderrdata) = p.communicate()
print stdoutdata
print stderrdata
But those commands don't seem to be executing in the shell. What am I missing? Also, the command shell appears regardless of whether shell is set to True or False.

I found this warning in subprocess's document,
Warning Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
So my suggestion is to use communicate to send your command.
import subprocess
p = subprocess.Popen('startup.cmd', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
command = 'get startup.xml\n'
command += 'startserver\n'
(stdoutdata, stderrdata) = p.communicate(command)
print stdoutdata
print stderrdata

This is a new process, so one cannot communicate directly with Popen.