We Keep Coding

How to insert os.system output into a text file? [duplicate]

This question already has answers here:
Running shell command and capturing the output
(21 answers)
Closed 2 years ago.
I want to get the stdout in a variable after running the os.system call.
Lets take this line as an example:
batcmd="dir"
result = os.system(batcmd)
result will contain the error code (stderr 0 under Windows or 1 under some linux for the above example).
How can I get the stdout for the above command without using redirection in the executed command?

If all you need is the stdout output, then take a look at subprocess.check_output():
import subprocess
batcmd="dir"
result = subprocess.check_output(batcmd, shell=True)
Because you were using os.system(), you'd have to set shell=True to get the same behaviour. You do want to heed the security concerns about passing untrusted arguments to your shell.
If you need to capture stderr as well, simply add stderr=subprocess.STDOUT to the call:
result = subprocess.check_output([batcmd], stderr=subprocess.STDOUT)
to redirect the error output to the default output stream.
If you know that the output is text, add text=True to decode the returned bytes value with the platform default encoding; use encoding="..." instead if that codec is not correct for the data you receive.

These answers didn't work for me. I had to use the following:
import subprocess
p = subprocess.Popen(["pwd"], stdout=subprocess.PIPE)
out = p.stdout.read()
print out
Or as a function (using shell=True was required for me on Python 2.6.7 and check_output was not added until 2.7, making it unusable here):
def system_call(command):
p = subprocess.Popen([command], stdout=subprocess.PIPE, shell=True)
return p.stdout.read()

import subprocess
string="echo Hello world"
result=subprocess.getoutput(string)
print("result::: ",result)

I had to use os.system, since subprocess was giving me a memory error for larger tasks. Reference for this problem here. So, in order to get the output of the os.system command I used this workaround:
import os
batcmd = 'dir'
result_code = os.system(batcmd + ' > output.txt')
if os.path.exists('output.txt'):
fp = open('output.txt', "r")
output = fp.read()
fp.close()
os.remove('output.txt')
print(output)

I would like to expand on the Windows solution. Using IDLE with Python 2.7.5, When I run this code from file Expts.py:
import subprocess
r = subprocess.check_output('cmd.exe dir',shell=False)
print r
...in the Python Shell, I ONLY get the output corresponding to "cmd.exe"; the "dir" part is ignored. HOWEVER, when I add a switch such as /K or /C ...
import subprocess
r = subprocess.check_output('cmd.exe /K dir',shell=False)
print r
...then in the Python Shell, I get all that I expect including the directory listing. Woohoo !
Now, if I try any of those same things in DOS Python command window, without the switch, or with the /K switch, it appears to make the window hang because it is running a subprocess cmd.exe and it awaiting further input - type 'exit' then hit [enter] to release. But with the /K switch it works perfectly and returns you to the python prompt. Allrightee then.
Went a step further...I thought this was cool...When I instead do this in Expts.py:
import subprocess
r = subprocess.call("cmd.exe dir",shell=False)
print r
...a new DOS window pops open and remains there displaying only the results of "cmd.exe" not of "dir". When I add the /C switch, the DOS window opens and closes very fast before I can see anything (as expected, because /C terminates when done). When I instead add the /K switch, the DOS window pops open and remain, AND I get all the output I expect including the directory listing.
If I try the same thing (subprocess.call instead of subprocess.check_output) from a DOS Python command window; all output is within the same window, there are no popup windows. Without the switch, again the "dir" part is ignored, AND the prompt changes from the python prompt to the DOS prompt (since a cmd.exe subprocess is running in python; again type 'exit' and you will revert to the python prompt). Adding the /K switch prints out the directory listing and changes the prompt from python to DOS since /K does not terminate the subprocess. Changing the switch to /C gives us all the output expected AND returns to the python prompt since the subprocess terminates in accordance with /C.
Sorry for the long-winded response, but I am frustrated on this board with the many terse 'answers' which at best don't work (seems because they are not tested - like Eduard F's response above mine which is missing the switch) or worse, are so terse that they don't help much at all (e.g., 'try subprocess instead of os.system' ... yeah, OK, now what ??). In contrast, I have provided solutions which I tested, and showed how there are subtle differences between them. Took a lot of time but...
Hope this helps.

commands also works.
import commands
batcmd = "dir"
result = commands.getoutput(batcmd)
print result
It works on linux, python 2.7.

How can I echo Python loop processing info to the command line?

I am trying to set up a (iron)python script that shows some processing information to the command line. Something like:
list = ["a", "b", "c"]
for i in list:
cmd /k echo str(list.index(i)) #just echo index and leave window open so user can see progression
The challenge is to send multiple echo commands to the same instance of the command line.
Note: I can't just use print(i) in this case, because this ironpython script will be running inside a visual programming application, that does not allow printing to the command line. The goal of the script is that users can see progression during heavy operations.
I have come across multiple threads that suggest subprocess for this. I've seen and tested a lot of examples but I can get none of them to work. I could really use some help with that.
For example, when I do this...
import subprocess
proc=subprocess.Popen("echo hello world", shell=True, stdout=subprocess.PIPE)
.... then I can see that 'hello world' is being returned, so the echo was probably successful. But I never see a shell window pop up.
Edit 1
Some more test results: method 1 gives me a good result (command is executed and screen is left open for reading); method 2: adding stdout opens the shell and leaves the screen open but without a visible result - theres nothing to read; method 3: adding stdin flashes the screen (closes it too soon)
method 1:
process = Popen("cmd /k dir")
method2:
process = Popen("cmd /k dir",stdout=subprocess.PIPE)
method 3:
process = Popen('cmd /k dir',stdin=subprocess.PIPE)
Edit 2
Method 4 - creates two separate cmd windows, one showing a, one showing b. Not it.
process = Popen("cmd /k echo a")
process = Popen("cmd /k echo b")
Method 5 - adding process.stdin.write after stdin - just flashes
process = Popen('cmd /k echo a',stdin=subprocess.PIPE)
process.stdin.write("cmd /k echo b")
It seems so simple what I'm looking for but its giving me a headache...

What you are trying to do is not as simple as you think. Because the CMD window is a different process from your IronPython script, to do anything interactive (like showing the progress of a script), you need to use some form of inter-process communication.
For example, you could write a server script to run in the CMD window and then send messages to it from your IronPython script. But then you have to worry about how to start the server, how to stop it, and how to synchronise with the IronPython client script. You can see an example of this kind of inter-process communication in this StackOverflow answer.
Before trying something like that I would make sure that there's no way to display progress in your visual programming application, as staying inside the same process is likely to be easier.

Close the terminal window after doing something

I want to just print some information and call an application e.g. notepad.
from subprocess import call
print("Opening Notepad++")
call([r"C:\Program Files (x86)\Notepad++\notepad++.exe"])
exit()
Problem now is that the terminal window doesn't automatically close. It stays open until I close the notepad window. How can I make the terminal window disappear automatically.

use Popen like so
import subprocess
subprocess.Popen(r'C:\Program Files (x86)\Notepad++\notepad++.exe', \
stdout=subprocess.PIPE, shell=False, creationflags = 0x08000000)

You need to call the notepad command with start COMMAND, like in Linux we use COMMAND & to fork the process into the background. in windows we use the start COMMAND
So you code refactored:
from subprocess import call
print("Opening Notepad++")
call([r"start C:\Program Files (x86)\Notepad++\notepad++.exe"])
exit()
Although note I don't have a windows machine to test on.

You could use pythonw.exe:
pythonw script.py
Or change its extension to pyw e.g. script.pyw and double click on it.
If you do that you should print "Opening Notepad++" to a popup window. See: Python Notification Popup that disappears

Call to several batch files through CMD doesn't block

I'm trying to call several install.bat files one after another with Python trough CMD.
It is necessary that each bat file be displayed in an interactive console window because it asks for some users instructions and that the python program only resume after each CMD process is resolved
Each install.bat file can take a pretty long time to finish its process.
My code is the following :
for game in games :
print("----------- Starting conversion for %s -----------" %game)
subprocess.call("start cmd /C " + "Install.bat", cwd=os.path.join(gamesDosDir,game), shell=True)
print("end")
But the console windows inside the shell are launched all at once and the "end" message appears event before any of them is finished, whereas I would like them appearing one by one and not go to the n+1 one until the n one is finished and the console window closed (either by user or automatically /K or /C then).
I understand this is some problems using CMD as call should be blocking. How to resolve that? Additionally, if possible how to keep it exactly the same and add 'Y' and 'Y' as default user input?

The most common way to start a batch file (or more generally a CLI command) if to pass it as an argument to cmd /c. After you comment I can assume that you need to use start to force the creation of a (new) command window.
In that case the correct way is to add the /wait option to the start command: it will force the start command to wait the end of its subprocess:
subprocess.call("start /W cmd /C " + "Install.bat", cwd=os.path.join(gamesDosDir,game),
shell=True)
But #eryksun proposed a far cleaner way. On Windows, .bat files can be executed without shell = True, and creationflags=CREATE_NEW_CONSOLE is enough to ensure a new console is created. So above line could simply become:
subprocess.call("Install.bat", cwd=os.path.join(gamesDosDir,game),
creationflags = subprocess.CREATE_NEW_CONSOLE)

Powershell and Python - How to run a command as admin

I am writing a code in python in which I need to get the internet traffic by software's name. It's required of me to use the cmd command netstat -nb, command which requires elevation. I have to keep it simple, something of one line or so, no long batch or powershell scripts. It's preferable if I use only the subprocess python library.
I have got two lines of code that work halfway of what I need:
subprocess.check_output('powershell Start-Process netstat -ArgumentList "-nb" -Verb "runAs"', stdout=subprocess.PIPE, shell=True)
The problem in this one is that a new window it's opened and all the data I need is lost. Maybe there's a way of not opening another window or saving the output from the new window?
subprocess.check_output('powershell Invoke-Command {cmd.exe -ArgumentList "/c netstat -nb"}', stdout=subprocess.PIPE, shell=True)
This one I have the output in the same window but I don't have elevation so I don't get any results... Maybe there is a way of getting elevation without opening a new window or so?
Thank you for your help, hope my question was clear enough.

Create a batch file to perform the task with captured output to a temp file:
[donetstat.bat]
netstat -nb > ".\donetstat.tmp"
Then execute that in your program:
[yourprogram.py]
subprocess.check_output('powershell Start-Process cmd -ArgumentList "/c ".\donetstat.tmp" -Verb "runAs"', stdout=subprocess.PIPE, shell=True)
It would probably be a bit more bullet-resistent to get the TEMP environment variable and use it for a fully-qualified tempfile location:
netstat -nb > "%TEMP%.\donetstat.tmp"
And then get do the same in your Python script.
Once you've created the tempfile, you should be able to process it in Python.
If this needs to be durable with multiple worker processes, add some code to ensure you have a unique tempfile for each process.