We Keep Coding

Run command in CMD via python and extract the data

I am trying to use the below code to run a command and extract the data from the cmd.
the file with the commands and data is a txt file. (let me know if I should change it or use an excel if better).
the commands look something like this: ping "host name" which would result in some data in the cmd.there is list of these in the file. so it would ping "hostname1" then line two ping "hostname2"..etc
THE QUESTION: I want it to run every line individually and extract the results from the cmd and store them in a txt file or excel file - Ideally I want all the results in the same file. is this possible? and how?
here is the code so far:
root_dir = pathlib.Path(r"path to file here")
cmds_file = root_dir.joinpath('actual file here with commands and data')
#fail = []
cmds = cmds_file.read_text().splitlines()
try:
for cmd in cmds:
args = cmd.split()
print(f"\nRunning: {args[0]}")
output = subprocess.check_output(args)
print(output.decode("utf-8"))
out_file = root_dir.joinpath(f"Name of file where I want results printed in")
out_file.write_text(output.decode("utf-8"))
except:
pass

You can use a module called subprocess import subprocess
Then you can define a variable like this
run = subprocess.run(command_to_execute, capture_output=True)
After that you can do print(run.stdout) to print the command output.
If you want to write it to a file you can do this after you run the above code
with open("PATH TO YOUR FILE", "w") as file:
file.write(run.stdout)
This should write a file which contains the output of your command
After that close the file using file.close() and reopen it but in "a" mode
with open("PATH TO YOUR FILE", "a") as file:
file.write(\n + run.stdout)
This should append data to your file.
Remember to close the file just for best practice, I have some bad memorys about not closing the file after I opened it :D

My plan is simple:
Open input, output file
Read input file line by line
Execute the command and direct the output to the output file
#!/usr/bin/env python3
import pathlib
import shlex
import subprocess
cmds_file = pathlib.Path(__file__).with_name("cmds.txt")
output_file = pathlib.Path(__file__).with_name("out.txt")
with open(cmds_file, encoding="utf-8") as commands, open(output_file, "w", encoding="utf-8") as output:
for command in commands:
command = shlex.split(command)
output.write(f"\n# {shlex.join(command)}\n")
output.flush()
subprocess.run(command, stdout=output, stderr=subprocess.STDOUT, encoding="utf-8")
Notes
Use shlex.split() to simulate the bash shell's command split
The line output.write(...) is optional. You can remove it
With subprocess.run(...), the stdout=output will redirect the command's output to the file. You don't have to do anything.
Update
I updated the subprocess.run line to redirect stderr to stdout, so error will show.

Open a shell file and replace a sting inside it using python

Here I transfer a shell script from my sytem to raspberry pi using ssh and run the script on Rpi
python3 paswd.py | scp replace.py pi#192.168.1.$i:
python3 paswd.py |ssh -tt pi#192.168.1.$i 'chmod u+x ~/'replace.py' && exit'
This happens and it shows no errors
Replace.py
def replace():
fout=open("launch.sh","rt")
for line in fout:
fout.close
fout=open("launch.sh","wt")
fout.write(line.replace('chromium-browser','chromium-browser --kiosk '))
fout.close
replace()
launch.sh
echo "connected, launching browser"
chromium-browser & disown
After the replace.py is executed am not geting the require results, I get an empty launch.sh or no changes at all. As said it shows no errors. Please let me know where is my mistake. The saame code worked perfectly for .txt file, it replaces "chromium-browser" with "chromium-browser --kiosk" but on .sh file am not getting it.
Thanks for the help

This will solve your problem.
def replace():
file_path = "launch.sh"
find = "chromium-browser"
replacement = "chromium-browser --kiosk"
with open(file_path) as f:
s = f.read()
s = s.replace(find, replacement)
with open(file_path, "w") as f:
f.write(s)
replace()

Python subprocess display logs on terminal and save in file

I am running a Python script using subprocess and willing to save output to a file as well as show live logs on terminal.
I have written below code and its saving logs in file but not showing live script execution logs on terminal.
TCID = sys.argv[1]
if TCID == "5_2_5_3":
output = subprocess.check_output([sys.executable, './script.py'])
with open('scriptout.log', 'wb') as outfile:
outfile.write(output)

I think this will fix your issue
import subprocess
outputfile = open('scriptout.log', 'a')
process = subprocess.Popen(["ping", "127.0.0.1"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
output = process.stdout.readline()
if output == b'' and process.poll() is not None:
break
if output:
out = output.decode()
outputfile.write(out)
print(out, end="")
outputfile.close()
also I tried
import subprocess
output = subprocess.check_output(["ping", "127.0.0.1"])
with open('scriptout.log', 'wb') as outfile:
print(output)
outfile.write(output)
but it outputs after command execution ends. Also I want try with logging module but I don't know how to use it sorry :(

python subprocess is overwriting file used for stdout - I need it to append to the file (windows)

I want to append the STDOUT of subprocess.call() to an existing file. My code below overwrites the file -
log_file = open(log_file_path, 'r+')
cmd = r'echo "some info for the log file"'
subprocess.call(cmd, shell=True, stdout=log_file, stderr=STDOUT)
log_file.close()
I'm looking for the equivalent of >> in subprocess.call() or subprocess.Popen(). It's driving me crazy trying to find it..
UPDATE:
Following the answers so far I've updated my code to
import subprocess
log_file = open('test_log_file.log', 'a+')
cmd = r'echo "some info for the log file\n"'
subprocess.call(cmd, shell=True, stdout=log_file, stderr=subprocess.STDOUT)
log_file.close()
I'm running this code from the command line in windows -
C:\users\aidan>test_subprocess.py
This adds the text to the log file. When I run the script again, nothing new is added. It still seems to be overwriting the file..

Use the 'a' append mode instead:
log_file = open(log_file_path, 'a+')
If you still see previous content overwritten, perhaps Windows needs you to explicitly seek to the end of the file; open as 'r+' or 'w' and seek to the end of the file:
import os
log_file = open(log_file_path, 'r+')
log_file.seek(0, os.SEEK_END)

Modify how you open log_file_path. You are opening the file for reading and writing 'r+'. Use the 'a' append mode instead of 'r+':
log_file = open(log_file_path, 'a+')

how to direct output into a txt file in python in windows

import itertools
variations = itertools.product('abc', repeat=3)
for variations in variations:
variation_string = ""
for letter in variations:
variation_string += letter
print (variation_string)
How can I redirect output into a txt file (on windows platform)?

From the console you would write:
python script.py > out.txt
If you want to do it in Python then you would write:
with open('out.txt', 'w') as f:
f.write(something)
Obviously this is just a trivial example. You'd clearly do more inside the with block.

You may also redirect stdout to your file directly in your script as print writes by default to sys.stdout file handler. Python provides a simple way to to it:
import sys # Need to have acces to sys.stdout
fd = open('foo.txt','w') # open the result file in write mode
old_stdout = sys.stdout # store the default system handler to be able to restore it
sys.stdout = fd # Now your file is used by print as destination
print 'bar' # 'bar' is added to your file
sys.stdout=old_stdout # here we restore the default behavior
print 'foorbar' # this is printed on the console
fd.close() # to not forget to close your file

In window command prompt, this command will store output of program.py into file output.txt
python program.py > output.txt

If it were me, I would use David Heffernan's method above to write your variable to the text file (because other methods require the user to use a command prompt).
import itertools
file = open('out.txt', 'w')
variations = itertools.product('abc', repeat=3)
for variations in variations:
variation_string = ""
for letter in variations:
variation_string += letter
file.write(variation_string)
file.close()

you may use >>
log = open("test.log","w")
print >> log, variation_string
log.close()

Extension to David's answer
If you are using PyCharm,
Go to Run --> Edit Configurations --> Logs --> Check mark Save console
output to file --> Enter complete path --> Apply