We Keep Coding

Run command in CMD via python and extract the data

I am trying to use the below code to run a command and extract the data from the cmd.
the file with the commands and data is a txt file. (let me know if I should change it or use an excel if better).
the commands look something like this: ping "host name" which would result in some data in the cmd.there is list of these in the file. so it would ping "hostname1" then line two ping "hostname2"..etc
THE QUESTION: I want it to run every line individually and extract the results from the cmd and store them in a txt file or excel file - Ideally I want all the results in the same file. is this possible? and how?
here is the code so far:
root_dir = pathlib.Path(r"path to file here")
cmds_file = root_dir.joinpath('actual file here with commands and data')
#fail = []
cmds = cmds_file.read_text().splitlines()
try:
for cmd in cmds:
args = cmd.split()
print(f"\nRunning: {args[0]}")
output = subprocess.check_output(args)
print(output.decode("utf-8"))
out_file = root_dir.joinpath(f"Name of file where I want results printed in")
out_file.write_text(output.decode("utf-8"))
except:
pass

You can use a module called subprocess import subprocess
Then you can define a variable like this
run = subprocess.run(command_to_execute, capture_output=True)
After that you can do print(run.stdout) to print the command output.
If you want to write it to a file you can do this after you run the above code
with open("PATH TO YOUR FILE", "w") as file:
file.write(run.stdout)
This should write a file which contains the output of your command
After that close the file using file.close() and reopen it but in "a" mode
with open("PATH TO YOUR FILE", "a") as file:
file.write(\n + run.stdout)
This should append data to your file.
Remember to close the file just for best practice, I have some bad memorys about not closing the file after I opened it :D

My plan is simple:
Open input, output file
Read input file line by line
Execute the command and direct the output to the output file
#!/usr/bin/env python3
import pathlib
import shlex
import subprocess
cmds_file = pathlib.Path(__file__).with_name("cmds.txt")
output_file = pathlib.Path(__file__).with_name("out.txt")
with open(cmds_file, encoding="utf-8") as commands, open(output_file, "w", encoding="utf-8") as output:
for command in commands:
command = shlex.split(command)
output.write(f"\n# {shlex.join(command)}\n")
output.flush()
subprocess.run(command, stdout=output, stderr=subprocess.STDOUT, encoding="utf-8")
Notes
Use shlex.split() to simulate the bash shell's command split
The line output.write(...) is optional. You can remove it
With subprocess.run(...), the stdout=output will redirect the command's output to the file. You don't have to do anything.
Update
I updated the subprocess.run line to redirect stderr to stdout, so error will show.

How do you use a python variable in popen()?

Im trying to record docker stats for every file in the mydata directory. For example if one of the files is names piano.txt I would like the output file to be piano_stuff.txt. This is what I have so far:
import subprocess
import signal
import os
for file_name in os.listdir('mydata'):
data_txt = "./" + file_name.split(".")[0] + "_stuff.txt"
dockerStats = subprocess.Popen("docker stats --format {{.MemUsage}} >> ${data_txt}", shell=True)
os.killpg(os.getpgid(dockerStats.pid), signal.SIGTERM)

Don't use shell=True. Open the file locally, and pass the file object as the stdout argument. You can also use the --no-stream option to have the command exit after producing one line of output, rather than asynchronously trying to kill the process as soon as possible. (You might get multiple lines of output, or you might get none, depending on when the OS schedules the Docker process to run.)
with open(data_txt, "a") as f:
subprocess.run(["docker", "stats", "--format", "{{.MemUsage}}", "--no-stream"], stdout=f)

Passing a temporary file to Popen subprocess?

I am trying to pass a temporary file to another python file and print the results of the second python file. I notice, however - i'm not gettin anything back from the other file. Is it possible to pass temporary files to subprocesses?
When I try reading from the tmp file I can see values are being written to it, but it appears that it is not being received by the other file. in the other file I've simply written (t = input(), for var in t: print(t))
Example Output
tmp file gen code:
for block in input_list: #iterating through blocklist
# open the file here
f = tempfile.TemporaryFile(mode='w+')
for val in block: #write to the file
f.write(val)
#print(val)
f.seek(0)
#print(f.read())
user_code= subprocess.Popen(['python', 'test_code.py'], stdin=f, stdout=PIPE)
user_code.wait()
mssg = user_code.communicate()
print(mssg)
print(mssg[0].decode('utf-8').strip())

Parse information from Linux file to Windows using python

I am trying to parse some contents from my Linux environment and dump them into an excel file in my Windows environment using python.
eg: foo/bar/myfile.txt has some contents I want to stick to an excel file in my windows env C:foo\bar\myfile.txt
I know how to extract the information I need but I cannot find a solution to create a file in my Windows OS from my linux env in python. any little info helps. Thanks!

import csv
import sys
import subprocess
def env_key_values(host):
"""
The subprocess.Popen will execute the env command on the remote
Linux server then return the results.
"""
ssh = subprocess.Popen(["ssh", host, "env"],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
"""
Read each line from stdout selecting only the varname=value lines.
Split the line on the '=' character. Use the first element as the
Linux environment variable name. Use the remaining elements as the value.
yield the result.
"""
for line in ssh.stdout:
str_line = line[:-1].decode("utf-8")
if "=" in str_line:
key_values = str_line.split("=")
values = "=".join(key_values[1:])
yield (key_values[0], values)
if __name__ == "__main__":
"""
Open desired file in write mode, do not enforce EOL character.
Create a CSV writer specifying the Windows EOL.
Use \t character for delimiter as the specified file in the question
used a .txt extension and the user wishes to import the file into Excel.
"""
with open("file.txt", "w", newline=None) as fout:
csv_out = csv.writer(fout, delimiter="\t" lineterminator="\r\n")
csv_out.writerow(["VARNAME","VALUE"])
for row in env_key_values("ldssbox01"):
csv_out.writerow(row)

How to provide input to a python subprocess call that expects filename, rather than variable?

I'm trying to call a shell script (segment.sh) within python.
The syntax that produce correct results at the console is:
> ./segment.sh ctb file.txt utf-8 0
As can be seen, this shell script expects an text file as input. What I am trying to do is to open up the file and read its content in python (later preferably from an HTML POST form), and somehow pass the variable containing the content to the python subprocess
call.
The following function is not working. But if I just provide a file name such as:
Popen(["/bin/bash", "./segment.sh", "ctb", "file.txt", "utf-8", "0"])`
then, it will work, but I want to pass the input from a variable. Would you please give me some pointers?
def pySegment(text):
op = subprocess.Popen(["/bin/bash", "./segment.sh", "ctb", "utf-8", "0"],
stdout = subprocess.PIPE,
stdin = subprocess.PIPE,
stderr = subprocess.STDOUT,
)
results = op.communicate(input=text)[0]
return results
if __name__ == "__main__":
filename = "./file.txt"
text = open(filename).read()
result = pySegment(text)
print result

I suggest to use a named pipe:
import os, tempfile, shutil, subprocess
temp_dir = tempfile.mkdtemp()
filename = os.path.join(temp_dir, 'file.txt')
text = '<text>'
os.mkfifo(filename)
try:
subprocess.Popen(('segment.sh', 'ctf', filename, 'utf-8', '0'))
with open(filename, 'w') as f:
f.write(text)
finally:
shutil.rmtree(temp_dir)
The named pipe will offer the same interface of a file without really creating that file as you need.