I am trying to parse some contents from my Linux environment and dump them into an excel file in my Windows environment using python.
eg: foo/bar/myfile.txt has some contents I want to stick to an excel file in my windows env C:foo\bar\myfile.txt
I know how to extract the information I need but I cannot find a solution to create a file in my Windows OS from my linux env in python. any little info helps. Thanks!
import csv
import sys
import subprocess
def env_key_values(host):
"""
The subprocess.Popen will execute the env command on the remote
Linux server then return the results.
"""
ssh = subprocess.Popen(["ssh", host, "env"],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
"""
Read each line from stdout selecting only the varname=value lines.
Split the line on the '=' character. Use the first element as the
Linux environment variable name. Use the remaining elements as the value.
yield the result.
"""
for line in ssh.stdout:
str_line = line[:-1].decode("utf-8")
if "=" in str_line:
key_values = str_line.split("=")
values = "=".join(key_values[1:])
yield (key_values[0], values)
if __name__ == "__main__":
"""
Open desired file in write mode, do not enforce EOL character.
Create a CSV writer specifying the Windows EOL.
Use \t character for delimiter as the specified file in the question
used a .txt extension and the user wishes to import the file into Excel.
"""
with open("file.txt", "w", newline=None) as fout:
csv_out = csv.writer(fout, delimiter="\t" lineterminator="\r\n")
csv_out.writerow(["VARNAME","VALUE"])
for row in env_key_values("ldssbox01"):
csv_out.writerow(row)
Related
I am trying to use the below code to run a command and extract the data from the cmd.
the file with the commands and data is a txt file. (let me know if I should change it or use an excel if better).
the commands look something like this: ping "host name" which would result in some data in the cmd.there is list of these in the file. so it would ping "hostname1" then line two ping "hostname2"..etc
THE QUESTION: I want it to run every line individually and extract the results from the cmd and store them in a txt file or excel file - Ideally I want all the results in the same file. is this possible? and how?
here is the code so far:
root_dir = pathlib.Path(r"path to file here")
cmds_file = root_dir.joinpath('actual file here with commands and data')
#fail = []
cmds = cmds_file.read_text().splitlines()
try:
for cmd in cmds:
args = cmd.split()
print(f"\nRunning: {args[0]}")
output = subprocess.check_output(args)
print(output.decode("utf-8"))
out_file = root_dir.joinpath(f"Name of file where I want results printed in")
out_file.write_text(output.decode("utf-8"))
except:
pass
You can use a module called subprocess import subprocess
Then you can define a variable like this
run = subprocess.run(command_to_execute, capture_output=True)
After that you can do print(run.stdout) to print the command output.
If you want to write it to a file you can do this after you run the above code
with open("PATH TO YOUR FILE", "w") as file:
file.write(run.stdout)
This should write a file which contains the output of your command
After that close the file using file.close() and reopen it but in "a" mode
with open("PATH TO YOUR FILE", "a") as file:
file.write(\n + run.stdout)
This should append data to your file.
Remember to close the file just for best practice, I have some bad memorys about not closing the file after I opened it :D
My plan is simple:
Open input, output file
Read input file line by line
Execute the command and direct the output to the output file
#!/usr/bin/env python3
import pathlib
import shlex
import subprocess
cmds_file = pathlib.Path(__file__).with_name("cmds.txt")
output_file = pathlib.Path(__file__).with_name("out.txt")
with open(cmds_file, encoding="utf-8") as commands, open(output_file, "w", encoding="utf-8") as output:
for command in commands:
command = shlex.split(command)
output.write(f"\n# {shlex.join(command)}\n")
output.flush()
subprocess.run(command, stdout=output, stderr=subprocess.STDOUT, encoding="utf-8")
Notes
Use shlex.split() to simulate the bash shell's command split
The line output.write(...) is optional. You can remove it
With subprocess.run(...), the stdout=output will redirect the command's output to the file. You don't have to do anything.
Update
I updated the subprocess.run line to redirect stderr to stdout, so error will show.
Im trying to record docker stats for every file in the mydata directory. For example if one of the files is names piano.txt I would like the output file to be piano_stuff.txt. This is what I have so far:
import subprocess
import signal
import os
for file_name in os.listdir('mydata'):
data_txt = "./" + file_name.split(".")[0] + "_stuff.txt"
dockerStats = subprocess.Popen("docker stats --format {{.MemUsage}} >> ${data_txt}", shell=True)
os.killpg(os.getpgid(dockerStats.pid), signal.SIGTERM)
Don't use shell=True. Open the file locally, and pass the file object as the stdout argument. You can also use the --no-stream option to have the command exit after producing one line of output, rather than asynchronously trying to kill the process as soon as possible. (You might get multiple lines of output, or you might get none, depending on when the OS schedules the Docker process to run.)
with open(data_txt, "a") as f:
subprocess.run(["docker", "stats", "--format", "{{.MemUsage}}", "--no-stream"], stdout=f)
I'm trying to create a .txt file with some data, and I want the file name to be the current time. But when I run my code it creates an empty file instead, without any file-type. Here is the code in question:
filename = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d_%H:%M')
with open('%s.txt' % filename, 'w') as open_file:
# writing to file
It seems to ignore the ".txt" part because if i write the code like this it works just fine:
with open('filename.txt', 'w') as open_file:
It runs fine on my machine (Ubuntu 16.04 and python 3.5)
import datetime
import time
filename = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d_%H:%M')
with open('%s.txt' % filename, 'w') as file:
file.write('code written')
please provide more info
And yes i am getting .txt in my file name
In windows, you cant use : in the filename. It stops the creation of the file when it reaches the colon.
I need to remove row line numbers from a large collection of Word .doc/.docx files as part of a (Python) data processing pipeline.
I am aware of solutions to do this in C# using Word.Interop (e.g. Is it possible to use Microsoft.Office.Interop.Word to programatically remove line numbering from a Word document?) but it would be great to achieve this e.g. using LibreOffice in --headless mode (before evaluating MS Word + wine solutions).
For a single file, with the UI, one can follow https://help.libreoffice.org/Writer/Line_Numbering, but I need to do this for a lot of files, so a macro/script/command line solution to
1) cycle through a set of files
2) remove row numbers and save the result to file
and triggered with e.g. a Python subprocess call would be great, or even with calls to the Python API (https://help.libreoffice.org/Common/Scripting).
To perform line removal for a list of files in the working directory (and put the resulting output into pdfs) run LibreOffice in a Linux command line:
soffice --headless --accept="socket,host=localhost,port=2002;urp;StarOffice.ServiceManager"
and then in the Python interpreter
import uno
import socket
import os
import subprocess
from pythonscript import ScriptContext
from com.sun.star.beans import PropertyValue
# list docfiles in working dir
files = [x for x in os.listdir('.') if x.endswith(".docx")]
# iterate on files
for file in files:
localContext = uno.getComponentContext()
resolver = localContext.ServiceManager.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext)
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
smgr = ctx.ServiceManager
desktop = smgr.createInstanceWithContext("com.sun.star.frame.Desktop", ctx)
# open file
model = desktop.loadComponentFromURL(uno.systemPathToFileUrl(os.path.realpath(file)), "_blank", 0, ())
# remove line numbers
model.getLineNumberingProperties().IsOn = False
# prepare to save output to pdf
XSCRIPTCONTEXT = ScriptContext(ctx, None, None)
p = PropertyValue()
p.Name = 'FilterName'
p.Value = 'writer_pdf_Export'
oDoc = XSCRIPTCONTEXT.getDocument()
# create pdf
oDoc.storeToURL("file://" + os.getcwd() + "/" + file + ".pdf", tuple([p]))
This should create pdf files with no line numbering in your working directory.
Useful links:
Add line numbers and export to pdf via macro on OpenOffice forums
LineNumberingProperties documentation
Info on running a macro from the command line
This is the python script will do. The question is how to call the external cmd file within the function?
Read a CSV file in the directory.
If the content in 6th column is equal to 'approved', then calls an
external windows script 'TransferProd.cmd'
.
def readCSV(x):
#csvContents is a list in the global scope that will contain lists of the
#items on each line of the specified CSV file
try:
global csvContents
file = open(csvDir + x + '.csv', 'r') #Opens the CSV file
csvContents = file.read().splitlines() #Appends each line of the CSV file to csvContents
#----------------------------------------------------------------------
#This takes each item in csvContents and splits it at "," into a list.
#The list created replaces the item in csvContents
for y in range(0,len(csvContents)):
csvContents[y] = csvContents[y].lower().split(',')
if csvContents[y][6] == 'approved':
***CALL TransferProd.cmd***
file.close()
return
except Exception as error:
log(logFile, 'An error has occurred in the readCSV function: ' + str(error))
raise
Take a look at the subprocess module.
import subprocess
p = subprocess.Popen(['TransferProd.cmd'])
You can specify where you want output/errors to go (directly to a file or to a file-like object), pipe in input, etc.
import os
os.system('TransferProd.cmd')
This works in both unix/windows flavors as it send the commands to the shell. There are some variations in returned values though! Check here.
If you don't need output of the command you could use: os.system(cmd)
The better solution is to use:
from subprocess import Popen, PIPE
proc = Popen(cmd, shell = True, close_fds = True)
stdout, stderr = proc.communicate()