I made some script for modify text.
But I can't make result.
below is my script.
i just begin study python.
I think my script didn't work because f = open('find_c_volume_show.txt', 'w')
please help me.
import sys
from itertools import islice
def next_n_lines(file_opened, N):
return [x.strip() for x in islice(file_opened, N)]
field_line = 1
num = 0
N = 9
split_line = field_line / N
strings = ("Vserver", "Volume Name", "Used Size", "Available Size", "Volume Size", "Aggregate Name", "Space Saved by Storage Efficiency")
f = open('find_c_volume_show.txt', 'w')
for line in open("c_volume_show.txt"):
if any(s in line for s in strings):
field1,field2 = line.strip().split(':')
field_line += 1
f.write(field2 + '\n')
f.close()
f = open('find_c_volume_show.txt', 'w')
f.write("Vserver,Volume Name,Aggregate Name,Volume Size,Available Size,Used Size,Space Saved\n")
with open('find_c_volume_show.txt', 'w') as result:
while num < split_line:
num += 1
lines = next_n_lines(result, N)
f.write('{}'.format(','.join(lines)) +'\n' )
f.close()
below is my text file. text file has data multi data.
Vserver Name: FAS8040-ZZZZ
Volume Name: vol0
Aggregate Name: Node1_aggr0
Volume Size: 466.6GB
Available Size: 436.2GB
Filesystem Size: 466.6GB
Total User-Visible Size: 466.6GB
Used Size: 30.40GB
Used Percentage: 6%
Node name: FAS8040-ZZZZ
Space Saved by Storage Efficiency: 0B
Percentage Saved by Storage Efficiency: 0%
Space Saved by Deduplication: 0B
Space Saved by Compression: 0B
Percentage Space Saved by Compression: 0%
Total Physical Used Size: 22.37GB
I want result like below.
Vserver,Volume Name,Aggregate Name,Volume Size,Available Size,Used Size,Space Saved
FAS8040-ZZZZ,vol0,Node1_aggr0,466.6GB,435.7GB,30.92GB,0B
FAS8040-YYYY,vol0,Node2_aggr0,466.6GB,428.7GB,37.91GB,0B
FAS8040-XXXX,vol0,Node2_aggr0,466.6GB,428.7GB,37.91GB,0B
The problem is that each time you open file with open(filename,'w'), it gets erased. You can use "temporal" file with different name to store results of your first for loop, or I would suggest to aggregate content of each line to list, and then write it straight away.
Additionally, you have problems with your "split_line" value, it is always 0. I guess what you mean is len(strings).
Here is a code:
import sys
strings = ("Vserver", "Volume Name", "Used Size", "Available Size", "Volume Size", "Aggregate Name", "Space Saved by Storage Efficiency")
with open('find_c_volume_show.txt', 'w') as f:
f.write("Vserver,Volume Name,Aggregate Name,Volume Size,Available Size,Used Size,Space Saved,Snapshot,Total Used Size\n")
row = []
for line in open("c_volume_show.txt"):
if any(s in line for s in strings):
field1,field2 = line.strip().split(':')
row.append(field2)
print(row)
if len(row) == len(strings):
f.write('{}'.format(','.join(row)) +'\n' )
print(row)
row = []
The new-ish pathlib module (available in Python >= 3.4) is a much, much easier way for reading and writing files than the traditional open() function. It is also great for working with path-like objects (both in Windows and for other OSes).
from pathlib import Path
afile = Path(r'c:\temp\foo.bar') # absolute path to file
To read a file located at a path, you can just grab the text directly from the Path object!
contents = afile.read_text()
content_lines = contents.split('\n')
...and WRITE text directly!
data = '\n'.join(content_lines)
afile.write_text(data) # overwrites existing file
You can also use its open method rather than the open function:
with afile.open() as f:
dostuff(f)
The with statement is a context manager. It automatically "cleans up" by closing the file when you are done (no matter what happens- even if there is an error).
It's Paths - Paths all the way down
Here is more about the Path library copied from another of my answers.
To simplify: you can build up any path (directory and file path objects are treated exactly the same) as an object, which can be an absolute path object or a relative path object. You can use raw strings to make complex paths (i.e., r'string') and pathlib will be very forgiving. However, note that there are better ways to build up paths than raw strings (see further down).
Here are examples:
from pathlib import Path
Path(r'c:\temp\foo.bar') # absolute path
Path(r'c:/temp/foo.bar') # same absolute path
Path('foo.bar') # different path, RELATIVE to current directory
Path('foo.bar').resolve() # resolve converts to absolute path
Path('foo.bar').exists() # check to see if path exists
Note that if you're on Windows pathlib forgives you for using the "wrong slash" in the second example. See discussion at the end about why you should probably always use the forward slash.
Simple displaying of some useful paths- such as the current working directory and the user home- works like this:
# Current directory (relative):
cwd = Path() # or Path('.')
print(cwd)
# Current directory (absolute):
cwd = Path.cwd()
print(cwd)
# User home directory:
home = Path.home()
print(home)
# Something inside the current directory
file_path = Path('some_file.txt') # relative path; or
file_path = Path()/'some_file.txt' # also relative path
file_path = Path().resolve()/Path('some_file.txt') # absolute path
print(file_path)
To navigate down the file tree, you can do things like this. Note that the first object, home, is a Path and the rest are just strings:
afile = home/'Documents'/'Project Documentation'/'file.txt' # or
afile = home.join('Documents','Project Documentation','file.txt')
Check to see if it is a file or a directory (and exists) this way:
afile.is_dir()
afile.is_file()
Make a new, empty file without opening it like this (silently replaces any existing file):
afile.touch()
To make the file only if it doesn't exist, use exist_ok=False:
try:
afile.touch(exist_ok=False)
except FileExistsError:
# file exists
Make a new directory (under the current directory, Path()) like this:
Path().mkdir('new/dir') # get errors if Path()/`new` doesn't exist
Path().mkdir('new/dir', parents=True) # will make Path()/`new` if it doesn't exist
Path().mkdir('new/dir', exist_ok=True) # errors ignored if `dir` already exists
Get the file extension or filename of a path this way:
afile.suffix # empty string if no extension
afile.stem # note: works on directories too
Use name for the entire last part of the path (stem and extension if they are there):
afile.name # note: works on directories too
Rename a file using the with_name method (which returns the same path object but with a new filename):
new_file = afile.with_name('file_new.txt')
You can iterate through all the "stuff' in a directory like so using iterdir:
all_the_things = list(Path().iterdir()) # returns a list of Path objects
Sidebar: backslashes (\)
Be careful when using backslashes in a path string, especially ending a path with a backslash. As with any string, Python will read that terminating backslash as an escape character even in raw input mode. Observe:
>>> r'\'
File "<stdin>", line 1
r'\'
^
SyntaxError: EOL while scanning string literal
So this will give a pretty cryptic error message if you are not aware of this issue:
>>> Path(r'C:\')
File "<stdin>", line 1
Path(r'\')
^
SyntaxError: EOL while scanning string literal
The reason for this error is that \' is assumed to be a single quotation in the string. This works fine: '\'' (the second single quotation ends the string).
If you insist on using backslashes, be sure to use raw input mode or you will run into problems. For example, the '\t' character represents a tab. So when you do this (without raw input):
>>> Path('C:\temp')
You are putting a tab character into your path. This is perfectly legal and Python won't complain until you do something that causes Windows to try turning it into a real Windows path:
>>> Path('C:\temp').resolve()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\temp'
This is also a very cryptic error if you do not know what is going on! Best to avoid the backslash characters altogether when messing about with paths.
It's work.
I change strings content.
Thank you so much.
strings = ("Vserver Name:", "Volume Name:", "Aggregate Name:", "Volume Size:", "Available Size:", " Used Size:", " Used Percentage:", "Node name:", "Space Saved by Storage Efficiency:")
with open('find_c_volume_show.txt', 'w') as f:
f.write("Vserver,Volume,Aggregate,Total,Avail,Used,UsedP,Node,Saved\n")
row = []
for line in open("c_volume_show.txt"):
if any(s in line for s in strings):
field1,field2 = line.strip().split(':')
row.append(field2.strip())
if len(row) == len(strings):
f.write('{}'.format(','.join(row)) +'\n' )
row = []
f.close()
Related
I am try to create some temporal files and make some operations on them inside a loop. Then I will access the information on all of the temporal files. And do some operations with that information. For simplicity I brought the following code that reproduces my issue:
import tempfile
tmp_files = []
for i in range(40):
tmp = tempfile.NamedTemporaryFile(suffix=".txt")
with open(tmp.name, "w") as f:
f.write(str(i))
tmp_files.append(tmp.name)
string = ""
for tmp_file in tmp_files:
with open(tmp_file, "r") as f:
data = f.read()
string += data
print(string)
ERROR:
with open(tmp_file, "r") as f: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpynh0kbnw.txt'
When I look on /tmp directory (with some time.sleep(2) on the loop) I see that the file is deleted and only one is preserved. And for that the error.
Of course I could handle to keep all the files with the flag tempfile.NamedTemporaryFile(suffix=".txt", delete=False). But that is not the idea. I would like to hold the temporal files just for the running time of the script. I also could delete the files with os.remove. But my question is more why this happen. Because I expected that the files hold to the end of the running. Because I don't close the file on the execution (or do I?).
A lot of thanks in advance.
tdelaney does already answer your actual question.
I just would like to offer you an alternative to NamedTemporaryFile. Why not creating a temporary folder which is removed (with all files in it) at the end of the script?
Instead of using a NamedTemporaryFile, you could use tempfile.TemporaryDirectory. The directory will be deleted when closed.
The example below uses the with statement which closes the file handle automatically when the block ends (see John Gordon's comment).
import os
import tempfile
with tempfile.TemporaryDirectory() as temp_folder:
tmp_files = []
for i in range(40):
tmp_file = os.path.join(temp_folder, f"{i}.txt")
with open(tmp_file, "w") as f:
f.write(str(i))
tmp_files.append(tmp_file)
string = ""
for tmp_file in tmp_files:
with open(tmp_file, "r") as f:
data = f.read()
string += data
print(string)
By default, a NamedTemporaryFile deletes its file when closed. its a bit subtle, but tmp = tempfile.NamedTemporaryFile(suffix=".txt") in the loop causes the previous file to be deleted when tmp is reassigned. One option is to use the delete=False parameter. Or, just keep the file open and seek to the beginning after the write.
NamedTemporaryFile is already a file object - you can write to it directly without reopening. Just make sure the mode is "write plus" and in text, not binary mode. Put the code an a try/finally block to make sure the files are really deleted at the end.
import tempfile
tmp_files = []
try:
for i in range(40):
tmp = tempfile.NamedTemporaryFile(suffix=".txt", mode="w+")
tmp.write(str(i))
tmp.seek(0)
tmp_files.append(tmp)
string = ""
for tmp_file in tmp_files:
data = tmp_file.read()
string += data
finally:
for tmp_file in tmp_files:
tmp_file.close()
print(string)
I am trying to create a file in a certain directory, and save the name of that file with today's date.
I am having some issue, where the file is created, but the title line that I want to write in, does not work.
from datetime import datetime
today = datetime.now().date().strftime('%Y-%m-%d')
g = open(path_prefix+today+'.csv', 'w+')
if os.stat(path_prefix+today+'.csv').st_size == 0: # this checks if file is empty
g = open(path_prefix+today+'.csv', 'w+')
g.write('Title\r\n')
path_prefix is just a path to the directory I am saving in /Users/name/Documents/folder/subfolder/
I am expecting a file 2019-08-22.csv to be saved in the directory given by path_prefix with a title as specified in the last line of the code above.
What I am getting is an empty file, and if I run the code again then the title is appended into the file.
As mentioned by #sampie777 I was not losing the file after writing to it, which is why the changes were not being saved when I opened the file. Adding close in an extra line solves the issue that I was having
from datetime import datetime
today = datetime.now().date().strftime('%Y-%m-%d')
g = open(path_prefix+today+'.csv', 'w+')
if os.stat(path_prefix+today+'.csv').st_size == 0: #this checks if file is empty
g = open(path_prefix+today+'.csv', 'w+')
g.write('Title\r\n')
g.close()
I am sure there are plenty of other ways to do this
You need to close the file before the content will be written to it. So call
g.close().
I can suggest to use:
with open(path_prefix+today+'.csv', 'w+') as g:
g.write('...')
This will automatically handle closing the file for you.
Also, why are you opening the file two times?
Tip: I see you are using path_prefix+today+'.csv' a lot. Create a variable for this, so you're code will be a lot easier to maintain.
Suggested refactor of the last lines:
output_file_name = path_prefix + today + '.csv' # I prefer "{}{}.csv".format(path_prefix, today) or "%s%s.csv" % (path_prefix, today)
is_output_file_empty = os.stat(output_file_name).st_size == 0
with open(output_file_name, 'a') as output_file:
if is_output_file_empty:
output_file.write('Title\r\n')
For more information, see this question: Correct way to write line to file?
and maybo also How to check whether a file is empty or not?
I haven't used Python in a while, but by doing a quick bit of research, this seems like it could work:
# - Load imports
import os
import os.path
from datetime import datetime
# - Get the date
dateToday = datetime.now().date()
# - Set the savePath / path_prefix
savePath = 'C:/Users/name/Documents/folder/subfolder/'
fileName = dateToday.strftime("%Y-%m-%d") # - Convert 'dateToday' to string
# - Join path and file name
completeName = os.path.join(savePath, fileName + ".csv")
# - Check for file
if (not path.exists(completeName)):
# - If it doesn't exist, write to it and then close
with (open(completeName, 'w+') as file):
file.write('Title\r\n')
else:
print("File already exists")
I want the absolute path of the file selected as input file (from file browser in the form) using the python code below:
for attr, document in request.files.iteritems():
orig_filename = document.filename
print os.path.abspath(orig_filename)
mhash = get_hash_for_doc(orig_filename)
This prints the path of current working directory along(where the python script is executing) with the 'orig_filename' appended to it, which is the wrong path. I am using python 2.7, flask 0.12 under linux OS.
The requirement is to find the hash value of the file before uploading it to the server to check deduplication. So I need to use the algorithm by passing the file selected for hashing to another function as:
def get_hash_for_doc(orig_filename):
mhash = None
hash = sha1()#md5()
with open(mfile, "rb") as f:
for chunk in iter(lambda: f.read(128 * hash.block_size), b""):
hash.update(chunk)
mhash = hash.hexdigest()
return mhash
In this function I want to read file from absolute path of the orig_filename before uploading. Avoided all other code checks here.
First you need to create a temp file to simulate this required file then make your process on it
import tempfile, os
try:
fd, tmp = tempfile.mkstemp()
with os.fdopen(fd, 'w') as out:
out.write(file.read())
mhash = get_hash_for_doc(tmp)
finally:
os.unlink(tmp)
If you want to find a folder/file.ext, for an input file, simply use 'os.path.abspath' like:
savefile = os.path.abspath(Myinputfile)
when "Myinputfile" is a variable that contains the relative path and file name. For instance, derived from an argument define by the user.
But if you prefer to have absolute address of the folder, without file name try this:
saveloc = os.path.dirname(os.path.realpath(Myinputfile))
You can use pathlib to find the absolute path of the selected file.
I am trying to write a Python script that takes the contents of a text file, and copies it into a new file that the program creates itself.
This is the code I am testing at the moment:
from datetime import datetime
errorLogPath = datetime.strftime(datetime.now(), '%Y%m%d_%H:%M') + ".log"
with open("Report.log") as logFile:
with open(errorLogPath, 'w') as errorLog:
for line in logFile:
errorLog.write(line)
Currently the new file is created, but it is completely blank and has the wrong filename. The filename should be YYYYMMDD_HH:MM.log instead I am getting a filename which does not show the minutes and the file is empty.
EDIT: Removed an unnecessary if statement, but the code is still not functioning :\
The simplest way to copy the file in python without using shutil module is:
with open("Report.log") as logFile, open(errorLogPath, 'w') as errorLog:
errorlog.writelines(logFile)
To use the shutil module:
import shutil
shutil.copy("Report.log", errorLogPath)
The problem is in your path name, : is a reserved characters in windows, here is the whole list:
< (less than)
> (greater than)
: (colon)
" (double quote)
/ (forward slash)
\ (backslash)
| (vertical bar or pipe)
? (question mark)
(asterisk)
the colon is referred as:
A disk designator with a backslash, for example "C:\" or "d:\".
Therefore, the correct solution will be to change your errorLogPath to remove the : character.
Then, the best way to copy a file is yo use copy
from datetime import datetime
from shutil import copy
error_log_path = datetime.strftime(datetime.now(), '%Y%m%d_%H_%M') + ".log"
log_file_path = "Report.log"
copy(log_file_path, error_log_path)
Note:
You can open several files with a single with statement.
It is better not to use lower_case rather than camelCase for variable's name in python.
try this, this has worked for me:
from datetime import datetime
import csv
errorLogPath = datetime.strftime(datetime.now(), '%Y%m%d_%H:%M') + ".log"
ff = open(errorLogPath, 'w')
csvwriter = csv.writer(ff)
with open("Report.log","r") as logFile:
reader = csv.reader(logFile)
for line in reader:
if "ROW" in line:
csvwriter.writerow(line)
else:
continue
ff.close()
I am building a python script to basically edit lots of files by means of searching and replacing words in the file.
There is an original file named: C:\python 3.5/remedy line 1.ahk
There is a file containing the words I want to replace (search words) in the original document and a text file that has the list of the new words that I would like to be placed into the final document.
The script then runs and works perfect. The final document is then created and named based on a line in the final text file (code begins on line 72). A way so I can tell what the final product is by looking at it. This file is originally named output = open("C:\python 3.5\output.ahk", 'w') and later in the script it is renamed based on line 37 in the script. All that works fine.
So the seemingly simple part left that I can't seem to figure out is how to take this one file and move it to a directory where it belongs. That directory is created based on the same line in that the file gets its name from (code starts on line 82). How do I simply move my file into a directory that has been created by the script, i.e. based on a variable (code starts on line 84 for this) so the name of the file is based on a variable.
import shutil
#below is where your modified file sits, before we move it into it's own directory named dst, based on a variable #mainnewdir
srcdir = r'C:\python 3.5/'+(justfilename)
dst = (mainnewdir)+(justfilename)
shutil.copyfile(src, dst)
Why does it format it with extra \ in the code?
Why does it seem to not give me a error if I use a / vs. a \ slash?
Here is the entire code, like I said only the last part of moving the file does not work:
import os
import linecache
import sys
import string
import re
## information/replacingvalues.txt this is the text of the values you want in your final document
#information = open("C:\python 3.5\replacingvalues.txt", 'r')
information = open("C:\python 3.5/replacingvalues.txt", 'r')
# information = open("C:\Program Files (x86)\Python35- 32\Scripts\Text_Find_and_Replace\information/replacingvalues.txt",
# Text_Find_and_Replace\Result/output.txt This is the dir and the sum or final document
# output = open("C:\python 3.5\output.ahk", 'w')
createblank = open ("C:\python 3.5/output.ahk", 'w')
createblank.close()
output = open("C:\python 3.5\output.ahk", 'w')
# field = open("C:\Program Files (x86)\Python35- 32\Scripts\Text_Find_and_Replace\Field/values.txt"
# Field is the file or words you will be replacing
field = open("C:\python 3.5/values.txt", 'r')
# modified code for autohot key
# Text_Find_and_Replace\Test/remedy line 1.ahk is the original doc you want modified
with open("C:\python 3.5/remedy line 1.ahk", 'r') as myfile:
inline = myfile.read()
## remedy line 1.ahk
informations = []
fields = []
dictionary = {}
i = 0
for line in information:
informations.append(line.splitlines())
for lines in field:
fields.append(lines.split())
i = i + 1;
if (len(fields) != len(informations)):
print("replacing values and values have different numbers")
exit();
else:
for i in range(0, i):
rightvalue = str(informations[i])
rightvalue = rightvalue.strip('[]')
rightvalue = rightvalue[1:-1]
leftvalue = str(fields[i])
leftvalue = leftvalue.strip('[]')
leftvalue = leftvalue.strip("'")
dictionary[leftvalue] = rightvalue
robj = re.compile('|'.join(dictionary.keys()))
result = robj.sub(lambda m: dictionary[m.group(0)], inline)
output.write(result)
information.close;
output.close;
field.close;
output.close()
import os
import linecache
linecache.clearcache()
newfilename= linecache.getline("C:\python 3.5/remedy line 1.txt",37)
filename = ("C:\python 3.5/output.ahk")
os.rename(filename, newfilename.strip())
#os.rename(filename, newfilename.strip()+".ahk")
linecache.clearcache()
############## below will create a new directory based on the the word or words in line 37 of the txt file.
newdirname= linecache.getline("C:\python 3.5/remedy line 1.txt",37)
#newpath = r'C:\pythontest\automadedir'
#below removes the /n ie new line raw assci
justfilename = (newdirname).strip()
#below removes the .txt from the rest of the justfilename..
autocreateddir = (justfilename).strip(".txt")
# below is an example of combining a string and a variable
# below makes the variable up that will be the name of the new directory based on reading line 37 of a text file above
mainnewdir= r'C:\pythontest\automadedir/'+(autocreateddir)
if not os.path.exists(mainnewdir):
os.makedirs(mainnewdir)
linecache.clearcache()
# ####################################################
#below is where your modified file sits, before we move it into it's own directory named dst, based on a variable #mainnewdir
srcdir = r'C:\python 3.5/'+(justfilename)
dst = (mainnewdir)+(justfilename)
shutil.copyfile(src, dst)
backslashes do not have a mind of their own.
When you paste windows paths as-is and they contain \n, r, \b, \x, \v, \U (python 3), (refer to table here for all of them), you're just using escape sequences without noticing it.
When the escape sequence doesn't exist (ex \p) it works. But when it's known the filenames are often invalid. Which explains the apparent randomness of the issue.
To be able to safely paste windows paths without changing/escaping them, just use the raw prefix:
my_file = r"C:\temp\foo.txt"
so the backslashes won't be interpreted. One exception though: if string ends with backslash you still have to double it.