If you need any more info just Let Me Know
I have a python script that adds a string after each line on a CSV file. the line file_lines = [''.join([x.strip(), string_to_add, '\n']) for x in f.readlines()] is the trouble maker. For each file line it will add the string and then add a new line after each time the string is added.
Here is the script:
#Adding .JPG string to the end of each line for the Part Numbers
string_to_add = ".JPG"
#Open the file and join the .JPG to the current lines
with open("PartNums.csv", 'r') as f:
file_lines = [''.join([x.strip(), string_to_add, '\n']) for x in f.readlines()]
#Writes to the file until its done
with open("PartNums.csv", 'w') as f:
f.writelines(file_lines)
The script works and does what it is supposed to, however my issue is later on in this larger script. This script outputs into a CSV file and it looks like this:
X00TB0001.JPG
X01BJ0003.JPG
X01BJ0004.JPG
X01BJ0005.JPG
X01BJ0006.JPG
X01BJ0007.JPG
X01BJ0008.JPG
X01BJ0026.JPG
X01BJ0038.JPG
X01BJ0039.JPG
X01BJ0040.JPG
X01BJ0041.JPG
...
X01BJ0050.JPG
X01BJ0058.JPG
X01BJ0059.JPG
X01BJ0060.JPG
X01BJ0061.JPG
X01BJ0170.JPG
X01BJ0178.JPG
Without the \n in that line the csv file output looks like this file_lines = [''.join([x.strip(), string_to_add]) for x in f.readlines()]:
X00TB0001.JPGX01BJ0003.JPGX01BJ0004.JPGX01BJ0005.JPGX01BJ0006.JPG
The issue is when I go to read this file later and move files with it using this script:
#If the string matches a file name move it to a new directory
dst = r"xxx"
with open('PicsWeHave.txt') as my_file:
for filename in my_file:
src = os.path.join(XXX") # .strip() to avoid un-wanted white spaces
#shutil.copy(src, os.path.join(dst, filename.strip()))
shutil.copy(os.path.join(src, filename), os.path.join(dst, filename))
When I run this whole Script it works until it has to move the files I get this error:
FileNotFoundError: [Errno 2] No such file or directory: 'XXX\\X15SL0447.JPG\n'
I know the file exist however the '\n' should not be there and that's why I am asking how can I still get everything on a new line and not have \n after each name so when I move the file the strings match.
Thank You For Your Help!
As they said above you should use .strip():
shutil.copy(os.path.join(src, filename.strip()), os.path.join(dst, filename.strip()))
This way it gives you the file name or string you need and then it removes anything else.
I have a huge report file with some data where i have to do some data processing on lines starting with the code "MLT-TRR"
For now i have extracted all the lines in my script that start with that code and placed them in a separate file. The new file looks like this- Rules.txt.
MLT-TRR Warning C:\Users\Di\Pictures\SavedPictures\top.png 63 10 Port is not registered [Folder: 'Picture']
MLT-TRR Warning C:\Users\Di\Pictures\SavedPictures\tree.png 315 10 Port is not registered [Folder: 'Picture.first_inst']
MLT-TRR Warning C:\Users\Di\Pictures\SavedPictures\top.png 315 10 Port is not registered [Folder: 'Picture.second_inst']
MLT-TRR Warning C:\Users\Di\Pictures\SavedPictures\tree.png 317 10 Port is not registered [Folder: 'Picture.third_inst']
MLT-TRR Warning C:\Users\Di\Pictures\SavedPictures\top.png 317 10 Port is not registered [Folder: 'Picture.fourth_inst']
For each of these lines i have to extract the data that lies after "[Folder: 'Picture" If there is no data after "[Folder: 'Picture" as in the case of my first line, then skip that line and move on to the next line.
I also want to extract the file names for each of those lines- top.txt, tree.txt
I couldnt think of a simpler method to do this as this involves a loop and gets messier.
Is there any way out i can do this? extracting just the file paths and the ending data of each line.
import os
import sys
from os import path
import numpy as np
folder_path = os.path.dirname(os.path.abspath(__file__))
inFile1 = 'Rules.txt'
inFile2 = 'TopRules.txt'
def open_file(filename):
try:
with open(filename,'r') as f:
targets = [line for line in f if "MLT-TRR" in line]
print targets
f.close()
with open(inFile1, "w") as f2:
for line in targets:
f2.write(line + "\n")
f2.close()
except Exception,e:
print str(e)
exit(1)
if __name__ == '__main__':
name = sys.argv[1]
filename = sys.argv[1]
open_file(filename)
To extract the filenames and other data, you should be able to use a regular expression:
import re
for line in f:
match = re.match(r"^MLT-TRR.*([A-Za-z]:\\[-A-Za-z0-9_:\\.]+).*\[Folder: 'Picture\.(\w+)']", line)
if match:
filename = match.group(1)
data = match.group(2)
This assumes that the data after 'Picture. only contains alphanumeric characters and underscores. And you may have to change the allowed characters in the filename part [A-Za-z0-9_:\\.] if you have weird filenames. It also assumes the filenames start with the Windows drive letter (so absolute paths), to make it easier to distinguish from other data in the line.
If you just want the basename of the filename, then after extracting it you can use os.path.basename or pathlib.Path.name.
I had a very similar problem and solved it by searching for the specific line 'key', in your case MLT-TRR" with regex and then specifying which 'bytes' to take from that line. I then append the selected data to an array.
import re #Import the regex function
#Make empty arrays:
P190=[] #my file
shot=[] #events in my file (multiple lines of text for each event)
S011east=[] #what I want
S011north #another thing I want
#Create your regex:
S011=re.compile(r"^S0\w*\W*11\b")
#search and append:
#Open P190 file
with open(import_file_path,'rt') as infile:
for lines in infile:
P190.append(lines.rstrip('\n'))
#Locate specific lines and extract data
for line in P190:
if S011.search(line)!= None:
easting=line[47:55]
easting=float(easting)
S011east.append(easting)
northing=line[55:64]
northing=float(northing)
S011north.append(northing)
If you set up regex to look for "MLT_TRR ????? Folder: 'Picture.'" then it should skip any lines that don't have any further information.
For the second part of your question.
I doubt your file names are a constant length so the above method won't work as you can't specify a number of bytes to extract.This code extracts the name and extension from a file path, you could apply it to whatever you extract from each line.
import os
tail=os.path.basename(import_file_path) #Get file name from path
I am trying to create a file in a certain directory, and save the name of that file with today's date.
I am having some issue, where the file is created, but the title line that I want to write in, does not work.
from datetime import datetime
today = datetime.now().date().strftime('%Y-%m-%d')
g = open(path_prefix+today+'.csv', 'w+')
if os.stat(path_prefix+today+'.csv').st_size == 0: # this checks if file is empty
g = open(path_prefix+today+'.csv', 'w+')
g.write('Title\r\n')
path_prefix is just a path to the directory I am saving in /Users/name/Documents/folder/subfolder/
I am expecting a file 2019-08-22.csv to be saved in the directory given by path_prefix with a title as specified in the last line of the code above.
What I am getting is an empty file, and if I run the code again then the title is appended into the file.
As mentioned by #sampie777 I was not losing the file after writing to it, which is why the changes were not being saved when I opened the file. Adding close in an extra line solves the issue that I was having
from datetime import datetime
today = datetime.now().date().strftime('%Y-%m-%d')
g = open(path_prefix+today+'.csv', 'w+')
if os.stat(path_prefix+today+'.csv').st_size == 0: #this checks if file is empty
g = open(path_prefix+today+'.csv', 'w+')
g.write('Title\r\n')
g.close()
I am sure there are plenty of other ways to do this
You need to close the file before the content will be written to it. So call
g.close().
I can suggest to use:
with open(path_prefix+today+'.csv', 'w+') as g:
g.write('...')
This will automatically handle closing the file for you.
Also, why are you opening the file two times?
Tip: I see you are using path_prefix+today+'.csv' a lot. Create a variable for this, so you're code will be a lot easier to maintain.
Suggested refactor of the last lines:
output_file_name = path_prefix + today + '.csv' # I prefer "{}{}.csv".format(path_prefix, today) or "%s%s.csv" % (path_prefix, today)
is_output_file_empty = os.stat(output_file_name).st_size == 0
with open(output_file_name, 'a') as output_file:
if is_output_file_empty:
output_file.write('Title\r\n')
For more information, see this question: Correct way to write line to file?
and maybo also How to check whether a file is empty or not?
I haven't used Python in a while, but by doing a quick bit of research, this seems like it could work:
# - Load imports
import os
import os.path
from datetime import datetime
# - Get the date
dateToday = datetime.now().date()
# - Set the savePath / path_prefix
savePath = 'C:/Users/name/Documents/folder/subfolder/'
fileName = dateToday.strftime("%Y-%m-%d") # - Convert 'dateToday' to string
# - Join path and file name
completeName = os.path.join(savePath, fileName + ".csv")
# - Check for file
if (not path.exists(completeName)):
# - If it doesn't exist, write to it and then close
with (open(completeName, 'w+') as file):
file.write('Title\r\n')
else:
print("File already exists")
I made some script for modify text.
But I can't make result.
below is my script.
i just begin study python.
I think my script didn't work because f = open('find_c_volume_show.txt', 'w')
please help me.
import sys
from itertools import islice
def next_n_lines(file_opened, N):
return [x.strip() for x in islice(file_opened, N)]
field_line = 1
num = 0
N = 9
split_line = field_line / N
strings = ("Vserver", "Volume Name", "Used Size", "Available Size", "Volume Size", "Aggregate Name", "Space Saved by Storage Efficiency")
f = open('find_c_volume_show.txt', 'w')
for line in open("c_volume_show.txt"):
if any(s in line for s in strings):
field1,field2 = line.strip().split(':')
field_line += 1
f.write(field2 + '\n')
f.close()
f = open('find_c_volume_show.txt', 'w')
f.write("Vserver,Volume Name,Aggregate Name,Volume Size,Available Size,Used Size,Space Saved\n")
with open('find_c_volume_show.txt', 'w') as result:
while num < split_line:
num += 1
lines = next_n_lines(result, N)
f.write('{}'.format(','.join(lines)) +'\n' )
f.close()
below is my text file. text file has data multi data.
Vserver Name: FAS8040-ZZZZ
Volume Name: vol0
Aggregate Name: Node1_aggr0
Volume Size: 466.6GB
Available Size: 436.2GB
Filesystem Size: 466.6GB
Total User-Visible Size: 466.6GB
Used Size: 30.40GB
Used Percentage: 6%
Node name: FAS8040-ZZZZ
Space Saved by Storage Efficiency: 0B
Percentage Saved by Storage Efficiency: 0%
Space Saved by Deduplication: 0B
Space Saved by Compression: 0B
Percentage Space Saved by Compression: 0%
Total Physical Used Size: 22.37GB
I want result like below.
Vserver,Volume Name,Aggregate Name,Volume Size,Available Size,Used Size,Space Saved
FAS8040-ZZZZ,vol0,Node1_aggr0,466.6GB,435.7GB,30.92GB,0B
FAS8040-YYYY,vol0,Node2_aggr0,466.6GB,428.7GB,37.91GB,0B
FAS8040-XXXX,vol0,Node2_aggr0,466.6GB,428.7GB,37.91GB,0B
The problem is that each time you open file with open(filename,'w'), it gets erased. You can use "temporal" file with different name to store results of your first for loop, or I would suggest to aggregate content of each line to list, and then write it straight away.
Additionally, you have problems with your "split_line" value, it is always 0. I guess what you mean is len(strings).
Here is a code:
import sys
strings = ("Vserver", "Volume Name", "Used Size", "Available Size", "Volume Size", "Aggregate Name", "Space Saved by Storage Efficiency")
with open('find_c_volume_show.txt', 'w') as f:
f.write("Vserver,Volume Name,Aggregate Name,Volume Size,Available Size,Used Size,Space Saved,Snapshot,Total Used Size\n")
row = []
for line in open("c_volume_show.txt"):
if any(s in line for s in strings):
field1,field2 = line.strip().split(':')
row.append(field2)
print(row)
if len(row) == len(strings):
f.write('{}'.format(','.join(row)) +'\n' )
print(row)
row = []
The new-ish pathlib module (available in Python >= 3.4) is a much, much easier way for reading and writing files than the traditional open() function. It is also great for working with path-like objects (both in Windows and for other OSes).
from pathlib import Path
afile = Path(r'c:\temp\foo.bar') # absolute path to file
To read a file located at a path, you can just grab the text directly from the Path object!
contents = afile.read_text()
content_lines = contents.split('\n')
...and WRITE text directly!
data = '\n'.join(content_lines)
afile.write_text(data) # overwrites existing file
You can also use its open method rather than the open function:
with afile.open() as f:
dostuff(f)
The with statement is a context manager. It automatically "cleans up" by closing the file when you are done (no matter what happens- even if there is an error).
It's Paths - Paths all the way down
Here is more about the Path library copied from another of my answers.
To simplify: you can build up any path (directory and file path objects are treated exactly the same) as an object, which can be an absolute path object or a relative path object. You can use raw strings to make complex paths (i.e., r'string') and pathlib will be very forgiving. However, note that there are better ways to build up paths than raw strings (see further down).
Here are examples:
from pathlib import Path
Path(r'c:\temp\foo.bar') # absolute path
Path(r'c:/temp/foo.bar') # same absolute path
Path('foo.bar') # different path, RELATIVE to current directory
Path('foo.bar').resolve() # resolve converts to absolute path
Path('foo.bar').exists() # check to see if path exists
Note that if you're on Windows pathlib forgives you for using the "wrong slash" in the second example. See discussion at the end about why you should probably always use the forward slash.
Simple displaying of some useful paths- such as the current working directory and the user home- works like this:
# Current directory (relative):
cwd = Path() # or Path('.')
print(cwd)
# Current directory (absolute):
cwd = Path.cwd()
print(cwd)
# User home directory:
home = Path.home()
print(home)
# Something inside the current directory
file_path = Path('some_file.txt') # relative path; or
file_path = Path()/'some_file.txt' # also relative path
file_path = Path().resolve()/Path('some_file.txt') # absolute path
print(file_path)
To navigate down the file tree, you can do things like this. Note that the first object, home, is a Path and the rest are just strings:
afile = home/'Documents'/'Project Documentation'/'file.txt' # or
afile = home.join('Documents','Project Documentation','file.txt')
Check to see if it is a file or a directory (and exists) this way:
afile.is_dir()
afile.is_file()
Make a new, empty file without opening it like this (silently replaces any existing file):
afile.touch()
To make the file only if it doesn't exist, use exist_ok=False:
try:
afile.touch(exist_ok=False)
except FileExistsError:
# file exists
Make a new directory (under the current directory, Path()) like this:
Path().mkdir('new/dir') # get errors if Path()/`new` doesn't exist
Path().mkdir('new/dir', parents=True) # will make Path()/`new` if it doesn't exist
Path().mkdir('new/dir', exist_ok=True) # errors ignored if `dir` already exists
Get the file extension or filename of a path this way:
afile.suffix # empty string if no extension
afile.stem # note: works on directories too
Use name for the entire last part of the path (stem and extension if they are there):
afile.name # note: works on directories too
Rename a file using the with_name method (which returns the same path object but with a new filename):
new_file = afile.with_name('file_new.txt')
You can iterate through all the "stuff' in a directory like so using iterdir:
all_the_things = list(Path().iterdir()) # returns a list of Path objects
Sidebar: backslashes (\)
Be careful when using backslashes in a path string, especially ending a path with a backslash. As with any string, Python will read that terminating backslash as an escape character even in raw input mode. Observe:
>>> r'\'
File "<stdin>", line 1
r'\'
^
SyntaxError: EOL while scanning string literal
So this will give a pretty cryptic error message if you are not aware of this issue:
>>> Path(r'C:\')
File "<stdin>", line 1
Path(r'\')
^
SyntaxError: EOL while scanning string literal
The reason for this error is that \' is assumed to be a single quotation in the string. This works fine: '\'' (the second single quotation ends the string).
If you insist on using backslashes, be sure to use raw input mode or you will run into problems. For example, the '\t' character represents a tab. So when you do this (without raw input):
>>> Path('C:\temp')
You are putting a tab character into your path. This is perfectly legal and Python won't complain until you do something that causes Windows to try turning it into a real Windows path:
>>> Path('C:\temp').resolve()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\temp'
This is also a very cryptic error if you do not know what is going on! Best to avoid the backslash characters altogether when messing about with paths.
It's work.
I change strings content.
Thank you so much.
strings = ("Vserver Name:", "Volume Name:", "Aggregate Name:", "Volume Size:", "Available Size:", " Used Size:", " Used Percentage:", "Node name:", "Space Saved by Storage Efficiency:")
with open('find_c_volume_show.txt', 'w') as f:
f.write("Vserver,Volume,Aggregate,Total,Avail,Used,UsedP,Node,Saved\n")
row = []
for line in open("c_volume_show.txt"):
if any(s in line for s in strings):
field1,field2 = line.strip().split(':')
row.append(field2.strip())
if len(row) == len(strings):
f.write('{}'.format(','.join(row)) +'\n' )
row = []
f.close()
I'm trying to write a Python script that will take any playlist and recreate it on another file structure. I have it written now so that all the filenames are stripped off the original playlist and put into a file. That works. Then the function findsong() is supposed to walk thru the new directory and find the same songs and make a new playlist based on the new directory structure.
Here's where it gets weird. If I use 'line' as my argument in the line 'If line in files' I get an empty new playlist. If I use ANY file that I know is there as the argument the entire playlist is recreated, not just the file I used as the argument. That's how I have it set up in this code. I cannot figure out this weird behavior. As long as the file exists, the whole playlist is recreated with the new paths. Wut??
Here is the code:
import os
def check():
datafile = open('testlist.m3u')
nopath = open('nopath.txt', 'w')
nopath.truncate()
for line in datafile:
if 'mp3' in line:
nopath.write(os.path.basename(line))
if 'wma' in line:
nopath.write(os.path.basename(line))
nopath.close()
def findsong():
nopath = open('nopath.txt')
squeezelist = open('squeezelist.m3u' ,'w')
squeezelist.truncate()
for line in nopath:
print line
for root, dirs, files in os.walk("c:\\Documents and Settings\\"):
print files
if ' Tuesday\'s Gone.mp3' in files:
squeezelist.write(os.path.join(root, line))
squeezelist.close()
check()
findsong()
When you iterate over the lines in a file Python retains the trailing newlines \n. You'll want to strip those off:
for line in nopath:
line = line.rstrip()