So i wrote a little program in python which allows me to take a .csv file, filter out the lines i need and then export these into a new .txt file.
This worked quite well, so i decided to make it more user friendly by allowing the user to select the file that should be converted by himself through the console (command line).
My problem: The file is imported as a .csv file but not exported as a .txt file which leads to my program overwriting the original file which will be emptied because of a step in my program which allows me to delete the first two lines of the output text.
Does anyone know a solution for this?
Thanks :)
import csv
import sys
userinput = raw_input('List:')
saveFile = open(userinput, 'w')
with open(userinput, 'r') as file:
reader = csv.reader(file)
count = 0
for row in reader:
print(row[2])
saveFile.write(row[2] + ' ""\n')
saveFile.close()
saveFile = open(userinput, 'r')
data_list = saveFile.readlines()
saveFile.close()
del data_list[1:2]
saveFile = open(userinput, 'w')
saveFile.writelines(data_list)
saveFile.close()
Try This:
userinput = raw_input('List:')
f_extns = userinput.split(".")
saveFile = open(f_extns[0]+'.txt', 'w')
I think you probably just want to save the file with a new name, this Extracting extension from filename in Python talks about splitting out the extension so then you can just add your own extension
you would end up with something like
name, ext = os.path.splitext(userinput)
saveFile = open(name + '.txt', 'w')
You probably just need to change the extension of the output file. Here is a solution that sets the output file extension to .txt; if the input file is also .txt then there will be a problem, but for all other extensions of the input file this should work.
import csv
import os
file_name = input('Name of file:')
# https://docs.python.org/3/library/os.path.html#os.path.splitext
# https://stackoverflow.com/questions/541390/extracting-extension-from-filename-in-python
file_name, file_ext_r = os.path.splitext(file_name)
file_ext_w = '.txt'
file_name_r = ''.format(file_name, file_ext_r)
file_name_w = ''.format(file_name, file_ext_w)
print('File to read:', file_name_r)
print('File to write:', file_name_w)
with open(file_name_r, 'r') as fr, open(file_name_w, 'w') as fw:
reader = csv.reader(fr)
for i, row in enumerate(reader):
print(row[2])
if i >= 2:
fw.write(row[2] + ' ""\n')
I also simplified your logic to avoid writting the first 2 lines to the output file; no need to read and write the output file again.
Does this work for you?
Related
I'm creating new files from originally existing ones in the mdp folder by changing a couple of lines in those files using python. I need to do this for 1000 files. Can anyone suggest a for loop which reads all files and changes them and creates new in one go?
This way I have to change the the number followed by 'md_' in the path and it's tedious because there are a 1000 files here.
I tried using str() but there was a 'could not read file error'
fin = open("/home/abc/xyz/mdp/md_1.mdp", "rt")
fout = open("/home/abc/xyz/middle/md_1.mdp", "wt")
for line in fin:
fout.write(line.replace('integrator = md', 'integrator
= md-vv'))
fin = open("/home/abc/xyz/middle/md_1.mdp", "rt")
fout = open("/home/abc/xyz/mdb/md_1.mdp", "wt")
for line in fin:
fout.write(line.replace('dt = 0.001', 'dt
= -0.001'))
fin.close()
fout.close()
os.listdir(path) is your friend:
import os
sourcedir = "/home/abc/xyz/mdp"
destdir = "/home/abc/xyz/middle"
for filename in os.listdir(sourcedir):
if not filename.endswith(".mdp"):
continue
source = os.path.join(sourcedir, filename)
dest = os.path.join(destdir, filename)
# with open(xxx) as varname makes sure the file(s)
# will be closed whatever happens in the 'with' block
# NB text mode is the default, and so is read mode
with open(source) as fin, open(dest, "w") as fout:
# python files are iterable... avoids reading
# the whole file in memory at once
for line in fin:
# will only work for those exact strings,
# you may want to use regexps if number of
# whitespaces vary etc
line = line.replace("dt = 0.001", "dt = -0.001")
line = line.replace(
'integrator = md',
'integrator = md-vv'
)
fout.write(line)
Assuming you want to edit all files that are located in the mdp folder you could do something like this.
import os
dir = "/home/abc/xyz/mdp/"
for filename in os.listdir(dir):
with open(dir + filename, "r+") as file:
text = file.read()
text = text.replace("dt = 0.001", "dt = -0.001")
file.seek(0)
file.write(text)
file.truncate()
This will go through every file and change it using str.replace().
If there are other files in the mdp folder that you do not want to edit, you could use and if-statement to check for the correct file name. Add something like this to encase the with open statement.
if filename.startswith("md_")
Essentially the data is temperatures from 4 different states over the course of 12 months, so there is 48 files to be populated into my folder on my desktop directory. But I am not sure how to take the data being pulled from the web and then take the files being saved in my program to be sent to the directory of my desktop. That's what I am confused about, how to take the files being created on in my program and send them to a folder on my desktop.
I am copying the data from the web, cleaning it up, then saving it into a file, then taking that file and wanting to save it to a folder on my desktop.
Here is the code:
import urllib
def accessData(Id, Month):
url = "https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=" + str(Id) + "&year=2017&month=" + str(Month) + "&graphspan=month&format=1"
infile = urllib.urlopen(url)
readLineByLine = infile.readlines()
infile.close()
return readLineByLine
f = open('stations.csv', 'r')
for line in f.readlines():
vals = line.split(',')
for j in range(1,13): # accessing months here from 1 to 12, b/c 13 exclusive
data = accessData(line, j)
filename = "{}-0{}-2017.csv".format(vals[0], j)
print(str(filename))
row_count = len(data)
for i in range(2, row_count):
if(data[i] != '<br>\n' and data[i] != '\n'):
writeFile = open(filename, 'w')
writeFile.write(data[i])
openfile = open(Desktop, writeFile , 'r')
file.close()
Have you tried running the script from your desktop. It looks like you haven't specified a directory. So maybe running from your desktop should output your results to your current working directory.
Alternatively, you could try use the in-built os library.
import os
os.getcwd() # to get the current working directory
os.chdir(pathname) # change your working directory to the path specified.
This would change your working directory to the place you want to save your files.
Also, in regards to the last four lines of your code. file is not open, so you cannot close this. Also, I do not believe you need the openfile statement.
writeFile = open(filename, 'w')
writeFile.write(data[i])
openfile = open(Desktop, writeFile , 'r')
file.close()
Try this instead.
with open(filename, 'w') as writeFile:
for i in range(2, row_count):
if(data[i] != '<br>\n' and data[i] != '\n'):
writeFile.write(data[i])
Using this approach you shouldn't need to close the file. 'w' is to write as if a new file, change this to 'a' if you need to append to the file.
You just need to provide writeFile.write() with the path to your destination file, rather than just the filename (which will otherwise be saved into your current working directory.)
Try something like:
f = open('stations.csv', 'r')
target_dir = "/path/to/your/Desktop/folder/"
for line in f.readlines():
...
# We can open the file outside your inner "row" loop
# using the combination of the path to your Desktop
# and your filename
with open(target_dir+filename, 'w') as writeFile:
for i in range(2, row_count):
if(data[i] != '<br>\n' and data[i] != '\n'):
writeFile.write(data[i])
# The "writeFile" object will close automatically outside the
# "with ... " block
As others have mentioned, you could approach this two different ways:
1) Run the script directly from the directory to which you would like to save the files. Then you would just need to specify the full path to the .csv file you are reading.
2) You could provide the full path to where you would like to save the files when you write them, however this seems more intensive and unnecessary.
On another note, when opening files for the purpose of reading/writing them, use with to simply open the file for as long as you need it, then when you exit the with statement, the file will automatically be closed.
Here is an example of Option 1 with some clean-up:
import urllib
def accessData(Id, Month):
url = "https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=" + str(Id) + "&year=2017&month=" + str(Month) + "&graphspan=month&format=1"
infile = urllib.urlopen(url)
readLineByLine = infile.readlines()
infile.close()
return readLineByLine
with open('Path to File' + 'stations.csv', 'r') as f:
for line in f.readlines():
vals = line.split(',')
for j in range(1,13):
data = accessData(line, j)
filename = "{}-0{}-2017.csv".format(vals[0], j)
with open(filename, 'w') as myfile:
for i in range(2, len(data)):
if data[i]!='<br>\n' and data[i]!='\n':
myfile.write(data[i])
print(filename + ' - Completed')
I have a small script, and I am trying to parse data from text file to Excel file instead of going till the counter which is till 166 it stops at 134 and then doesn't do anything.
I have a file close operation also but it doesn't close the file and looks like the script continues to run.
Any thoughts? What am I doing wrong ?
path = ('C:\\Users\\40081\\PycharmProjects\\abcd')
#file_name = open('parsed_DUT1.txt', 'r')
file_name = 'parsed_DUT1.txt'
count=1
for line in file_name:
inputfile = open(file_name)
outputfile = open("parsed_DUT1" + '.xls', 'w+')
while count < 166:
for line in inputfile:
text = "TestNum_" + str(count*1)
if text in line :
#data = text[text.find(" ")+1:].split()[0]
outputfile.writelines(line)
count = count+1
inputfile.close()
outputfile.close()
w+
Opens a file for both writing and reading. Overwrites the existing
file if the file exists. If the file does not exist, creates a new
file for reading and writing.
You are opening the output file in w+ mode, that overwrites it everytime. Try with
outputfile = open("parsed_DUT1" + '.xls', 'a') # 'a' opens a file for appending.
I also suggest you to deal with files with with statement:
with open(file_name) as inputfile, open("parsed_DUT1" + '.xls', 'a') as outputfile:
# do stuff with input and output files
Output:
Sorry, this was being awfully awkward when I trying to paste my Python code into the code box on this forum post.
Code:
# update three quotes to a file
file_name = "my_quote.txt"
# create a file called my_quote.txt
new_file = open(file_name, 'w')
new_file.close()
def update_file(file_name, quote):
# First open the file
new_file = open(file_name, 'w')
new_file.write("This is an update\n")
new_file.write(quote)
new_file.write("\n\n")
# now close the file
new_file.close()
for index in range(3):
quote = input("Enter your favorite quote: ")
update_file(file_name, quote)
# Now print the contents to the screen
new_file = open(file_name, 'r')
print(new_file.read())
# And finally close the file
new_file.close(
You should be using append instead of write. When you use write, it creates a new file regardless of what was there before. Try new_file = open(file_name, 'a')
Why is it only writing last input to txt?
Everytime you do open(file_name, 'w') it clears the contents of the file and begins to write from the start of the file.
If you would like to append new content to that file do
open(file_name, 'a')
I guess you should use a instead of w to append to file:
new_file = open(file_name, 'a')
And read the docs before asking of course ;)
I'm trying to collect specific lines from multiple text (nmap) documents and then create a new file with them all in a table format. I haven't gotten to the table part yet because I can't get the append to work.
#imports
import os
#Change directories
os.chdir ("M:\\Daily Testing")
#User Input
print "What is the name of the system being scanned?"
sys = raw_input("> ")
#Subfolder selected
os.chdir (sys)
os.chdir ("RESULTS")
#variables
tag = ["/tcp", "/udp"]
fout = [sys + " Weekly Summary.csv"]
import glob
for filename in glob.glob("*.nmap"):
with open(filename, "rU") as f:
for line in f:
if not line.strip():
continue
for t in tag:
if t in line:
fout.write(line)
else:
continue
You have neglected to open a file to append to (fout is a list, not a file object, therefore it has no .write() method).
Change the line
fout = [sys + " Weekly Summary.csv"]
to
with open(sys+" Weekly Summary.csv", "w") as fout:
and indent the following lines accordingly.
So, something like this:
<snip>
import glob
with open(sys + " Weekly Summary.csv", "w") as fout:
for filename in glob.glob("*.nmap"):
with open(filename, "rU") as f:
for line in f:
if not line.strip():
continue
for t in tag:
if t in line:
fout.write(line)
else:
continue