Related
I'm trying to alter multiple text files using a python scrip but I'm getting, not all, but some empty files as output. How to solve this?
def tratador_arquivo(arquivo):
with open(arquivo, 'r+',encoding="utf8") as inputtext:
for ponto in inputtext:
saida="saida_"+arquivo
with open(saida, 'w') as saidatemp:
saidatemp.write(ponto.replace('. ','.\n'))
import os
pasta = os.listdir('/Users/gabri/Desktop/Textos Imóveis')
os.chdir('/Users/gabri/Desktop/Textos Imóveis')
for arquivo in pasta:
tratador_arquivo(arquivo)
Expanding abarnert's answer, you may get the whole code as follows:
def tratador_arquivo(arquivo):
saida = "saida_" + arquivo
with open(arquivo, 'r+',encoding="utf8") as inputtext, open(saida, 'w') as saidatemp:
for ponto in inputtext:
saidatemp.write(ponto.replace('. ', '.\n'))
Also mind the indentation error of the function content in the code you provided.
Look at your loop:
for ponto in inputtext:
saida="saida_"+arquivo
with open(saida, 'w') as saidatemp:
saidatemp.write(ponto.replace('. ','.\n'))
This re-opens the same file over and over, once for each line of the input file. Since you open it in w mode, that truncates the file, erasing whatever you'd previously written there, and replaces it with just the (transformed) newest line. So, at the end of the loop, your output file only has the (transformed) last line of the input file.
This is probably always wrong (although it's hard to say that for sure when I don't know what exactly you're trying to do). But in cases where the input file ends with a blank line, it will be especially obvious that it's wrong, because the only thing in the output file will be that blank line.
What you probably want to do is this:
saida="saida_"+arquivo
with open(saida, 'w') as saidatemp:
for ponto in inputtext:
saidatemp.write(ponto.replace('. ','.\n'))
In other words, just open the file once, and keep writing new lines to it.
I am trying to loop through a text file and apply some logic but I am not able to loop through the text file. So currently I have a text file that is structured like this:
--- section1 ---
"a","b","c"
"d","e","f"
--- section2 ---
"1","2","3"
"4","5","6"
--- section3 ---
"12","12","12"
"11","11","11"
I am trying to filter out the first line which contains '---' and convert the lines below into json until the next '---' line appear in the text document.
However I got this error " fields1 = next(file).split(',') StopIteration
with open(fileName,'r') as file:
for line in file:
if line.startswith('-') and 'section1' in line:
while '---' not in next(file):
fields1 = next(file).split(',')
for x in range(0,len(fields1)):
testarr.append({
config.get('test','test'): fields1[x]
})
with open(test_dir,'w') as test_file:
json.dump(testarr, test_file)
Any idea why my code is not working or how i can solve the error ?
The cause of your error is that you are misusing the file object genrator by calling next on it twice as often as you think. Each call to next gets a line and returns it. Therefore, while '---' not in next(file): fields1 = next(file).split(',') gets a line, checks it for ---, then gets another line and tries to parse it. This means that you are able to skip a line containing the --- by having it come up in the second next. In that case you will get to the end of the file before you find the line you are looking for. StopIteration is how iterators normally indicate that their input has been exhausted.
There are a couple of other issues you may want to address in your code:
Using next on a generator like a file when you are already inside a for loop may cause undefined behavior. You may be getting away with it this time, but it is not good practice in general. The main reason you are getting away with it, by the way, is possibly that you never actually return control to the for loop once the while is triggered, and not that files are particularly permissive in this regard.
The inner with that dumps your data to a file is inside your while loop. That means that the file you open with 'w' permissions will get truncated for every iteration of the while (i.e., each line in the file). As the array grows, the output will actually appear fine, but you probably want to move that out of the inner loop.
The simplest solution would be to rewrite the code in two loops: one to find the start of the part you care about, and the other to process it until the end is found.
Something like this:
test_arr = []
with open(fileName, 'r') as file:
for line in file:
if line.startswith('---') and 'section1' in line:
break
for line in file:
if '---' in line:
break
fields1 = line.split(',')
for item in fields1:
testarr.append({config.get('test','test'): item})
with open(test_dir,'w') as test_file:
json.dump(testarr, test_file)
EDIT:
Taking #tripleee's advice, I have removed the regex check for the start line. While regex gives great precision and flexibility for finding a specific pattern, it is really overkill for this example. I would like to point out that if you are looking for a section other than section1, or if section1 appears after some other lines with dashes, you will absolutely need this two-loop approach. The one-loop solutions in the other answers will not work in a non-trivial case.
Looks like you are overcomplicating matters massively. The next inside the inner while loop I imagine is tripping up the outer for loop, but that's just unnecessary anyway. You are already looping over lines; pick the ones you want, then quit when you're done.
with open(fileName,'r') as inputfile:
for line in inputfile:
if line.startswith('-') and 'section1' in line:
continue
elif line.startswith('-'):
break
else:
testarr.append({config.get('test', 'test'): x
for x in line.split(',')})
with open(test_dir,'w') as test_file:
json.dump(testarr, test_file)
I hope I got the append right, as I wanted to also show you how to map the split fields more elegantly, but I'm not sure I completely understand what your original code did. (I'm guessing you'll want to trim the \n off the end of the line before splitting it, actually. Also, I imagine you want to trim the quotes from around each value. x.strip('"') for x in line.rstrip('\n').split(','))
I also renamed file to inputfile to avoid clashing with the reserved keyword file.
If you want to write more files, basically, add more states in the loop and move the write snippet back inside the loop. I don't particularly want to explain how this is equivalent to a state machine but it should not be hard to understand: with two states, you are skipping or collecting; to extend this, add one more state for the boundary when flipping back, where you write out the collected data and reinitialize the collected lines to none.
next() raises a StopIteration exception when the iterator is exhausted. In other words, your code gets to the end of the file, and you call next() again, and there's nothing more for it to return, so it raises that exception.
As for how to solve your problem, I think this might be what you want:
with open(fileName, 'r') as file:
for line in file:
if line.startswith('---'):
if 'section1' in line:
continue
else:
break
fields1 = line.split(',')
for x in range(len(fields1)):
testarr.append({
config.get('test', 'test'): fields1[x]
})
with open(test_dir, 'w') as test_file:
json.dump(testarr, test_file)
The exact question to this problem is:
*Create a file with a 20 lines of text and name it “lines.txt”. Write a program to read this a file “lines.txt” and write the text to a new file, “numbered_lines.txt”, that will also have line numbers at the beginning of each line.
Example:
Input file: “lines.txt”
Line one
Line two
Expected output file:
1 Line one
2 Line two
I am stuck, and this is what I have so far. I am a true beginner to Python and my instructor does not make things very clear. Critique and help much appreciated.
file_object=open("lines.txt",'r')
for ln in file_object:
print(ln)
count=1
file_input=open("numbered_lines.txt",'w')
for Line in file_object:
print(count,' Line',(str))
count=+1
file_object.close
file_input.close
All I get for output is the .txt file I created stating lines 1-20. I am very stuck and honestly have very little idea about what I am doing. Thank you
You have all the right parts, and you're almost there:
When you do
for ln in file_object:
print(ln)
you've exhausted the contents of that file, and you won't be able to read them again, like you try to do later on.
Also, print does not write to a file, you want file_input.write(...)
This should fix all of that:
infile = open("lines.txt", 'r')
outfile = open("numbered_lines.txt", 'w')
line_number = 1
for line in infile:
outfile.write(str(line_number) + " " + line)
infile.close()
outfile.close()
However, here is a more pythonic way to do it:
with open("lines.txt") as infile, open("numbered_lines.txt", 'w') as outfile:
for i, line in enumerate(infile, 1):
outfile.write("{} {}".format(i, line))
Good first try, and with that, I can go through your code and explain what you did right (or wrong)
file_object=open("lines.txt",'r')
for ln in file_object:
print(ln)
This is fine, though generally you want to put a space before and after assignments (you are assigning the results of open to file_object) and add a space after a,` when separating arguments, so you might want to write that like so:
file_object = open("lines.txt", 'r')
for ln in file_object:
print(ln)
However, at this point the internal reference in the file_object have reached the end of the file, so if you wish to reuse the same object, you need to seek back to the beginning position, which is 0. As your assignment only states write to the file (and not on the screen), the above loop should be omitted from the file (but I get what you want to do, you want to see the contents of the file immediately though sometimes instructors are pretty strict on what they accept). Moving on:
count=1
file_input=open("numbered_lines.txt",'w')
for Line in file_object:
Looks pretty normal so far, again, minor formatting issues. In Python, typically we name all variables lower-case, as names with Capitalization are generally reserved for class names (if you wish to, you may read about them). Now we enter into the loop you got
print(count,' Line',(str))
This prints not quite what you want. as ' Line' is enclosed inside a quote, it is treated as a string literal - so it's treated literally as text and not code. Given that you had assigned Line, you want to take out the quotes. The (str) at the end simply just print out the string object and it definitely is not what you want. Also, you forgot to specify the file you want to print to. By default it will print to the screen, but you want to print it to the the numbered_lines.txt file which you had opened and assigned to file_input. We will correct this later.
count=+1
If you format this differently, you are assigning +1 to count. I am guessing you wanted to use the += operator to increment it. Remember this on your quiz/tests.
Finally:
file_object.close
file_input.close
They are meant to be called as functions, you need to invoke them by adding parentheses at the end with arguments, but as close takes no arguments, there will be nothing inside the parentheses. Putting everything together, the complete corrected code for your program should look like this
file_object = open("lines.txt", 'r')
count = 1
file_input = open("numbered_lines.txt", 'w')
for line in file_object:
print(count, line, file=file_input)
count += 1
file_object.close()
file_input.close()
Run the program. You will notice that there is an extra empty line between every line of text. This is because by default the print function adds a new line end character; the line you got from the file included a new-line character at the end (that's what make them lines, right?) so we don't have to add our own here. You can of course change it to an empty string. That line will look like this.
print(count, line, file=file_input, end='')
Naturally, other Python programmers will tell you that there are Pythonic ways, but you are just starting out, don't worry too much about them (although you can definitely pick up on this later and I highly encourage you to!)
The right way to open a file is using a with statement:
with open("lines.txt",'r') as file_object:
... # do something
That way, the context manager introduced by with will close your file at the end of "something " or in case of exception.
Of course, you can close the file yourself if you are not familiar with that. Not that close is a method: to call it you need parenthesis:
file_object.close()
See the chapter 7.2. Reading and Writing Files, in the official documentation.
In the first loop you're printing the contents of the input file. This means that the file contents have already been consumed when you get to the second loop. (Plus the assignment didn't ask you to print the file contents.)
In the second loop you're using print() instead of writing to a file. Try file_input.write(str(count) + " " + Line) (And file_input seems like a bad name for a file that you will be writing to.)
count=+1 sets count to +1, i.e. positive one. I think you meant count += 1 instead.
At the end of the program you're calling .close instead of .close(). The parentheses are important!
There is one answer to this question:
Getting "newline inside string" while reading the csv file in Python?
But this didn't work when I used the accepted answer.
If the answer in the above link doesn't work and you have opened multiple files during the execution of your code, go back and make sure you have closed all your previous files when you were done with them.
I had a script that opened and processed multiple files. Then at the very end, it kept throwing a _csv.Error in the same manner that Amit Pal saw.
My code runs about 500 lines and has three stages where it processes multiple files in succession. Here's the section of code that gave the error. As you can see, the code is plain vanilla:
f = open('file.csv')
fread = csv.reader(f)
for row in fread:
do something
And the error was:
for row in fread:
_csv.Error: newline inside string
So I told the script to print what the row....OK, that's not clear, here's what I did:
print row
f = open('file.csv')
fread = csv.reader(f)
for row in fread:
do something
Interestingly, what printed was the LAST LINE from one of the previous files I had opened and processed.
What made this really weird was that I used different variable names, but apparently the data was stuck in a buffer or memory somewhere.
So I went back and made sure I closed all previously opened files and that solved my problem.
Hope this helps someone.
I'm learning PyGTK and I'm making a Text Editor (That seems to be the hello world of pygtk :])
Anyways, I have a "Save" function that writes the TextBuffer to a file. Looks something like
try:
f = open(self.working_file_path, "rw+")
buff = self._get_buffer()
f.write(self._get_text())
#update modified flag
buff.set_modified(False)
f.close()
except IOError as e:
print "File Doesnt Exist so bring up Save As..."
......
Basically, if the file exist, write the buffer to it, if not bring up the Save As Dialog.
My question is: What is the best way to "update" a file. I seem to only be able to append to the end of a file. I've tried various file modes, but I'm sure I'm missing something.
Thanks in advance!
You can open a file in "r+" mode, which allows you to both read and write to the file, and to seek to particular positions and write there. This probably doesn't help you do what I think you want though; it sounds like you're wanting to only write out the changed data?
Remember that on the disk the file isn't stored as a series of extensible lines, it's just a sequence of bytes; some of those bytes indicate line-endings, but the next line follows on immediately. So if you edit the first line in the file and you write the new first line out, unless the new one happens to be exactly the same length as the old one the second line now won't be in the right place, so you'll need to move it (and have taken a copy of it first if the new line you wrote out was longer than the original). And this now means that the next line isn't in the right position either... and so on until you've had to read in and write out the entire rest of the file.
In practice you almost never write only part of an existing file unless you can simply append more data; if you need to "alter" a file you read it in, alter it in memory, and write it back out or you read in the file in pieces (often line by line) and then write out to a new file as you go (and then possibly move the new file over the top of the original). The first approach is easiest, the second is better for not having to hold the whole thing in memory at once.
At the point where you write to the file, your location is at the end of the file, so you need to seek back to the beginning. Then, you will overwrite the file, but this may leave old content at the end, so you also need to truncate the file.
Additionally, the mode you're specifying ('rw+') is invalid, and I get IOErrors when I try to do some operations on files opened with it. I believe that you want mode 'r+' ("Open for reading and writing. The stream is positioned at the beginning of the file."). 'w+' is similar, but would create the file if it didn't exist.
So, what you're looking for might be code like this:
try:
f = open(self.working_file_path, "r+")
buff = self._get_buffer()
f.seek(0)
f.truncate()
f.write(self._get_text())
#update modified flag
buff.set_modified(False)
f.close()
except IOError as e:
print "File Doesnt Exist so bring up Save As..."
......
However, you may want to modify this code to correctly catch and handle errors while truncating and writing the file, rather than assuming that all IOErrors in this section are non-existant-file errors from the call to open.
Read the file in as a list, add an element to the start of it, write it all out. Something like this.
f = open(self.working_file_path, "r+")
flist = f.readlines()
flist.insert(0, self._get_text())
f.seek(0)
f.writelines(flist)