I've been previously writing code for a quiz program with a text file that stores all of the participants' results. The code that converts the text file to a dictionary and the text file itself are shown below:
Code:
import collections
from collections import defaultdict
scores_guessed = collections.defaultdict(lambda: collections.deque(maxlen=4))
with open('GuessScores.txt') as f:
for line in f:
name,val = line.split(":")
scores_guessed[name].appendleft(int(val))
for k in sorted(scores_guessed):
print("\n"+k," ".join(map(str,scores_guessed[k])))
writer = open('GuessScores.txt', 'wb')
for key, value in scores_guessed.items():
output = "%s:%s\n" % (key,value)
writer.write(output)
The text file appears like this:
Jack:10
Dave:20
Adam:30
Jack:40
Adam:50
Dave:60
Jack:70
Dave:80
Jack:90
Jack:100
Dave:110
Dave:120
Adam:130
Adam:140
Adam:150
Now, when I run the program code, the dictionary appears like this:
Adam 150 140 130 50
Dave 120 110 80 60
Jack 100 90 70 40
Now, this arranges the dictionary into order of highest scores, and the top 4 scores!
I want the python IDLE to overwrite the GuessScores.txt to this:
Adam:150
Adam:140
Adam:130
Adam:50
Dave:120
Dave:110
Dave:80
Dave:60
Jack:100
Jack:90
Jack:70
Jack:40
BUT when I run the code, this error appears:
Traceback (most recent call last):
File "/Users/Ahmad/Desktop/Test Files SO copy/readFile_prompt.py", line 16, in <module>
writer.write(output)
TypeError: 'str' does not support the buffer interface
The GuessScores.txt file is empty because it cannot write to the file, since there is the error above.
Why is this happening? And what is the fix? I have asked this previously but there were numerous issues. I am running Python 3.3.2 on a Mac 10.8 Mavericks iMac, if that makes any help.
Thanks,
Delbert.
The first issue is that you are trying to write text to a file that you opened in binary mode. In 3.x, this will no longer work. "text" vs. "binary" used to mean very little (only affecting line-ending translation, so no difference at all on some systems). Now it means like what it sounds like: a file opened in text mode is one whose contents are to be treated like text with some specific encoding, and a file opened in binary mode is one whose contents are to be treated as a sequence of bytes.
Thus, you need open('GuessScores.txt', 'w'), not open('GuessScores.txt', 'wb').
That said, you really should be using with blocks to manage the files, and you're going to have to write code that actually formats the dictionary content in the way you want. I assume you intend to output in sorted name order, and you need to iterate over each deque and write a line for each item. Something like:
with open('GuessScores.txt', 'w') as f:
for name, scores in sorted(scores_guessed.items()):
for score in scores:
f.write("{}:{}\n".format(name, score))
(Note also the new-style formatting.)
If necessary, you can explicitly specify the encoding of the file in the open call, with the encoding keyword parameter. (If you don't know what I mean by "encoding", you must learn. I'm serious. Drop everything and look it up.)
The writing problem has to do with the b in your open function. You've opened it in binary mode, so only bytes can be written. You can either remove the b or call bytes on output to give it the right type. You have a logic error anyway though. When I run it on Python 2.7, the output to GuessedScores.txt is this:
Dave:deque([120,110,80,60],maxlen=4)
Jack:deque([100, 90, 70, 40], maxlen=4)
Adam:deque([150, 140, 130, 50], maxlen=4)
So your values are the whole deques, not the individual scores. You'll have to format them, similar to how you did in your print statement.
Related
I am new to Python and am wondering how to address the following attribute error. I believe I need to define/declare the file variable? Thanks for any suggestions, here is my script:
AttributeError Traceback (most recent call last)
in
51
52 # Write methods to print to Financial_Analysis_Summary
---> 53 file.write("Financial Analysis")
54 file.write("\n")
55 file.write("----------------------------")
AttributeError: 'str' object has no attribute 'write'
From your code and error, I think You've defined the variable 'file' as a string. Aslo there is no attribute write() in the class str. Hence, the reason for this error. For more information, include the whole script i.e., mainly the use of variable 'file'. I think you can use print() to print the above mentioned details or create a new class with a method inside to print your desired things
It looks like you've somehow defined file as a string rather than a file. What you should to is define it thus:
summary_file=open("C:/someFolder/someOtherFolder/Financial_Analysis_Summary.txt",
mode='r+', encoding='utf8')
and then write to it.
The first argument to the open function is the file path. The mode is how you want to access the file: 'r' lets you read the file and nothing else (and throws a FileNotFoundError if the file doesn't yet exist; the others just create it), 'r+' lets you write to the file while leaving its preexisting text in place (although if you write to the middle of the file you'll still overwrite whatever was there), 'w' deletes what was in the file and lets you write to it, 'a' lets you write text only to the end of the file, 'w+' and 'a+' are the same as w and a except they let you read from the file; you can add b to the end of any of these to interact with the file in the form of bytes rather than strings. The encoding should only matter if you plan to use Unicode characters, in which case set it to the same encoding you'll use to view the file (usually 'utf8') to avoid garbling non-ASCII characters.
There is several errors "a bytes-like object is required, not 'str'. But none of them is related to mine. I open the file using open(filename, "w"), not "wb". The code is like 150 lines long. The beginning of the code is to assign the input of command line into parser args.
The args.result is an empty txt file which I want to write my result onto.
I open it using open.
I think the following code should be enough to illustrate my question. Before the line 65, the code is writing two functions to be used in calculation but I think it should be irrelevant to the bug.
In the code, I manually create the file 'save/results/result.txt' in the command terminal. Then I open the file in the line 132.
The remaining code is
A interesting bug happens that the line 158 runs OK. "begin training\n" can be written into file. Then for the line 165, during the first time of loop, it is OK and "aa\n" can be written into file. But during the second loop, the program end with an error "a bytes-like" object is required, not 'str'. The error message is as following.
Anyone could provide a help of that?
Quite thanks.
I've never had trouble with similar code, but if you want a quick fix, I'd bet making a string named f_text and adding to it within the for loop, then writing all of it to the file after the last iteration of the for loop would be a simple way to circumvent the problem.
IE:
f_text = ""
for epoch in range(your_range):
# Do calculations
print(the_stuff_you_wanted_to)
f_text += the_stuff_you_wanted_to + "\n"
f.write(f_text)
I'm mostly posting this to act as a quick fix though. I feel that there's probably a better solution and could help more if you show more of your code, like where you actually initialize f.
I'm using Python to recreate a program that have been written in Fortran 95, the program opens a binary file, containing only float numbers, and read a specific value, it works just fine in Fortran, when I execute the code, I get 284.69 for example.
Although, when I try to do the same in Python, reading the entire first line of the file, I get characters like these:
Y{�C�x�Cz~�C�x�C�j�C�r�C�v�Ch�Ck�CVx�C
Here is how I open the file and read the values:
f = open(args.model_files[0], "r").readlines()
print str(f[0])
I can't provide a file as example, because it is too big, but I affirm that there is only float numbers.
I would like to at least understand what type of characters I'm getting, or what I'm doing wrong when opening the file, any suggestion is welcome.
I'm trying to run the following code but for some reason I get the following error: "TypeError: limit must be an integer".
Reading csv data file
import sys
import csv
maxInt = sys.maxsize
decrement = True
while decrement:
decrement = False
try:
**csv.field_size_limit(maxInt)**
except OverflowError:
maxInt = int(maxInt/10)
decrement = True
with open("Data.csv", 'rb') as textfile:
text = csv.reader(textfile, delimiter=" ", quotechar='|')
for line in text:
print ' '.join(line)
The error occurs in the starred line. I have only added the extra bit above the csv read statement as the file was too large to read normally. Alternatively, I could change the file to a text file from csv but I'm not sure whether this will corrupt the data further I can't actually see any of the data as the file is >2GB and hence costly to open.
Any ideas? I'm fairly new to Python but I'd really like to learn a lot more.
I'm not sure whether this qualifies as an answer or not, but here are a few things:
First, the csv reader automatically buffers per line of the CSV, so the file size shouldn't matter too much, 2KB or 2GB, whatever.
What might matter is the number of columns or amount of data inside the fields themselves. If this CSV contains War and Peace in each column, then yeah, you're going to have an issue reading it.
Some ways to potentially debug are to run print sys.maxsize, and to just open up a python interpreter, import sys, csv and then run csv.field_size_limit(sys.maxsize). If you are getting some terribly small number or an exception, you may have a bad install of Python. Otherwise, try to take a simpler version of your file. Maybe the first line, or the first several lines and just 1 column. See if you can reproduce the smallest possible case and remove the variability of your system and the file size.
On Windows 7 64bit with Python 2.6, maxInt = sys.maxsize returns 9223372036854775807L which consequently results in a TypeError: limit must be an integer when calling csv.field_size_limit(maxInt). Interestingly, using maxInt = int(sys.maxsize) does not change this. A crude workaround is to simlpy use csv.field_size_limit(2147483647) which of course cause issues on other platforms. In my case this was adquat to identify the broken value in the CSV, fix the export options in the other application and remove the need for csv.field_size_limit().
-- originally posted by user roskakori on this related question
I'm parsing a 20Gb file and outputting lines that meet a certain condition to another file, however occasionally python will read in 2 lines at once and concatenate them.
inputFileHandle = open(inputFileName, 'r')
row = 0
for line in inputFileHandle:
row = row + 1
if line_meets_condition:
outputFileHandle.write(line)
else:
lstIgnoredRows.append(row)
I've checked the line endings in the source file and they check out as line feeds (ascii char 10). Pulling out the problem rows and parsing them in isolation works as expected. Am I hitting some python limitation here? The position in the file of the first anomaly is around the 4GB mark.
Quick google search for "python reading files larger than 4gb" yielded many many results. See here for such an example and another one which takes over from the first.
It's a bug in Python.
Now, the explanation of the bug; it's not easy to reproduce because it depends both on the internal FILE buffer size and the number of chars passed to fread().
In the Microsoft CRT source code, in open.c, there is a block starting with this encouraging comment "This is the hard part. We found a CR at end of buffer. We must peek ahead to see if next char is an LF."
Oddly, there is an almost exact copy of this function in Perl source code:
http://perl5.git.perl.org/perl.git/blob/4342f4d6df6a7dfa22a470aa21e54a5622c009f3:/win32/win32.c#l3668
The problem is in the call to SetFilePointer(), used to step back one position after the lookahead; it will fail because it is unable to return the current position in a 32bit DWORD. [The fix is easy; do you see it?]
At this point, the function thinks that the next read() will return the LF, but it won't because the file pointer was not moved back.
And the work-around:
But note that Python 3.x is not affected (raw files are always opened in binary mode and CRLF translation is done by Python); with 2.7, you may use io.open().
The 4GB mark is suspiciously near the maximum value that can be stored in a 32-bit register (2**32).
The code you've posted looks fine by itself, so I would suspect a bug in your Python build.
FWIW, the snippet would be a little cleaner if it used enumerate:
inputFileHandle = open(inputFileName, 'r')
for row, line in enumerate(inputFileHandle):
if line_meets_condition:
outputFileHandle.write(line)
else:
lstIgnoredRows.append(row)