In python I'm seeing evidence that fp.readlines() is closing the file when I try to access the fp later in the program. Can you confirm this behavior, do I need to re-open the file again later if I also want to read from it again?
Is the file closed? is similar, but didn't answer all of my questions.
import sys
def lines(fp):
print str(len(fp.readlines()))
def main():
sent_file = open(sys.argv[1], "r")
lines(sent_file)
for line in sent_file:
print line
this returns:
20
Once you have read a file, the file pointer has been moved to the end and no more lines will be 'found' beyond that point.
Re-open the file or seek back to the start:
sent_file.seek(0)
Your file is not closed; a closed file raises an exception when you attempt to access it:
>>> fileobj = open('names.txt')
>>> fileobj.close()
>>> fileobj.read()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file
It doesn't close the file, but it does read the lines in it so they cannot be read again without reopening the file or setting the file pointer back to the beginning with fp.seek(0).
As evidence that it doesn't close the file, try changing the function to actually close the file:
def lines(fp):
print str(len(fp.readlines()))
fp.close()
You will get the error:
Traceback (most recent call last):
File "test5.py", line 16, in <module>
main()
File "test5.py", line 12, in main
for line in sent_file:
ValueError: I/O operation on closed file
It won't be closed, but the file will be at the end. If you want to read its contents a second time then consider using
f.seek(0)
You may want to use the with statement and context manager:
>>> with open('data.txt', 'w+') as my_file: # This will allways ensure
... my_file.write('TEST\n') # that the file is closed.
... my_file.seek(0)
... my_file.read()
...
'TEST'
If you use a normal call, remember to close it manually (in theory python closes file objects and garbage collect them as needed):
>>> my_file = open('data.txt', 'w+')
>>> my_file.write('TEST\n') # 'del my_file' should close it and garbage collect it
>>> my_file.seek(0)
>>> my_file.read()
'TEST'
>>> my_file.close() # Makes shure to flush buffers to disk
Related
I'm trying to load data from a pickled object into a list, but despite opening the file, I am receiving
Traceback (most recent call last):
File "/path/to/file.py", line 18, in <module>
data.append(pickle.load(file))
ValueError: peek of closed file
I assumed that I missed something in opening the file, but I looked and what I have seemed fine to me (this is my first foray into IO with pickle)
# load data to list
with open('tasks.txt', 'rb') as file:
data = []
while True:
try:
data.append(pickle.load(file))
except EOFError:
break
file.close()
Am I handling the opening wrong, or is it something else?
You closed the file after the first load; remove the file.close() entirely (the with statement already handles that), and it should work fine.
How do I read from a named pipe in Python 3.5.3?
The pipe's name and location is /tmp/shairport-sync-metadata and it can be viewed by anybody, but only modified by the shairport-sync user.
Other websites and questions say to use os.mkfifo("pipe-location") but I've been getting this error:
>>> import os
>>> os.mkfifo("/tmp/shairport-sync-metadata")
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
os.mkfifo("/tmp/shairport-sync-metadata")
FileExistsError: [Errno 17] File exists
Is there a way to get around this? Sorry for the nooby question.
os.mkfifo is used to create fifo. Use open to open/read fifo already exist:
with open('/tmp/shairport-sync-metadata') as f: # add `rb` for binary mode
# line-by-line read
for line in f:
print(line)
# f.read(1024) # to read 1024 characters
import unicodecsv
engagement_file=r'G:\college\udacity\intro to data analitics\datasets\daily_engagement.csv'
enrollment_file=r'G:\college\udacity\intro to data analitics\datasets\enrollments.csv'
project_submissions_file=r'G:\college\udacity\intro to data analitics\datasets\project_submissions.csv'
def csv_to_list(csv_file):
with open(csv_file,'rb') as f:
reader=unicodecsv.DictReader(f)
return list(reader)
daily_engagement=csv_to_list(engagement_file)
enrollment=csv_to_list(enrollment_file)
project_submissions=csv_to_list(project_submissions_file)
on executing this piece of code I get following errors
Traceback (most recent call last):
File "G:\college\udacity\intro to data analitics\data_analytis_csv_to_list.py", line 10, in <module>
daily_engagement=csv_to_list(engagement_file)
File "G:\college\udacity\intro to data analitics\data_analytis_csv_to_list.py", line 8, in csv_to_list
return list(reader)
File "C:\ProgramData\Anaconda2\lib\site-packages\unicodecsv\py2.py", line 217, in next
row = csv.DictReader.next(self)
File "C:\ProgramData\Anaconda2\lib\csv.py", line 108, in next
row = self.reader.next()
File "C:\ProgramData\Anaconda2\lib\site-packages\unicodecsv\py2.py", line 117, in next
row = self.reader.next()
ValueError: I/O operation on closed file
I dont know how to solve it ,I m new to python
thanks in advance
When using with open() as f: in python the file f is only open inside the with clause. That is the point of using it; it provides automatic file closing and cleaning in a easy and readable way.
If you want to work on the file either open it without the with clause (that is plain opening a file) or do the operations on that file inside the clause, calling it directly as f.
You need to move your return under your with statement. Once control flow has gone out of the with statement, Python automatically closes the file for you. That means any file I/O you have to do needs to be done under the context manager:
def csv_to_list(csv_file):
with open(csv_file,'rb') as f:
reader = unicodecsv.DictReader(f)
return list(reader) # return the file under the context manager
So, i've been writing this program that takes a HTMl file, replaces some text and puts the return back into a different file in a different directory.
This error happened.
Traceback (most recent call last):
File "/Users/Glenn/jack/HTML_Task/src/HTML Rewriter.py", line 19, in <module>
with open (os.path.join("/Users/Glenn/jack/HTML_Task/src", out_file)):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/posixpath.py", line 89, in join
genericpath._check_arg_types('join', a, *p)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/genericpath.py", line 143, in _check_arg_types
(funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'TextIOWrapper'
Below is my code. Has anyone got any solutions I could implement, or should I kill it with fire.
import re
import os
os.mkdir ("dest")
file = open("2016-06-06_UK_BackToSchool.html").read()
text_filtered = re.sub(r'http://', '/', file)
print (text_filtered)
with open ("2016-06-06_UK_BackToSchool.html", "wt") as out_file:
print ("testtesttest")
with open (os.path.join("/Users/Glenn/jack/HTML_Task/src", out_file)):
out_file.write(text_filtered)
os.rename("/Users/Glenn/jack/HTML_Task/src/2016-06-06_UK_BackToSchool.html", "/Users/Glenn/jack/HTML_Task/src/dest/2016-06-06_UK_BackToSchool.html")
with open (os.path.join("/Users/Glenn/jack/HTML_Task/src", out_file)):
Here out_file if TextIOWrapper, not string.
os.path.join takes string as arguments.
Do not use keywords name as variable. file is keyword.
Do not use space in between function call os.mkdir ("dest")
try to change this:
with open ("2016-06-06_UK_BackToSchool.html", "wt") as out_file
on this:
with open ("2016-06-06_UK_BackToSchool.html", "w") as out_file:
or this:
with open ("2016-06-06_UK_BackToSchool.html", "wb") as out_file:
I have problem with this code:
file = tempfile.TemporaryFile(mode='wrb')
file.write(base64.b64decode(data))
file.flush()
os.fsync(file)
# file.seek(0)
f = gzip.GzipFile(mode='rb', fileobj=file)
print f.read()
I dont know why it doesn't print out anything. If I uncomment file.seek then error occurs:
File "/usr/lib/python2.5/gzip.py", line 263, in _read
self._read_gzip_header()
File "/usr/lib/python2.5/gzip.py", line 162, in _read_gzip_header
magic = self.fileobj.read(2)
IOError: [Errno 9] Bad file descriptor
Just for information this version works fine:
x = open("test.gzip", 'wb')
x.write(base64.b64decode(data))
x.close()
f = gzip.GzipFile('test.gzip', 'rb')
print f.read()
EDIT: For wrb problem. It doesn't give me an error when initialize it. Python 2.5.2.
>>> t = tempfile.TemporaryFile(mode="wrb")
>>> t.write("test")
>>> t.seek(0)
>>> t.read()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 9] Bad file descriptor
'wrb' is not a valid mode.
This works fine:
import tempfile
import gzip
with tempfile.TemporaryFile(mode='w+b') as f:
f.write(data.decode('base64'))
f.flush()
f.seek(0)
gzf = gzip.GzipFile(mode='rb', fileobj=f)
print gzf.read()
Some tips:
You can't .seek(0) or .read() a gzip file in wrb mode or wb or w+b. GzipFile class __init__ set itself to READ or WRITE only by looking at the first character of wrb (set itself to WRITE for this case).
When doing f = gzip.GzipFile(mode='rb', fileobj=file) your real file is file not f, I understood that after reading GzipFile class definition.
A working example for me was:
from tempfile import NamedTemporaryFile
import gzip
with NamedTemporaryFile(mode='w+b', delete=True, suffix='.txt.gz', prefix='f') as t_file:
gzip_file = gzip.GzipFile(mode='wb', fileobj=t_file)
gzip_file.write('SOMETHING HERE')
gzip_file.close()
t_file.seek(0)
# Do something here with your t_file, maybe send it to an external storage or:
print t_file.read()
I hope this can be useful for someone out there, took a lot of my time to make it work.