I am just beginning with python with lpthw and had a specific question for closing a file.
I can open a file with:
input = open(from_file)
indata = input.read()
#Do something
indata.close()
However, if I try to simplify the code into a single line:
indata = open(from_file).read()
How do I close the file I opened, or is it already automatically closed?
Thanks in advance for the help!
You simply have to use more than one line; however, a more pythonic way to do it would be:
with open(path_to_file, 'r') as f:
contents = f.read()
Note that with what you are doing before, you could miss closing the file if an exception was thrown. The 'with' statement here will cause it be closed even if an exception is propagated out of the 'with' block.
Files are automatically closed when the relevant variable is no longer referenced. It is taken care of by Python garbage collection.
In this case, the call to open() creates a File object, of which the read() method is run. After the method is executed, no reference to it exists and it is closed (at least by the end of script execution).
Although this works, it is not good practice. It is always better to explicitly close a file, or (even better) to follow the with suggestion of the other answer.
Related
Will codes like this close the f.txt safely?
for line in open('f.txt', 'r'):
pass
It runs correctly, but I'm worrying that the opened file will not be closed safely.
Best practice is to use like below:
with open(filename,'r') as file_obj:
# Do stuff with file_obj here
This will make sure that your file gets closed once you come out of with block.
It is good practice to use the with keyword when dealing with file objects. The advantage is that the file is properly closed after its suite finishes, even if an exception is raised at some point.
with open(filename, 'r') as f:
read_data = f.read()
if you are not using with statement then you should call f.close().If you don’t explicitly close a file, Python’s garbage collector will eventually destroy the object and close the open file for you, but the file may stay open for a while
This simple code
# This code will BLANK the file 'myfile'!
with open('myfile', 'w') as file:
raise Exception()
rather than merely throwing an exception, deletes all data in "myfile", although no actual write operation is even attempted.
This is dangerous to say the least, and certainly not how other languages treat such situations.
How I can prevent this? Do I have to handle every possible exception in order to be certain that the target file will not be blanked by some unforeseen condition? Surely there must be a standard pattern to solve this problem. And, above all: What is happening here in the first place?
You are opening a file for writing. It is that simple action that blanks the file, regardless of what else you do with it. From the open() function documentation:
'w'
open for writing, truncating the file first
Emphasis mine. In essence, the file is empty because you didn't write anything to it, not because you opened it.
Postpone opening the file to a point where you actually have data to write if you don't want this to happen. Writing a list of strings to a file is not going to cause exceptions at the Python level.
Alternatively, write to a new file, and rename (move) it afterwards to replace the original. Renaming a file as left to the OS.
The statement open('myfile', 'w') will delete all the contents on execution i.e. truncate the file.
If you want to retain the lines you have to use open('myfile', 'a'). Here the a option is for append.
Opening a file for writing erases the contents. Best way to avoid lost of data, not only in case of exceptions, also computer shutdown, etc. is to create a new temporary file and rename the file to the original name, when everything is done.
yourfile = "myfile"
try:
with tempfile.NamedTemporaryFile(dir=os.path.dirname(yourfile) or '.', delete=False) as output:
do_something()
except Exception:
handle_exception()
else:
os.rename(output.name, yourfile)
with open(file, 'rb') as readerfile:
reader = csv.reader(readerfile)
In the above syntax, can I perform the first and second line together? It seems unnecessary to use 2 variables ('readerfile' and 'reader' above) if I only need to use the latter.
Is the former variable ('readerfile') ever used?
Can I use the same variable name for both is that bad form?
You can do:
reader = csv.reader(open(file, 'rb'))
but that would mean you are not closing your file explicitly.
with open(file, 'rb') as readerfile:
The first line opens the file and stores the file object in readerfile. The with statement ensures that the file is closed when you exit the block by any means, including exceptions.
reader = csv.reader(readerfile)
The second line creates a CSV reader object using the file object. It needs the file object (otherwise where would it read the data from?). Of course you could conceivably store it in the same variable
readerfile = csv.reader(readerfile)
if you wanted to (and don't plan on using the file object again), but this will likely lead to confusion for readers of your code.
Note that you haven't read anything yet! You still need to iterate over the reader object in order to get the data that you're interested in, and if you close the file before that happens then the reader object won't work. The file object is used behind the scenes by the reader object, even if you "hide" it by overwriting the readerfile variable.
Lastly, if you really want to do everything on one line, you could conceivably define a function that abstracts the with statement:
def with1(context, func):
with context as x:
return func(x)
Now you can write this as one line:
data = with1(open(file, 'rb'), lambda readerfile: list(csv.reader(readerfile)))
It's by no means clearer, however.
This is not recommended at all
Why is it important to use one line?
Most python programmers know well the benefits of using the with statement. Keep in mind that readers might be lazy (that is -read line by line-) on some cases. You want to be able to handle the file with the correct statement, ensuring the correct closing, even if errors arise.
Nevertheless, you can use a one liner for this, as stated in other answers:
reader = csv.reader(open(file, 'rb'))
So basically you want a one-liner?
reader = csv.reader(open(file, 'rb'))
As said before, the problem with that is with open() allows you to do the following steps in one time:
Open the file
Do what you want with the file (inside your open block)
Close the file (that is implicit and you don't have to specify it)
If you don't use with open but directly open, you file stays opened until the object is garbage collected, and that could lead to unpredicted behaviour in some cases.
Plus, your original code (two lines) is much more readable than a one-liner.
If you put them together, then the file won't be closed automatically -- but that often doesn't really matter, since it will be closed automatically when the script terminates.
It's not common to need to reference the raw file once acsv.readerinstance has been created from (except possibly to explicitly close it if you're not using awithstatement).
If you use the same variable name for both, it will probably work because thecsv.readerinstance will still hold a reference to the file object, so it won't be garbage collected until the program ends. It's not a commonly idiom, however.
Since csv files are often processed sequentially, the following can be a fairly concise way to do it since thecsv.readerinstance frequently doesn't really need to be given a variable name and it will close the file properly even if an exception occurs:
with open(file, 'rb') as readerfile:
for row in csv.reader(readerfile):
process the data...
Opening & closing file using file object:
fp=open("ram.txt","w")
fp.close()
If we want to Open & close file without using file object ,i.e;
open("ram.txt","w")
Do we need to write close("poem.txt") or writing close() is fine?
None of them are giving any error...
By only writing close() ,How it would understand to what file we are referencing?
For every object in memory, Python keeps a reference count. As long as there are no more references to an object around, it will be garbage collected.
The open() function returns a file object.
f = open("myfile.txt", "w")
And in the line above, you keep a reference to the object around in the variable f, and therefore the file object keeps existing. If you do
del f
Then the file object has no references anymore, and will be cleaned up. It'll be closed in the process, but that can take a little while which is why it's better to use the with construct.
However, if you just do:
open("myfile.txt")
Then the file object is created and immediately discarded again, because there are no references to it. It's gone, and closed. You can't close it anymore, because you can't say what exactly you want to close.
open("myfile.txt", "r").readlines()
To evaluate this whole expression, first open is called, which returns a file object, and then the method readlines is called on that. Then the result of that is returned. As there are now no references to the file object, it is immediately discarded again.
I would use with open(...), if I understand the question correctly.
This answer might help you What is the python keyword "with" used for?.
In answer to your actual question... a file object (what you get back when you call open) has the reference to the file in it. So when you do something like:
fp = open(myfile, 'w')
fp.write(...)
fp.close()
Everything in the above, including both write and close, know they reference myfile because that's the file that fp is associated with. I'm not sure what fp.close(myfile) actually does, but it certainly doesn't need the filename after it's open.
Better constructions like
with open(myfile,'w') as fp:
fp.write(...)
don't change this; in this case, fp is also a context manager, but still contains the pointer to myfile; there's no need to remind it.
When I do
open('output1.txt', 'w').write("Hello guys")
A file called output1.txt is immediatly created and contains the string "Hello guys".
But when I do
openvar = open('output2.txt', 'w')
openvar.write("Hello guys")
Then only the file output2.txt is created. The text "Hello guys" will only be seen on the output2.txt when I do openvar.close().
Why is this behaviour different only because of an extra variable assignment?
Python detects that the file object is not referenced anymore in your first case so the garbage collector will collect it and call its destructor which closes the file.
In the second case the file object still exists so it's not closed automatically.
You should always close your files when you area done. The with statement makes this pretty easy:
with open('output.txt', 'w') as f:
f.write('Hello')
As soon as the block is left, the file is closed again - even if the code inside the block raises an exception.
If you need to keep the file open for some reason (e.g. because you are going to write more data), you can .flush() it to force the system to empty the write buffer and actually writes it to the file.
In the first case garbage collector will close file for you. There are no references to that file. In the second case you have created a reference to the file. You have to manualy close it, or it will be closed by garbage collector when reference is destroyed.