This question already has answers here:
Is there a need to close files that have no reference to them?
(6 answers)
Closed 4 years ago.
Usually it is required to call the .close() method on a file object or use the "with open" construct to make sure it gets closed.
However, I am wondering if writing to a file like this leaves the file closed:
open(path,'w').writelines(fileContents)
No, open(path,'w').writelines(fileContents) does not close the file. It's left in an open state, even though the file object is anonymous and it's not assigned to anything. CPython will clean up the handle if it goes out of scope.
Furthermore, I believe garbage collection would have to occur, which may or may not happen at the end of a block (it usually does, but there's no mandate or guarantee AFAIK). Other Python interpreters may or may not close the file when it goes out of scope, so don't rely on that mechanism for general Python code.
It's best to just use the with idiom and be done with it. There are very few reasons not to use with when dealing with file descriptors.
Related
This question already has answers here:
python: complexity of os.path.exists with a ext4 filesystem?
(2 answers)
Closed 4 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
I have a ton of folders nested inside one another.
What is time complexity of Python's os.path.exists() ?
Does it change if used with different OS ?
os.path.exists just performs a system call and returns True if the path points to an existing file or directory.
Python seems to performa an lstat() system call on the given path.
If the operation only consists in a lookup in a hash table, then the cost is O(1), but it may depend on the Operating System and in how is it implemented internally.
Given you are asking the OS if a single file exists, it does not need to do any algorithmic logic, or go down your path... I don't see how it could be anything else than O(1).
The standard answer to "how can I ensure a file is closed in Python" is to wrap the commands in a "with" statement, so that the destructor is called upon exiting the "with" block.
But what about a case where you can't do that because the file handle needs to remain open across a large swath of code? For example, you open the file handle in an object constructor, saving it to an object property, and then referring to the file handle in many object methods.
It would be possible to move the opening of the file handle to the methods themselves, but basically in that case I'd be opening/closing the file every time a method is called, which is far less efficient.
I have tried placing a "close" command in the object destructor (the "del" method), but this does not work.
A dirty but easy win is to keep record of file names when you open them, and make sure file.close(...) appears at the end. Keep a list or wrapping the open() function (suggest by this post) may do the job.
Post check what files are open in Python
suggests several solutions like wrapping the built-in file object, command line method and module psutil, etc, maybe some of them would fit in your situations.
If I do (in Python):
text = open("filename").read()
is the file automatically closed?
The garbage collector would be activated at some point, but you cannot be certain of when unless you force it.
The best way to ensure that the file is closed when you go out of scope just do this:
with open("filename") as f: text = f.read()
also one-liner but safer.
In CPython (the reference Python implementation) the file will be automatically closed. CPython destroys objects as soon as they have no references, which happens at the end of the statement at the very latest.
In other Python implementations this may not happen right away since they may rely on the memory management of an underlying virtual machine, or use some other memory management strategy entirely (see PyParallel for an interesting example).
Python, the language, does not specify any particular form of memory management, so you can't rely on the file being closed in the general case. Use the with statement to explicitly specify when it will be closed if you need to rely on it.
In practice, I often use this approach in short-lived scripts where it doesn't really matter when the file gets closed.
Since you have no reference on the open file handle,
CPython will close it automatically either during garbage collection or at program exit. The problem here is that you don't have any guarantees about when that will occur, which is why the with open(...) construct is preferred.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Is close() necessary when using iterator on a Python file object
for line in open("processes.txt").readlines():
doSomethingWith(line)
Take that code for example. There's nothing to call close() on. So does it close itself automatically?
Files will close when the corresponding object is deallocated. The sample you give depends on that; there is no reference to the object, so the object will be removed and the file will be closed.
Important to note is that there isn't a guarantee made as to when the object will be removed. With CPython, you have reference counting as the basis of memory management, so you would expect the file to close immediately. In, say, Jython, the garbage collector is not guaranteed to run at any particular time (or even at all), so you shouldn't count on the file being closed and should instead close the file manually or (better) use a with statement.
AFAIK they don't. In order to have autoclosing, you need to use a context manager, such as with
Although the object itself may be reclaimed by garbage collection and closed, there is no definite time to when garbage collection occurs.
with open("processes.txt") as openfile:
<do stuff>
Say you have:
def my_func():
fh = open(...)
try:
print fh.read()
finally:
fh.close()
My first question is: Is it worth having the try/finally (or with) statement? Isn't the file closed anyway when the function terminates (via garbage collection)?
I came across this after reading a recipe form Martelli's "python cookbook" where
all_the_text = open('thefile.txt').read()
comes with the comment: "When you do so, you no longer have a reference to the file object as soon as the reading operation finishes. In practice, Python notices the lack of a reference at once, and immediately closes the file."
My function example is almost the same. You do have a reference, it's just that the reference has a very short life.
My second question is: What does "immediately" in Martelli's statement mean? Even though you don't have a reference at all, doesn't the file closing happen at garbage collection time anyway?
It is good practice to close the file yourself. Using the with statement leads to clean code and it automatically closes the file (which is a Good Thing).
Even though Python is a high-level programming language, you still need to be in control of what you're doing. As a rule of thumb: if you open a file, it also needs to be closed. There's never a good reason to be sloppy in your code :-)
Regarding your second question: it won't run immediately, it'll run when the garbage collector decides it is time to run. When the file object is deallocated Python will close the file. Here are some articles on garbage collection in Python (also see the gc module), it's an interesting read.
It also shows that Python's garbage collection uses a threshold based on the number of allocated and deallocated objects before it decides to garbage collect. If your file is big then Python might hold the file open longer than necessary because the garbage collection code might not have run yet.