So I'm having an issue with using the zipfile module in Python. Currently when I try to compress a KML file to create a new KMZ file I'm missing the last few lines. It doesn't seem to matter how long the KML is. I assume this is because zipfile isn't writing the last zipped block.
kmz = zipfile.ZipFile(kmzPath , 'w')
kmz.write(kmlPath, 'CORS.kml', zipfile.ZIP_DEFLATED)
And yes before you ask I have imported zlib to do the compression. I've tried to use zlib at the lower level too but have the same issue. I'm stuck.
Any ideas?
Make sure that you called
kmz.close()
after the .write(...) command, otherwise the full contents of the file won't be flushed to disk. To make sure this happens automatically, always use the with context manager, as the file will be closed when the loop is exited:
with zipfile.ZipFile(kmzPath, 'w') as kmz:
kmz.write(kmlPath, 'CORS.kml', zipfile.ZIP_DEFLATED)
This is just a guess, but according to the zipfile documentation:
You must call close() before exiting your program or essential records will not be written.
You don't indicate that you are in fact calling kmz.close() - could that be the problem?
Related
Here is my code of accessing&editing the file:
def edit_default_settings(self, setting_type, value):
with open("cam_settings.json", "r") as f:
cam_settings = json.load(f)
cam_settings[setting_type] = value
with open("cam_settings.json", 'w') as f:
json.dump(cam_settings, f, indent=4)
I use It in a program that runs for several hours in a day, and once in a ~week I'm noticing, that cam_settings.json file becoming empty (literally empty, the file explorer shows 0 bytes), but can't imagine how that is possible
Would be glad to hear some comments on what could go wrong
I can't see any issues with the code itself, but there can be an issue with the execution environment. Are you running the code in a multi-threaded environment or running multiple instances of the same program at once?
This situation can arise if this code is executed parallelly and multiple threads/processes try to access the file at the same time. Try logging each time the function was executed and if the function was executed successfully. Try exception handlers and error logging.
If this is a problem, using buffers or singleton pattern can solve the issue.
As #Chels said, the file is truncated when it's opened with 'w'. That doesn't explain why it stays that way; I can only imagine that happening if your code crashed. Maybe you need to check logs for code crashes (or change how your code is run so that crash reasons get logged, if they aren't).
But there's a way to make this process safer in case of crashes. Write to a separate file and then replace the old file with the new file, only after the new file is fully written. You can use os.replace() for this. You could do this simply with a differently-named file:
with open(".cam_settings.json.tmp", 'w') as f:
json.dump(cam_settings, f, indent=4)
os.replace(".cam_settings.json.tmp", "cam_settings.json")
Or you could use a temporary file from the tempfile module.
When openning a file with the "w" parameter, everytime you will write to it, the content of the file will be erased. (You will actually replace what's written already).
Not sure if this is what you are looking for, but could be one of the reasons why "cam_settings.json" becomes empty after the call of open("cam_settings.json", 'w')!
In such a case, to append some text, use the "a" parameter, as:
open("cam_settings.json", 'a')
I have the following problem:
I have a folder with files.
I want to write into those files their respective file path + filename (home/text.txt) .
How can I achieve this in python?
Thanks in advance for your time and help!
with open('FOLDER_NAME/FILE_NAME','w+') as f:
Assuming the python file is in the same path as the folder.
You can use:
..
to move back a directory.
file = open("path", "w+")
file.write("string you want to write in there")
file.close
With w+ it is for reading and writing to a file, existing data will be overwritten.
Of course, as Landon said, you can simply do this by using with, which will close the file for you after you are done writing to it:
with open("path") as file:
file.write("same string here")
This second snippet only takes up 2 lines, and it is the common way of opening a file.
However if you want append to instead of overwriting a file, use a+ this will open and allow you to read and append. Which means existing data will still be there, whatever you write will be added to the end. The file will also be created if it doesn’t exist.
Read more:
https://www.geeksforgeeks.org/reading-writing-text-files-python/
Correct way to write line to file?
I am writing a bit torrent client and allocate space on hard drive for each file I have to download.
File can be huge and I want to overwrite some piece of it without deleting all content.
I read some other answers, but they suggest creating a temporary space and then copying changes back. That would be too heavy process.
I am sure there must be some utility.
Since you are talking about a download file, you should consider the storage of downloaded data in a temporary file(chunks). Refer : streaming & text-file
If I read you question (without python), I would say you could use OS specific commands to overwrite parts of a file.
For example in Linux, use sed or awk.
Refer : Inserting-a-line-in-a-file-at-a-specific-location
I found a solution, using OS (Linux) specific system call:
fd = os.open(path, os.O_WRONLY)
os.lseek(fd, offset, os.SEEK_SET)
os.write(fd, piece)
os.close(fd)
This did overwrite file without nullifying it.
But when had:
with open(path, 'wb') as file:
file.seek(offset)
file.write(piece)
It was deleting all other content.
Program is not portable this way but I don't need it to be.
for some reason the readline() function in my following code seems to print nothing.
fileName = input()
fileName += ".txt"
fileA = open(fileName, 'a+')
print("Opened", fileA.name)
line = fileA.readline()
print(line)
fileA.close()
I'm using PyCharm, and I've been attempting to access 'file.txt' which is located inside my only PyCharm project folder. It contains the following:
Opened file!!
I have no idea what is wrong, and I can't find any relevant information for my problem whatsoever. Any help is appreciated.
Because you opened the file in a+ mode, the file pointer starts at the end of the file. After all, that is where you would normally append text.
If you want to read from the top, you need to place fileA.seek(0) just before you call readline:
fileA.seek(0)
line = fileA.readline()
Doing so sets the pointer to the top of the file.
Note: After reading the comments, it appears that you only need to do this if you are running a Windows machine. Those using a *nix system should not have this problem.
I hope I framed the question right. I am trying to force myself to be a better programmer. By better I mean efficient. I want to write a program to identify the files in a directory and read each file for further processing. After some shuffling I got to this:
for file in os.listdir(dir):
y=open(dir+'\\'+file,'r').readlines()
for line in y:
pass
y.close()
It should be no surprise that I get an AttributeError since y is a list. I didn't think about that when I wrote the snippet.
I am thinking about this and am afraid that I have five open files (there are five files in the directory specified by dir.
I can fix the code so it runs and I explicitly close the files after opening them. I am curious if I need to or if Python handles closing the file in the next iteration of the loop. If so then I only need to write:
for file in os.listdir(dir):
y=open(dir+'\\'+file,'r').readlines()
for line in y:
pass
I am guessing that it(python) does handle this effortlessly. The reason I think that this might be handled is that I have changed the object/thing that y is referencing. When I start the second iteration there are no more memory references to the file that was opened and read using the readlines method.
Python will close open files when they get garbage-collected, so generally you can forget about it -- particularly when reading.
That said, if you want to close explicitely, you could do this:
for file in os.listdir(dir):
f = open(dir+'\\'+file,'r')
y = f.readlines()
for line in y:
pass
f.close()
However, we can immediately improve this, because in python you can iterate over file-like objects directly:
for file in os.listdir(dir):
y = open(dir+'\\'+file,'r')
for line in y:
pass
y.close()
Finally, in recent python, there is the 'with' statement:
for file in os.listdir(dir):
with open(dir+'\\'+file,'r') as y:
for line in y:
pass
When the with block ends, python will close the file for you and clean it up.
(you also might want to look into os.path for more pythonic tools for manipulating file names and directories)
Don't worry about it. Python's garbage collector is good, and I've never had a problem with not closing file-pointers (for read operations at least)
If you did want to explicitly close the file, just store the open() in one variable, then call readlines() on that, for example..
f = open("thefile.txt")
all_lines = f.readlines()
f.close()
Or, you can use the with statement, which was added in Python 2.5 as a from __future__ import, and "properly" added in Python 2.6:
from __future__ import with_statement # for python 2.5, not required for >2.6
with open("thefile.txt") as f:
print f.readlines()
# or
the_file = open("thefile.txt")
with the_file as f:
print f.readlines()
The file will automatically be closed at the end of the block.
..but, there are other more important things to worry about in the snippets you posted, mostly stylistic things.
Firstly, try to avoid manually constructing paths using string-concatenation. The os.path module contains lots of methods to do this, in a more reliable, cross-platform manner.
import os
y = open(os.path.join(dir, file), 'r')
Also, you are using two variable names, dir and file - both of which are built-in functions. Pylint is a good tool to spot things like this, in this case it would give the warning:
[W0622] Redefining built-in 'file'