I need to create a temp file with some contents for test:
def test_something():
tmp_path = make_temp_file('hello\nworld')
do_something(tmp_path)
but the following code I used to make the file automatically deletes the file on close:
def make_temp_file(text):
with tempfile.NamedTemporaryFile() as tmp_file:
tmp_file.write(text.encode('utf-8'))
tmp_file.flush()
return tmp_file.name
so, in do_something(tmp_path) the file was not found. How can I prevent deleting the tmp file on close? I'll manually delete it after the test.
There is a delete parameter which defaults to true. So the code should be tempfile.NamedTemporaryFile(delete=False) in the example.
Thanks to #juanpa.arrivillaga.
Related
I am attempting to create and write to a temporary file on Windows OS using Python. I have used the Python module tempfile to create a temporary file.
But when I go to write that temporary file I get an error Permission Denied. Am I not allowed to write to temporary files?! Am I doing something wrong? If I want to create and write to a temporary file how should should I do it in Python? I want to create a temporary file in the temp directory for security purposes and not locally (in the dir the .exe is executing).
IOError: [Errno 13] Permission denied: 'c:\\users\\blah~1\\appdata\\local\\temp\\tmpiwz8qw'
temp = tempfile.NamedTemporaryFile().name
f = open(temp, 'w') # error occurs on this line
NamedTemporaryFile actually creates and opens the file for you, there's no need for you to open it again for writing.
In fact, the Python docs state:
Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).
That's why you're getting your permission error. What you're probably after is something like:
f = tempfile.NamedTemporaryFile(mode='w') # open file
temp = f.name # get name (if needed)
Use the delete parameter as below:
tmpf = NamedTemporaryFile(delete=False)
But then you need to manually delete the temporary file once you are done with it.
tmpf.close()
os.unlink(tmpf.name)
Reference for bug: https://github.com/bravoserver/bravo/issues/111
regards,
Vidyesh
Consider using os.path.join(tempfile.gettempdir(), os.urandom(24).hex()) instead. It's reliable, cross-platform, and the only caveat is that it doesn't work on FAT partitions.
NamedTemporaryFile has a number of issues, not the least of which is that it can fail to create files because of a permission error, fail to detect the permission error, and then loop millions of times, hanging your program and your filesystem.
The following custom implementation of named temporary file is expanded on the original answer by Erik Aronesty:
import os
import tempfile
class CustomNamedTemporaryFile:
"""
This custom implementation is needed because of the following limitation of tempfile.NamedTemporaryFile:
> Whether the name can be used to open the file a second time, while the named temporary file is still open,
> varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).
"""
def __init__(self, mode='wb', delete=True):
self._mode = mode
self._delete = delete
def __enter__(self):
# Generate a random temporary file name
file_name = os.path.join(tempfile.gettempdir(), os.urandom(24).hex())
# Ensure the file is created
open(file_name, "x").close()
# Open the file in the given mode
self._tempFile = open(file_name, self._mode)
return self._tempFile
def __exit__(self, exc_type, exc_val, exc_tb):
self._tempFile.close()
if self._delete:
os.remove(self._tempFile.name)
This issue might be more complex than many of you think. Anyway this was my solution:
Make use of atexit module
def delete_files(files):
for file in files:
file.close()
os.unlink(file.name)
Make NamedTemporaryFile delete=False
temp_files = []
result_file = NamedTemporaryFile(dir=tmp_path(), suffix=".xlsx", delete=False)
self.temp_files.append(result_file)
Register delete_files as a clean up function
atexit.register(delete_files, temp_files)
tempfile.NamedTemporaryFile() :
It creates and opens a temporary file for you.
f = open(temp, 'w') :
You are again going to open the file which is already open and that's why you are getting Permission Denied error.
If you really wants to open the file again then you first need to close it which will look something like this-
temp= tempfile.NamedTemporaryFile()
temp.close()
f = open(temp.name, 'w')
Permission was denied because the file is Open during line 2 of your code.
close it with f.close() first then you can start writing on your tempfile
Is there a way for Python to close that the file is already open file.
Or at the very least display a popup that file is open or a custom written error message popup for permission error.
As to avoid:
PermissionError: [Errno 13] Permission denied: 'C:\\zf.csv'
I've seen a lot of solutions that open a file then close it through python. But in my case. Lets say I left my csv open and then tried to run the job.
How can I make it so it closes the currently opened csv?
I've tried the below variations but none seem to work as they expect that I have already opened the csv at an earlier point through python. I suspect I'm over complicating this.
f = 'C:\\zf.csv'
file.close()
AttributeError: 'str' object has no attribute 'close'
This gives an error as there is no reference to opening of file but simply strings.
Or even..
theFile = open(f)
file_content = theFile.read()
# do whatever you need to do
theFile.close()
As well as:
fileobj=open('C:\\zf.csv',"wb+")
if not fileobj.closed:
print("file is already opened")
How do I close an already open csv?
The only workaround I can think of would be to add a messagebox, though I can't seem to get it to detect the file.
filename = "C:\\zf.csv"
if not os.access(filename, os.W_OK):
print("Write access not permitted on %s" % filename)
messagebox.showinfo("Title", "Close your CSV")
Try using a with context, which will manage the close (__exit__) operation smoothly at the end of the context:
with open(...) as theFile:
file_content = theFile.read()
You can also try to copy the file to a temporary file, and open/close/remove it at will. It requires that you have read access to the original, though.
In this example I have a file "test.txt" that is write-only (chmod 444) and it throws a "Permission denied" error if I try writing to it directly. I copy it to a temporary file that has "777" rights so that I can do what I want with it:
import tempfile, shutil, os
def create_temporary_copy(path):
temp_dir = tempfile.gettempdir()
temp_path = os.path.join(temp_dir, 'temp_file_name')
os.chmod(temp_path, 0o777); # give full access to the tempfile so we can copy
shutil.copy2(path, temp_path) # copy the original into the temp one
os.chmod(temp_path, 0o777); # replace permissions from the original file
return temp_path
path = "./test.txt" # original file
copy_path = create_temporary_copy(path) # temp copy
with open(copy_path, "w") as g: # can do what I want with it
g.write("TEST\n")
f = open("C:/Users/amol/Downloads/result.csv", "r")
print(f.readlines()) #just to check file is open
f.close()
# here you can add above print statement to check if file is closed or not. I am using python 3.5
I know that it is possible to create a temporary file, and write the data of the file I wish to copy to it. I was just wondering if there was a function like:
create_temporary_copy(file_path)
There isn't one directly, but you can use a combination of tempfile and shutil.copy2 to achieve the same result:
import tempfile, shutil, os
def create_temporary_copy(path):
temp_dir = tempfile.gettempdir()
temp_path = os.path.join(temp_dir, 'temp_file_name')
shutil.copy2(path, temp_path)
return temp_path
You'll need to deal with removing the temporary file in the caller, though.
This isn't quite as concise, and I imagine there may be issues with exception safety, (e.g. what happens if 'original_path' doesn't exist, or the temporary_copy object goes out of scope while you have the file open) but this code adds a little RAII to the clean up. The difference here to using NamedTemporaryFile directly is that rather than ending up with a file object, you end up with a file, which is occasionally desirable (e.g. if you plan to call out to other code to read it, or some such.)
import os,shutil,tempfile
class temporary_copy(object):
def __init__(self,original_path):
self.original_path = original_path
def __enter__(self):
temp_dir = tempfile.gettempdir()
base_path = os.path.basename(self.original_path)
self.path = os.path.join(temp_dir,base_path)
shutil.copy2(self.original_path, self.path)
return self.path
def __exit__(self,exc_type, exc_val, exc_tb):
os.remove(self.path)
in your code you'd write:
with temporary_copy(path) as temporary_path_to_copy:
... do stuff with temporary_path_to_copy ...
# Here in the code, the copy should now have been deleted.
The following is more concise (OP's ask) than the selected answer. Enjoy!
import tempfile, shutil, os
def create_temporary_copy(path):
tmp = tempfile.NamedTemporaryFile(delete=True)
shutil.copy2(path, tmp.name)
return tmp.name
A variation on #tramdas's answer, accounting for the fact that the file cannot be opened twice on windows. This version ignores the preservation of the file extension.
import os, shutil, tempfile
def create_temporary_copy(src):
# create the temporary file in read/write mode (r+)
tf = tempfile.TemporaryFile(mode='r+b', prefix='__', suffix='.tmp')
# on windows, we can't open the the file again, either manually
# or indirectly via shutil.copy2, but we *can* copy
# the file directly using file-like objects, which is what
# TemporaryFile returns to us.
# Use `with open` here to automatically close the source file
with open(src,'r+b') as f:
shutil.copyfileobj(f,tf)
# display the name of the temporary file for diagnostic purposes
print 'temp file:',tf.name
# rewind the temporary file, otherwise things will go
# tragically wrong on Windows
tf.seek(0)
return tf
# make a temporary copy of the file 'foo.txt'
name = None
with create_temporary_copy('foo.txt') as temp:
name = temp.name
# prove that it exists
print 'exists', os.path.isfile(name) # prints True
# read all lines from the file
i = 0
for line in temp:
print i,line.strip()
i += 1
# temp.close() is implicit using `with`
# prove that it has been deleted
print 'exists', os.path.isfile(name) # prints False
A slight variation (in particular I needed the preserve_extension feature for my use case, and I like the "self-cleanup" feature):
import os, shutil, tempfile
def create_temporary_copy(src_file_name, preserve_extension=False):
'''
Copies the source file into a temporary file.
Returns a _TemporaryFileWrapper, whose destructor deletes the temp file
(i.e. the temp file is deleted when the object goes out of scope).
'''
tf_suffix=''
if preserve_extension:
_, tf_suffix = os.path.splitext(src_file_name)
tf = tempfile.NamedTemporaryFile(suffix=tf_suffix)
shutil.copy2(src_file_name, tf.name)
return tf
Hello
My error is produced in generating a zip file. Can you inform what I should do?
main.py", line 2289, in get
buf=zipf.read(2048)
NameError: global name 'zipf' is not defined
The complete code is as follows:
def addFile(self,zipstream,url,fname):
# get the contents
result = urlfetch.fetch(url)
# store the contents in a stream
f=StringIO.StringIO(result.content)
length = result.headers['Content-Length']
f.seek(0)
# write the contents to the zip file
while True:
buff = f.read(int(length))
if buff=="":break
zipstream.writestr(fname,buff)
return zipstream
def get(self):
self.response.headers["Cache-Control"] = "public,max-age=%s" % 86400
start=datetime.datetime.now()-timedelta(days=20)
count = int(self.request.get('count')) if not self.request.get('count')=='' else 1000
from google.appengine.api import memcache
memcache_key = "ads"
data = memcache.get(memcache_key)
if data is None:
a= Ad.all().filter("modified >", start).filter("url IN", ['www.koolbusiness.com']).filter("published =", True).order("-modified").fetch(count)
memcache.set("ads", a)
else:
a = data
dispatch='templates/kml.html'
template_values = {'a': a , 'request':self.request,}
path = os.path.join(os.path.dirname(__file__), dispatch)
output = template.render(path, template_values)
self.response.headers['Content-Length'] = len(output)
zipstream=StringIO.StringIO()
file = zipfile.ZipFile(zipstream,"w")
url = 'http://www.koolbusiness.com/list.kml'
# repeat this for every URL that should be added to the zipfile
file =self.addFile(file,url,"list.kml")
# we have finished with the zip so package it up and write the directory
file.close()
zipstream.seek(0)
# create and return the output stream
self.response.headers['Content-Type'] ='application/zip'
self.response.headers['Content-Disposition'] = 'attachment; filename="list.kmz"'
while True:
buf=zipf.read(2048)
if buf=="": break
self.response.out.write(buf)
That is probably zipstream and not zipf. So replace that with zipstream and it might work.
i don't see where you declare zipf?
zipfile? Senthil Kumaran is probably right with zipstream since you seek(0) on zipstream before the while loop to read chunks of the mystery variable.
edit:
Almost certainly the variable is zipstream.
zipfile docs:
class zipfile.ZipFile(file[, mode[, compression[, allowZip64]]])
Open a ZIP file, where file can be either a path to a file (a string) or
a file-like object. The mode parameter
should be 'r' to read an existing
file, 'w' to truncate and write a new
file, or 'a' to append to an existing
file. If mode is 'a' and file refers
to an existing ZIP file, then
additional files are added to it. If
file does not refer to a ZIP file,
then a new ZIP archive is appended to
the file. This is meant for adding a
ZIP archive to another file (such as
python.exe).
your code:
zipsteam=StringIO.StringIO()
create a file-like object using StringIO which is essentially a "memory file" read more in docs
file = zipfile.ZipFile(zipstream,w)
opens the zipfile with the zipstream file-like object in 'w' mode
url = 'http://www.koolbusiness.com/list.kml'
# repeat this for every URL that should be added to the zipfile
file =self.addFile(file,url,"list.kml")
# we have finished with the zip so package it up and write the directory
file.close()
uses the addFile method to retrieve and write the retrieved data to the file-like object and returns it. The variables are slightly confusing because you pass a zipfile to the addFile method which aliases as zipstream (confusing because we are using zipstream as a StringIO file-like object). Anyways, the zipfile is returned, and closed to make sure everything is "written".
It was written to our "memory file", which we now seek to index 0
zipstream.seek(0)
and after doing some header stuff, we finally reach the while loop that will read our "memory-file" in chunks
while True:
buf=zipstream.read(2048)
if buf=="": break
self.response.out.write(buf)
You need to declare:
global zipf
right after your
def get(self):
line. you are modifying a global variable, and this is the only way python knows what you are doing.
I want to use tempfile.NamedTemporaryFile() to write some contents into it and then open that file. I have written following code:
tf = tempfile.NamedTemporaryFile()
tfName = tf.name
tf.seek(0)
tf.write(contents)
tf.flush()
but I am unable to open this file and see its contents in Notepad or similar application. Is there any way to achieve this? Why can't I do something like:
os.system('start notepad.exe ' + tfName)
at the end.
I don't want to save the file permanently on my system. I just want the contents to be opened as a text in Notepad or similar application and delete the file when I close that application.
This could be one of two reasons:
Firstly, by default the temporary file is deleted as soon as it is closed. To fix this use:
tf = tempfile.NamedTemporaryFile(delete=False)
and then delete the file manually once you've finished viewing it in the other application.
Alternatively, it could be that because the file is still open in Python Windows won't let you open it using another application.
Edit: to answer some questions from the comments:
As of the docs from 2 when using delete=False the file can be removed by using:
tf.close()
os.unlink(tf.name)
You can also use it with a context manager so that the file will be closed/deleted when it goes out of scope. It will also be cleaned up if the code in the context manager raises.
import tempfile
with tempfile.NamedTemporaryFile() as temp:
temp.write('Some data')
temp.flush()
# do something interesting with temp before it is destroyed
Here is a useful context manager for this.
(In my opinion, this functionality should be part of the Python standard library.)
# python2 or python3
import contextlib
import os
#contextlib.contextmanager
def temporary_filename(suffix=None):
"""Context that introduces a temporary file.
Creates a temporary file, yields its name, and upon context exit, deletes it.
(In contrast, tempfile.NamedTemporaryFile() provides a 'file' object and
deletes the file as soon as that file object is closed, so the temporary file
cannot be safely re-opened by another library or process.)
Args:
suffix: desired filename extension (e.g. '.mp4').
Yields:
The name of the temporary file.
"""
import tempfile
try:
f = tempfile.NamedTemporaryFile(suffix=suffix, delete=False)
tmp_name = f.name
f.close()
yield tmp_name
finally:
os.unlink(tmp_name)
# Example:
with temporary_filename() as filename:
os.system('echo Hello >' + filename)
assert 6 <= os.path.getsize(filename) <= 8 # depending on text EOL
assert not os.path.exists(filename)