shutil.make_archive permission issue - python

I have a functionality in my Python which backups a particular directory everyday. The backup steps include compressing the directory. I am using shutil.make_archive to compress the directory.
But now I am facing a permission issue when the directory I compress contains files which I do not have access to.
In this case, I just want to skip that file and then go ahead with compression of the remaining files. How do I achieve this ?
I searched SO and came across this answer, which shows how to zip a directory from using zipfile library. In this case, I can just check if a particular file throws an exception and ignore it.
Is this the only way or is there an easier way ?
My code (just uses shutil.make_archive):
shutil.make_archive(filename, "zip", foldername)
The code in the answer which I have modified for my use:
def zipdir(path, ziph):
# ziph is zipfile handle
for root, dirs, files in os.walk(path):
for file in files:
try:
ziph.write(os.path.join(root, file))
except PermissionError as e:
continue
if __name__ == '__main__':
zipf = zipfile.ZipFile('Python.zip', 'w', zipfile.ZIP_DEFLATED)
zipdir('tmp/', zipf)
zipf.close()
Please tell me if there is an easier/efficient solution to my issue.

I know this is ancient history, but you can use copytree and pass in a custom copy function that suppresses the permissions errors:
import os
#Adapted from python shutil source code.
def copyIgnorePermissionError(src, dst, *, follow_symlinks=True):
"""Copy data and metadata. Return the file's destination.
Metadata is copied with copystat(). Please see the copystat function
for more information.
The destination may be a directory.
If follow_symlinks is false, symlinks won't be followed. This
resembles GNU's "cp -P src dst".
"""
if os.path.isdir(dst):
dst = os.path.join(dst, os.path.basename(src))
#try the copy. If it gives us any lip about permission errors,
#we're going to just suppress them here. If for some reason
#a non-permission error happens, then raise it.
try:
copyfile(src, dst, follow_symlinks=follow_symlinks)
copystat(src, dst, follow_symlinks=follow_symlinks)
except PermissionError:
pass
except Exception as e:
raise(e)
finally:
return dst
#copy the source to some safe destination
shutil.copytree(aSource, aCopyDestination, copy_function=copyIgnorePermissionError)
That will make a copy of all the files and directories that you can access at your permission level. Then you can use shutil.make_archive to create the archive of this copy, then delete the copy.

Related

Access denied from "shutil.rmtree('dir_path')", despite emptying dir_path's contents [duplicate]

How can I delete the contents of a local folder in Python?
The current project is for Windows, but I would like to see *nix also.
import os, shutil
folder = '/path/to/folder'
for filename in os.listdir(folder):
file_path = os.path.join(folder, filename)
try:
if os.path.isfile(file_path) or os.path.islink(file_path):
os.unlink(file_path)
elif os.path.isdir(file_path):
shutil.rmtree(file_path)
except Exception as e:
print('Failed to delete %s. Reason: %s' % (file_path, e))
You can simply do this:
import os
import glob
files = glob.glob('/YOUR/PATH/*')
for f in files:
os.remove(f)
You can of course use an other filter in you path, for example : /YOU/PATH/*.txt for removing all text files in a directory.
You can delete the folder itself, as well as all its contents, using shutil.rmtree:
import shutil
shutil.rmtree('/path/to/folder')
shutil.rmtree(path, ignore_errors=False, onerror=None)
Delete an entire directory tree; path must point to a directory (but not a symbolic link to a directory). If ignore_errors is true, errors resulting from failed removals will be ignored; if false or omitted, such errors are handled by calling a handler specified by onerror or, if that is omitted, they raise an exception.
Expanding on mhawke's answer this is what I've implemented. It removes all the content of a folder but not the folder itself. Tested on Linux with files, folders and symbolic links, should work on Windows as well.
import os
import shutil
for root, dirs, files in os.walk('/path/to/folder'):
for f in files:
os.unlink(os.path.join(root, f))
for d in dirs:
shutil.rmtree(os.path.join(root, d))
I'm surprised nobody has mentioned the awesome pathlib to do this job.
If you only want to remove files in a directory it can be a oneliner
from pathlib import Path
[f.unlink() for f in Path("/path/to/folder").glob("*") if f.is_file()]
To also recursively remove directories you can write something like this:
from pathlib import Path
from shutil import rmtree
for path in Path("/path/to/folder").glob("**/*"):
if path.is_file():
path.unlink()
elif path.is_dir():
rmtree(path)
Using rmtree and recreating the folder could work, but I have run into errors when deleting and immediately recreating folders on network drives.
The proposed solution using walk does not work as it uses rmtree to remove folders and then may attempt to use os.unlink on the files that were previously in those folders. This causes an error.
The posted glob solution will also attempt to delete non-empty folders, causing errors.
I suggest you use:
folder_path = '/path/to/folder'
for file_object in os.listdir(folder_path):
file_object_path = os.path.join(folder_path, file_object)
if os.path.isfile(file_object_path) or os.path.islink(file_object_path):
os.unlink(file_object_path)
else:
shutil.rmtree(file_object_path)
This:
removes all symbolic links
dead links
links to directories
links to files
removes subdirectories
does not remove the parent directory
Code:
for filename in os.listdir(dirpath):
filepath = os.path.join(dirpath, filename)
try:
shutil.rmtree(filepath)
except OSError:
os.remove(filepath)
As many other answers, this does not try to adjust permissions to enable removal of files/directories.
Using os.scandir and context manager protocol in Python 3.6+:
import os
import shutil
with os.scandir(target_dir) as entries:
for entry in entries:
if entry.is_dir() and not entry.is_symlink():
shutil.rmtree(entry.path)
else:
os.remove(entry.path)
Earlier versions of Python:
import os
import shutil
# Gather directory contents
contents = [os.path.join(target_dir, i) for i in os.listdir(target_dir)]
# Iterate and remove each item in the appropriate manner
[shutil.rmtree(i) if os.path.isdir(i) and not os.path.islink(i) else os.remove(i) for i in contents]
Notes: in case someone down voted my answer, I have something to explain here.
Everyone likes short 'n' simple answers. However, sometimes the reality is not so simple.
Back to my answer. I know shutil.rmtree() could be used to delete a directory tree. I've used it many times in my own projects. But you must realize that the directory itself will also be deleted by shutil.rmtree(). While this might be acceptable for some, it's not a valid answer for deleting the contents of a folder (without side effects).
I'll show you an example of the side effects. Suppose that you have a directory with customized owner and mode bits, where there are a lot of contents. Then you delete it with shutil.rmtree() and rebuild it with os.mkdir(). And you'll get an empty directory with default (inherited) owner and mode bits instead. While you might have the privilege to delete the contents and even the directory, you might not be able to set back the original owner and mode bits on the directory (e.g. you're not a superuser).
Finally, be patient and read the code. It's long and ugly (in sight), but proven to be reliable and efficient (in use).
Here's a long and ugly, but reliable and efficient solution.
It resolves a few problems which are not addressed by the other answerers:
It correctly handles symbolic links, including not calling shutil.rmtree() on a symbolic link (which will pass the os.path.isdir() test if it links to a directory; even the result of os.walk() contains symbolic linked directories as well).
It handles read-only files nicely.
Here's the code (the only useful function is clear_dir()):
import os
import stat
import shutil
# http://stackoverflow.com/questions/1889597/deleting-directory-in-python
def _remove_readonly(fn, path_, excinfo):
# Handle read-only files and directories
if fn is os.rmdir:
os.chmod(path_, stat.S_IWRITE)
os.rmdir(path_)
elif fn is os.remove:
os.lchmod(path_, stat.S_IWRITE)
os.remove(path_)
def force_remove_file_or_symlink(path_):
try:
os.remove(path_)
except OSError:
os.lchmod(path_, stat.S_IWRITE)
os.remove(path_)
# Code from shutil.rmtree()
def is_regular_dir(path_):
try:
mode = os.lstat(path_).st_mode
except os.error:
mode = 0
return stat.S_ISDIR(mode)
def clear_dir(path_):
if is_regular_dir(path_):
# Given path is a directory, clear its content
for name in os.listdir(path_):
fullpath = os.path.join(path_, name)
if is_regular_dir(fullpath):
shutil.rmtree(fullpath, onerror=_remove_readonly)
else:
force_remove_file_or_symlink(fullpath)
else:
# Given path is a file or a symlink.
# Raise an exception here to avoid accidentally clearing the content
# of a symbolic linked directory.
raise OSError("Cannot call clear_dir() on a symbolic link")
As a oneliner:
import os
# Python 2.7
map( os.unlink, (os.path.join( mydir,f) for f in os.listdir(mydir)) )
# Python 3+
list( map( os.unlink, (os.path.join( mydir,f) for f in os.listdir(mydir)) ) )
A more robust solution accounting for files and directories as well would be (2.7):
def rm(f):
if os.path.isdir(f): return os.rmdir(f)
if os.path.isfile(f): return os.unlink(f)
raise TypeError, 'must be either file or directory'
map( rm, (os.path.join( mydir,f) for f in os.listdir(mydir)) )
I used to solve the problem this way:
import shutil
import os
shutil.rmtree(dirpath)
os.mkdir(dirpath)
To delete all files inside a folder a I use:
import os
for i in os.listdir():
os.remove(i)
To delete all the files inside the directory as well as its sub-directories, without removing the folders themselves, simply do this:
import os
mypath = "my_folder" #Enter your path here
for root, dirs, files in os.walk(mypath, topdown=False):
for file in files:
os.remove(os.path.join(root, file))
# Add this block to remove folders
for dir in dirs:
os.rmdir(os.path.join(root, dir))
# Add this line to remove the root folder at the end
os.rmdir(mypath)
You might be better off using os.walk() for this.
os.listdir() doesn't distinguish files from directories and you will quickly get into trouble trying to unlink these. There is a good example of using os.walk() to recursively remove a directory here, and hints on how to adapt it to your circumstances.
If you are using a *nix system, why not leverage the system command?
import os
path = 'folder/to/clean'
os.system('rm -rf %s/*' % path)
I konw it's an old thread but I have found something interesting from the official site of python. Just for sharing another idea for removing of all contents in a directory. Because I have some problems of authorization when using shutil.rmtree() and I don't want to remove the directory and recreate it. The address original is http://docs.python.org/2/library/os.html#os.walk. Hope that could help someone.
def emptydir(top):
if(top == '/' or top == "\\"): return
else:
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
os.remove(os.path.join(root, name))
for name in dirs:
os.rmdir(os.path.join(root, name))
I had to remove files from 3 separate folders inside a single parent directory:
directory
folderA
file1
folderB
file2
folderC
file3
This simple code did the trick for me: (I'm on Unix)
import os
import glob
folders = glob.glob('./path/to/parentdir/*')
for fo in folders:
file = glob.glob(f'{fo}/*')
for f in file:
os.remove(f)
Hope this helps.
Yet Another Solution:
import sh
sh.rm(sh.glob('/path/to/folder/*'))
Well, I think this code is working. It will not delete the folder and you can use this code to delete files having the particular extension.
import os
import glob
files = glob.glob(r'path/*')
for items in files:
os.remove(items)
Pretty intuitive way of doing it:
import shutil, os
def remove_folder_contents(path):
shutil.rmtree(path)
os.makedirs(path)
remove_folder_contents('/path/to/folder')
use this function
import os
import glob
def truncate(path):
files = glob.glob(path+'/*.*')
for f in files:
os.remove(f)
truncate('/my/path')
Use the method bellow to remove the contents of a directory, not the directory itself:
import os
import shutil
def remove_contents(path):
for c in os.listdir(path):
full_path = os.path.join(path, c)
if os.path.isfile(full_path):
os.remove(full_path)
else:
shutil.rmtree(full_path)
Answer for a limited, specific situation:
assuming you want to delete the files while maintainig the subfolders tree, you could use a recursive algorithm:
import os
def recursively_remove_files(f):
if os.path.isfile(f):
os.unlink(f)
elif os.path.isdir(f):
for fi in os.listdir(f):
recursively_remove_files(os.path.join(f, fi))
recursively_remove_files(my_directory)
Maybe slightly off-topic, but I think many would find it useful
I resolved the issue with rmtree makedirs by adding time.sleep() between:
if os.path.isdir(folder_location):
shutil.rmtree(folder_location)
time.sleep(.5)
os.makedirs(folder_location, 0o777)
the easiest way to delete all files in a folder/remove all files
import os
files = os.listdir(yourFilePath)
for f in files:
os.remove(yourFilePath + f)
This should do the trick just using the OS module to list and then remove!
import os
DIR = os.list('Folder')
for i in range(len(DIR)):
os.remove('Folder'+chr(92)+i)
Worked for me, any problems let me know!

PermissionError: [Errno 13] Permission denied # PYTHON

i am trying to copy a file from one folder to another, but i am getting "PermissionError: [Errno 13] Permission denied". I am working within my home directory and i am the administrator of the PC. Went through many other previous posts .. tried all the options that are to my knowledge (newbie to programming) ... need some help.
import os
import shutil
src = "C:\\Users\\chzia\\Scripts\\test" # the file lab.txt is in this folder that needs to be copied to testcp folder.
dst = "C:\\Users\\chzia\\Scripts\\testcp"
for file in os.listdir(src):
src_file = os.path.join(src, file)
dst_file = os.path.join(dst, file)
#shutil.copymode(src, dst) # i have tried these options too same error
#shutil.copyfile(src, dst) # i have tried these options too same error
shutil.copy(src, dst)
My target is to create an .exe that copies a file from the network location to a specific folder on a pc where the .exe is run.
Thanks in advance for all the support and help.
Perhaps try to use shutil.copyfile instead:
shutil.copyfile(src, dst)
Similar old topic on Why would shutil.copy() raise a permission exception when cp doesn't?
I am sure I am late, but I ran into the same problem.
I noticed that in my case the problem is that the subfolder already exists.
If I delete the folder at the start (it is OK in my case).
import os
import shutil
dst = "C:\\Users\\chzia\\Scripts\\testcp" # target folder
def checkFolder(path):
try:
os.stat(path)
shutil.rmtree(path)
except:
os.mkdir(path)
checkFolder(dst)
If you Googled the exception and ended-up here, remember to provide the absolute/full path when using copy & copyfile from shutil. For example,
abs_src_path = os.path.abspath(relative_file_path)
abs_dst_path = os.path.abspath(relative_dst_path)
shutil.copy(abs_src_path , abs_dst_path)
In the question above that is already done, but you might be the one who is mislead by the error message.

Use Python to automate copying files to their corresponding target folders(shutil.copytree doesn't work as intended)

I wanted to automate the process of copying files (in their target folders) to their corresponding source folders (which has the same folder structure as the source) located in a different directory on computer...I tried to use python's shutil.copytree, but that will copy all the target folders into the source folders and the Python documentation said that "The destination directory, named by dst, must not already exist" (which, in my case, break the rule). So what I wanted to do is to only copy the target files to the corresponding folder, so that the source and target files would end up staying in the same folder...Is it possible to do by using python?Here I attached to the question a screenshot to further explain what I meant. Thank you so much for your help! At the meantime, I'll try to do more research about it too!
Here's the modified version of shutil.copytree which does not create directory (removed os.makedirs call).
import os
from shutil import Error, WindowsError, copy2, copystat
def copytree(src, dst, symlinks=False, ignore=None):
names = os.listdir(src)
if ignore is not None:
ignored_names = ignore(src, names)
else:
ignored_names = set()
# os.makedirs(dst)
errors = []
for name in names:
if name in ignored_names:
continue
srcname = os.path.join(src, name)
dstname = os.path.join(dst, name)
try:
if symlinks and os.path.islink(srcname):
linkto = os.readlink(srcname)
os.symlink(linkto, dstname)
elif os.path.isdir(srcname):
copytree(srcname, dstname, symlinks, ignore)
else:
# Will raise a SpecialFileError for unsupported file types
copy2(srcname, dstname)
# catch the Error from the recursive copytree so that we can
# continue with other files
except Error, err:
errors.extend(err.args[0])
except EnvironmentError, why:
errors.append((srcname, dstname, str(why)))
try:
copystat(src, dst)
except OSError, why:
if WindowsError is not None and isinstance(why, WindowsError):
# Copying file access times may fail on Windows
pass
else:
errors.append((src, dst, str(why)))
if errors:
raise Error, errors
Using mock (or unittest.mock in Python 3.x), you can temporarily disable os.makedirs by replacing os.makedirs with a Mock object (See unittest.mock.patch):
from shutil import copytree
import mock # import unittest.mock as mock in Python 3.x
with mock.patch('os.makedirs'):
copytree('PlaceB', 'PlaceA')
I just found a rather easy way to make it happen.
We can use ditto command to merge the 2 folders together.
ditto PlaceB PlaceA

Python: folder creation when copying files

I'm trying to create a shell script that will copy files from one computer (employee's old computer) to another (employee's new computer). I have it to the point where I can copy files over, thanks to the lovely people here, but I'm running into a problem - if I'm going from, say, this directory that has 2 files:
C:\Users\specificuser\Documents\Test Folder
....to this directory...
C:\Users\specificuser\Desktop
...I see the files show up on the Desktop, but the folder those files were in (Test Folder) isn't created.
Here is the copy function I'm using:
#copy function
def dir_copy(srcpath, dstpath):
#if the destination path doesn't exist, create it
if not os.path.exists(dstpath):
os.makedir(dstpath)
#tag each file to the source path to create the file path
for file in os.listdir(srcpath):
srcfile = os.path.join(srcpath, file)
dstfile = os.path.join(dstpath, file)
#if the source file path is a directory, copy the directory
if os.path.isdir(srcfile):
dir_copy(srcfile, dstfile)
else: #if the source file path is just a file, copy the file
shutil.copyfile(srcfile, dstfile)
I know I need to create the directory on the destination, I'm just not quite sure how to do it.
Edit: I found that I had a type (os.makedir instead of os.mkdir). I tested it, and it creates directories like it's supposed to. HOWEVER I'd like it to create the directory one level up from where it's starting. For example, in Test Folder there is Sub Test Folder. It has created Sub Test Folder but won't create Test Folder because Test Folder is not part of the dstpath. Does that make sense?
You might want to look at shutil.copytree(). It performs the recursive copy functionality, including directories, that you're looking for. So, for a basic recursive copy, you could just run:
shutil.copytree(srcpath, dstpath)
However, to accomplish your goal of copying the source directory to the destination directory, creating the source directory inside of the destination directory in the process, you could use something like this:
import os
import shutil
def dir_copy(srcpath, dstdir):
dirname = os.path.basename(srcpath)
dstpath = os.path.join(dstdir, dirname)
shutil.copytree(srcpath, dstpath)
Note that your srcpath must not contain a slash at the end for this to work. Also, the result of joining the destination directory and the source directory name must not already exist, or copytree will fail.
This is a common problem with file copy... do you intend to just copy the contents of the folder or do you want the folder itself copied. Copy utilities typically have a flag for this and you can too. I use os.makedirs so that any intermediate directories are created also.
#copy function
def dir_copy(srcpath, dstpath, include_directory=False):
if include_directory:
dstpath = os.path.join(dstpath, os.path.basename(srcpath))
os.makedirs(dstpath, exist_ok=True)
#tag each file to the source path to create the file path
for file in os.listdir(srcpath):
srcfile = os.path.join(srcpath, file)
dstfile = os.path.join(dstpath, file)
#if the source file path is a directory, copy the directory
if os.path.isdir(srcfile):
dir_copy(srcfile, dstfile)
else: #if the source file path is just a file, copy the file
shutil.copyfile(srcfile, dstfile)
import shutil
import os
def dir_copy(srcpath, dstpath):
try:
shutil.copytree(srcpath, dstpath)
except shutil.Error as e:
print('Directory not copied. Error: %s' % e)
except OSError as e:
print('Directory not copied. Error: %s' % e)
dir_copy('/home/sergey/test1', '/home/sergey/test2')
I use this script to backup (copy) my working folder. It will skip large files, keep folder structure (hierarchy) and create destination folders if they don't exist.
import os
import shutil
for root, dirs, files in os.walk(the_folder_copy_from):
for name in files:
if os.path.getsize(os.path.join(root, name))<10*1024*1024:
target=os.path.join("backup", os.path.relpath(os.path.join(root, name),start=the_folder_copy_from))
print(target)
os.makedirs(os.path.dirname(target),exist_ok=True)
shutil.copy(src=os.path.join(root, name),dst=target)
print("Done")

How to delete the contents of a folder?

How can I delete the contents of a local folder in Python?
The current project is for Windows, but I would like to see *nix also.
import os, shutil
folder = '/path/to/folder'
for filename in os.listdir(folder):
file_path = os.path.join(folder, filename)
try:
if os.path.isfile(file_path) or os.path.islink(file_path):
os.unlink(file_path)
elif os.path.isdir(file_path):
shutil.rmtree(file_path)
except Exception as e:
print('Failed to delete %s. Reason: %s' % (file_path, e))
You can simply do this:
import os
import glob
files = glob.glob('/YOUR/PATH/*')
for f in files:
os.remove(f)
You can of course use an other filter in you path, for example : /YOU/PATH/*.txt for removing all text files in a directory.
You can delete the folder itself, as well as all its contents, using shutil.rmtree:
import shutil
shutil.rmtree('/path/to/folder')
shutil.rmtree(path, ignore_errors=False, onerror=None)
Delete an entire directory tree; path must point to a directory (but not a symbolic link to a directory). If ignore_errors is true, errors resulting from failed removals will be ignored; if false or omitted, such errors are handled by calling a handler specified by onerror or, if that is omitted, they raise an exception.
Expanding on mhawke's answer this is what I've implemented. It removes all the content of a folder but not the folder itself. Tested on Linux with files, folders and symbolic links, should work on Windows as well.
import os
import shutil
for root, dirs, files in os.walk('/path/to/folder'):
for f in files:
os.unlink(os.path.join(root, f))
for d in dirs:
shutil.rmtree(os.path.join(root, d))
I'm surprised nobody has mentioned the awesome pathlib to do this job.
If you only want to remove files in a directory it can be a oneliner
from pathlib import Path
[f.unlink() for f in Path("/path/to/folder").glob("*") if f.is_file()]
To also recursively remove directories you can write something like this:
from pathlib import Path
from shutil import rmtree
for path in Path("/path/to/folder").glob("**/*"):
if path.is_file():
path.unlink()
elif path.is_dir():
rmtree(path)
Using rmtree and recreating the folder could work, but I have run into errors when deleting and immediately recreating folders on network drives.
The proposed solution using walk does not work as it uses rmtree to remove folders and then may attempt to use os.unlink on the files that were previously in those folders. This causes an error.
The posted glob solution will also attempt to delete non-empty folders, causing errors.
I suggest you use:
folder_path = '/path/to/folder'
for file_object in os.listdir(folder_path):
file_object_path = os.path.join(folder_path, file_object)
if os.path.isfile(file_object_path) or os.path.islink(file_object_path):
os.unlink(file_object_path)
else:
shutil.rmtree(file_object_path)
This:
removes all symbolic links
dead links
links to directories
links to files
removes subdirectories
does not remove the parent directory
Code:
for filename in os.listdir(dirpath):
filepath = os.path.join(dirpath, filename)
try:
shutil.rmtree(filepath)
except OSError:
os.remove(filepath)
As many other answers, this does not try to adjust permissions to enable removal of files/directories.
Using os.scandir and context manager protocol in Python 3.6+:
import os
import shutil
with os.scandir(target_dir) as entries:
for entry in entries:
if entry.is_dir() and not entry.is_symlink():
shutil.rmtree(entry.path)
else:
os.remove(entry.path)
Earlier versions of Python:
import os
import shutil
# Gather directory contents
contents = [os.path.join(target_dir, i) for i in os.listdir(target_dir)]
# Iterate and remove each item in the appropriate manner
[shutil.rmtree(i) if os.path.isdir(i) and not os.path.islink(i) else os.remove(i) for i in contents]
Notes: in case someone down voted my answer, I have something to explain here.
Everyone likes short 'n' simple answers. However, sometimes the reality is not so simple.
Back to my answer. I know shutil.rmtree() could be used to delete a directory tree. I've used it many times in my own projects. But you must realize that the directory itself will also be deleted by shutil.rmtree(). While this might be acceptable for some, it's not a valid answer for deleting the contents of a folder (without side effects).
I'll show you an example of the side effects. Suppose that you have a directory with customized owner and mode bits, where there are a lot of contents. Then you delete it with shutil.rmtree() and rebuild it with os.mkdir(). And you'll get an empty directory with default (inherited) owner and mode bits instead. While you might have the privilege to delete the contents and even the directory, you might not be able to set back the original owner and mode bits on the directory (e.g. you're not a superuser).
Finally, be patient and read the code. It's long and ugly (in sight), but proven to be reliable and efficient (in use).
Here's a long and ugly, but reliable and efficient solution.
It resolves a few problems which are not addressed by the other answerers:
It correctly handles symbolic links, including not calling shutil.rmtree() on a symbolic link (which will pass the os.path.isdir() test if it links to a directory; even the result of os.walk() contains symbolic linked directories as well).
It handles read-only files nicely.
Here's the code (the only useful function is clear_dir()):
import os
import stat
import shutil
# http://stackoverflow.com/questions/1889597/deleting-directory-in-python
def _remove_readonly(fn, path_, excinfo):
# Handle read-only files and directories
if fn is os.rmdir:
os.chmod(path_, stat.S_IWRITE)
os.rmdir(path_)
elif fn is os.remove:
os.lchmod(path_, stat.S_IWRITE)
os.remove(path_)
def force_remove_file_or_symlink(path_):
try:
os.remove(path_)
except OSError:
os.lchmod(path_, stat.S_IWRITE)
os.remove(path_)
# Code from shutil.rmtree()
def is_regular_dir(path_):
try:
mode = os.lstat(path_).st_mode
except os.error:
mode = 0
return stat.S_ISDIR(mode)
def clear_dir(path_):
if is_regular_dir(path_):
# Given path is a directory, clear its content
for name in os.listdir(path_):
fullpath = os.path.join(path_, name)
if is_regular_dir(fullpath):
shutil.rmtree(fullpath, onerror=_remove_readonly)
else:
force_remove_file_or_symlink(fullpath)
else:
# Given path is a file or a symlink.
# Raise an exception here to avoid accidentally clearing the content
# of a symbolic linked directory.
raise OSError("Cannot call clear_dir() on a symbolic link")
As a oneliner:
import os
# Python 2.7
map( os.unlink, (os.path.join( mydir,f) for f in os.listdir(mydir)) )
# Python 3+
list( map( os.unlink, (os.path.join( mydir,f) for f in os.listdir(mydir)) ) )
A more robust solution accounting for files and directories as well would be (2.7):
def rm(f):
if os.path.isdir(f): return os.rmdir(f)
if os.path.isfile(f): return os.unlink(f)
raise TypeError, 'must be either file or directory'
map( rm, (os.path.join( mydir,f) for f in os.listdir(mydir)) )
I used to solve the problem this way:
import shutil
import os
shutil.rmtree(dirpath)
os.mkdir(dirpath)
To delete all files inside a folder a I use:
import os
for i in os.listdir():
os.remove(i)
To delete all the files inside the directory as well as its sub-directories, without removing the folders themselves, simply do this:
import os
mypath = "my_folder" #Enter your path here
for root, dirs, files in os.walk(mypath, topdown=False):
for file in files:
os.remove(os.path.join(root, file))
# Add this block to remove folders
for dir in dirs:
os.rmdir(os.path.join(root, dir))
# Add this line to remove the root folder at the end
os.rmdir(mypath)
You might be better off using os.walk() for this.
os.listdir() doesn't distinguish files from directories and you will quickly get into trouble trying to unlink these. There is a good example of using os.walk() to recursively remove a directory here, and hints on how to adapt it to your circumstances.
If you are using a *nix system, why not leverage the system command?
import os
path = 'folder/to/clean'
os.system('rm -rf %s/*' % path)
I konw it's an old thread but I have found something interesting from the official site of python. Just for sharing another idea for removing of all contents in a directory. Because I have some problems of authorization when using shutil.rmtree() and I don't want to remove the directory and recreate it. The address original is http://docs.python.org/2/library/os.html#os.walk. Hope that could help someone.
def emptydir(top):
if(top == '/' or top == "\\"): return
else:
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
os.remove(os.path.join(root, name))
for name in dirs:
os.rmdir(os.path.join(root, name))
I had to remove files from 3 separate folders inside a single parent directory:
directory
folderA
file1
folderB
file2
folderC
file3
This simple code did the trick for me: (I'm on Unix)
import os
import glob
folders = glob.glob('./path/to/parentdir/*')
for fo in folders:
file = glob.glob(f'{fo}/*')
for f in file:
os.remove(f)
Hope this helps.
Yet Another Solution:
import sh
sh.rm(sh.glob('/path/to/folder/*'))
Well, I think this code is working. It will not delete the folder and you can use this code to delete files having the particular extension.
import os
import glob
files = glob.glob(r'path/*')
for items in files:
os.remove(items)
Pretty intuitive way of doing it:
import shutil, os
def remove_folder_contents(path):
shutil.rmtree(path)
os.makedirs(path)
remove_folder_contents('/path/to/folder')
use this function
import os
import glob
def truncate(path):
files = glob.glob(path+'/*.*')
for f in files:
os.remove(f)
truncate('/my/path')
Use the method bellow to remove the contents of a directory, not the directory itself:
import os
import shutil
def remove_contents(path):
for c in os.listdir(path):
full_path = os.path.join(path, c)
if os.path.isfile(full_path):
os.remove(full_path)
else:
shutil.rmtree(full_path)
Answer for a limited, specific situation:
assuming you want to delete the files while maintainig the subfolders tree, you could use a recursive algorithm:
import os
def recursively_remove_files(f):
if os.path.isfile(f):
os.unlink(f)
elif os.path.isdir(f):
for fi in os.listdir(f):
recursively_remove_files(os.path.join(f, fi))
recursively_remove_files(my_directory)
Maybe slightly off-topic, but I think many would find it useful
Other methods I tried with os and glob package, I had permission issue but with this I had no permission issue plus one less package usage. Probably fail if sub directory exist.
import os
def remove_files_in_folder(folderPath):
# loop through all the contents of folder
for filename in os.listdir(folderPath):
# remove the file
os.remove(f"{folderPath}/{filename}")
remove_files_in_folder('./src/inputFiles/tmp')
Folder structure
root
|
+-- main.py
|
+-- src
|
+-- inputFiles
|
+-- tmp
|
+-- file1.txt
+-- img1.png
I resolved the issue with rmtree makedirs by adding time.sleep() between:
if os.path.isdir(folder_location):
shutil.rmtree(folder_location)
time.sleep(.5)
os.makedirs(folder_location, 0o777)
the easiest way to delete all files in a folder/remove all files
import os
files = os.listdir(yourFilePath)
for f in files:
os.remove(yourFilePath + f)
This should do the trick just using the OS module to list and then remove!
import os
DIR = os.list('Folder')
for i in range(len(DIR)):
os.remove('Folder'+chr(92)+i)
Worked for me, any problems let me know!

Categories