In python, I understand that I can delete multiple files with the same name using the following command for eg:
for f in glob.glob("file_name_*.txt"):
os.remove(f)
And that a single directory can be deleted with shutil.rmtree('/path/to/dir') - and that this command will delete the directory even if the directory is not empty. On the other hand, os.rmdir() needs that the directory be empty.
I actually want to delete multiple directories with the same name, and they are not empty. So, I am looking for something like
shutil.rmtree('directory_*')
Is there a way to do this with python?
You have all of the pieces: glob() iterates, and rmtree() deletes:
for path in glob.glob("directory_*"):
shutil.rmtree(path)
This will throw OSError if one of the globbed paths names a file, or for any other reason that rmtree() can fail. You can add error handling as you see fit, once you decide how you want to handle the errors. It doesn't make sense to add error handling unless you know what you want to do with the error, so I have left error handling out.
Related
Suppose I am going to remove a folder from the blob from databricks. however when the folder is not empty , it throws the error "non-empty" directory is not permitted using dbutils.fs.rm("wabs://..../folder")
Even has error include recurse=True
How to solve it?
Note: Delete the file or directory (optionally recursively delete all files in the directory). This call throws an exception with IO_ERROR if the path is a non-empty directory and recursive is set to false or on other similar errors.
For more details, refer similar SO thread.
Hope this helps.
According to here, the below function helps me solve this problem.
def delete_mounted_dir(dirname):
files=dbutils.fs.ls(dirname)
for f in files:
if f.isDir():
delete_mounted_dir(f.path)
dbutils.fs.rm(f.path, recurse=True)
I have what I think is a basic python error. I am building several graphs with the networkx module, and I need to write their edgelists in different gexf files (for gephi). Since I have to perform these operations multiple times I am doing this in a for loop, and I get an error while writing the files.
I need a graph (therefore, a different output file) for each element of the owner column of a dataframe.
for owner in df.owner.unique():
sdf=df[df['owner']==owner]
sG=nx.Graph()
sG.add_nodes_from(sdf['col1'])
sG.add_nodes_from(sdf['col2'])
i=0
while i < len(sdf):
sG.add_edge(sdf.iloc[i,0],sdf.iloc[i,1], weight=sdf.iloc[i,2])
i+=1
with open('com_{}.gexf'.format(owner),'x') as f:
nx.write_gexf(sG,f)
On the first iteration I get a
FileNotFoundError: [Errno 2] No such file or directory
error, suggesting that this is not the right way to create, name and write the files in a loop. What is the right way to do this?
if owner contains a slash, for example "foo/bar", then open will first try to navigate to directory com_foo before creating file bar.gexf. If com_foo doesn't exist, then this exception will occur.
One possible solution is to replace all slashes in owner with a less objectionable character.
with open('com_{}.gexf'.format(owner.replace("/", "_")),'x') as f:
How do I create directories in Python given a list of files with paths that may or may not exist?
I am downloading some files from s3 that may or may not exist in a certain directory. Based upon these keys that represent a potentially deeply nested directory that may or may not exist.
So based upon the key /a/number/of/nested/dirs/file.txt how can I created /a/number/of/nested/dirs/ if they do not exist and do it in a way that doesn't take forever to check for each file in the list.
I am doing this because if the local parent directories do not already exist get_contents_to_filename breaks.
My Final Solution Using Answer:
for file_with_path in files_with_paths:
try:
if not os.path.exists(file_with_path):
os.makedirs(file_with_path)
site_object.get_contents_to_filename(file_with_path)
except:
pass
Simply use os.makedirs(). This will create all intermediate directories if needed.
Recursive directory creation function. Like mkdir(), but makes all
intermediate-level directories needed to contain the leaf directory.
Raises an error exception if the leaf directory already exists or
cannot be created. The default mode is 0777 (octal). On some systems,
mode is ignored. Where it is used, the current umask value is first
masked out.
This question already has answers here:
Python changing file permissions when not wanted
(2 answers)
Closed 9 years ago.
for some reason this python script no longer works now. The script changes the folder permission to read only after it has been run? It runs once and deletes all the files in the folder but when it runs again it gets a Windows error 5 Access denied due to the script changing the permissions to read only on the folder. I can't see why it does this or how to avoid it?
The thing is i didn't write this script and know nothing about python. how would you change it to avoid this issue. Please could you give an example with the code in the script, i wouldn't know where to place it. thanks for the help!
import os
import shutil
for root, dirs, files in os.walk(eg.globals.tvzip):
for f in files:
os.remove(os.path.join(root, f))
for d in dirs:
shutil.rmtree(os.path.join(root, d))
for root, dirs, files in os.walk(eg.globals.tvproc):
for f in files:
os.remove(os.path.join(root, f))
for d in dirs:
shutil.rmtree(os.path.join(root, d))
I don't care whether you wrote this code or not, it makes no sense, and trying to make it work without fixing it is a silly idea.
First, if you want to remove an entire directory tree, don't try to walk the tree and remove each subtree before walking it. Just remove the whole tree:
shutil.rmtree(eg.globals.tvzip)
shutil.rmtree(eg.globals.tvproc)
If you want to remove all of the contents of the tree, but not the root itself, don't use os.walk, just os.listdir:
for p in os.listdir(eg.globals.tvzip):
shutil.rmtree(os.path.join(eg.globals.tvzip, p)
for p in os.listdir(eg.globals.tvproc):
shutil.rmtree(os.path.join(eg.globals.tvproc, p)
That will remove any errors caused by your code stepping on its own toes, trying to keep a directory open for its walk and trying to delete it at the same time.
If you still get errors, it could be because some of the files are read-only, but it could just as easily be because some other program has them open. The only way you will be able to debug that is to know which files, so you can examine them.
The exceptions that you get should include the full pathname to the file that failed in their output—in fact, you showed one in one of your other questions:
WindowsError: [Error 5] Access is denied: 'C:\\zDump\\TVzip\\Elem.avi'
So, how do you know what the problem is?
You can open C:\zDump\TVzip in Explorer and look at Elem.avi and see if it's read-only. Or you can use the DOS prompt, if you know how to do that.
To determine whether it's being kept open by another program, you need a third-party tool. The GUI tool Process Explorer and the command-line tool Handle, both from Sysinternals and published by Microsoft, are probably the simplest.
If you want to delete a whole tree of flies, shutil.rmtree will do it for you - you don't need to walk the list of files deleting them.
If you're trying to not delete the top level directory, you should add a check for that. According to the docs, you will be given the top-level directory:
os.walk(top, topdown=True, onerror=None, followlinks=False)
Generate
the file names in a directory tree by walking the tree either top-down
or bottom-up. For each directory in the tree rooted at directory top
(including top itself), it yields a 3-tuple (dirpath, dirnames,
filenames).
Could something else than your script be setting these folders read only? Perhaps you're deleting them, then getting Access Denied because they don't exist, or something else is re-creating them that way?
I'm trying to remove a directory using python but I do not want to recursively remove the whole directory path in the process: i.e
/home/dir/dir/dirtoberemoved
So I don't want to remove anything at a higher level just the one directory and all its contents. I've been looking on stackoverflow to research this question and most answers have included using the shutil module which I am unfamiliar with, looking at the python documentation for the module it says 'Delete an entire directory tree'
If I do something like this:
if os.path.exists("/home/dir/dir/dirtoberemoved");
shutil.rmtree("/home/dir/dir/dirtoberemoved");
or
shutil.rmtree("/dirtoberemoved");
Will the entire path be removed? If so is there any good way just to delete one non-empty directory in python without deleting higher level directories?
You need to specify the whole path to the directory to be removed. Only the last part of the path will be deleted, the /home/dir/dir/ part will be untouched.
The deletion refers to any sub-directories contained within the named path, so if there is a /home/dir/dir/dirtoberemoved/foo sub-directory it'll be removed together with it's parent.