I have some code that I would like to run on multiple files, each contained in its own subdirectory. I'd like to write some additional code to ask the user for a subdirectory name, opens the subdirectory, and run the code on a file contained within it (there is only one file contained in each subdirectory). Can anyone help?
with this loop you can manipulate all the file through the subfolders for the root folder
import os
for root, dirs, files in os.walk('Your path here'):
for file in files:
//your code here//
Related
I am working on a data cleanup in a network drive. The drive has 1000+ folders, and those folders have several subfolders. The script that I got from G4G (seen below) prompts me to select a folder. I can click on one of my 1000+ folders, and the data is cleaned up properly (duplicates are deleted). However, I'd like to loop the command through the whole drive to avoid clicking on folders for hours. I cannot select the drive as my folder because duplicate file names between the first folders in the drive should not be considered duplicates.
EDIT:
I'll give an example to clarify.
Z:/Folder1 and Z:/Folder2 both have several files named "text.txt," immediately inside of the folders and within the subdirectories of the folders. Folder1 and Folder2, amongst all "text.txt" files immediately inside and within its subdirectories, should each be left with one "text.txt." If the current script is applied to Folder1 and Folder2 individually, then the desired result of one "text.txt" file existing in Folder1 and one existing in Folder2 is accomplished. If the script is applied to the Z drive, then between Folder1 and Folder2, there would only be one "text.txt," and one of the folders would be without a file named "text.txt."
How can I apply this script to each first folder in the drive without having to manually click on each folder?
from tkinter.filedialog import askdirectory
# Importing required libraries.
from tkinter import Tk
import os
import hashlib
from pathlib import Path
# We don't want the GUI window of
# tkinter to be appearing on our screen
Tk().withdraw()
# Dialog box for selecting a folder.
file_path = askdirectory(title="Select a folder")
# Listing out all the files
# inside our root folder.
list_of_files = os.walk(file_path)
# In order to detect the duplicate
# files we are going to define an empty dictionary.
unique_files = dict()
for root, folders, files in list_of_files:
# Running a for loop on all the files
for file in files:
# Finding complete file path
file_path = Path(os.path.join(root, file))
# Converting all the content of
# our file into md5 hash.
Hash_file = hashlib.md5(open(file_path, 'rb').read()).hexdigest()
# If file hash has already #
# been added we'll simply delete that file
if Hash_file not in unique_files:
unique_files[Hash_file] = file_path
else:
if file.endswith((".txt",".bmp")):
os.remove(file_path)
print(f"{file_path} has been deleted")
Maybe you should run it for your drive and use if/else to skip first folder
list_of_files = os.walk("your drive")
for root, folders, files in list_of_files:
if root != "your drive":
for file in files:
# ... code ...
This way you can skip also other (sub)folders.
OR you can use next() to skip some element from os.walk() because os.walk() doesn't give directly list with all elements but generator.
list_of_files = os.walk("your drive")
next(list_of_files) # skip first item
for root, folders, files in list_of_files:
for file in files:
# ... code ...
I am trying to figure out how to generate an excel workbook for each subfolder in my directory while skipping the folders that are empty. My directory structure is below.
So it would start with Folder A, execute my lines of code to create an excel file using Folder A's contents, then move to Folder B, execute my lines of code to create a separate excel file using Folder B's contents, then move to Folder C and skip it since it's empty, and continue on.
How do I loop through each folder in this manner and keep going when a folder is empty?
I would greatly appreciate the help!
myscript.py
folderA
- report1.xlsx
- report2.xlsx
folderB
- report1.xlsx
- report2.xlsx
folderC
** EMPTY **
folderD
- report1.xlsx
- report2.xlsx
Something like this maybe?
from pathlib import Path
from itertools import groupby
def by_folder(path):
return path.parent
for folder, files in groupby(Path("path/to/root/dir").rglob("*.xlsx"), key=by_folder):
print(f"Gonna merge these files from {folder}: ")
for file in files:
print(f"{file.name}")
print()
We recursively search for .xlsx files in the root directory, and group files into lists based on the immediate common parent folder. If a folder is empty, it won't be matched by the glob pattern.
You can use the os.listdir() method to list everything that's inside a folder. Only bad thing is that it'll get all files, so you may get a problem to get "inside a file" when actually you want to get inside all folder.
The following code loop through all subfolders, skip files, and print the name of all folders that are not empty.
for folder in os.listdir("."):
try:
if len( os.listdir("./"+folder) )>0:
print(
folder
)
except:
pass
I have a folder structure where I have a few txt files in a few folders, and then I have to copy all the folders where I found txt files to another folder. My root folder is always different, so to find it I use os.path.dirname(os.path.abspath(file)) and search all the folders from there. Unfortunately I'm stuck here, because I never use any file handling in python.
From your question I'm assuming you have certain folders at a specific location and the folders contain specific txt files. Now you want to move all the folders containing text files to a specific location.
You can do this:
Walk through all the folders in the master_folder.
Walk through files in the sub-folders, and if any file has a .txt extension, it moves the folder to the target location:
import os
import shutil
for dirpath, dirs, files in os.walk('path_to_master_folder'):
for filename in files:
if filename.endswith('txt'):
shutil.move(dirpath, 'path_to_destination_folder')
I use os.renmae to rename files and move them about, but i am failing at doing the following task.
I have a main folder containing sub-folders with the structure below.
Main folder "Back", containing sub-folders named with letters and numbers e.g. A01, A02, B01, B02, etc.. inside each of those folders is a set of files, amongst them is a file called "rad" so a file path example looks something like this:
Back/A01/rad
/A02/rad
/B01/rad
.../rad
I have another sub-folder called "rads" inside the main "Back"
Back/rads
What i want to do is copy all the rad files only, from each of the folders in back and move them to the folder "rads" and name each rad file based on the folder it came from.
e.g. rad_A01, rad_A02, rad_B01, etc...
I couldnt really figure out how to increase the folder number when i move the files.
os.rename ("Back//rad, Back/rads/rad_")
I thought of making a list of all the names of the files and then do something like from x in list, do os.rename (), but i didnt know how to tell python to name the file according to the subfolder it came from as they are not a continuous series..
Any help is appreciated.
Thanks in advance
import os
for subdir, dirs, files in os.walk('./Back/'):
for file in files:
filepath = subdir+os.sep+file
if filepath.endswith("rad.txt"):
par_dir = os.path.split(os.path.dirname(filepath))[1]
os.system('cp '+filepath+' ./Back/rads/rad_'+par_dir)
save this python file beside Back directory and it should work fine!
This code iterates over each files in each subdirectory of Back, checks all files with name rad.txt, appends name of parent directory and copy to the rads folder.
Note: I saved rad files with .txt extension, change it accordingly.
I've been trying to figure out how I can loop through a root directory with sub directories and search for a file. If a specific file exists then run a script.
File structure example:
Root
--Folder 1
----TEST.txt
--Folder 2
----[no files]
--Folder 3
----TEST.txt
What I am trying to achieve is having the .py file in Root. When run it will loop over each folder in Root, if the file TEST.txt then do some processing.
Notes:
There will always be folders in Root
Where processing is needed there will be a file called TEST.txt
There will definitely be some folders that do not have TEST.txt
Pseudo code:
From Root open Folder 1. If TEST.txt is there then do some cool stuff and then 'cd ../' and repeat process but look in Folder 2.
Stop looping when all Folders have been checked.
This code should be able to search all folders and sub folders for a file.
import os
thisdir = os.getcwd()
for root, dirs, files in os.walk(thisdir):
if 'TEST.txt' in files:
#do some processing
Joining 'root' and the file name should be able to give you access to that file if you want to execute or analyze it somehow.