I have a folder that contains a lot of subfolders, with images saved as png in each folder :
For example :
emotion\angry
emotion\disgusted
emotion\fearful
emotion\happy
I can remove the images in one of the folder using the below code :
folder_path = (r'C:\Users\emotion\angry')
test = os.listdir(folder_path)
for images in test:
if images.endswith(".png"):
os.remove(os.path.join(folder_path, images))
How do I create a loop to loop through each subfolder in emotion/?
As I do not want to manually write out all the code to clear all the folders...
You can list files using glob patterns and delete them using your normal os.remove.
import os
import glob
fileList = glob.glob('C:\Users\emotion\*\*.png')
for filePath in fileList:
try:
os.remove(filePath)
except:
print("Error while deleting file : ", filePath)
def all_dirs(directory):
# Returning list with all directories in selected directory
dir_list = []
for dir in [x[0] for x in os.walk(directory)]:
dir_list.append(dir)
return dir_list
This solution should work for every OS.
Related
I work as an IT help desk associate and have to constantly clear out disk space for VDIs. I want to write a code that can clear out specific folders all at once to save time. I wrote a code, but only have it to run one folder at a time. Is there a way for me to make it a function and just make variables for each path and have them all cleared at once. Sample:
import os
#path to delete
path = r"C:\Users\jwals\test test"
#will check if the path is a dir or not
if os.path.exists(path):
#iterating through subfolders
for root_folder, folders, files in os.walk(path):
#checking for files
for file in files:
#file path
file_path = os.path.join(root_folder, file)
#delete the files in that path
if not os.remove(file_path):
#will print if successful
print(f"{file_path} deleted successfully")
#unsuccessful message (if this prints something went wrong)
else:
print(f"Unable to delete the {file_path}")
IIUIC, you can simply do this:
import os
#paths to delete
paths = [r"C:\Users\jwals\test test", r"C:\Users\jwals\another test"]
def delete_files(path):
#will check if the path is a dir or not
if os.path.exists(path):
#iterating through subfolders
for root_folder, folders, files in os.walk(path):
#checking for files
for file in files:
#file path
file_path = os.path.join(root_folder, file)
...
for path in paths:
delete_files(path)
I plan to move the pictures in all subfolders (as shown in the picture) under the train file train/LUAD to another new folder train_new/LUAD. There are .jpg images in each subfolder such as the first one in the picture TCGA-05-4249-01Z-00-DX1.9fce0297-cc19-4c04-872c-4466ee4024db.
import os
import shutil
count = 0
def moveFiles(path, disdir):
dirlist = os.listdir(path)
for i in dirlist:
child = os.path.join('%s/%s' % (path, i))
if os.path.isfile(child):
imagename, jpg = os.path.splitext(i)
shutil.copy(child, os.path.join(disdir, imagename + ".jpg"))
continue
moveFiles(child, disdir)
if __name__ == '__main__':
rootimage = '/content/drive/MyDrive/stat841_final_data/train/LUAD'
disdir = '/content/drive/MyDrive/stat841_final_data/train_new/LUAD'
moveFiles(rootimage, disdir)
But it does not work. I only got image from the last subfolder except for other subfolders in my new folder train_new/LUAD...
Just to clarify, you want to move (not copy) images from a nested file structure to a new folder, without nesting?
Be aware that this could overwrite images, if multiple images share the same name!
import pathlib
def move_files(source_folder:pathlib.Path, target_folder:pathlib.Path):
target_folder.mkdir(parents=True, exist_ok=True)
for image_file in source_folder.rglob("*.jpg"): # recursively find image paths
image_file.rename(target_folder.joinpath(image_file.name))
If you are unsure maybe use the copy function first, so you won't lose your original data:
import pathlib
import shutil
def move_files(source_folder:pathlib.Path, target_folder:pathlib.Path):
target_folder.mkdir(parents=True, exist_ok=True)
for image_file in source_folder.rglob("*.jpg"): # recursively find image paths
shutil.copy(image_file, target_folder.joinpath(image_file.name))
I have the following example dataset output from pandas.
What I would like to do in an efficient way is using glob to search the filename in the associated main folder and sub folder only and not to loop through all the main folders/ sub folders as per my current code. I need this to then match against a main folders and sub folder I have and if it matches then it copies the file. I have code that works but it is very inefficient and has to go through all folders/sub folders for each search. The code is as below;At the moment, main_folder and searchdate are lists.filenames_i_want, is also the list that I will be matching to. Any way i can make it go straight to the folder/subfolder e.g if i provided this as CSV input?
import itertools
import glob
import shutil
from pathlib import Path
filenames_i_want = Search_param
main_folder=locosearch
searchfolder= Search_date
TargetFolder = r'C:\ELK\LOGS\XX\DEST'
for directory,folder in itertools.product(main_folder, searchfolder):
files = glob.glob('Z:/{}/{}/asts_data_logger/*.bz2'.format(directory, folder))
for f in files:
current_path = Path(f)
cpn = current_path.name
if current_path.name in filenames_i_want:
print(f"found target file: {f}")
shutil.copy2(f, TargetFolder)
I created a column and used the fields to make an absolute path and then used tuples to go through each row and shutil to copy
TargetFolder = r'C:\ELK\LOGS\ATH\DEST'
for row in df.itertuples():
search = row.Search
try:
shutil.copy2(search, TargetFolder)
except Exception as e:
print(e)
I have a folder of images, they have random names. What i want to do is change the images names to numbers for example 1.jpg 2.jpg 3.jpg and so on till the images are done.
what you need is os.listdir() to list all items in a folder and os.rename to rename those items.
import os
contents = os.listdir()
for i, filename in enumerate(contents):
os.rename(filename, i) # or i.jpg or whatever which is beyond that scope
import os
path = '/Path/To/Directory'
files = os.listdir(path)
i = 1
for file in files:
os.rename(os.path.join(path, file), os.path.join(path, str(i)+'.jpg'))
i = i+1
This can be done using os library:
If the folder has images only, no other files, you can run in the correct folder:
import os
count = 1
for picture in os.listdir():
os.rename(picture, f'{count}.jpg')
count += 1
You can read more about os here: https://docs.python.org/3/library/os.html
I have been working this challenge for about a day or so. I've looked at multiple questions and answers asked on SO and tried to 'MacGyver' the code used for my purpose, but still having issues.
I have a directory (lets call it "src\") with hundreds of files (.txt and .xml). Each .txt file has an associated .xml file (let's call it a pair). Example:
src\text-001.txt
src\text-001.xml
src\text-002.txt
src\text-002.xml
src\text-003.txt
src\text-003.xml
Here's an example of how I would like it to turn out so each pair of files are placed into a single unique folder:
src\text-001\text-001.txt
src\text-001\text-001.xml
src\text-002\text-002.txt
src\text-002\text-002.xml
src\text-003\text-003.txt
src\text-003\text-003.xml
What I'd like to do is create an associated folder for each pair and then move each pair of files into its respective folder using Python. I've already tried working from code I found (thanks to a post from Nov '12 by Sethdd, but am having trouble figuring out how to use the move function to grab pairs of files. Here's where I'm at:
import os
import shutil
srcpath = "PATH_TO_SOURCE"
srcfiles = os.listdir(srcpath)
destpath = "PATH_TO_DEST"
# grabs the name of the file before extension and uses as the dest folder name
destdirs = list(set([filename[0:9] for filename in srcfiles]))
def create(dirname, destpath):
full_path = os.path.join(destpath, dirname)
os.mkdir(full_path)
return full_path
def move(filename, dirpath):
shutil.move(os.path.join(srcpath, filename)
,dirpath)
# create destination directories and store their names along with full paths
targets = [
(folder, create(folder, destpath)) for folder in destdirs
]
for dirname, full_path in targets:
for filename in srcfile:
if dirname == filename[0:9]:
move(filename, full_path)
I feel like it should be easy, but Python isn't something I work with everyday and it's been a while since my scripting days... Any help would be greatly appreciated!
Thanks,
WK2EcoD
Use the glob module to interate all of the 'txt' files. From that you can parse and create the folders and copy the files.
The process should be as simple as it appears to you as a human.
for file_name in os.listdir(srcpath):
dir = file_name[:9]
# if dir doesn't exist, create it
# move file_name to dir
You're doing a lot of intermediate work that seems to be confusing you.
Also, insert some simple print statements to track data flow and execution flow. It appears that you have no tracing output so far.
You can do it with os module. For every file in directory check if associated folder exists, create if needed and then move the file. See the code below:
import os
SRC = 'path-to-src'
for fname in os.listdir(SRC):
filename, file_extension = os.path.splitext(fname)
if file_extension not in ['xml', 'txt']:
continue
folder_path = os.path.join(SRC, filename)
if not os.path.exists(folder_path):
os.mkdir(folderpath)
os.rename(
os.path.join(SRC, fname),
os.path.join(folder_path, fname)
)
My approach would be:
Find the pairs that I want to move (do nothing with files without a pair)
Create a directory for every pair
Move the pair to the directory
#! /usr/bin/env python
# -*- coding: utf-8 -*-
import os, shutil
import re
def getPairs(files):
pairs = []
file_re = re.compile(r'^(.*)\.(.*)$')
for f in files:
match = file_re.match(f)
if match:
(name, ext) = match.groups()
if ext == 'txt' and name + '.xml' in files:
pairs.append(name)
return pairs
def movePairsToDir(pairs):
for name in pairs:
os.mkdir(name)
shutil.move(name+'.txt', name)
shutil.move(name+'.xml', name)
files = os.listdir()
pairs = getPairs(files)
movePairsToDir(pairs)
NOTE: This script works when called inside the directory with the pairs.