i'm having difficulties trying to read from sub-folders that are inside a folder. What im trying to do is: i have my path "C:\Dataset" which has 2 folders inside them and inside both folders, i have person names that have pictures for example: "C:\Dataset\Drunk\JohnDoe\Pic1", "C:\Dataset\Sober\JaneDoe\Pic1". I want to be able to read each picture and store them in a path variable.
At the moment, what i got so far, basically, i get the images as long as they are inside Drunk and Sober only, for instance: 'C:\Dataset\Drunk\Pic1', and the code that i am using to do is this:
DATADIR = "C:\Dataset"
CATEGORIES = ["Positive", "Negative"]
for category in CATEGORIES:
path = os.path.join(DATADIR, category)
for img in os.listdir(path):
img_path = os.path.join(path,img)
img_data = open(img_path, 'r').read()
break
break
Basically, what i am trying to do is that when i iterate inside Drunk folder it also iterates inside the inner folders, reading the path to the pictures that are in C:\Dataset\Drunk\JohnDoe\nthPic, C:\Dataset\Drunk\JoeDoe\nthPic, C:\Dataset\Drunk and Sober \nthJoe\nthPic C:\Dataset\Drunk\JamesDoe\nthPic. Therefore, when I do the reading, it grabs the whole folder map
This is basically what my goal is.
You need one nesting more:
It saves all images in the dictionary images, key is the full path.
DATADIR = "C:\Dataset"
CATEGORIES = ["Drunk", "Sober"]
images = {}
for category in CATEGORIES:
path = os.path.join(DATADIR, category)
for person in os.listdir(path):
personfolder = os.path.join(path, person):
for imgname in os.listdir(personfolder):
fullpath = os.path.join(personfolder, imgname)
images[fullpath] = open(fullpath, 'r').read()
Related
I plan to move the pictures in all subfolders (as shown in the picture) under the train file train/LUAD to another new folder train_new/LUAD. There are .jpg images in each subfolder such as the first one in the picture TCGA-05-4249-01Z-00-DX1.9fce0297-cc19-4c04-872c-4466ee4024db.
import os
import shutil
count = 0
def moveFiles(path, disdir):
dirlist = os.listdir(path)
for i in dirlist:
child = os.path.join('%s/%s' % (path, i))
if os.path.isfile(child):
imagename, jpg = os.path.splitext(i)
shutil.copy(child, os.path.join(disdir, imagename + ".jpg"))
continue
moveFiles(child, disdir)
if __name__ == '__main__':
rootimage = '/content/drive/MyDrive/stat841_final_data/train/LUAD'
disdir = '/content/drive/MyDrive/stat841_final_data/train_new/LUAD'
moveFiles(rootimage, disdir)
But it does not work. I only got image from the last subfolder except for other subfolders in my new folder train_new/LUAD...
Just to clarify, you want to move (not copy) images from a nested file structure to a new folder, without nesting?
Be aware that this could overwrite images, if multiple images share the same name!
import pathlib
def move_files(source_folder:pathlib.Path, target_folder:pathlib.Path):
target_folder.mkdir(parents=True, exist_ok=True)
for image_file in source_folder.rglob("*.jpg"): # recursively find image paths
image_file.rename(target_folder.joinpath(image_file.name))
If you are unsure maybe use the copy function first, so you won't lose your original data:
import pathlib
import shutil
def move_files(source_folder:pathlib.Path, target_folder:pathlib.Path):
target_folder.mkdir(parents=True, exist_ok=True)
for image_file in source_folder.rglob("*.jpg"): # recursively find image paths
shutil.copy(image_file, target_folder.joinpath(image_file.name))
I have a folder that contains a lot of subfolders, with images saved as png in each folder :
For example :
emotion\angry
emotion\disgusted
emotion\fearful
emotion\happy
I can remove the images in one of the folder using the below code :
folder_path = (r'C:\Users\emotion\angry')
test = os.listdir(folder_path)
for images in test:
if images.endswith(".png"):
os.remove(os.path.join(folder_path, images))
How do I create a loop to loop through each subfolder in emotion/?
As I do not want to manually write out all the code to clear all the folders...
You can list files using glob patterns and delete them using your normal os.remove.
import os
import glob
fileList = glob.glob('C:\Users\emotion\*\*.png')
for filePath in fileList:
try:
os.remove(filePath)
except:
print("Error while deleting file : ", filePath)
def all_dirs(directory):
# Returning list with all directories in selected directory
dir_list = []
for dir in [x[0] for x in os.walk(directory)]:
dir_list.append(dir)
return dir_list
This solution should work for every OS.
import os
import shutil
import glob
i = 0
Source = "L:\\source\\GIS\\Natural_Resource\\Trees\\TreeWatch\\2019\\TreePhotos"
destination = "L:\\source\\Natural Resources\\Tree_Folders\\MULTIPLE_TREE_NAMES\\Photos"
destination_list = os.listdir(Source)
data_dir_list = os.listdir(destination)
for fileName in destination_list:
if not os.path.isdir(os.path.join(Source, fileName)):
for prefix in data_dir_list:
if fileName.startswith(prefix):
shutil.move(os.path.join(Source, fileName),
os.path.join(destination, prefix, fileName))
print (fileName + " copied")`
i=i+1`
print ("Total Photos Copied:", i)
I am trying to take a folder full of photos taken over a year, separate them out to the correct folder based on a number the photo is labeled as, corresponding the the "tree's" number and then once it has moved into the corresponding numbered folder within that folder i have a photo folder that the photos for that specific "tree" will need to be placed in. The code above moves it to the corresponding numbered folder, but I don't know how to get it into the photos folder then to be able to loop back out of it and continue with the other ~600 photos
replace the shutil.move line with:
os.makedirs(os.path.join(destination, prefix, "photos"), exist_ok=True)
shutil.move(os.path.join(Source, fileName),
os.path.join(destination, prefix, "photos", fileName))
And it ought to work. This basically creates a new folder called "photos" if it doesn't already exist, and moves the file there instead. I hope that's what you wanted
I have two folders with images. Let the two folder names A and B. A contains 100 files and B has only 80 files. Both the files have the same name. I want to save only the 80 files from A which has the same correspondence to B in folder C.
Here is a part of my code. However, it is throwing error :
Required argument 'img' (pos 2) not found.
path1= '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/images'
path_mask= '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/masks'
save_path = '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/exp'
for file in os.listdir(path1):
for file1 in os.listdir(path_mask):
img_name = file[:-4]
mask_name =file1[:-4]
if img_name == mask_name:
cv2.imwrite(os.path.join(save_path,img_name))
Your issue here is that you are not passing a file object to cv2.imwrite(os.path.join(save_path,img_name)) when trying to perform the copy; that's what the error is telling you.
However, your current approach includes a nested for loop which will give poor performance. If you only want to know the files that the directories have in common, you can create a set of the file names in each directory and find the intersection. Then you just need to iterate through the common files and copy them over (as said in the comments, there's no need for cv2 here - they may be images but they're just regular files that can be copied).
import os
from shutil import copyfile
dir_1 = 'A'
dir_2 = 'B'
output_dir = 'C'
files_1 = os.listdir(dir_1)
files_2 = os.listdir(dir_2)
# Find the common files between both
common_files = set(files_1).intersection(files_2)
# Copy the common files over.
for file in common_files:
copyfile(os.path.join(dir_1, file),
os.path.join(output_dir, file))
If the reason that you are stripping the last characters from the files in os.listdir is because the files have the same name but different extensions, you only need to make two small modifications (where here I'm assuming the extension is .png that needs to be added back later):
files_1 = [item[:-4] for item in os.listdir(dir_1)]
files_2 = [item[:-4] for item in os.listdir(dir_2)]
And:
for file in common_files:
file = file + '.png' # Add the extension back on to the file name
copyfile(os.path.join(dir_1, file),
os.path.join(output_dir, file))
The any() method returns True if any element of an iterable is True. If not, any() returns False. shutil.copy - Copies the file src to the file or directory dst.
import os
import shutil
def read_file(folderName,folderPath):
''' Return list of files name '''
path = folderPath+folderName
return [file for file in os.listdir(path)]
def save_file(soureFolderName,destFolderName,folderPath,fileName):
''' Save file on destination folder'''
try:
source_path = folderPath+soureFolderName+"/"+fileName
dest_path = folderPath+destFolderName+"/"+fileName
shutil.copy(source_path, dest_path)
except Exception as e:
print(e)
base_path = '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/'
folder_images_files = read_file('images',base_path)
folder_masks_file = read_file('masks',base_path)
for file_1 in folder_images_files:
#Check folder A file is exists in folder B
if any(file_1 == file_2 for file_2 in folder_masks_file):
save_file("images","exp",base_path,file_1)
I need to separate the images in folder with it's filename in python.
for example, i have a folder of images named as,
0_source.jpg, 0_main.jpg,0_cell.jpg, 1_net.jpg, 1_cells.jpg, 2_image.jpg,2_name.jpg
i want to separate these images and save in different folder like:
folder1:
0_source.jpg, 0_main.jpg, 0_cell.jpg
folder 2:
1_net.jpg, 1_cells.jpg
folder 3:
2_image.jpg, 2_name.jpg
I tried to look around, but seems none of them fits what i really needed.
hope someone here could help.
I'm open to any ideas, recommendation and suggestion, thank you.
The main idea is to use a regular expression to extract the folder name starting from the image name. Then build the folders and move the images
import shutil
import re
import os
images = ["0_source.jpg", "0_main.jpg", "0_cell.jpg", "1_net.jpg", "1_cells.jpg", "2_image.jpg", "2_name.jpg"]
def build_images_folders(image_names: [str]):
folder_name_regex = re.compile(r"^(\d)._*") # Regular expression which extract the folder starting from the image_name name
for image_name in image_names:
regex_match = folder_name_regex.match(image_name)
if regex_match:
folder_index = regex_match.group(1) # Extract the folder name
folder_name = "{folder_prefix}{folder_index}".format(folder_prefix="folder_", folder_index=folder_index) # Build the folder name
os.makedirs(folder_name, exist_ok=True) # Build the folder. If already exists, don't raise exception
src_image_path = image_name
dst_image_path = os.path.join(folder_name, image_name)
shutil.move(src_image_path, dst_image_path)
else:
error = "Invalid image name format \"{image_name}\". Expecting <number>_<string>".format(image_name=image_name)
raise ValueError(error)
if __name__ == '__main__':
build_images_folders(images