We Keep Coding

Dataset created with tf.data.Dataset.list_files - how can I access the individual file paths while using the map function?

I am trying to build an image classification pipeline based on this tutorial here, starting at "Using tf.data for finer control".
There are some necessary differences between my implementation and the one in the tutorial:
All my images are in one folder (not separated by class in different folders), in the code this folder is called path_to_imagefiles
The information, about the label is stored in a csv-file, I made a df_metadata in my code below that shows how it is structured if I read the csv-file.
I pass my data as a list of absolut paths to the images (in the code called list_of_imagefile_paths) to tf.data.Dataset.list_files.
There is no way around those changes because I require this workflow for my own data later on.
My problem is that I don't know how to change the code to get the labels from df_metadata instead of the file-path (which is how it is done in the tutorial).
I tried to change the get_label(file_path) function from the tutorial the following way:
def get_label(file_path):
parts = tf.strings.split(file_path, os.path.sep) #same as in the tutorial
image_name = parts[-1].numpy() #get the name of the image
df_image = df_metadata[df_metadata.values == image_name] #find the image name in the dataframe
one_hot = df_image.iloc[-1, 3] == class_names #df_image.iloc[-1, 3] gets the label
# Integer encode the label
return tf.argmax(one_hot)
Now the issue is, that image_name = parts[-1].numpy() doesn't seem to result in 'image1.jpg', which I would need to make it work. I get the following error: 'AttributeError: 'Tensor' object has no attribute 'numpy'
How can I access the information stored in df_metadata to get my labels? Any ideas?
Here is the whole code:
import tensorflow as tf
import pandas as pd
import os
import matplotlib.pyplot as plt
def get_label(file_path):
#This is the part where I need help
parts = tf.strings.split(file_path, os.path.sep)
image_name = parts[-1]
df_image = df_metadata[df_metadata.values == image_name]
one_hot = df_image.iloc[-1, 3] == class_names
# Integer encode the label
return tf.argmax(one_hot)
def decode_img(img):
# Convert the compressed string to a 3D uint8 tensor
img = tf.io.decode_jpeg(img, channels=3)
# Resize the image to the desired size
return tf.image.resize(img, [IMG_SIZE, IMG_SIZE])
def process_path(file_path):
label = get_label(file_path)
# Load the raw data from the file as a string
img = tf.io.read_file(file_path)
img = decode_img(img)
return img, label
#setup paths
path_to_imagefiles = r'P:\somefolder\somesubfolder\all_flower_photos'
#setup parameters
IMG_SIZE = 180
AUTOTUNE = tf.data.AUTOTUNE
batch_size = 64
#creating the df_metadata for demonstration purposes, I actually read them from a csv-file
metadata = [['image1.jpg', 100, 200, 'rose'], ['image2.jpg', 100, 200, 'tulip'],
['image3.jpg', 100, 200, 'sunflower'], ['image4.jpg', 100, 200, 'dandelion'],
['image5.jpg', 100, 200, 'daisy'], ['image6.jpg', 100, 200, 'rose']]
df_metadata = pd.DataFrame(metadata)
list_of_imagefile_names = df_metadata.iloc[:,0].to_list()
list_of_imagefile_paths = []
for image_name in list_of_imagefile_names:
list_of_imagefile_paths.append(os.path.join(path_to_imagefiles, image_name))
list_ds = tf.data.Dataset.list_files(list_of_imagefile_paths, shuffle=False)
list_ds = list_ds.shuffle(len(list_ds), reshuffle_each_iteration=False)
class_names = df_metadata.iloc[:,3].unique()
val_size = int(len(list_ds)*0.2)
train_ds = list_ds.skip(val_size)
val_ds = list_ds.take(val_size)
train_ds = train_ds.map(process_path, num_parallel_calls=AUTOTUNE)
val_ds = val_ds.map(process_path, num_parallel_calls=AUTOTUNE)

How to load images and text labels for CNN regression from different folders

I have two folders, X_train and Y_train. X_train is images, Y_train is vector and .txt files. I try to train CNN for regression.
I could not figure out how to take data and train the network. When i use "ImageDataGenerator" , it suppose that X_train and Y_train folders are classes.
import os
import tensorflow as tf
os.chdir(r'C:\\Data')
from glob2 import glob
x_files = glob('X_train\\*.jpg')
y_files = glob('Y_rain\\*.txt')
Above, i found destination of them, how can i take them and be ready for model.fit ? Thank you.

Makes sure x_files and y_files are sorted together, then you can use something like this:
import tensorflow as tf
from glob2 import glob
import os
x_files = glob('X_train\\*.jpg')
y_files = glob('Y_rain\\*.txt')
target_names = ['cat', 'dog']
files = tf.data.Dataset.from_tensor_slices((x_files, y_files))
imsize = 128
def get_label(file_path):
label = tf.io.read_file(file_path)
return tf.cast(label == target_names, tf.int32)
def decode_img(img):
img = tf.image.decode_jpeg(img, channels=3)
img = tf.image.convert_image_dtype(img, tf.float32)
img = tf.image.resize(images=img, size=(imsize, imsize))
return img
def process_path(file_path):
label = get_label(file_path)
img = tf.io.read_file(file_path)
img = decode_img(img)
return img, label
train_ds = files.map(process_path).batch(32)
Then, train_ds can be passed to model.fit() and will return batches of 32 pairs of images, labels.

How to build a Custom Data Generator for Keras/tf.Keras where X images are being augmented and corresponding Y labels are also images

I am working on Image Binarization using UNet and have a dataset of 150 images and their binarized versions too. My idea is to augment the images randomly to make them look like they are differentso I have made a function which inserts any of the 4-5 types of Noises, skewness, shearing and so on to an image. I could have easily used
ImageDataGenerator(preprocess_function=my_aug_function) to augment the images but the problem is that my y target is also an image. Also, I could have used something like:
train_dataset = (
train_dataset.map(
encode_single_sample, num_parallel_calls=tf.data.experimental.AUTOTUNE
)
.batch(batch_size)
.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)
)
But it has 2 problems:
With larger dataset, it'll blow up the memory as data needs to be already in the memory
This is the crucial part that I need to augment the images on the go to make it look like I have a huge dataset.
Another Solution could be saving augmented images to a directory and making them 30-40K and then loading them. It would be silly thing to do.
Now the idea part is that I can use Sequence as the parent class but How can I keep on augmenting and generating new images on the fly with respective Y binarized images?
I have an idea as the below code. Can somebody help me with the augmentation and generation of y images. I have my X_DIR, Y_DIR where image names for binarised and original are same but stored in different directories.
class DataGenerator(tensorflow.keras.utils.Sequence):
def __init__(self, files_path, labels_path, batch_size=32, shuffle=True, random_state=42):
'Initialization'
self.files = files_path
self.labels = labels_path
self.batch_size = batch_size
self.shuffle = shuffle
self.random_state = random_state
self.on_epoch_end()
def on_epoch_end(self):
'Updates indexes after each epoch'
# Shuffle the data here
def __len__(self):
return int(np.floor(len(self.files) / self.batch_size))
def __getitem__(self, index):
# What do I do here?
def __data_generation(self, files):
# I think this is responsible for Augmentation but no idea how should I implement it and how does it works.

Custom Image Data Generator
load Directory data into dataframe for CustomDataGenerator
def data_to_df(data_dir, subset=None, validation_split=None):
df = pd.DataFrame()
filenames = []
labels = []
for dataset in os.listdir(data_dir):
img_list = os.listdir(os.path.join(data_dir, dataset))
label = name_to_idx[dataset]
for image in img_list:
filenames.append(os.path.join(data_dir, dataset, image))
labels.append(label)
df["filenames"] = filenames
df["labels"] = labels
if subset == "train":
split_indexes = int(len(df) * validation_split)
train_df = df[split_indexes:]
val_df = df[:split_indexes]
return train_df, val_df
return df
train_df, val_df = data_to_df(train_dir, subset="train", validation_split=0.2)
Custom Data Generator
import tensorflow as tf
from PIL import Image
import numpy as np
class CustomDataGenerator(tf.keras.utils.Sequence):
''' Custom DataGenerator to load img
Arguments:
data_frame = pandas data frame in filenames and labels format
batch_size = divide data in batches
shuffle = shuffle data before loading
img_shape = image shape in (h, w, d) format
augmentation = data augmentation to make model rebust to overfitting
Output:
Img: numpy array of image
label : output label for image
'''
def __init__(self, data_frame, batch_size=10, img_shape=None, augmentation=True, num_classes=None):
self.data_frame = data_frame
self.train_len = len(data_frame)
self.batch_size = batch_size
self.img_shape = img_shape
self.num_classes = num_classes
print(f"Found {self.data_frame.shape[0]} images belonging to {self.num_classes} classes")
def __len__(self):
''' return total number of batches '''
self.data_frame = shuffle(self.data_frame)
return math.ceil(self.train_len/self.batch_size)
def on_epoch_end(self):
''' shuffle data after every epoch '''
# fix on epoch end it's not working, adding shuffle in len for alternative
pass
def __data_augmentation(self, img):
''' function for apply some data augmentation '''
img = tf.keras.preprocessing.image.random_shift(img, 0.2, 0.3)
img = tf.image.random_flip_left_right(img)
img = tf.image.random_flip_up_down(img)
return img
def __get_image(self, file_id):
""" open image with file_id path and apply data augmentation """
img = np.asarray(Image.open(file_id))
img = np.resize(img, self.img_shape)
img = self.__data_augmentation(img)
img = preprocess_input(img)
return img
def __get_label(self, label_id):
""" uncomment the below line to convert label into categorical format """
#label_id = tf.keras.utils.to_categorical(label_id, num_classes)
return label_id
def __getitem__(self, idx):
batch_x = self.data_frame["filenames"][idx * self.batch_size:(idx + 1) * self.batch_size]
batch_y = self.data_frame["labels"][idx * self.batch_size:(idx + 1) * self.batch_size]
# read your data here using the batch lists, batch_x and batch_y
x = [self.__get_image(file_id) for file_id in batch_x]
y = [self.__get_label(label_id) for label_id in batch_y]
return tf.convert_to_tensor(x), tf.convert_to_tensor(y)

You can use libraries like albumentations and imgaug, both are good but I have heard there are issues with random seed with albumentations.
Here's an example of imgaug taken from the documentation here:
seq = iaa.Sequential([
iaa.Dropout([0.05, 0.2]), # drop 5% or 20% of all pixels
iaa.Sharpen((0.0, 1.0)), # sharpen the image
iaa.Affine(rotate=(-45, 45)), # rotate by -45 to 45 degrees (affects segmaps)
iaa.ElasticTransformation(alpha=50, sigma=5) # apply water effect (affects segmaps)
], random_order=True)
# Augment images and segmaps.
images_aug = []
segmaps_aug = []
for _ in range(len(input_data)):
images_aug_i, segmaps_aug_i = seq(image=image, segmentation_maps=segmap)
images_aug.append(images_aug_i)
segmaps_aug.append(segmaps_aug_i)
You are going in the right way with the custom generator. In __getitem__, make a batch using batch_x = self.files[index:index+batch_size] and same with batch_y, then augment them using X,y = __data_generation(batch_x, batch_y) which will load images(using any library you like, I prefer opencv), and return the augmented pairs (and any other manipulation).
Your __getitem__ will then return the tuple (X,y)

You can use ImageDataGenerator even if your label is an image.
Here is a simple example of how you can do that:
Code:
# Specifying your data augmentation here for both image and label
image_datagen = tf.keras.preprocessing.image.ImageDataGenerator()
mask_datagen = tf.keras.preprocessing.image.ImageDataGenerator()
# Provide the same seed and keyword arguments to the flow methods
seed = 1
image_generator = image_datagen.flow_from_directory(
data_dir,
class_mode=None,
seed=seed)
mask_generator = mask_datagen.flow_from_directory(
data_dir,
class_mode=None,
seed=seed)
# Combine the image and label generator.
train_generator = zip(image_generator, mask_generator)
Now, if you iterate over it you will get:
for image, label in train_generator:
print(image.shape,label.shape)
break
Output:
(32, 256, 256, 3) (32, 256, 256, 3)
You can use this train_generator with fit() command.
Code:
model.fit_generator(
train_generator,
steps_per_epoch=2000,
epochs=50)
With flow_from_directory your memory won't be cluttered and Imagedatagenerator will take care of the augmentation part.

Create a mixed data generator (images,csv) in keras

I am building a model with multiple inputs as shown in pyimagesearch, however I can't load all images into RAM and I am trying to create a generator that uses flow_from_directory and get from a CSV file all the extra attributes for each image being processed.
Question: How do I get the attributes from the CSV to correspond with the images in each batch from the image generator?
def get_combined_generator(images_dir, csv_dir, split, *args):
"""
Creates train/val generators on images and csv data.
Arguments:
images_dir : string
Path to a directory with subdirectories for each class.
csv_dir : string
Path to a directory containing train/val csv files with extra attributes.
split : string
Current split being used (train, val or test)
"""
img_width, img_height, batch_size = args
datagen = ImageDataGenerator(
rescale=1. / 255)
generator = datagen.flow_from_directory(
f'{images_dir}/{split}',
target_size=(img_width, img_height),
batch_size=batch_size,
shuffle=True,
class_mode='categorical')
df = pd.read_csv(f'{csv_dir}/{split}.csv', index_col='image')
def my_generator(image_gen, data):
while True:
i = image_gen.batch_index
batch = image_gen.batch_size
row = data[i * batch:(i + 1) * batch]
images, labels = image_gen.next()
yield [images, row], labels
csv_generator = my_generator(generator, df)
return csv_generator

I found a solution based on Luke's answer using a custom generator
import random
import pandas as pd
import numpy as np
from glob import glob
from keras.preprocessing import image as krs_image
# Create the arguments for image preprocessing
data_gen_args = dict(
horizontal_flip=True,
brightness_range=[0.5, 1.5],
shear_range=10,
channel_shift_range=50,
rescale=1. / 255,
)
# Create an empty data generator
datagen = ImageDataGenerator()
# Read the image list and csv
image_file_list = glob(f'{images_dir}/{split}/**/*.JPG', recursive=True)
df = pd.read_csv(f'{csv_dir}/{split}.csv', index_col=csv_data[0])
random.shuffle(image_file_list)
def custom_generator(images_list, dataframe, batch_size):
i = 0
while True:
batch = {'images': [], 'csv': [], 'labels': []}
for b in range(batch_size):
if i == len(images_list):
i = 0
random.shuffle(images_list)
# Read image from list and convert to array
image_path = images_list[i]
image_name = os.path.basename(image_path).replace('.JPG', '')
image = krs_image.load_img(image_path, target_size=(img_height, img_width))
image = datagen.apply_transform(image, data_gen_args)
image = krs_image.img_to_array(image)
# Read data from csv using the name of current image
csv_row = dataframe.loc[image_name, :]
label = csv_row['class']
csv_features = csv_row.drop(labels='class')
batch['images'].append(image)
batch['csv'].append(csv_features)
batch['labels'].append(label)
i += 1
batch['images'] = np.array(batch['images'])
batch['csv'] = np.array(batch['csv'])
# Convert labels to categorical values
batch['labels'] = np.eye(num_classes)[batch['labels']]
yield [batch['images'], batch['csv']], batch['labels']

I would suggest creating a custom generator given this relatively specific case. Something like the following (modified from a similar answer here) should suffice:
import os
import random
import pandas as pd
def generator(image_dir, csv_dir, batch_size):
i = 0
image_file_list = os.listdir(image_dir)
while True:
batch_x = {'images': list(), 'other_feats': list()} # use a dict for multiple inputs
batch_y = list()
for b in range(batch_size):
if i == len(image_file_list):
i = 0
random.shuffle(image_file_list)
sample = image_file_list[i]
image_file_path = sample[0]
csv_file_path = os.path.join(csv_dir,
os.path.basename(image_file_path).replace('.png', '.csv'))
i += 1
image = preprocess_image(cv2.imread(image_file_path))
csv_file = pd.read_csv(csv_file_path)
other_feat = preprocess_feats(csv_file)
batch_x['images'].append(image)
batch_x['other_feats'].append(other_feat)
batch_y.append(csv_file.loc[image_name, :]['class'])
batch_x['images'] = np.array(batch_x['images']) # convert each list to array
batch_x['other_feats'] = np.array(batch_x['other_feats'])
batch_y = np.eye(num_classes)[batch['labels']]
yield batch_x, batch_y
Then, you can use Keras's fit_generator() function to train your model.
Obviously, this assumes you have csv files with the same names as your image files, and that you have some custom preprocessing functions for images and csv files.

tensorflow - how to input a directory by dataset api

I am new to tensorflow, and here is my situation: I have lots of folders and each contains several images. I need my training input to be folders(each time 2 folders), and each time 4 images inside a folder be selected for training.
I have tried Dataset api, and tried to use the map or flat_map function, but I failed to read images inside a folder.
Here is part of my codes:
def parse_function(filename):
print(filename)
batch_data = []
batch_label = []
dir_path = os.path.join(data_path, str(filename))
imgs_list = os.listdir(dir_path)
random.shuffle(imgs_list)
imgs_list = imgs_list * 4 #each time select 4 images
for i in range(img_num):
img_path = os.path.join(dir_path, imgs_list[i])
image_string = tf.read_file(img_path)
image_decoded = tf.image.decode_image(image_string)
image_resized = tf.image.resize_images(image_decoded, [224, 224])
batch_data.append(image_resized)
batch_label.append(label)
return batch_data, batch_label
dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))
dataset = dataset.map(_parse_function)
where filename is a list of folder name like '123456', labels is list of label like 0 or 1.