We Keep Coding

Dataset created with tf.data.Dataset.list_files - how can I access the individual file paths while using the map function?

I am trying to build an image classification pipeline based on this tutorial here, starting at "Using tf.data for finer control".
There are some necessary differences between my implementation and the one in the tutorial:
All my images are in one folder (not separated by class in different folders), in the code this folder is called path_to_imagefiles
The information, about the label is stored in a csv-file, I made a df_metadata in my code below that shows how it is structured if I read the csv-file.
I pass my data as a list of absolut paths to the images (in the code called list_of_imagefile_paths) to tf.data.Dataset.list_files.
There is no way around those changes because I require this workflow for my own data later on.
My problem is that I don't know how to change the code to get the labels from df_metadata instead of the file-path (which is how it is done in the tutorial).
I tried to change the get_label(file_path) function from the tutorial the following way:
def get_label(file_path):
parts = tf.strings.split(file_path, os.path.sep) #same as in the tutorial
image_name = parts[-1].numpy() #get the name of the image
df_image = df_metadata[df_metadata.values == image_name] #find the image name in the dataframe
one_hot = df_image.iloc[-1, 3] == class_names #df_image.iloc[-1, 3] gets the label
# Integer encode the label
return tf.argmax(one_hot)
Now the issue is, that image_name = parts[-1].numpy() doesn't seem to result in 'image1.jpg', which I would need to make it work. I get the following error: 'AttributeError: 'Tensor' object has no attribute 'numpy'
How can I access the information stored in df_metadata to get my labels? Any ideas?
Here is the whole code:
import tensorflow as tf
import pandas as pd
import os
import matplotlib.pyplot as plt
def get_label(file_path):
#This is the part where I need help
parts = tf.strings.split(file_path, os.path.sep)
image_name = parts[-1]
df_image = df_metadata[df_metadata.values == image_name]
one_hot = df_image.iloc[-1, 3] == class_names
# Integer encode the label
return tf.argmax(one_hot)
def decode_img(img):
# Convert the compressed string to a 3D uint8 tensor
img = tf.io.decode_jpeg(img, channels=3)
# Resize the image to the desired size
return tf.image.resize(img, [IMG_SIZE, IMG_SIZE])
def process_path(file_path):
label = get_label(file_path)
# Load the raw data from the file as a string
img = tf.io.read_file(file_path)
img = decode_img(img)
return img, label
#setup paths
path_to_imagefiles = r'P:\somefolder\somesubfolder\all_flower_photos'
#setup parameters
IMG_SIZE = 180
AUTOTUNE = tf.data.AUTOTUNE
batch_size = 64
#creating the df_metadata for demonstration purposes, I actually read them from a csv-file
metadata = [['image1.jpg', 100, 200, 'rose'], ['image2.jpg', 100, 200, 'tulip'],
['image3.jpg', 100, 200, 'sunflower'], ['image4.jpg', 100, 200, 'dandelion'],
['image5.jpg', 100, 200, 'daisy'], ['image6.jpg', 100, 200, 'rose']]
df_metadata = pd.DataFrame(metadata)
list_of_imagefile_names = df_metadata.iloc[:,0].to_list()
list_of_imagefile_paths = []
for image_name in list_of_imagefile_names:
list_of_imagefile_paths.append(os.path.join(path_to_imagefiles, image_name))
list_ds = tf.data.Dataset.list_files(list_of_imagefile_paths, shuffle=False)
list_ds = list_ds.shuffle(len(list_ds), reshuffle_each_iteration=False)
class_names = df_metadata.iloc[:,3].unique()
val_size = int(len(list_ds)*0.2)
train_ds = list_ds.skip(val_size)
val_ds = list_ds.take(val_size)
train_ds = train_ds.map(process_path, num_parallel_calls=AUTOTUNE)
val_ds = val_ds.map(process_path, num_parallel_calls=AUTOTUNE)

image_dataset_from_directory using a subset of sub-directories

I have downloaded the MINC dataset for material classification which consists of 23 cateogories. However, I am only interested in a subset of the categories (e.g. [wood, foliage, glass, hair])
Is it possible to get a subset of the data using tf.keras.preprocessing.image_dataset_from_directory?
I have tried tf.keras.preprocessing.image_dataset_from_directory(folder_dir, label_mode="categorical", class_names=["wood", "foliage", "glass", "hair"]) but it give this error The `class_names` passed did not match the names of the subdirectories of the target directory.
Is there a way to get a subset of the directories without deleting or modifying the folders? I know datagen.flow_from_directory is able to do it but keras says that it is deprecated and I should use image_dataset_from_directory.

There are two ways of doing this the first way is to do this by generator, but that process is costly, there is another way of doing this called Using tf.data for finer control. You can check this out at this link
https://www.tensorflow.org/tutorials/load_data/images
But, I will show you a brief demo that how you can load only the folders of your choice.
So, let's start...
#First import some libraries which are needed
import os
import glob
import tensorflow as tf
import matplotlib.pyplot as plt
I am taking only two classes of "Cats" vs "Dogs" you can take more than two classes...
batch_size = 32
img_height = 180
img_width = 180
#define your data directory where your dataset is placed
data_dir = path to your datasetfolder
#Now, here define a list of names for your dataset, like I am only loading cats and dogs... you can fill it with more if you have more
dataset_names = ['cats' , 'dogs']
#Now, glob the list of images in these two directories (cats & Dogs)
list_files = [glob.glob(data_dir + images + '/*.jpg') for images in folders]
list_files = list_files[0] + list_files[1]
image_count = len(list_files)
#Now, here pass this list to a tf.data.Dataset
list_files = tf.data.Dataset.from_tensor_slices(list_files)
#Now, define your class names to labels your dataset later...
class_names = ['cats', 'dogs']
#Now, here define the validation, test, train etc.
val_size = int(image_count * 0.2)
train_ds = list_files.skip(val_size)
val_ds = list_files.take(val_size)
#To get labels
def get_label(file_path):
# Convert the path to a list of path components
parts = tf.strings.split(file_path, os.path.sep)
parts = tf.strings.substr(parts, -4, 4)[0]
one_hot = parts == class_names
# Integer encode the label
return tf.argmax(one_hot)
def decode_img(img):
# Convert the compressed string to a 3D uint8 tensor
img = tf.io.decode_jpeg(img, channels=3)
# Resize the image to the desired size
return tf.image.resize(img, [img_height, img_width])
def process_path(file_path):
label = get_label(file_path)
# Load the raw data from the file as a string
img = tf.io.read_file(file_path)
img = decode_img(img)
return img, label
#Use Dataset.map to create a dataset of image, label pairs:
# Set `num_parallel_calls` so multiple images are loaded/processed in parallel.
train_ds = train_ds.map(process_path, num_parallel_calls=tf.data.AUTOTUNE)
val_ds = val_ds.map(process_path, num_parallel_calls=tf.data.AUTOTUNE)
#Configure dataset for performance
def configure_for_performance(ds):
ds = ds.cache()
ds = ds.shuffle(buffer_size=1000)
ds = ds.batch(batch_size)
ds = ds.prefetch(buffer_size=tf.data.AUTOTUNE)
return ds
train_ds = configure_for_performance(train_ds)
val_ds = configure_for_performance(val_ds)
#Visualize the data
image_batch, label_batch = next(iter(train_ds))
plt.figure(figsize=(10, 10))
for i in range(9):
ax = plt.subplot(3, 3, i + 1)
plt.imshow(image_batch[i].numpy().astype("uint8"))
label = label_batch[i]
plt.title(class_names[label])
plt.axis("off")
Output:
Link to the COLAB file is:
https://colab.research.google.com/drive/1oUNuGVDWDLqwt_YQ80X-CBRL6kJ_YhUX?usp=sharing

Create a mixed data generator (images,csv) in keras

I am building a model with multiple inputs as shown in pyimagesearch, however I can't load all images into RAM and I am trying to create a generator that uses flow_from_directory and get from a CSV file all the extra attributes for each image being processed.
Question: How do I get the attributes from the CSV to correspond with the images in each batch from the image generator?
def get_combined_generator(images_dir, csv_dir, split, *args):
"""
Creates train/val generators on images and csv data.
Arguments:
images_dir : string
Path to a directory with subdirectories for each class.
csv_dir : string
Path to a directory containing train/val csv files with extra attributes.
split : string
Current split being used (train, val or test)
"""
img_width, img_height, batch_size = args
datagen = ImageDataGenerator(
rescale=1. / 255)
generator = datagen.flow_from_directory(
f'{images_dir}/{split}',
target_size=(img_width, img_height),
batch_size=batch_size,
shuffle=True,
class_mode='categorical')
df = pd.read_csv(f'{csv_dir}/{split}.csv', index_col='image')
def my_generator(image_gen, data):
while True:
i = image_gen.batch_index
batch = image_gen.batch_size
row = data[i * batch:(i + 1) * batch]
images, labels = image_gen.next()
yield [images, row], labels
csv_generator = my_generator(generator, df)
return csv_generator

I found a solution based on Luke's answer using a custom generator
import random
import pandas as pd
import numpy as np
from glob import glob
from keras.preprocessing import image as krs_image
# Create the arguments for image preprocessing
data_gen_args = dict(
horizontal_flip=True,
brightness_range=[0.5, 1.5],
shear_range=10,
channel_shift_range=50,
rescale=1. / 255,
)
# Create an empty data generator
datagen = ImageDataGenerator()
# Read the image list and csv
image_file_list = glob(f'{images_dir}/{split}/**/*.JPG', recursive=True)
df = pd.read_csv(f'{csv_dir}/{split}.csv', index_col=csv_data[0])
random.shuffle(image_file_list)
def custom_generator(images_list, dataframe, batch_size):
i = 0
while True:
batch = {'images': [], 'csv': [], 'labels': []}
for b in range(batch_size):
if i == len(images_list):
i = 0
random.shuffle(images_list)
# Read image from list and convert to array
image_path = images_list[i]
image_name = os.path.basename(image_path).replace('.JPG', '')
image = krs_image.load_img(image_path, target_size=(img_height, img_width))
image = datagen.apply_transform(image, data_gen_args)
image = krs_image.img_to_array(image)
# Read data from csv using the name of current image
csv_row = dataframe.loc[image_name, :]
label = csv_row['class']
csv_features = csv_row.drop(labels='class')
batch['images'].append(image)
batch['csv'].append(csv_features)
batch['labels'].append(label)
i += 1
batch['images'] = np.array(batch['images'])
batch['csv'] = np.array(batch['csv'])
# Convert labels to categorical values
batch['labels'] = np.eye(num_classes)[batch['labels']]
yield [batch['images'], batch['csv']], batch['labels']

I would suggest creating a custom generator given this relatively specific case. Something like the following (modified from a similar answer here) should suffice:
import os
import random
import pandas as pd
def generator(image_dir, csv_dir, batch_size):
i = 0
image_file_list = os.listdir(image_dir)
while True:
batch_x = {'images': list(), 'other_feats': list()} # use a dict for multiple inputs
batch_y = list()
for b in range(batch_size):
if i == len(image_file_list):
i = 0
random.shuffle(image_file_list)
sample = image_file_list[i]
image_file_path = sample[0]
csv_file_path = os.path.join(csv_dir,
os.path.basename(image_file_path).replace('.png', '.csv'))
i += 1
image = preprocess_image(cv2.imread(image_file_path))
csv_file = pd.read_csv(csv_file_path)
other_feat = preprocess_feats(csv_file)
batch_x['images'].append(image)
batch_x['other_feats'].append(other_feat)
batch_y.append(csv_file.loc[image_name, :]['class'])
batch_x['images'] = np.array(batch_x['images']) # convert each list to array
batch_x['other_feats'] = np.array(batch_x['other_feats'])
batch_y = np.eye(num_classes)[batch['labels']]
yield batch_x, batch_y
Then, you can use Keras's fit_generator() function to train your model.
Obviously, this assumes you have csv files with the same names as your image files, and that you have some custom preprocessing functions for images and csv files.

Create tensorflow dataset from image local directory

I have a very huge database of images locally, with the data distribution like each folder cointains the images of one class.
I would like to use the tensorflow dataset API to obtain batches de data without having all the images loaded in memory.
I have tried something like this:
def _parse_function(filename, label):
image_string = tf.read_file(filename, "file_reader")
image_decoded = tf.image.decode_jpeg(image_string, channels=3)
image = tf.cast(image_decoded, tf.float32)
return image, label
image_list, label_list, label_map_dict = read_data()
dataset = tf.data.Dataset.from_tensor_slices((tf.constant(image_list), tf.constant(label_list)))
dataset = dataset.shuffle(len(image_list))
dataset = dataset.repeat(epochs).batch(batch_size)
dataset = dataset.map(_parse_function)
iterator = dataset.make_one_shot_iterator()
image_list is a list where the path (and name) of the images have been appended and label_list is a list where the class of each image has been appended in the same order.
But the _parse_function does not work, the error that I recibe is:
ValueError: Shape must be rank 0 but is rank 1 for 'file_reader' (op: 'ReadFile') with input shapes: [?].
I have googled the error, but nothing works for me.
If I do not use the map function, I just recibe the path of the images (which are store in image_list), so I think that I need the map function to read the images, but I am not able to make it works.
Thank you in advance.
EDIT:
def read_data():
image_list = []
label_list = []
label_map_dict = {}
count_label = 0
for class_name in os.listdir(base_path):
class_path = os.path.join(base_path, class_name)
label_map_dict[class_name]=count_label
for image_name in os.listdir(class_path):
image_path = os.path.join(class_path, image_name)
label_list.append(count_label)
image_list.append(image_path)
count_label += 1

The error is in this line dataset = dataset.repeat(epochs).batch(batch_size) Your pipeline adds batchsize as a dimension to input.
You need to batch your dataset after map function like this
dataset = tf.data.Dataset.from_tensor_slices((tf.constant(image_list), tf.constant(label_list)))
dataset = dataset.shuffle(len(image_list))
dataset = dataset.repeat(epochs)
dataset = dataset.map(_parse_function).batch(batch_size)

Tensorflow tutorial on MNIST

This Tensorflow tutorial loads an already existing dataset (MNIST) into the code. Instead of that I want to insert my own training and testing images.
def main(unused_argv):
# Load training and eval data
mnist = tf.contrib.learn.datasets.load_dataset("mnist")
train_data = mnist.train.images # Returns np.array
train_labels = np.asarray(mnist.train.labels, dtype=np.int32)
eval_data = mnist.test.images # Returns np.array
eval_labels = np.asarray(mnist.test.labels, dtype=np.int32)
It says it returns an np array of raw pixel values.
My question:
1. How do I create such a numpy array for my own image set?
I want to do this so I can directly substitute my numpy array instead of this MNIST data in the sample code and train the model on my data (0-9 and A-Z).
EDIT: On further analysis, I've realized that the pixel values in mnist.train.images and mnist.test.images have been normalized between 0 to 1 from 0 to 255 ( I suppose) How does this normalization help?
Folder structure: Training and testing folder are in the same folder
Training folder:
--> 0
-->Image_Of_0.png
--> 1
-->Image_Of_1.png
.
.
.
--> Z
-->Image_Of_Z.png
Testing folder:
--> 0
-->Image_Of_0.png
--> 1
-->Image_Of_1.png
.
.
.
--> Z
-->Image_Of_Z.png
Code I wrote:
Names = [['C:\\Users\\xx\\Project\\training-images', 'train',9490], ['C:\\Users\\xx\\Project\\test-images', 'test',3175]]
#9490 is the number of training files in total (All the PNGs)
#3175 is the number of testing files in total (All the PNGs)
for name in Names:
FileList = []
for dirname in os.listdir(name[0]):
path = os.path.join(name[0], dirname)
for filename in os.listdir(path):
if filename.endswith(".png"):
FileList.append(os.path.join(name[0], dirname, filename))
print(FileList)
## Creates list of all PNG files in training and testing folder
x_data = np.array([np.array(cv2.imread(filename)) for filename in FileList])
pixels = x_data.flatten().reshape(name[2], 2352) #2352 = 28 * 28 * 3 image
print(pixels)
Can the pixels array created be supplied as the training and testing data i.e would it have the same format as the data being supplied in the sample code?
2. Similarly what numpy array must be created for all the labels? (Folder names)

1. How do I create such a numpy array for my own image set?
TensorFlow accepts data in multiple ways (tf.data, feed_dict, QueueRunner).
What you should be using is TFRecord which is accessible via tf.data API. It is also recommended format. Lets say you have folder containing images and you want to convert it to tfrecord file.
import tensorflow as tf
import numpy as np
import glob
from PIL import Image
# Converting the values into features
# _int64 is used for numeric values
def _int64_feature(value):
return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))
# _bytes is used for string/char values
def _bytes_feature(value):
return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
tfrecord_filename = 'something.tfrecords'
# Initiating the writer and creating the tfrecords file.
writer = tf.python_io.TFRecordWriter(tfrecord_filename)
# Loading the location of all files - image dataset
# Considering our image dataset has apple or orange
# The images are named as apple01.jpg, apple02.jpg .. , orange01.jpg .. etc.
images = glob.glob('data/*.jpg')
for image in images[:1]:
img = Image.open(image)
img = np.array(img.resize((32,32)))
label = 0 if 'apple' in image else 1
feature = { 'label': _int64_feature(label),
'image': _bytes_feature(img.tostring()) }
#create an example protocol buffer
example = tf.train.Example(features=tf.train.Features(feature=feature))
#writing the serialized example.
writer.write(example.SerializeToString())
writer.close()
Now to read this tfrecord file and do stuff
import tensorflow as tf
import glob
reader = tf.TFRecordReader()
filenames = glob.glob('*.tfrecords')
filename_queue = tf.train.string_input_producer(
filenames)
_, serialized_example = reader.read(filename_queue)
feature_set = { 'image': tf.FixedLenFeature([], tf.string),
'label': tf.FixedLenFeature([], tf.int64)
}
features = tf.parse_single_example( serialized_example, features= feature_set )
label = features['label']
with tf.Session() as sess:
print sess.run([image,label])
Here's an example for MNIST in tensorflow/examples
Cheers!