I am unable to decide on feeding video data to the keras model. I'd like to use a DataGenerator for this case like ImageDataGenerator. From this answer I gather, ImageDataGenerator would not be suitable for this.
I have looked at this github repo for a VideoGenerator in keras which uses .npy files in directories. But the downside is, data augmentation is absent at the moment. How do I go about accomplishing this?
Is there no way I can use ImageDataGenerator?
Supposedly, I split all videos into frames and then load directories with .jpg files instead, how would that fare?
If I write a custom data generator using this data generator tutorial, how do I arrange this partition dict? My data consists of .avi files.
You might find this example helpful.
First you create your tensor of timestep data and then you reshape it for the LSTM network.
Assuming that your dataset consists of sorted frames in that order:
data/
class0/
img001.jpg
img002.jpg
...
class1/
img001.jpg
img002.jpg
...
Related
As an engineer student works towards DSP and ML fields, I am working on an audio classification project with inputs being short clips (4 sec.) of instruments like bass, keyboard, guitar, etc. (NSynth Dataset by the Magenta team at Google).
The idea is to convert all the short clips (.wav files) to spectrograms or melspectrograms then apply a CNN to train the model.
However, my questions is since the entire dataset is large (approximately 23GB), I wonder if I should firstly convert all the audio files to images like PNG then apply CNN. I feel like this can take a lot of time, and it will double the storage space for my input data as now it is audio + image (maybe up to 70GB).
Thus, I wonder if there is any workaround here that can speed the process.
Thanks in advance.
Preprocessing is totally worth it. You will very likely end up, running multiple experiments before your network will work as you want it to and you don't want to waste time pre-processing the features every time, you want to change a few hyper-parameters.
Rather than using PNG, I would rather save directly PyTorch tensors (torch.save that uses Python's standard pickling protocols) or NumPy arrays (numpy.savez saves serialized arrays into a zip file). If you are concerned with disk space, you can consider numpy.save_compressed.
I have data stored in this format
img01_blue.tif
img01_yellow.tif
img01_red.tif
...
imgn_blue.tif
imgn_yellow.tif
imgn_red.tif
with each images being split into 3 images with different channels, indicated by their suffixes.
Now I want to feed them to CNN built by Keras - Python.
Because the data is large and already structured, I feed them in batch by ImageGenerator and flow_from_directory without preprocessing by beforehand.
I want to merge multiple files into 1 single input, each in different channels, can I do that using Keras tool or I have to preprocess the data by other packages first?
The ImageGenerator.flow_from_directory assumes you have single image files. You will have to pre-process your data and merge the files into a single one. If you like to keep the files separate, then you will have to write your own data generator that handles the data you have. But it would wiser to pre-process, here is a post that provides a starting point.
I have saved my large array of images and their labels in HDF5 format using this link:
Saving and loading a large number of images (data) into a single HDF5 file
which gives me following keys.
list of datasets:
['test_img', 'test_labels', 'train_img', 'train_labels', 'train_mean', 'val_img', 'val_labels']
Now, I want to provide training data and training labels to a convolutional neural network (VGG-16 or ResNet) for training purpose and also want to validate and test my result using CNN. How can I input my data from HDF5 file into CNN?
Have a look at this open-source project tftables.
This other thread also has a great detailed answer by mikkola, explaining how to use your own generator with the new Tensorflow Dataset API otherwise.
I want to train a network from data that I have in a Dataframe. I have image paths and classes.
There are many image classes so splitting them into directories is not an option.
I only found beside the flow_from_directory function, the flow function which takes into an argument a numpy array and classes. However, the data will not even fit in RAM.
Is there any solution using Keras ?
Edit 1:
Issue referenced in Github
https://github.com/keras-team/keras/issues/3295
https://github.com/keras-team/keras/issues/3338
I am new to Tensorflow and to implementing deep learning. I have a dataset of images (images of the same object).
I want to train a Neural Network model using python and Tensorflow for object detection.
I am trying to import the data to Tensorflow but I am not sure what is the right way to do it.
Most of the tutorials available online are using public datasets (i.e. MNIST), which importing is straightforward but not helpful in the case where I need to use my own data.
Is there a procedure or tutorial that i can follow?
There are many ways to import images for training, you can use Tensorflow but these will be imported as Tensorflow objects, which you won't be able to visualize until you run the session.
My favorite tool to import images is skimage.io.imread. The imported images will have the dimension (width, height, channels)
Or you can use importing tool from scipy.misc.
To resize images, you can use skimage.transform.resize.
Before training, you will need to normalize all the images to have the values between 0 and 1. To do that, you simply divide the images by 255.
The next step is to one hot encode your labels to be an array of 0s and 1s.
Then you can build and train your CNN.
You could create a data directory containing one subdirectory per image class containing the respective image files and use flow_from_directory of tf.keras.preprocessing.image.ImageDataGenerator.
A tutorial on how to use this can be found in the Keras Blog.