I have been getting this error and cannot figure it out.
ValueError: Cannot reshape a tensor with 48032 elements to shape [32,1] (32 elements) for 'Reshape' (op: 'Reshape') with input shapes: [32,1501], [2] and with input tensors computed as partial shapes: input[1] = [32,1].
What I am doing is trying to use a tripleloss function from the tensorflow_addons library using an example from here
https://www.tensorflow.org/addons/tutorials/losses_triplet
I pretty much copied it and change the data.
My data set contains 1501 different classes separated into folders for each class. I am using a data generator from tf.data.Dataset which seems to work fine too.
This is what I have
BATCH_SIZE = 32
train_datagen = ImageDataGenerator(
preprocessing_function=preprocess_input,
shear_range=0,
rotation_range=20,
zoom_range=0.15,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
ds = tf.data.Dataset.from_generator(generator=train_datagen.flow_from_directory,
args=[train_dir, (224, 224), 'categorical'],
output_types=(tf.float32, tf.float32),
output_shapes=([32, 224,224,3], [32,1501]))
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(filters=64, kernel_size=2, padding='same', activation='relu', input_shape=(224, 224, 3)),
tf.keras.layers.MaxPooling2D(pool_size=2),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Conv2D(filters=32, kernel_size=2, padding='same', activation='relu'),
tf.keras.layers.MaxPooling2D(pool_size=2),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(256, activation=None), # No activation on final dense layer
tf.keras.layers.Lambda(lambda x: tf.math.l2_normalize(x, axis=1)) # L2 normalize embeddings
])
model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
loss=tfa.losses.TripletSemiHardLoss())
history = model.fit(ds, epochs=45, verbose=1, callbacks=None)
Its pretty much a verbatim copy other than the dataset.
Do I have to make a map function like ds.map(function)?
exactly the problem I ran into.
As it is stated in https://www.tensorflow.org/addons/api_docs/python/tfa/losses/TripletSemiHardLoss
"We expect labels y_true to be provided as 1-D integer Tensor with shape [batch_size] of multi-class integer labels."
ImageDataGenerator produces [batchsize,nclasses] tensor which has to be preprocessed to be fed into TripletSemiHardLoss.
I personally just made own trainig function instead of model.fit:
for e in range(EPOCHS):
print('Epoch', e)
for b in range(int(STEPS_PER_EPOCH)):
batch=train_data_gen.next()
x_batch=batch[0]
y_batch=np.argmax(batch[1],axis=1) # <- class labels: y_true: 1-D integer
history=model.fit(x_batch, y_batch) # 1 step fit
print(e,b)
which does train the model, however at the moment I'm strugling with the loss value, which is random between 0 and 1 at every step. Must be the gradients being lost. Looking into it.
Edit1:
actually, this thing works:
image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)
train_data_gen = image_generator.flow_from_directory(directory=str(data_dir),
batch_size=BATCH_SIZE,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH),
classes = list(CLASS_NAMES),
color_mode='grayscale',
class_mode='sparse')
model.fit(train_data_gen, epochs=10)
The magic was to use class_mode='sparse'
Related
I created a CNN using Tensorflow to identify pneumonia and sometimes it returns a very small number as a prediction. why is this happening?
I have attached the link for the dataset
Here I how I process and load the data.
from tensorflow.keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator( rescale = 1.0/255. )
val_datagen = ImageDataGenerator( rescale = 1.0/255. )
test_datagen = ImageDataGenerator( rescale = 1.0/255. )
train_generator = train_datagen.flow_from_directory('/kaggle/input/chest-xray-pneumonia/chest_xray/chest_xray/train/',
batch_size=20,
class_mode='binary',
target_size=(350, 350))
validation_generator = val_datagen.flow_from_directory('/kaggle/input/chest-xray-pneumonia/chest_xray/chest_xray/val/',
batch_size=20,
class_mode = 'binary',
target_size = (350, 350))
test_generator = test_datagen.flow_from_directory('/kaggle/input/chest-xray-pneumonia/chest_xray/chest_xray/test/',
batch_size=20,
class_mode = 'binary',
target_size = (350, 350
And here the Model, compile and fit functions
import tensorflow as tf
model = tf.keras.models.Sequential([
# Note the input shape is the desired size of the image 150x150 with 3 bytes color
tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(350, 350, 3)),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
# Flatten the results to feed into a DNN
tf.keras.layers.Flatten(),
# 512 neuron hidden layer
tf.keras.layers.Dense(1024, activation='relu'),
# Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')
tf.keras.layers.Dense(1, activation='sigmoid')
])
compile the model
from tensorflow.keras.optimizers import RMSprop
model.compile(optimizer=RMSprop(learning_rate=0.001),
loss='binary_crossentropy',
metrics = ['accuracy'])
model fit
history = model.fit(train_generator,
validation_data=validation_generator,
steps_per_epoch=200,
epochs=2000,
validation_steps=200,
callbacks=[callbacks],
verbose=2)
The evaluation metrics as followings, loss: 0.2351 - accuracy: 0.9847
The prediction shows a very small number for the negative pneumonia, and for positive it shows more than .50.
I have two questions:
why I get a very small number as 2.xxxx * 10e-20?
why I can't get the following values as null?
val_acc = history.history[ 'val_accuracy' ]
val_loss = history.history['val_loss' ]
I see that there is no problem with your code, neither with the results you get.
This is a binary classification problem (2 classes: Positive or negative pneumonia), and the output of your model is one neurone giving values between 0 and 1.
So if the output is higher than 0.5, this means positive pneumonia. Otherwise, when you have a very small value like 2 * 10e-20 this means that it's negative pneumonia.
For your second question, you are not supposed to have accuracy and loss values to be null simply because the model is well trained and has 98% accuracy on training data.
I'm doing a ML model that takes pixel values from a numpy array as training and testing data. I defined a function that divides the dataset into images and labels. My task is to use Image Generator for data augmentation and then train the model. Everything goes smoothly until I am to train the model. It keeps giving me errors about the loss function used. When I use categorical_crossentropy it says I can either use 'sparse_categorical_crossentropy' or use function to_categorical. Well I tried both and there were still errors so I decided to try and use tf.convert_to_tensor() on my labels but now I get a shape error:
ValueError: A target array with shape (126, 25, 2) was passed for an output of shape (None, 3) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.
This is my code:
training_labels = tf.convert_to_tensor(training_labels)
testing_labels = tf.convert_to_tensor(testing_labels)
# Create an ImageDataGenerator and do Image Augmentation
train_datagen = ImageDataGenerator(
rescale = 1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest'
)
validation_datagen = ImageDataGenerator(rescale = 1./255)
train_generator = train_datagen.flow(training_images,
training_labels,
batch_size=126
)
validation_generator = validation_datagen.flow(
testing_images,
testing_labels,
batch_size=126
)
# Keep These
print(training_images.shape)
print(testing_images.shape)
# Their output should be:
# (27455, 28, 28, 1)
# (7172, 28, 28, 1)
And here goes the model:
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(28, 28, 1)),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(3, activation='softmax')
])
# Compile Model.
model.compile(loss = 'categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
# Train the Model
history = model.fit_generator(train_generator, validation_data=validation_generator, epochs=2)
model.evaluate(testing_images, testing_labels, verbose=0)
I got stuck with it, I googled for solution but with no success. Can you please help me somehow make a move?
Thanks a lot!
When using categorical cross entropy as the loss function, the labels should be one hot encoded and hence the number of neurons present in the final layer should be equal to the number of classes present in the dataset and that's the error you are getting. Since the number of output neurons is 3, I'm guessing you have 3 classes and hence the shape of training_labels/testing_labels should be (num of images in train/test, 3).
Below is a small snippet for cifar dataset.
from tf.keras.utils import to_categorical
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
num_classes = 10
# convert to one hot encoding
# shape will be (60000, 10) since there are 60000 images and 10 classes in cifar
y_train = to_categorical(y_train, num_classes)
# shape will be (10000, 10) since there are 10000 images and 10 classes in cifar
y_test = to_categorical(y_test, num_classes)
datagen = ImageDataGenerator(
featurewise_center=True,
featurewise_std_normalization=True,
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True)
# compute quantities required for featurewise normalization
# (std, mean, and principal components if ZCA whitening is applied)
datagen.fit(x_train)
history = model.fit_generator(datagen.flow(x_train, y_train, batch_size=32), epochs=2)
I am implementing resnet-50 on Kaggle and I am getting a value error. Kindly help me out
train_dir='../input/project/data/train'
test_dir='../input/project/data/test'
train_datagen=ImageDataGenerator(rescale=1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
test_datagen = ImageDataGenerator(rescale = 1./255)
train_generator = train_datagen.flow_from_directory(
train_dir,
color_mode='grayscale',
target_size=(28,28),
class_mode='binary',
batch_size=32,
)
test_generator = test_datagen.flow_from_directory(
test_dir,
color_mode='grayscale',
target_size=(28,28),
class_mode='binary',
batch_size=32,
shuffle='False',
)
model = Sequential()
model.add(ResNet50(include_top=False, pooling='avg', weights=resnet_weights_path,input_tensor=Input(shape=(224,224,3))))
model.add(Flatten())
model.add(BatchNormalization())
model.add(Dense(2048, activation='relu'))
model.add(BatchNormalization())
model.add(Dense(1024, activation='relu'))
model.add(BatchNormalization())
model.add(Dense(2, activation='sigmoid'))
model.layers[0].trainable = False
I am training a binary classifier and I am getting the error below
ValueError: Cannot assign to variable conv3_block1_0_conv/kernel:0 due to variable shape (1, 1, 256, 512) and value shape (512, 128, 1, 1) are incompatible
You have given input_tensor=Input(shape=(224,224,3)) while defining the ResNet50 base model. But you are giving target_size=(28,28) in your train_generator and test_generator. The training image shape which ResNet50 receiving i.e. target_size is different from what it expects i.e. input_tensor. Change your target_size to match with the shape mentioned in the input_tensor. Also, ResNet50 expects color_mode to be rgb rather grayscale.
This is because of weights which you are using(weights=resnet_weights_path). You've to use latest trained model. Input Image size can be any as per pre-trained model guidelines
Below worked for me
n_h, n_w, n_c =(256, 256, 3)
weights_path = '../input/d/aeryss/keras-pretrained- models/ResNet50_NoTop_ImageNet.h5'
ResNet50 = keras.applications.ResNet50(weights=weights_path ,include_top=False, input_shape=(n_h, n_w, n_c))
It looks like you are using pre-trained weights on your model. You should set the skip_mismatch=True keyword on the model.load_weights function. After returning the model variable, please set the following code:
model.load_weights(weights, by_name=True, skip_mismatch=True)
Where weight is your pre-trained model weight. It should ignore any mismatch of your model and the pre-trained weights.
just starting out with ML, and wanted to create my own CNN to detect orientation of images with faces. I followed a tutorial to accept input images of 64x64x1, and here is my code:
from keras.models import Model
from keras.layers import Input
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers.convolutional import Conv2D
from keras.layers.pooling import MaxPooling2D
from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator()
train_it = datagen.flow_from_directory('firstThousandTransformed/', class_mode='categorical', batch_size=64, color_mode="grayscale")
val_it = datagen.flow_from_directory('validation/', class_mode='categorical', batch_size=64, color_mode="grayscale")
imageInput = Input(shape=(64,64,1))
conv1 = Conv2D(32, kernel_size=4, activation='relu')(imageInput)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
conv2 = Conv2D(16, kernel_size=4, activation='relu')(pool1)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
flat = Flatten()(pool2)
hidden1 = Dense(10, activation='relu')(flat)
output = Dense(4, activation='sigmoid')(hidden1)
model = Model(inputs=imageInput, outputs=output)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model.summary())
model.fit(train_it, steps_per_epoch=16, validation_data=val_it, validation_steps=8)
However, I get this error when I try to run:
Input to reshape is a tensor with 3810304 values, but the requested
shape requires a multiple of 2704 [[node model/flatten/Reshape
(defined at c:\Users\cdues\Desktop\kerasTutorial\orentationTry.py:33)
]] [Op:__inference_train_function_836]
Below is my model summary:
I need some help understanding what a Tensor shape is and where my code has gone wrong here. Just working through the tutorial with Keras, I didn't encounter Tensor shape and now I am sort of lost. Sorry for the basic question, can yall help a noobie out? Thanks!
Try using the target_size argument while calling flow_from_directory.
train_it = datagen.flow_from_directory('firstThousandTransformed/',
class_mode='categorical',
batch_size=64,
color_mode='grayscale',
target_size=(64,64))
val_it = datagen.flow_from_directory('validation/',
class_mode='categorical',
batch_size=64,
color_mode='grayscale',
target_size=(64,64))
This way you can reshape the images from the directories before feeding to the model.
First in ImageDataGenerator there is a parameter called rescale. Typically with pixel values in the range 0 to 255 rescale is set to 1/255 so pixel value fall in the range from 0 to 1. I recommend you use that. Documentation for ImageDataGenerator is here.. In flow from directory you can specify the image size with parameter target_size: Tuple of integers (height, width), default: (256, 256). Documentation is at location specified earlier. In your model you have 4 nodes in your output layer. This implies you are classifying images into one of 4 classes. If that is the case in model.compile you should use categorical cross entropy as the loss. Change the activation function in your output layer to softmax.
I'm having some trouble getting a SimpleRNN to run. I'm attempting to create a image classifier RNN using Keras using tensorflow. I read the images using the Keras ImageDataGenerator FlowFromDirectory method so I can control the batch size through it and passing my training set as my x value in the fit method. I believe my issue is with the input_shape parameter of the SimpleRNN layer. I understand it should be (batch size, time steps, input dimension) but I can't understand why the array that is given to the layer is (32, 813, 701, 3). Below is my current code.
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
training_set = train_datagen.flow_from_directory('../main_set2/train_set',
target_size=(813, 701),
batch_size=32,
class_mode='binary')
test_set = test_datagen.flow_from_directory('../main_set2/test_set',
target_size=(813, 701),
batch_size=32,
class_mode='binary')
timesteps = 20
model = Sequential()
model.add(SimpleRNN(units=32,
activation='tanh',
use_bias=True,
kernel_initializer='orthogonal',
recurrent_initializer='orthogonal',
input_shape=(timesteps, 1)
))
model.add(Dense(units=32, activation='relu'))
model.add(Dense(1, activation="relu"))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.fit(training_set,
epochs=50,
verbose=2,
validation_data=test_set,
steps_per_epoch=50,
validation_steps=20)
I'd like to understand what input_shape expects and what I'm doing wrong. Thanks for your help