I need to train a 3D_Unet model with (128x128x128) patches of 42 CT scans.
My input data is 128x128x128 for the CT scans and also for masks.
I extended the shape of arrays to (128, 128, 128, 1). Where 1 is the channel.
The problem is how to feed the model with my list of 40 4D-arrays?
How can I use the model.fit() or model.train_on_batch with the correct input shape specified in my Model?
project_name = '3D-Unet Segmentation of Lungs'
img_rows = 128
img_cols = 128
img_depth = 128
# smooth = 1
K.set_image_data_format('channels_last')
#corresponds to inputs with shape:
#(batch, spatial_dim1, spatial_dim2, spatial_dim3, channels)
def get_unet():
inputs = Input(shape=(img_depth, img_rows, img_cols, 1))
conv1 = Conv3D(32, (3, 3, 3), activation='relu', padding='same')(inputs)
conv1 = Conv3D(32, (3, 3, 3), activation='relu', padding='same')(conv1)
pool1 = MaxPooling3D(pool_size=(2, 2, 2))(conv1)
conv2 = Conv3D(64, (3, 3, 3), activation='relu', padding='same')(pool1)
conv2 = Conv3D(64, (3, 3, 3), activation='relu', padding='same')(conv2)
pool2 = MaxPooling3D(pool_size=(2, 2, 2))(conv2)
....
model = Model(inputs=[inputs], outputs=[conv10])
model.summary()
#plot_model(model, to_file='model.png')
model.compile(optimizer=Adam(lr=1e-5, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.000000199),
loss='binary_crossentropy', metrics=['accuracy'])
return model
for list of arrays as input
What should I specify in either .train_on_batch() or .fit()?
This is the error I get when using the .train_on_batch option:
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 42 arrays
model.train_on_batch(train_arrays_list, mask_arrays_list)
This is the error I get when using the .model.fit option after having increased the shape of arrays with axis=0.
UnboundLocalError: local variable 'batch_index' referenced before assignment
model.fit(train_arrays_list[0], mask_arrays_list[0],
batch_size=1,
epochs=50,
verbose=1,
shuffle=True,
validation_split=0.10,
callbacks=[model_checkpoint, csv_logger])
You have to transform your list of numpy arrays of shape (128, 128, 128, 1) into a stacked 5 dimensional numpy array of shape (42, 128, 128, 128, 1). You can do this with: model.fit(np.array(train_arrays_list), np.array(mask_arrays_list), batch_size=1, ...)
Related
I would like to train a convolutional neural network autoencoder on a csv file. The csv file contains pixel neighborhood position of an original image of 1024x1024.
When I try to train it, I have the following error that I don't manage to resolve.
ValueError: Input 0 of layer max_pooling2d is incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 1, 1024, 1024, 16). Any idea, what I am doing wrong in my coding?
Let's me explain my code:
My csv file has this structure:
0 0 1.875223e+01
1 0 1.875223e+01
2 0 2.637685e+01
3 0 2.637685e+01
4 0 2.637685e+01
I managed to load my dataset, extract the x, y, and value columns as NumPy arrays and extract the relevant columns as NumPy arrays
x = data[0].values
y = data[1].values
values = data[2].values
Then, I create an empty image with the correct dimensions and fill in the image with the pixel values
image = np.empty((1024, 1024))
for i, (xi, yi, value) in enumerate(zip(x, y, values)):
image[xi.astype(int), yi.astype(int)] = value
To use this array as input to my convolutional autoencoder I reshaped it to a 4D array with dimensions
# Reshape the image array to a 4D tensor
image = image.reshape((1, image.shape[0], image.shape[1], 1))
Finally, I declare the convolutional autoencoder structure, at this stage I have the error `incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 1, 1024, 1024, 16)'
import keras
from keras.layers import Input, Conv2D, MaxPooling2D, UpSampling2D
from keras.models import Model
# Define the input layer
input_layer = Input(shape=(1,image.shape[1], image.shape[2], 1))
# Define the encoder layers
x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_layer)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)
# Define the decoder layers
x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)
# Define the autoencoder model
autoencoder = Model(input_layer, decoded)
# Compile the model
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
# Reshape the image array to a 4D tensor
image = image.reshape((1, image.shape[0], image.shape[1], 1))
# Train the model
autoencoder.fit(image, image, epochs=50, batch_size=1, shuffle=True)
You should drop the first dimension in the input layer:
input_layer = Input(shape=(image.shape[1], image.shape[2], 1))
Training pool shape and input layer shape should be different. Input layer shape describes the shape of a single datapoint, while training pool shape describes whole dataset. Hence it has one more dimension with size equal to the number of data points in your dataset.
You should also drop second image.reshape since at that time image.shape == (1, 1024, 1024, 1) and doing image = image.reshape((1, image.shape[0], image.shape[1], 1)) tries to reshape that into (1, 1, 1024, 1) which is impossible.
And lastly you forgot to add padding='same' to one of the Conv2D layers. Add that to match output layer shape with your training label shape.
I'm pretty new to this AI model creation and I'm trying to create a activity recognition model using heatmaps using this reference Human activity recognition model
In the above link they're doing activity recognition by providing some video feed, the videos have been sliced into frames and training the model here in reference link.
But in my case I'm providing the sliced frames(images) directly for the training.
The problem is that when I'm providing my dataset for AI model creation, the input is not matching with expected input.
I'm getting this error
ValueError: Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 4, 8, 8, 3), found shape=(4, 8, 8, 3)
Here is my code
seed_constant = 27
np.random.seed(seed_constant)
random.seed(seed_constant)
tf.random.set_seed(seed_constant)
CLASSES_LIST = ["Forward", "Backward"]
SEQUENCE_LENGTH = 4
IMAGE_HEIGHT = 8
IMAGE_WIDTH = 8
DATASET_DIR = r"D:\ppl_count.tar (1)\ppl_count\ppl_count\datasets\ir\dataset_test"
Input = (IMAGE_WIDTH, IMAGE_HEIGHT)
def create_dataset():
"""
This function will extract the data of the selected classes and create the required dataset.
Returns:
features: A list containing the extracted frames of the videos.
labels: A list containing the indexes of the classes associated with the videos.
video_files_paths: A list containing the paths of the videos in the disk.
"""
images_dire = r"D:\ppl_count.tar (1)\ppl_count\ppl_count\datasets\ir\test_sample"
dataset_files = create_dataset_files(images_dir=images_dire, datasets_dir=DATASET_DIR,
split_size=100,
num_threads=1,
resize=Input, normalize=True)
return dataset_files
create_dataset()
data1 = numpy.load(r"D:\ppl_count.tar
(1)\ppl_count\ppl_count\datasets\ir\dataset_test\dataset0.npz",
allow_pickle=True)
features_ = data1.f.data
labels_ = data1.f.labels
one_hot_encoded_labels = to_categorical(labels_)
# splits data into train and test sets
features_train, features_test, labels_train, labels_test = train_test_split(features_,
one_hot_encoded_labels,
test_size=0.25,
shuffle=True,
random_state=seed_constant)
print("dataset_creation_success")
def create_convlstm_model():
"""
This function will construct the required convlstm model.
Returns:
model: It is the required constructed convlstm model.
"""
# We will use a Sequential model for model construction
model = Sequential()
# Define the Model Architecture.
# #######################################################################################################################
model.add(ConvLSTM2D(filters=4, kernel_size=(3, 3), activation='tanh', data_format="channels_last",
recurrent_dropout=0.2, return_sequences=True, input_shape=(SEQUENCE_LENGTH,
IMAGE_HEIGHT, IMAGE_WIDTH, 3)))
model.add(MaxPooling3D(pool_size=(1, 2, 2), strides=(1, 1, 1), padding='same', data_format='channels_last'))
model.add(TimeDistributed(Dropout(0.2)))
print(f'1_works')
model.add(ConvLSTM2D(filters=8, kernel_size=(3, 3), activation='tanh', data_format="channels_last",
recurrent_dropout=0.2, return_sequences=True, input_shape=(SEQUENCE_LENGTH,
IMAGE_HEIGHT, IMAGE_WIDTH, 3)))
model.add(MaxPooling3D(pool_size=(1, 2, 2), strides=(1, 1, 1), padding='same', data_format='channels_last'))
model.add(TimeDistributed(Dropout(0.2)))
print(f'2_works')
model.add(ConvLSTM2D(filters=14, kernel_size=(3, 3), activation='tanh', data_format="channels_last",
recurrent_dropout=0.2, return_sequences=True, input_shape=(SEQUENCE_LENGTH,
IMAGE_HEIGHT, IMAGE_WIDTH, 3)))
model.add(MaxPooling3D(pool_size=(1, 2, 2), strides=(1, 1, 1), padding='same', data_format='channels_last'))
model.add(TimeDistributed(Dropout(0.2)))
print(f'3_works')
model.add(ConvLSTM2D(filters=16, kernel_size=(3, 3), activation='tanh', padding='same', data_format="channels_last",
recurrent_dropout=0.2, return_sequences=True, input_shape=(SEQUENCE_LENGTH,
IMAGE_HEIGHT, IMAGE_WIDTH, 3)))
model.add(MaxPooling3D(pool_size=(1, 2, 2), strides=(1, 1, 1), padding='same', data_format='channels_last'))
model.add(TimeDistributed(Dropout(0.2)))
model.add(Flatten())
model.add(Dense(len(CLASSES_LIST), activation="softmax"))
# #######################################################################################################################
# Display the models summary.
model.summary()
# Return the constructed convlstm model.
return model
convlstm_model = create_convlstm_model()
# Display the success message.
print("Model Created Successfully!")
early_stopping_callback = EarlyStopping(monitor='val_loss', patience=10, mode='min',
restore_best_weights=True)
# Compile the model and specify loss function, optimizer and metrics values to the model
convlstm_model.compile(loss='categorical_crossentropy', optimizer='Adam', metrics=
["accuracy"])
# Start training the model.
convlstm_model_training_history = convlstm_model.fit(x=features_train, y=labels_train,
epochs=50, batch_size=4,
shuffle=True, validation_split=0.2,
callbacks=[early_stopping_callback])
**please do ignore some indentations as it's very difficult to post with indentations and provide any solution/reference where I can get answer from. **
tf.keras.layers.ConvLSTM2D expects input of shape 5D but you are giving 4D input.
I could reproduce your issue
import tensorflow as tf
SEQUENCE_LENGTH = 4
IMAGE_HEIGHT = 8
IMAGE_WIDTH = 8
input_shape = (SEQUENCE_LENGTH, IMAGE_HEIGHT, IMAGE_WIDTH,3)
x = tf.random.normal(input_shape)
y = tf.keras.layers.ConvLSTM2D(filters=4, kernel_size=(3, 3), activation='tanh', data_format="channels_last",
recurrent_dropout=0.2, return_sequences=True, input_shape=input_shape)(x)
print(y.shape)
Output
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-8-6a18c66320ed> in <module>()
5 x = tf.random.normal(input_shape)
6 y = tf.keras.layers.ConvLSTM2D(filters=4, kernel_size=(3, 3), activation='tanh', data_format="channels_last",
----> 7 recurrent_dropout=0.2, return_sequences=True, input_shape=input_shape)(x)
8 print(y.shape)
2 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
212 ndim = shape.rank
213 if ndim != spec.ndim:
--> 214 raise ValueError(f'Input {input_index} of layer "{layer_name}" '
215 'is incompatible with the layer: '
216 f'expected ndim={spec.ndim}, found ndim={ndim}. '
ValueError: Input 0 of layer "conv_lstm2d_4" is incompatible with the layer: expected ndim=5, found ndim=4. Full shape received: (4, 8, 8, 3)
Working sample code
import tensorflow as tf
SEQUENCE_LENGTH = 4
IMAGE_HEIGHT = 8
IMAGE_WIDTH = 8
input_shape = (16,SEQUENCE_LENGTH, IMAGE_HEIGHT, IMAGE_WIDTH,3)
x = tf.random.normal(input_shape)
y = tf.keras.layers.ConvLSTM2D(filters=4, kernel_size=(3, 3), activation='tanh', data_format="channels_last",
recurrent_dropout=0.2, return_sequences=True, input_shape=input_shape)(x)
print(y.shape)
Output
(16, 4, 6, 6, 4)
I am trying to implement the AutoEncoder developed on the keras documentation webpage (https://blog.keras.io/building-autoencoders-in-keras.html) that is using convolutional layers. On the example they use it for the MNIST dataset flattened (reshaped from 3 channels RGB to 1) but I want to use all 3 channels. The dataset I am using has different dimentions. So, what I tried to do is just changing the parts of the code in order to output and image from the decoder with dims = (128, 128, 3) but the problem is that I get negative loss (deeply negative) and I do not know what is happening. This is the chunk of code where I do so:
input_img = keras.Input(shape= input_dim)
x = layers.Conv2D(32, (3, 3), activation='relu', padding='same')(input_img)
x = layers.MaxPooling2D((2, 2), padding='same')(x)
x = layers.Conv2D(32, (3, 3), activation='relu', padding='same')(x)
encoded = layers.MaxPooling2D((2, 2), padding='same')(x)
x = layers.Conv2D(32, (3, 3), activation='relu', padding='same')(encoded)
x = layers.UpSampling3D((2))(x)
x = layers.Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = layers.UpSampling3D((2))(x)
decoded = layers.Conv2D(3, (3, 3), activation='sigmoid', padding='same')(x)
autoencoder = keras.Model(input_img, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
autoencoder.fit(x_train, x_train,
epochs=50,
batch_size=128,
shuffle=True,
validation_data=(x_test, x_test))
Input dim is equal to (128, 128, 3) and my data dimentions x_train.shape, x_test.shape, are equal to ((6000, 128, 128, 3), (1200, 128, 128, 3)).
Thanks in advance!
I created a model. but when I want the model to do the estimation, I get an error.
inputs = tf.keras.Input(shape=(512, 512,1))
conv2d_layer = tf.keras.layers.Conv2D(32, (2,2), padding='Same')(inputs)
conv2d_layer = tf.keras.layers.Conv2D(32, (2,2), activation='relu', padding='Same')(conv2d_layer)
bn_layer = tf.keras.layers.BatchNormalization()(conv2d_layer)
mp_layer = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(bn_layer)
drop = tf.keras.layers.Dropout(0.25)(mp_layer)
conv2d_layer = tf.keras.layers.Conv2D(64, (2,2), activation='relu', padding='Same')(drop)
conv2d_layer = tf.keras.layers.Conv2D(64, (2,2), activation='relu', padding='Same')(conv2d_layer)
bn_layer = tf.keras.layers.BatchNormalization()(conv2d_layer)
mp_layer = tf.keras.layers.MaxPooling2D(pool_size=(2,2), strides=(2,2))(bn_layer)
drop = tf.keras.layers.Dropout(0.25)(mp_layer)
flatten_layer = tf.keras.layers.Flatten()(drop)
dense_layer = tf.keras.layers.Dense(512, activation='relu')(flatten_layer)
drop = tf.keras.layers.Dropout(0.5)(dense_layer)
outputs = tf.keras.layers.Dense(2, activation='softmax')(drop)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name='tumor_model')
model.summary()
Train Images Shape (342, 512, 512, 1)
Train Labels Shape (342, 2)
Test Images Shape (38, 512, 512, 1)
Test Labels Shape (38, 2)
Problem Here:
pred = model.predict(test_images[12])
WARNING:tensorflow:Model was constructed with shape (None, 512, 512, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 512, 512, 1), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'"), but it was called on an input with incompatible shape (32, 512, 1, 1).
The error is telling you that test_images.shape is (32,512,1,1). Print out
test_images.shape then find out what is wrong with how you created the test_images
I am trying the following snippet on 64 images of size 28,28,1, but it throws
ValueError: Data cardinality is ambiguous
despite I think the dimensions of the tensors being correct.
loss = model_net.train_on_batch(Batch_X, Batch_Y)
print(type(Batch_X)):<class 'list'>
print(type(Batch_X[1][0])):<class 'numpy.ndarray'>
print(type(Batch_Y)):<class 'numpy.ndarray'>
print(np.shape(Batch_X)):(2, 64, 28, 28, 1)
print(np.shape(Batch_Y)):(64,)
Model is
input_shape=(28,28,1)
left_input = Input(input_shape)
right_input = Input(input_shape)
model = Sequential([
Conv2D(filters=64, kernel_size=(3, 3), activation='relu',input_shape=(28, 28, 1)),
MaxPool2D(pool_size=(2, 2), strides=2),
Conv2D(filters=64, kernel_size=(3, 3), activation='relu'),
MaxPool2D(pool_size=(2, 2), strides=2),
Flatten(),
Dense(units=4096, activation='sigmoid')])
model.summary()
encoded_l = model(left_input)
encoded_r = model(right_input)
subtracted = keras.layers.Subtract()([encoded_l, encoded_r])
prediction = Dense(1, activation='sigmoid')(subtracted)
model_net = Model(inputs=[left_input, right_input], outputs=prediction)
optimizer= Adam(learning_rate=0.0006)
model_net.compile(loss='binary_crossentropy', optimizer=optimizer)
plot_model(model_net, show_shapes=True, show_layer_names=True)
In the above you can see the model takes in 2 images simultaneously for forward pass; thus every element in Batch_X comprises of 2 matrixes. Any suggestions where I am likely making a mistake.