Related
I am a beginner in text processing techniques and I am trying to execute the below code.
from keras.layers import Dense, Input, GlobalMaxPooling1D
from keras.layers import Conv1D, MaxPooling1D, Embedding
from keras.models import Model
from keras.layers import Input, Dense, Embedding, Conv2D, MaxPooling2D, Dropout,concatenate
from keras.layers.core import Reshape, Flatten
from keras.callbacks import EarlyStopping
from keras.optimizers import Adam
from keras.models import Model
from keras import regularizers
sequence_length = trn_abs.shape[1]
filter_sizes = [3,4,5]
num_filters = 100
drop = 0.5
inputs = Input(shape=(sequence_length,))
embedding = embedding_layer(inputs)
reshape = Reshape((sequence_length,embedding_dim,1))(embedding)
conv_0 = Conv2D(num_filters, (filter_sizes[0], embedding_dim),activation='relu',kernel_regularizer=regularizers.l2(0.01))(reshape)
conv_1 = Conv2D(num_filters, (filter_sizes[1], embedding_dim),activation='relu',kernel_regularizer=regularizers.l2(0.01))(reshape)
conv_2 = Conv2D(num_filters, (filter_sizes[2], embedding_dim),activation='relu',kernel_regularizer=regularizers.l2(0.01))(reshape)
maxpool_0 = MaxPooling2D((sequence_length - filter_sizes[0] + 1, 1), strides=(1,1))(conv_0)
maxpool_1 = MaxPooling2D((sequence_length - filter_sizes[1] + 1, 1), strides=(1,1))(conv_1)
maxpool_2 = MaxPooling2D((sequence_length - filter_sizes[2] + 1, 1), strides=(1,1))(conv_2)
merged_tensor = concatenate([maxpool_0, maxpool_1, maxpool_2], axis=1)
flatten = Flatten()(merged_tensor)
reshape = Reshape((3*num_filters,))(flatten)
dropout = Dropout(drop)(flatten)
output = Dense(units=3, activation='softmax',kernel_regularizer=regularizers.l2(0.01))(dropout)
# this creates a model that includes
model = Model(inputs, output)
adam = Adam(lr=1e-3)
model.compile(loss='categorical_crossentropy',
optimizer=adam,
metrics=['acc'])
callbacks = [EarlyStopping(monitor='val_loss')]
model.fit(X_trn, trn[target_cols], epochs=100)
and I am getting the following error:
ValueError: A target array with shape (11203, 25) was passed for output of shape (None, 3) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.
Could anyone help me with this, I am new to stackoverflow too,so please accept my apologies for ill-formating of question.
It's really important that the number of neurons at the end of your neural network is the number of categories you have. So try this:
output = Dense(units=25, activation='softmax'...
This are my imports:
import tensorflow as tf
import keras
from keras.models import Sequential, Model
from keras.layers import Conv2D, Flatten, MaxPooling2D, Dense, Input, Reshape, Concatenate, GlobalAveragePooling2D, BatchNormalization, Dropout, Activation, GlobalMaxPooling2D
from keras.utils import Sequence
I defined this model:
def create_ST_layer(input_shape = (64, 128, 3)):
input_img = Input(shape=input_shape)
model = Conv2D(48, kernel_size=(5, 5), input_shape = input_shape, strides = (1, 1), activation = "relu")(input_img)
model = MaxPooling2D(pool_size=(2, 2), strides = (2, 2))(model)
model = Conv2D(32, kernel_size=(5, 5), strides = (1, 1), activation = "relu")(model)
model = MaxPooling2D(pool_size=(2, 2), strides = (2, 2))(model)
model = Dense(50, activation = "relu")(model)
model = Dense(6)(model)
return model
And created the model by:
model = create_ST_layer()
When I now try to get the summary of the model:
model.summary()
I get the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-7-5f15418b3570> in <module>()
----> 1 model.summary()
AttributeError: 'Tensor' object has no attribute 'summary'
Is there something wrong with my imports?
Thanks a lot!
I tested this on tensorflow 2.2.0 on Google Colab.
I would change a couple of things to start with. With the new tensorflow version, rather than importing keras you should import tensorflow.keras.
So your code would look like this for imports:
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Conv2D, Flatten, MaxPooling2D, Dense, Input, Reshape, Concatenate, GlobalAveragePooling2D, BatchNormalization, Dropout, Activation, GlobalMaxPooling2D
from tensorflow.keras.utils import Sequence
Also you need to call the following line to groups layers into an object with training and inference features. [Model link] : https://www.tensorflow.org/api_docs/python/tf/keras/Model
So your complete code would look something like this:
import tensorflow as tf
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Conv2D, Flatten, MaxPooling2D, Dense, Input, Reshape, Concatenate, GlobalAveragePooling2D, BatchNormalization, Dropout, Activation, GlobalMaxPooling2D
from tensorflow.keras.utils import Sequence
def create_ST_layer(input_shape = (64, 128, 3)):
input_img = Input(shape=input_shape)
model = Conv2D(48, kernel_size=(5, 5), input_shape = input_shape, strides = (1, 1), activation = "relu")(input_img)
model = MaxPooling2D(pool_size=(2, 2), strides = (2, 2))(model)
model = Conv2D(32, kernel_size=(5, 5), strides = (1, 1), activation = "relu")(model)
model = MaxPooling2D(pool_size=(2, 2), strides = (2, 2))(model)
model = Dense(50, activation = "relu")(model)
model = Dense(6)(model)
model = tf.keras.Model(inputs=input_img, outputs= model)
return model
model = create_ST_layer()
model.summary()
I get the following output with your model:
enter image description here
Because just adding layers one by one will create a Tensorflow graph. If you want to create Keras model, you should either-
add layers using model.add(). [link]
create Keras model after creating a Tensorflow graph by keras.models.Model().
Using 2nd method, you could do:
def create_ST_layer(input_shape = (64, 128, 3)):
input_img = Input(shape=input_shape)
model = Conv2D(48, kernel_size=(5, 5), input_shape = input_shape, strides = (1, 1), activation = "relu")(input_img)
model = MaxPooling2D(pool_size=(2, 2), strides = (2, 2))(model)
model = Conv2D(32, kernel_size=(5, 5), strides = (1, 1), activation = "relu")(model)
model = MaxPooling2D(pool_size=(2, 2), strides = (2, 2))(model)
model = Dense(50, activation = "relu")(model)
model = Dense(6)(model)
myModel = Model(input_img, model)
return myModel
model = create_ST_layer()
model.summary()
This is my code in order to join resnet50 model with this model (that I want to train on my dataset). I want to freeze layers of the resnet50 model ( see Trainable=false) in the code .
Here I'm importing resnet 50 model
``
import tensorflow.keras
import tensorflow as tf
from tensorflow.keras.applications.resnet50 import ResNet50
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictions
resnet50_imagnet_model = tensorflow.keras.applications.resnet.ResNet50(weights = "imagenet",
include_top=False,
input_shape = (150, 150, 3),
pooling='max')
``
Here I create my model
```
# freeze feature layers and rebuild model
for l in resnet50_imagnet_model.layers:
l.trainable = False
#construction du model
model5 = [
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(12, activation='softmax')
]
#Jointure des deux modeles
model_using_pre_trained_resnet50 = tf.keras.Sequential(resnet50_imagnet_model.layers + model5 )
```
Last line doesn't work and I have this error :
Input 0 of layer conv2_block1_3_conv is incompatible with the layer: expected axis -1 of input shape to have value 64 but received input with shape [None, 38, 38, 256
Thanks for help .
You can also use keras' functional API, like below
from tensorflow.keras.applications.resnet50 import ResNet50
import tensorflow as tf
resnet50_imagenet_model = ResNet50(include_top=False, weights='imagenet', input_shape=(150, 150, 3))
#Flatten output layer of Resnet
flattened = tf.keras.layers.Flatten()(resnet50_imagenet_model.output)
#Fully connected layer 1
fc1 = tf.keras.layers.Dense(128, activation='relu', name="AddedDense1")(flattened)
#Fully connected layer, output layer
fc2 = tf.keras.layers.Dense(12, activation='softmax', name="AddedDense2")(fc1)
model = tf.keras.models.Model(inputs=resnet50_imagenet_model.input, outputs=fc2)
Also refer this question.
I'm programming in python 3.7.5 using keras and TensorFlow 1.13.1
I want remove batch normalization layer from model coded below:
from keras import backend as K
from keras.callbacks import *
from keras.layers import *
from keras.models import *
from keras.utils import *
from keras.optimizers import Adadelta, RMSprop, Adam, SGD
from keras.callbacks import ModelCheckpoint
from keras.callbacks import TensorBoard
from config import *
def ctc_lambda_func(args):
iy_pred, ilabels, iinput_length, ilabel_length = args
# the 2 is critical here since the first couple outputs of the RNN
# tend to be garbage:
iy_pred = iy_pred[:, 2:, :] # no such influence
return K.ctc_batch_cost(ilabels, iy_pred, iinput_length, ilabel_length)
def CRNN_model(is_training=True):
inputShape = Input((width, height, 1), name='input') # base on Tensorflow backend
conv_1 = Conv2D(64, (3, 3), activation='relu', padding='same')(inputShape)
conv_2 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv_1)
#batchnorm_2 = BatchNormalization()(conv_2)
pool_2 = MaxPooling2D(pool_size=(2, 2))(conv_2)
conv_3 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool_2)
conv_4 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv_3)
#batchnorm_4 = BatchNormalization()(conv_4)
pool_4 = MaxPooling2D(pool_size=(2, 2))(conv_4)
conv_5 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool_4)
conv_6 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv_5)
pool_5 = MaxPool2D(pool_size=(2, 2))(conv_6)
#batchnorm_6 = BatchNormalization()(conv_6)
#bn_shape = batchnorm_6.get_shape()
#print(bn_shape)
#x_reshape = Reshape(target_shape=(int(bn_shape[1]), int(bn_shape[2] * bn_shape[3])))(batchnorm_6)
#drop_reshape = Dropout(0.25, name='d1')(x_reshape)
fl_1 = Flatten()(pool_5)
fc_1 = Dense(256, activation='relu')(fl_1)
#print(x_reshape.get_shape())
#print(fc_1.get_shape())
bi_LSTM_1 = Bidirectional(LSTM(256, return_sequences=True, kernel_initializer='he_normal'), merge_mode='sum')(fc_1)
bi_LSTM_2 = Bidirectional(LSTM(128, return_sequences=True, kernel_initializer='he_normal'), merge_mode='concat')(bi_LSTM_1)
#drop_rnn = Dropout(0.3, name='d2')(bi_LSTM_2)
fc_2 = Dense(label_classes, kernel_initializer='he_normal', activation='softmax')(bi_LSTM_2)
base_model = Model(inputs=[inputShape], outputs=fc_2)
labels = Input(name='the_labels', shape=[label_len], dtype='float32')
input_length = Input(name='input_length', shape=[1], dtype='int64')
label_length = Input(name='label_length', shape=[1], dtype='int64')
loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')([fc_2, labels, input_length, label_length])
if is_training:
return Model(inputs=[inputShape, labels, input_length, label_length], outputs=[loss_out]), base_model
else:
return base_model
but I get this error:
Traceback (most recent call last):
File "C:/Users/Babak/PycharmProjects/CRNN-OCR/captcha-recognition-master1/captcha-recognition-master/training.py", line 79, in <module>
model, base_model = CRNN_model(is_training=True)
File "C:\Users\Babak\PycharmProjects\CRNN-OCR\captcha-recognition-master1\captcha-recognition-master\model.py", line 51, in CRNN_model
bi_LSTM_1 = Bidirectional(LSTM(256, return_sequences=True, kernel_initializer='he_normal'), merge_mode='sum')(fc_1)
File "C:\Program Files\Python37\lib\site-packages\keras\layers\wrappers.py", line 437, in __call__
return super(Bidirectional, self).__call__(inputs, **kwargs)
File "C:\Program Files\Python37\lib\site-packages\keras\engine\base_layer.py", line 446, in __call__
self.assert_input_compatibility(inputs)
File "C:\Program Files\Python37\lib\site-packages\keras\engine\base_layer.py", line 342, in assert_input_compatibility
str(K.ndim(x)))
ValueError: Input 0 is incompatible with layer bidirectional_1: expected ndim=3, found ndim=2
Process finished with exit code 1
How can I remove batch norm layers which is commented. I note that I manually remove drop out layers. So assume that dropout are removed. I remove dropout layers without problem. But I have problem in removing batch normalization layers
As per the error code, LSTM layers expect 3D input tensors, but Dense outputs only 2D. Many possible fixes exist, but not all will work equally well:
Conv2D outputs 4D tensors, shaped (samples, height, width, channels)
LSTM expects input shaped (samples, timesteps, channels)
Thus, you need to somehow transform the (height, width) dimensions into timesteps
In existing research, image data is flattened and treated sequentially - however, channels remain untouched. Thus, a viable approach is to use Reshape to yield a 3D tensor shaped (samples, height*width, channels). Finally, as Dense cannot work with 3D data, you'll need the TimeDistributed wrapper that'll apply the same Dense weights to dim 1 of input - i.e. to timesteps:
pool_shapes = K.int_shape(pool_5)
fl_1 = Reshape((pool_shapes[1] * pool_shapes[2], pool_shapes[3]))(pool_5)
fc_1 = TimeDistributed(Dense(256, activation='relu'))(fl_1)
Lastly, return_sequences=True outputs a 3D tensor, which your output Dense cannot handle - so either use return_sequences=False to output 2D, or insert a Flatten before the Dense.
I have an issue with Recurrentshop and Keras. I am trying to use Concatenate and multidimensional tensors in a Recurrent Model, and I get dimension issue regardless of how I arrange the Input, shape and batch_shape.
Minimal code:
from keras.layers import *
from keras.models import *
from recurrentshop import *
from keras.layers import Concatenate
input_shape=(128,128,3)
x_t = Input(shape=(128,128,3,))
h_tm1 = Input(shape=(128,128,3, ))
h_t1 = Concatenate()([x_t, h_tm1])
last = Conv2D(3, kernel_size=(3,3), strides=(1,1), padding='same', name='conv2')(h_t1)
# Build the RNN
rnn = RecurrentModel(input=x_t, initial_states=[h_tm1], output=last, final_states=[last], state_initializer=['zeros'])
x = Input(shape=(128,128,3, ))
y = rnn(x)
model = Model(x, y)
model.predict(np.random.random((1, 128, 128, 3)))
ErrorCode:
ValueError: Shape must be rank 3 but it is rank 4 for 'recurrent_model_1/concatenate_1/concat' (op:ConcatV2) with input shapes: [?,128,3], [?,128,128,3], [].
Please help.
Try this (the changed lines are commented):
from recurrentshop import *
from keras.layers import Concatenate
x_t = Input(shape=(128, 128, 3,))
h_tm1 = Input(shape=(128, 128, 3,))
h_t1 = Concatenate()([x_t, h_tm1])
last = Conv2D(3, kernel_size=(3, 3), strides=(1, 1), padding='same', name='conv2')(h_t1)
rnn = RecurrentModel(input=x_t,
initial_states=[h_tm1],
output=last,
final_states=[last],
state_initializer=['zeros'])
x = Input(shape=(1, 128, 128, 3,)) # a series of 3D tensors -> 4D
y = rnn(x)
model = Model(x, y)
model.predict(np.random.random((1, 1, 128, 128, 3))) # a batch of x -> 5D