I need to modify output of last feature map of my second convolution layer.
Or add array to my conv layer output if it's possible.
Below is python script i created and example of desired change in output.
Thank you for your help!
import numpy as np
from keras import backend as K
num=18
m=11
n=50
k=3
np.random.seed(100)
features = np.random.rand(num,m,n,k)
model
input_shape=features.shape[1:]
model = Sequential()
model.add(Conv2D(2, kernel_size=(1, 3), strides=(1,1),activation='relu',input_shape=input_shape))
model.add(Conv2D(21, kernel_size=(1, 48), strides=(1,1),padding="valid",activation='relu'))
model.add(Conv2D(1, kernel_size=(1, 1), strides=(1, 1),activation='relu',padding="valid"))
model.add(Dense(1, activation='softmax'))
Adam = optimizers.Adam(lr=0.00003, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
model.compile(loss='mse',optimizer=Adam)
get_1st_layer_output = K.function([model.layers[0].input],
[model.layers[1].output])
layer_output = get_1st_layer_output([features])
Setting DESIRED layer_output values
I need to do it every propagation step.
for i in range(0,11):
layer_output[0][0][i][0][20]=0.1
print(layer_output[0][0][i][0][20])
I think I would use a concatenation with a constant tensor in that case. Unfortunately, I can't quite get it to work, but I'll share my work anyway to hopefully help you on your way.
import numpy as np
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Conv2D, Dense, Concatenate
from keras import optimizers
num=18
m=11
n=50
k=3
np.random.seed(100)
features = np.random.rand(num, m, n, k)
custom_tensor = K.constant(0.1, shape=(11, 48, 1))
input_shape = features.shape[1:]
input = keras.Input(shape=input_shape)
print(K.ndim(input))
layer0 = Conv2D(2, kernel_size=(1, 3), strides=(1,1),activation='relu')(input)
layer0_added = Concatenate(axis=-1)([layer0, custom_tensor])
layer1 = Conv2D(20, kernel_size=(1, 48), strides=(1,1),padding="valid",activation='relu')(layer0_added)
layer2 = Conv2D(1, kernel_size=(1, 1), strides=(1, 1),activation='relu',padding="valid")(layer1)
layer3 = Dense(1, activation='softmax')(layer2)
model = keras.models.model(layer0)
Adam = optimizers.Adam(lr=0.00003, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
model.compile(loss='mse', optimizer=Adam)
It produced an error
ValueError: `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 11, 48, 2), (11, 48, 1)]
But hopefully this helps you along anyway.
Related
Okay I'm new to this I'm trying to classify Traffic Sign images Im actually following a notebook from Kaggle, after building the Keras model
from keras.models import Sequential
from keras.layers import Conv2D, MaxPool2D, Dense, Flatten, Dropout
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=(5,5), activation='relu', input_shape=X_train.shape[1:]))
model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(rate=0.25))
model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(rate=0.25))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(rate=0.5))
model.add(Dense(43, activation='softmax'))
#Compilation of the model
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy']
)
now for the test part after fitting the model the code i found is like this
y_test=pd.read_csv("C:/Users/Louay/input/test.csv")
labels=y_test['Path'].to_numpy()
y_test=y_test['ClassId'].values
data=[]
for f in labels:
image=cv2.imread('C:/Users/Louay/input/Test/'+f.replace('Test/', ''))
image_from_array = Image.fromarray(image, 'RGB')
size_image = image_from_array.resize((height, width))
data.append(np.array(size_image))
X_test=np.array(data)
X_test = X_test.astype('float32')/255
pred = model.predict_classes(X_test)
this works and it predicts the classes of all images in the test set correctly my problem is when I tried to predict only 1 image from the test set okay I thought I would repeat the same image processing part and then use predict_classes() so my code should be like this
image=cv2.imread('C:/Users/Louay/input/Test/00000.png')
image_from_array = Image.fromarray(image, 'RGB')
size_image = image_from_array.resize((height, width))
test=np.array(size_image)
pred = model.predict_classes(test.astype('float32')/255)
okay I'm working on 1 image so I thought I don't need the data[] list where I append all processed images but when I run the code I get this error
ValueError: Input 0 of layer sequential_1 is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: [None, None, 3]
I know I'm doing something wrong but before correcting my code I really want to understand why am I getting this error what's causing it, what's actually happening?
Conv2D expects 4+D tensor with shape: batch_shape + (channels, rows, cols) if data_format='channels_first' or 4+D tensor with shape: batch_shape + (rows, cols, channels) if data_format='channels_last'.
Add extra dimension to your input using tf.expand_dims
Working sample code
import tensorflow as tf
image = tf.zeros([10,10,3])
input_shape = tf.expand_dims(image, axis=0).shape.as_list()
x = tf.random.normal(input_shape)
y = tf.keras.layers.Conv2D(
2, 3, activation='relu', input_shape=input_shape[1:])(x)
print(y.shape)
Output
(1, 8, 8, 2)
I trained a model to categorize pictures in two different types. Everything is working
quite good, but my Model can only do a specific prediction (1 or 0 in my case), but I am interested to have a prediction which is more like a probability (For example 90% 1 and 10% 0).
Where is the part of my code which I should change now? Is it something with the sigmoid function in the end which decides if its 1 or 0? Help would be nice. Thanks in advance.
import numpy as np
from keras.callbacks import TensorBoard
from keras import regularizers
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense, Conv2D, MaxPooling2D
from keras.optimizers import Adam
from keras.metrics import categorical_crossentropy
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras.layers.normalization import BatchNormalization
from utils import DataGenerator, PATH
train_path = 'Dataset/train'
valid_path = 'Dataset/valid'
test_path = 'Dataset/test'
model = Sequential()
model.add(Conv2D(16, (3, 3), input_shape=(640, 640, 1), padding='same', activation='relu',
kernel_regularizer=regularizers.l2(1e-4),
bias_regularizer=regularizers.l2(1e-4)))
model.add(MaxPooling2D(pool_size=(4, 4)))
model.add(Conv2D(32, (3, 3), activation='relu',
kernel_regularizer=regularizers.l2(1e-4),
bias_regularizer=regularizers.l2(1e-4)))
model.add(MaxPooling2D(pool_size=(5, 5)))
model.add(Conv2D(64, (3, 3), activation='relu',
kernel_regularizer=regularizers.l2(1e-4),
bias_regularizer=regularizers.l2(1e-4)))
model.add(MaxPooling2D(pool_size=(6, 6)))
model.add(Flatten())
model.add(Dense(64, activation='relu',
kernel_regularizer=regularizers.l2(1e-4),
bias_regularizer=regularizers.l2(1e-4)))
model.add(Dropout(0.3))
model.add(Dense(1, activation='sigmoid',
kernel_regularizer=regularizers.l2(1e-4),
bias_regularizer=regularizers.l2(1e-4)))
print(model.summary())
model.compile(loss='binary_crossentropy', optimizer=Adam(lr=1e-3), metrics=['accuracy'])
epochs = 50
batch_size = 16
datagen = DataGenerator()
datagen.load_data()
model.fit_generator(datagen.flow(batch_size=batch_size), epochs=epochs, validation_data=datagen.get_validation_data(),
callbacks=[TensorBoard(log_dir=PATH+'/tensorboard')])
#model.save_weights('first_try.h5')
model.save('second_try')
If I try to get a picture in my model like this:
path = 'train/clean/picturenumber2'
def prepare(filepath):
IMG_SIZE = 640
img_array = cv2.imread(filepath, cv2.IMREAD_GRAYSCALE)
new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))
return new_array.reshape(-1, IMG_SIZE, IMG_SIZE, 1)
model = tf.keras.models.load_model('second_try')
prediction = model.predict(prepare(path))
print(prediction)
I just get an output like this: [[1.]] Also if I put in a list with multiple pictures. The prediction itself seems to be working.
short answer :
change sigmoid activation function in the last layer to softmax
why ?
because sigmoid output range is 0.0 to 1.0, so to make a meaningful interpretation of this output, you choose an appropriate threshold above which represents the positive class and anything below as negative class.(for a binary classification problem)
even softmax has the same output range but the difference being its outputs are normalized class probabilities more on that here, so if your model outputs 0.99 on any given input, then it can be interpreted as the model is 99.0% confident that it is a positive class and 0.1% confident that it belongs to a negative class.
update :
as #amin suggested, if you need normalized probabilities you should do couple more changes for it to work.
modify your data generator to output 2 classes/labels instead of one.
change last Dense layer from 1 node to 2 nodes.
I used two different models for my case.
The case is classification for different type of surface defect.
The input shape is (200, 200, 1), and there are 6 classes.
The numbers of training data is 1440(240 for 1 class), and the number of validation data is 360(60 for 1 class).
The training process is very well with first model. Both training loss and validation loss are dropping very quickly.
After that, I want to use MobileNetV2 from keras for comparing the training result. The training loss and accuracy in MobileV2 are improve, but the validation accuracy stuck in 0.1667(the loss is bumpy).
I wonder know what causes this result, Can I call this situation 'Over-fitting'? Or just this model is too deep to my case?
First model:
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D, BatchNormalization
import parameter
input_shape = (parameter.IMAGE_SIZE_Y, parameter.IMAGE_SIZE_X, parameter.channel)
def MyModel():
model = Sequential()
model.add(Conv2D(16, (3, 3), input_shape = input_shape, padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Conv2D(128, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Flatten())
model.add(Dense(256, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(6, activation = 'softmax'))
model.summary()
return model
Second model:
from keras.models import Sequential, Model
from keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D, BatchNormalization, GlobalAveragePooling2D
from keras.optimizers import Adam
from keras.applications import MobileNetV2
import parameter
def MyMobileNetV2():
input_shape = (parameter.IMAGE_SIZE_X, parameter.IMAGE_SIZE_Y, parameter.channel)
model = MobileNetV2(input_shape = input_shape,
include_top = False,
weights = 'imagenet')
x = model.output
x = GlobalAveragePooling2D()(x)
x = BatchNormalization()(x)
x = Dense(1280, activation='relu')(x)
x = BatchNormalization()(x)
predictions = Dense(6, activation='softmax', kernel_initializer='random_uniform', bias_initializer='zeros')(x)
model = Model(inputs = model.input, outputs = predictions)
optimizer = Adam(lr=0.01)
loss = "categorical_crossentropy"
for layer in model.layers:
layer.trainable = True
model.compile(optimizer=optimizer,
loss=loss,
metrics=["accuracy"])
model.summary()
for i, layer in enumerate(model.layers):
print(i, layer.name, layer.trainable)
return model
I'm programming in python 3.7.5 using keras and TensorFlow 1.13.1
I want remove batch normalization layer from model coded below:
from keras import backend as K
from keras.callbacks import *
from keras.layers import *
from keras.models import *
from keras.utils import *
from keras.optimizers import Adadelta, RMSprop, Adam, SGD
from keras.callbacks import ModelCheckpoint
from keras.callbacks import TensorBoard
from config import *
def ctc_lambda_func(args):
iy_pred, ilabels, iinput_length, ilabel_length = args
# the 2 is critical here since the first couple outputs of the RNN
# tend to be garbage:
iy_pred = iy_pred[:, 2:, :] # no such influence
return K.ctc_batch_cost(ilabels, iy_pred, iinput_length, ilabel_length)
def CRNN_model(is_training=True):
inputShape = Input((width, height, 1), name='input') # base on Tensorflow backend
conv_1 = Conv2D(64, (3, 3), activation='relu', padding='same')(inputShape)
conv_2 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv_1)
#batchnorm_2 = BatchNormalization()(conv_2)
pool_2 = MaxPooling2D(pool_size=(2, 2))(conv_2)
conv_3 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool_2)
conv_4 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv_3)
#batchnorm_4 = BatchNormalization()(conv_4)
pool_4 = MaxPooling2D(pool_size=(2, 2))(conv_4)
conv_5 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool_4)
conv_6 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv_5)
pool_5 = MaxPool2D(pool_size=(2, 2))(conv_6)
#batchnorm_6 = BatchNormalization()(conv_6)
#bn_shape = batchnorm_6.get_shape()
#print(bn_shape)
#x_reshape = Reshape(target_shape=(int(bn_shape[1]), int(bn_shape[2] * bn_shape[3])))(batchnorm_6)
#drop_reshape = Dropout(0.25, name='d1')(x_reshape)
fl_1 = Flatten()(pool_5)
fc_1 = Dense(256, activation='relu')(fl_1)
#print(x_reshape.get_shape())
#print(fc_1.get_shape())
bi_LSTM_1 = Bidirectional(LSTM(256, return_sequences=True, kernel_initializer='he_normal'), merge_mode='sum')(fc_1)
bi_LSTM_2 = Bidirectional(LSTM(128, return_sequences=True, kernel_initializer='he_normal'), merge_mode='concat')(bi_LSTM_1)
#drop_rnn = Dropout(0.3, name='d2')(bi_LSTM_2)
fc_2 = Dense(label_classes, kernel_initializer='he_normal', activation='softmax')(bi_LSTM_2)
base_model = Model(inputs=[inputShape], outputs=fc_2)
labels = Input(name='the_labels', shape=[label_len], dtype='float32')
input_length = Input(name='input_length', shape=[1], dtype='int64')
label_length = Input(name='label_length', shape=[1], dtype='int64')
loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')([fc_2, labels, input_length, label_length])
if is_training:
return Model(inputs=[inputShape, labels, input_length, label_length], outputs=[loss_out]), base_model
else:
return base_model
but I get this error:
Traceback (most recent call last):
File "C:/Users/Babak/PycharmProjects/CRNN-OCR/captcha-recognition-master1/captcha-recognition-master/training.py", line 79, in <module>
model, base_model = CRNN_model(is_training=True)
File "C:\Users\Babak\PycharmProjects\CRNN-OCR\captcha-recognition-master1\captcha-recognition-master\model.py", line 51, in CRNN_model
bi_LSTM_1 = Bidirectional(LSTM(256, return_sequences=True, kernel_initializer='he_normal'), merge_mode='sum')(fc_1)
File "C:\Program Files\Python37\lib\site-packages\keras\layers\wrappers.py", line 437, in __call__
return super(Bidirectional, self).__call__(inputs, **kwargs)
File "C:\Program Files\Python37\lib\site-packages\keras\engine\base_layer.py", line 446, in __call__
self.assert_input_compatibility(inputs)
File "C:\Program Files\Python37\lib\site-packages\keras\engine\base_layer.py", line 342, in assert_input_compatibility
str(K.ndim(x)))
ValueError: Input 0 is incompatible with layer bidirectional_1: expected ndim=3, found ndim=2
Process finished with exit code 1
How can I remove batch norm layers which is commented. I note that I manually remove drop out layers. So assume that dropout are removed. I remove dropout layers without problem. But I have problem in removing batch normalization layers
As per the error code, LSTM layers expect 3D input tensors, but Dense outputs only 2D. Many possible fixes exist, but not all will work equally well:
Conv2D outputs 4D tensors, shaped (samples, height, width, channels)
LSTM expects input shaped (samples, timesteps, channels)
Thus, you need to somehow transform the (height, width) dimensions into timesteps
In existing research, image data is flattened and treated sequentially - however, channels remain untouched. Thus, a viable approach is to use Reshape to yield a 3D tensor shaped (samples, height*width, channels). Finally, as Dense cannot work with 3D data, you'll need the TimeDistributed wrapper that'll apply the same Dense weights to dim 1 of input - i.e. to timesteps:
pool_shapes = K.int_shape(pool_5)
fl_1 = Reshape((pool_shapes[1] * pool_shapes[2], pool_shapes[3]))(pool_5)
fc_1 = TimeDistributed(Dense(256, activation='relu'))(fl_1)
Lastly, return_sequences=True outputs a 3D tensor, which your output Dense cannot handle - so either use return_sequences=False to output 2D, or insert a Flatten before the Dense.
I'm writing Deep learning network in Keras, previously tested in Matlab. To avoid doing all the learning, I exported weights and biases of the final layer in Matlab as the .csv file, and want to use them in my network - so my network would just test score based on given weights, instead of whole learning.
Code:
import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, Input
from keras.layers import Convolution2D, MaxPooling2D
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
#from keras import backend as K
from keras.preprocessing import image
print("\n")
print("\n")
trained_weights = []
trained_bias = []
trained_weights = np.loadtxt('Weights.csv', delimiter=';')
trained_bias = np.loadtxt('Bias.csv', delimiter=';')
network_weights = np.array([[trained_weights]])
network_bias = np.array([[trained_bias]])
network_outputs = np.array([['a','c','d','e','f','g','h']])
# Load test image
img = load_img('note_a.png')
note = image.img_to_array(img)
#note = np.expand_dims(note, axis=0)
note = np.reshape(note, [1, 50, 30, 3])
# Model architecture
model = Sequential()
# Layer 1
model.add(Convolution2D(12, (6, 6), batch_size=1, input_shape=(50, 30, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
# Layer 2
model.add(Convolution2D(24, (6, 6), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
# Layer 3
model.add(Convolution2D(48, (6, 6), activation='relu'))
model.add(Flatten())
layer2=Dense(7, weights=[network_weights, network_bias], activation='softmax')
model.add(layer2)
model.summary()
print("\n")
print("\n")
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics = ['accuracy'])
#model.fit((note,network_outputs), batch_size=32, nb_epoch=10, verbose=1)
#score = model.evaluate(note, network_outputs)
I was trying to use
model.set_weights([network_outputs])
but it seems like it assigns a weights only to my first layer of network, so I just assigned weights to final layer. Result is the following error:
ValueError: Layer weight shape (672, 7) not compatible with provided weight shape (1, 1, 672, 7)
And this is quite confusing for me. How, by doing weights=[network_weights, network_bias] I get 4 dimensions? Is it because network_weights has dimensions [1,672], and network_bias=[1,7], which makes [1,1,672,7]?
And how I can properly resize this weights parameter?
np.array([[trained_weights]]) creates an array out of your data surrounded by 2 empty dimensions, so your final shape is (1, 1, x, y). Same for your trained_bias. Applying [network_weights, network_bias] does again surround your 2 arrays with a dimension, which does obviously not match.
I think you just need to clean all the unnecessary surroundings by using something like:
weights=[trained_weights, trained_bias]