I'm working on a sentiment analysis project in python with word2vec as an embedding method. (in my NON_ENGLISH corpus I considered every negative tweet with the 0 label, positive = 1 and neutral = 2)Since I'm really new in this field I don't have any idea as to why my model predicts everything negative. I surfed the net before and read something about the number of hidden layer and batches , etc. which related to this error but I'm not sure that I understood correctly.
here is my keras model:
model = Sequential()
model.add(Conv1D(32, kernel_size=3, activation='elu',
padding='same', input_shape=
(max_tweet_length,vector_size)))
model.add(Conv1D(32, kernel_size=3, activation='elu',
padding='same'))
model.add(Conv1D(32, kernel_size=3, activation='elu',
padding='same'))
model.add(Conv1D(32, kernel_size=3, activation='elu',
padding='same'))
model.add(Dropout(0.25))
model.add(Conv1D(32, kernel_size=2, activation='elu',
padding='same'))
model.add(Conv1D(32, kernel_size=2, activation='elu',
padding='same'))
model.add(Conv1D(32, kernel_size=2, activation='elu',
padding='same'))
model.add(Conv1D(32, kernel_size=2, activation='elu',
padding='same'))
model.add(Dropout(0.25))
model.add(Dense(256, activation='tanh'))
model.add(Dense(256, activation='tanh'))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(2, activation='sigmoid'))
thanks a lot!
Related
I want to use pre_trained model in federated learning as following code:
first I build my model and set the weights on model and then I freeze convolutional layers and remove 4 last layer.
def create_keras_model():
model = Sequential()
model.add(Conv2D(16, kernel_size=(3,3), strides=(1,1), padding='same', activation='relu', input_shape=(226,232,1)))
model.add(MaxPooling2D((2,2), strides=(2,2), padding='same'))
model.add(Conv2D(32, kernel_size=(3,3), strides=(1,1), padding='same', activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2), padding='same'))
model.add(Conv2D(64, kernel_size=(3,3), strides=(1,1), padding='same', activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2), padding='same'))
model.add(Conv2D(64, kernel_size=(3,3), strides=(1,1), padding='same', activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2), padding='same'))
model.add(Conv2D(128, kernel_size=(3,3), strides=(1,1), padding='same', activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2), padding='same'))
model.add(Conv2D(128, kernel_size=(3,3), strides=(1,1), padding='same', activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2), padding='same'))
model.add(Conv2D(256, kernel_size=(3,3), strides=(1,1), padding='same', activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2), padding='same'))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(10, activation='softmax'))
return model
keras_model = create_keras_model()
server_state=FileCheckpointManager(root_dir= '/content/drive/MyDrive',
prefix= 'federated_clustering',
step= 1,
keep_total= 1,
keep_first= True).load_checkpoint(structure=server_state,round_num=10)
keras_model.set_weights(server_state)
for layer in keras_model.layers[:-4]:
layer.trainable = False
model_pre = Model(inputs=keras_model.input,outputs=keras_model.layers[14].output)
next, I build new model.
def create_keras_model1():
model = Sequential()
model.add(model_pre)
model.add(Dense(256, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(2, activation='softmax'))
return model
def model_fn():
# We _must_ create a new model here, and _not_ capture it from an external
# scope. TFF will call this within different graph contexts.
keras_model = create_keras_model1()
return tff.learning.from_keras_model(
keras_model,
input_spec=preprocessed_example_dataset.element_spec,
loss=tf.keras.losses.SparseCategoricalCrossentropy(),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
but I get ValueError when I want to use tff.learning.build_federated_averaging_process.
iterative_process = tff.learning.build_federated_averaging_process(
model_fn,
client_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=0.001),
server_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=1.0))
ValueError: Your Layer or Model is in an invalid state. This can happen for the following cases:
1. You might be interleaving estimator/non-estimator models or interleaving models/layers made in tf.compat.v1.Graph.as_default() with models/layers created outside of it. Converting a model to an estimator (via model_to_estimator) invalidates all models/layers made before the conversion (even if they were not the model converted to an estimator). Similarly, making a layer or a model inside a a tf.compat.v1.Graph invalidates all layers/models you previously made outside of the graph.
2. You might be using a custom keras layer implementation with custom __init__ which didn't call super().__init__. Please check the implementation of <class 'keras.engine.functional.Functional'> and its bases.
please help me to fix it.
You need to create a new keras model during the invocation of model_fn, as the code comment suggests. It seems you are using model_pre which you have already created before that, which is likely the core problem.
I am working on the CNN model. In the given CNN model I can not handle how to divide the 4th layer into two streams and get output.
I also built a model in Keras.
def _build_model(self):
model = Sequential()
model.add(Conv2D(8, (3, 3), strides=4, padding='same', input_shape=self.state_size))
model.add(Activation('relu'))
model.add(Conv2D(2, (2, 2), strides=4, padding='same'))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(16, activation='relu'))
model.add(Dense(self.action_size, activation='relu'))
model.compile(loss='mse', optimizer=Adam())
return model
How to handle it? An example would be appreciated.
Im currently working on a little CNN with the Cifar-10 Dataset. I just updatet my code a little bit here and there and now it isnt working. I cant figure out the mistake. The prediction tells me "not an number". Couldnt find a answer for my proplem. So i cant post the Question without adding a little bit more text sooooo. Idk what i should write here. A good breakfast would be nice now. Coffe and Pancakes something like that. I hope i can poste the question now.
from keras.datasets import cifar10
import numpy as np
(x_training, y_training), (x_test,y_test) = cifar10.load_data()
x_training = x_training / 255.0
x_test = x_test / 255.0
%matplotlib inline
import matplotlib.pyplot as plt
plt.imshow(x_training[3])
plt.show
from keras.models import Sequential
from keras.layers import Dense, Flatten,Conv2D , MaxPooling2D, Dropout
import tensorflow as tf
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), input_shape=(32, 32, 3), activation="relu", padding="same"))
model.add(Conv2D(32, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(1, activation="sigmoid"))
model.compile(optimizer='RMSProp', loss="binary_crossentropy", metrics=['accuracy'])
model.summary()
model.fit(x_training, y_training,batch_size=128, epochs=10, shuffle = True )
model.evaluate(x_training, y_training)
results = model.predict(x_training[1].reshape(-1, 32, 32, 3))
results
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
'dog', 'frog', 'horse', 'ship', 'truck']
max = np.max(results)
max_position = np.argmax(results)
class_name_predict = class_names[max_position]
plt.imshow(x_training[1])
plt.show
test = class_name_predict
test
plt.imshow(x_training[1])
plt.show
x = class_names[y_training[1][0]]
x
There are some mistakes in your model:
The output layer for a multiclass problem must have a dimension equal to the number of classes with a softmax activation function
The standard losses for a multiclass problem are categorical_crossentropy and sparse_categoricalcrossentropy. categorical_crossentropy can be used when your target is one-hot encoded, sparse_categoricalcrossentropy is used when you have integer encoded labels (this is your case)
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), input_shape=(32, 32, 3), activation="relu", padding="same"))
model.add(Conv2D(32, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(len(class_names), activation="softmax"))
model.compile(optimizer='RMSProp', loss="sparse_categorical_crossentropy", metrics=['accuracy'])
model.summary()
here the running notebook
I want to use keras to make a CNN construct, but my input images' shape will different. After I use small input shape to learning, I recognize image shape will also different.
input_shape = (None, None, 3)
model = Sequential()
model.add(Conv2D(64, (3,3), input_shape=input_shape, padding='same', activation='relu'))
model.add(Conv2D(64, (3,3), padding='same', activation='relu'))
model.add(MaxPooling2D(2,2))
model.add(Conv2D(128, (3,3), padding='same', activation='relu'))
model.add(Conv2D(128, (3,3), padding='same', activation='relu'))
model.add(MaxPooling2D(2,2))
model.add(Conv2D(256, (3,3), padding='same', activation='relu'))
model.add(Conv2D(256, (3,3), padding='same', activation='relu'))
model.add(Conv2D(256, (3,3), padding='same', activation='relu'))
model.add(MaxPooling2D(2,2))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(MaxPooling2D(2,2))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(MaxPooling2D(2,2))
model.add(Flatten())
model.add(Dense(4000, activation='relu'))
model.add(Dense(4000, activation='relu'))
model.add(Dense(30, activation='relu'))
But the program execute to "Flatten()" error. What can I use? Thanks you very much.
You should reshape to some nice square shape...
I have a classifier model that I trained using 'theano' backend. The model works properly and I got the expected classification perforamance. The tensor size is Nx3x28x112 However, I would like to use the same classifier in another file (main_file.py) which contains a GANs implementation (with'tensorflow' backend). Thereby, I want to use the same classificer in the main_file.py and to change the input size of the tensor in order to be Nx28x112x3 (that is the proper input for the tensorflow backend). While the training procedure starts the performance is not close to the one I got with 'theano' and is close to random performance. My model looks like:
def createModel():
model = Sequential()
# The first two layers with 32 filters of window size 3x3
model.add(Conv2D(28, (3, 3), padding='same', activation='relu', input_shape=(28, 112, 3)))
# or input_shape = (3, 28, 112) in case of theano backend
model.add(Conv2D(28, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(nClasses, activation='softmax'))
return model
What should I do in order to make the model perform properly? Is there any fundamental difference when the backend is changing except the order of the input tensors?