So I have been reading other posts about Dense layers and the input shape and unfortunately am just not really grasping how to adjust the input shape. I am trying to replicate a model that is here:
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
import tensorflow as tf
model = Sequential()
model.add(tf.keras.Input(shape=input_shape))
model.add(Dense(64,
activation='tanh'))
model.add(Dense(64,
activation='tanh'))
model.add(Dropout(0.15))
model.compile(loss=root_mean_squared_error,
optimizer=tf.keras.optimizers.Adam(learning_rate))
My inputs have been in batches of 168 elements with 3 features each. To my understanding (which is very limited, I've been learning what I can as I go), this would leave me with an input shape of (168,3). When I input that the error code that comes out is
ValueError: Dimensions must be equal, but are 64 and 3 for '{{node root_mean_squared_error/sub}} =
Sub[T=DT_FLOAT](sequential/dropout/dropout/Mul_1, Cast)' with input shapes: [?,168,64], [?,1,3].
Is there something I am missing? When I do the same thing with an LSTM model I just put the 'input_shape' variable as a parameter in the first LSTM layer. Thank you in advance for helping me and maybe pointing me in the right direction.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout
model = Sequential()
model.add(tf.keras.Input(shape=(168,3)))
model.add(Dense(64,
activation='tanh'))
model.add(Dense(64,
activation='tanh'))
model.add(Dropout(0.15))
model.compile(loss="mean_squared_error",
optimizer=tf.keras.optimizers.Adam(0.1))
print(model.summary())
x = tf.constant(tf.random.normal(shape=(168,3)))
y = tf.constant(tf.random.normal(shape=(168,1)))
model.fit(x=x,y=y)
Related
I build a model consisting of one CNN and one LSTM. CNN to extract the features and pass that to LSTM. I am working with a Multi-class text classification problem, the input for CNN is TF-IDF.
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense,Dropout,Flatten,Activation
from tensorflow.keras.metrics import CategoricalAccuracy, AUC
from tensorflow import keras
import tensorflow as tf
from keras.layers import TimeDistributed
from tensorflow.keras.layers import Conv2D,MaxPooling2D,Conv1D,MaxPooling1D
from keras.layers.recurrent import LSTM
model = Sequential()
model.add(Conv1D(128, 5, activation='relu', input_shape=( 1500,1)))
model.add(MaxPooling1D(pool_size=2))
model.add(LSTM(10))
model.add(Dense(5, activation='softmax'))
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model.summary())
history=model.fit(X_train, y_train, epochs=20,validation_data=(X_test,y_test),batch_size=512)
The error as follows:
ValueError: Input 0 of layer sequential_33 is incompatible with the layer: expected axis -1 of input shape to have value 1 but received input with shape (None, 1, 1500)
the model expects an input_shape of 1, 1500 and you are giving one with 1500,1. change your input in the model from model.add(Conv1D(128, 5, activation='relu', input_shape=( 1500,1))) to model.add(Conv1D(128, 5, activation='relu', input_shape=( 1, 1500)))
but there might be more errors with the format after that i'm not sure if you can just add the lstm in your model.
I'm trying to train a simple model over some picture data that belongs to 10 classes.
The images are in B/W format (not gray scale), I'm using the image_dataset_from_directory to import the data into python as well as split it into validation/training sets.
My code is as below:
My Imports
import matplotlib.pyplot as plt
import numpy as np
import os
import PIL
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.layers import Dense
Read Image Data
trainDT = tf.keras.preprocessing.image_dataset_from_directory(
data_path,
labels="inferred",
label_mode="categorical",
class_names=['0','1','2','3','4','5','6','7','8','9'],
color_mode="grayscale",
batch_size=4,
image_size=(256, 256),
shuffle=True,
seed=44,
validation_split=0.1,
subset='validation',
interpolation="bilinear",
follow_links=False,
)
Model Creation/Compile/Fit
model = Sequential([
Dense(units=128, activation='relu', input_shape=(256,256,1), name='h1'),
Dense(units=64, activation='relu',name='h2'),
Dense(units=16, activation='relu',name='h3'),
layers.Flatten(name='flat'),
Dense(units=10, activation='softmax',name='out')
],name='1st')
model.summary()
model.compile(optimizer='adam' , loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(x=trainDT, validation_data=train_data, epochs=10, verbose=2)
The model training returns an error:
InvalidArgumentError Traceback (most recent call last)
....
/// anaconda paths and anaconda python code snippets in the error reporting \\\
....
InvalidArgumentError: Matrix size-incompatible: In[0]: [1310720,3], In[1]: [1,128]
[[node 1st/h1/Tensordot/MatMul (defined at <ipython-input-38-58d6507e2d35>:1) ]] [Op:__inference_test_function_11541]
Function call stack:
test_function
I don't understand where the size mismatch comes from, I've spent a few hours looking around for a solution and trying different things but nothing seems to work for me.
Appreciate any help, thank you in advance!
Dense layers expect flat input (not 3d tensor), but you are sending (256,256,1) shaped tensor into the first dense layer. If you want to use dense layers from the beginning then you will need to move the flatten to be the first layer or you will need to properly reshape your data.
model = tf.keras.Sequential([
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation="relu"),
tf.keras.layers.Dense(10, activation="softmax")
])
Also, the flatten between 2 dense layers makes no sense because the output of a dense layer is flat anyway.
From the structure of your model (especially the flatten placement), I assume that
those dense layers were supposed to be convolutional layers instead.
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(32, (3, 3), activation='relu'),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation="relu"),
tf.keras.layers.Dense(10, activation="softmax")
])
Convolutional layers can process 2D input and they will also produce more dimensional output which you need to flatten before passing it to the dense top (note that you can add more convolutional layers).
Hy mhk777 Hope you are doing well. Brother, I think that you are confusing dense layers with convolution layers. You have to apply some convolution layers to the image before giving it to dense layers. If you don't want to apply convolution than you have to give 2d array to the dense layer i.e (number of samples, data)
import tensorflow as tf
from tensorflow.keras import datasets, layers, models
model = models.Sequential()
# Here are convolutional Layer
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(256,256,1)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
# Here are your dense layers
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10))
model.summary()
model.compile(optimizer='adam' , loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(x=trainDT, validation_data=train_data, epochs=10, verbose=2)
Ciao,
I'm working with CNN 1d on Keras but I have tons of troubles with the input shape variable.
I have a time series of 100 timesteps and 5 features with boolean labels. I want to train a CNN 1d that works with a sliding window of length 10. This is a very simple code I wrote:
from keras.models import Sequential
from keras.layers import Dense, Conv1D
import numpy as np
N_FEATURES=5
N_TIMESTEPS=10
X = np.random.rand((100, N_FEATURES))
Y = np.random.randint(0,2, size=100)
# CNN
model.Sequential()
model.add(Conv1D(filter=32, kernel_size=N_TIMESTEPS, activation='relu', input_shape=N_FEATURES
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
My problem here is that I get the following error:
File "<ipython-input-2-43966a5809bd>", line 2, in <module>
model.add(Conv1D(filter=32, kernel_size=10, activation='relu', input_shape=N_FEATURES))
TypeError: __init__() takes at least 3 arguments (3 given)
I've also tried by passing to the input_shape the following values:
input_shape=(None, N_FEATURES)
input_shape=(1, N_FEATURES)
input_shape=(N_FEATURES, None)
input_shape=(N_FEATURES, 1)
input_shape=(N_FEATURES, )
Do you know what's wrong with the code or in general can you explain the logic behind in input_shape variable in Keras CNN?
The crazy thing is that my problem is the same of the following:
Keras CNN Error: expected Sequence to have 3 dimensions, but got array with shape (500, 400)
But I cannot solve it with the solution given in the post.
The Keras version is 2.0.6-tf
Thanks
This should work:
from keras.models import Sequential
from keras.layers import Dense, Conv1D
import numpy as np
N_FEATURES=5
N_TIMESTEPS=10
X = np.random.rand(100, N_FEATURES)
Y = np.random.randint(0,2, size=100)
# Create a Sequential model
model = Sequential()
# Change the input shape to input_shape=(N_TIMESTEPS, N_FEATURES)
model.add(Conv1D(filters=32, kernel_size=N_TIMESTEPS, activation='relu', input_shape=(N_TIMESTEPS, N_FEATURES)))
# If it is a binary classification then you want 1 neuron - Dense(1, activation='sigmoid')
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
Please see the comments before each line of code. Moreover, the input shape that Conv1D expects is (time_steps, feature_size_per_time_step). The translation of that for your code is (N_TIMESTEPS, N_FEATURES).
I'm trying to train a CNN on word vectors generated using the gensim library. After I have generated all of my data in numeric form, I try to pass it on to a CNN model using Keras when I get the following error:
ValueError: Error when checking input: expected conv2d_1_input to have 4 dimensions, but got array with shape (20000, 250, 50)
I've searched on this problem for hours, and all of the solutions posted for similar/same issues haven't been able to solve this error for me. Can anyone see where I'm going wrong with the input dimensions? I've generated some random numpy data that recreates the error:
import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Convolution2D, Flatten, Dropout
from keras.layers.embeddings import Embedding
from keras.preprocessing import sequence
from keras.callbacks import TensorBoard
t = np.random.rand(20000,250,50)
l = np.random.rand(20000,1)
embedding_vecor_length = 50
net = Sequential()
net.add(Convolution2D(64, 3,input_shape=(1,250,50),
data_format='channels_first'))
# Convolutional model (3x conv, flatten, 2x dense)
net.add(Convolution2D(32,(3), padding='same'))
net.add(Convolution2D(16,(3), padding='same'))
net.add(Convolution2D(8,(3), padding='same'))
net.add(Flatten())
net.add(Dropout(0.2))
net.add(Dense(180,activation='sigmoid'))
net.add(Dropout(0.2))
net.add(Dense(1,activation='sigmoid'))
net.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
tensorBoardCallback = TensorBoard(log_dir='./logs', write_graph=True)
net.summary()
net.fit(t, l, epochs=3, callbacks=[tensorBoardCallback], batch_size=64)
Convolutions use 4 dimensions. Considering you're using "channels_first":
Images
Channels
Side 1
Side 2
Your input is missing the channels.
t = np.random.rand(20000,1,250,50)
I have taken a standard ResNet50 model:
model = keras.applications.resnet50.ResNet50(include_top=False,
weights='imagenet',
classes=10,
input_shape=(224, 224, 3))
And added several dense layers of my own:
top_model = Sequential()
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(2, activation='softmax'))
model = Model(input=model.input, output=top_model(model.output))
This way it works great, however, when I want to delete last Dense and Dropout layers with model.pop() keras wont work well:
model.layers[-1].layers
[<keras.layers.core.Flatten at 0x16b5c00b8>,
<keras.layers.core.Dense at 0x16b5c0320>,
<keras.layers.core.Dropout at 0x16b5c02e8>,
<keras.layers.core.Dense at 0x16b5c0d68>]
model.layers[-1].pop()
model.layers[-1].pop()
model.layers[-1].layers
[<keras.layers.core.Flatten at 0x1ae6e5940>,
<keras.layers.core.Dense at 0x1ae6e9e10>]
model.layers[-1].outputs = [model.layers[-1].layers[-1].output]
model.outputs = model.layers[-1].outputs
model.layers[-1].layers[-1].outbound_nodes = []
Then I just compile the model and when trying to predict, I get an error:
You must feed a value for placeholder tensor 'flatten_7_input_12' with dtype float and shape [?,1,1,2048]
model.pop() takes care of all the underlying settings, including setting the model.output to the output of the new last layer. Therefore, you don't need to handle anything regarding the outputs.
Please also note that you are assigning to the model variable; thus, model.outputs is already referring to the extended model's output.
Here is a sample code that works fine on keras 2.0.6 with TensorFlow backend (1.4.0):
import keras
from keras.models import Sequential, Model
from keras.layers import *
import numpy as np
model = keras.applications.resnet50.ResNet50(include_top=False,
weights='imagenet',
classes=10,
input_shape=(224, 224, 3))
top_model = Sequential()
top_model.add(keras.layers.Flatten(input_shape=model.output_shape[1:]))
top_model.add(keras.layers.Dense(256, activation='relu'))
top_model.add(keras.layers.Dropout(0.5))
top_model.add(keras.layers.Dense(2, activation='softmax'))
model_extended = Model(input=model.input, output=top_model(model.output))
model_extended.layers[-1].pop()
model_extended.layers[-1].pop()
model_extended.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model_extended.predict(np.zeros((1, 224, 224, 3)))