I have a network in which the data comes from a stack of images and a vector of numbers.
I begin with two "branches": The images go through several convolutions and produce an output of shape (50, 50, 64). In the other branch I import the number and go:
x = Input(shape = (13)) # data vector is of length 13
x = Dense(50*50)(x)
x = Reshape((50,50))(x)
I now have 2 outputs from the branches - one is of shape (50, 50, 64) and the other of shape (50, 50, 1). How can I "stick" them together to get a collective (50, 50, 65) which I'd then Deconv2D?
You could use the keras Concatenate() layer as in follows:
import numpy as np
from keras import backend as K
from keras import layers
# create some dummy tensors with numpy and the keras backend
arr1 = K.constant(np.zeros((50, 50, 1)))
arr2 = K.constant(np.zeros((50, 50, 64)))
# and this is how you call the concatenate layer
combined = layers.Concatenate()([arr1, arr2])
# it should print this:
# Tensor("concatenate_1/concat:0", shape=(50, 50, 65), dtype=float32)
print(combined)
you can use numpy function : np.c_
try:
>>> x.shape
(50, 50, 64)
>>> y.shape
(50, 50, 1)
>>> z = np.c_[x,y]
>>> z.shape
(50, 50, 65)
Related
The keras model is like this:
input_x = Input(shape=input_shape)
x=Conv2D(...)(input_x)
...
y_pred1 = Conv2D(...)(x) # shape of (None, 80, 80, 2)
y_pred2 = Dense(...)(x) # shape of (None, 4)
y_merged = Concatenate(...)([y_pred1, y_pred2])
model = Model(input_x, y_merged)
y_pred1 and y_pred2 are the results I want the model to learn to predict.
But the loss function fcn1 for the y_pred1 branch need y_pred2 prediction results, so I have to concatenate the results of the two branches to get y_merged, so that fcn1 will have access to y_pred2.
The problem is, I want to use the Concatenate layer to concatenate the y_pred1 (None, 4) output with the y_pred2 (None, 80, 80, 2) output, but I don't know how to do that.
How can I reshape the (None, 4) to (None, 80, 80, 1)? For example, by filling the (None, 80, 80, 1) with the 4 elements in y_pred2 and zeros.
Is there any better solutions than using the Concatenate layer?
Maybe this extracted piece of code could help you:
tf.print(condi_input.shape)
# shape is TensorShape([None, 1])
condi_i_casted = tf.expand_dims(condi_input, 2)
tf.print(condi_i_casted.shape)
# shape is TensorShape([None, 1, 1])
broadcasted_val = tf.broadcast_to(condi_i_casted, shape=tf.shape(decoder_outputs))
tf.print(broadcasted_val.shape)
# shape is TensorShape([None, 23, 256])
When you want to broadcast a value, first think about what exactly you want to broadcast. In this example, condi_input has shape(None,1) and helped me as a condition for my encoder-decoder lstm network. To match all dimensionalities, of the encoder states of the lstm, first I had to use tf.expand_dims() to expand the condition value from a shape like [[1]] to [[[1]]].
This is what you need to do first. If you have a prediction as a softmax from the dense layers, you might want to use tf.argmax() first, so you only have one value, which is way easier to broadcast. However, its also possible with 4 but keep in mind, that the dimensions need to match. You cannot broadcast shape(None,4) to shape(None,6), but to shape(None,8) since 8 is devidable through 4.
Then you you can use tf.broadcast() to broadcast your value into the desired shape. Then you have two shapes, you can concatenate together.
hope this helps you out.
Figured it out, the code is like this:
input_x = Input(shape=input_shape)
x=Conv2D(...)(input_x)
...
y_pred1 = Conv2D(...)(x) # shape of (None, 80, 80, 2)
y_pred2 = Dense(4)(x) # (None, 4)
# =========transform to concatenate:===========
y_pred2_matrix = Lambda(lambda x: K.expand_dims(K.expand_dims(x, -1)))(y_pred2) # (None,4, 1,1)
y_pred2_matrix = ZeroPadding2D(padding=((0,76),(0,79)))(y_pred2_matrix) # (None, 80, 80,1)
y_merged = Concatenate(axis=-1)([y_pred1, y_pred2_matrix]) # (None, 80, 80, 3)
The 4 elements of y_pred2 can be indexed as y_merged[None, :4, 0, 2]
I have a medical imaging dataset with a dimension of (80,80,2900), each image is 80*80. First I loaded the mat file of the data as follow:
data = loadmat('cardiac-dig.mat')
images_LV = np.array (data['images_LV'])
val_data_size = 500
valid_images = images_LV[:,:,:val_data_size]
train_images = images_LV[:,:,val_data_size:]
valid_masks = masks[:,:,:val_data_size]
train_masks = masks[:,:,val_data_size:]
when I tried to fit the model using this:
model.fit(train_images , train_masks, epochs=2, batch_size=8)
I end up this error which says the input should be 4 dimensions:
Error when checking input: expected input_6 to have 4 dimensions, but got array with shape (80, 80, 2400)
I tried to reshape the input to 4 dimensions using:
images_LV = np.reshape(images_LV, (-1, 80,80,2900))
but I got other error:
Error when checking input: expected input_6 to have shape (80, 80, 1) but got array with shape (80, 80, 2400)
I think that the input should be like (2900,80,80,1)?
if you need more information I can share.
I found the solution. I wanted to post the answer to those may have the same issue:
based on the shape of my dataset, (80,80,2900), I needed to change the dimension from 3 to 4. Also, it was necessary to reshape the dataset as follow:
***images = np.swapaxes(images, 0, 2)
images = np.swapaxes(images, 1, 2)
images = np.reshape(images, (-1, 80,80,1))
print(images.shape)
\\(2900, 80, 80, 1)***
After above change, I simply pass the data as the input of the model:
***train_images = images[val_data_size:,:,:,:]
model.fit(train_images , train_masks, epochs=5, batch_size=8)***
I have two tensors.
v, shape=(50, 64, 128), dtype=float32
m, shape=(64, 50, 1), dtype=int32
Values in m are integers between 0 and 50 (<=49)
I want to use values of m, to get a specific tensor of v for the same index out of 64.
Resulting tensor is r: shape=(64, 50, 128), dtype=float32
For example values for r(i, j, 0-128) = v(m(i, j), i, 0-128)
The closest thing I see is tf.nn.embedding_lookup but I'm not sure how to use it for this use case
You can use the following tf.nn.embedding_lookup or tf.gather_nd methods to achieve your goals.
import tensorflow as tf
import numpy as np
m_np = np.random.randint(0,50,(64, 50, 1))
m = tf.constant(m_np)
n = tf.random.normal((50, 64, 128))
# Method 1
tmp = tf.nn.embedding_lookup(n,m[:,:,0]) # shape=(64,50,64,128)
tmp = tf.transpose(tmp,[1,3,0,2]) # shape=(50,128,64,64)
result1 = tf.transpose(tf.matrix_diag_part(tmp),[2,0,1]) # shape=(64,50,128)
# Method 2
indices = tf.tile(tf.reshape(tf.range(64),(-1,1,1)),(1,50,1)) # shape=(64,50,1)
indices = tf.concat([m,indices],axis=-1) # shape=(64,50,2)
result2 = tf.gather_nd(n,indices) # shape=(64,50,128)
with tf.Session() as sess:
# Randomly select a location for test
n_value,result_value = sess.run([n,result1])
print((n_value[m_np[5,4],5,:]==result_value[5,4]).all())
# True
I have an array X_train containing 9957 images. I am making a Convolutional network.The desired shape of the array for feeding into the model is (batchsize, channel, height, width)
X_train.shape #gives (9957, 60, 80, 3)
X_train[1].shape #gives (60, 80, 3)
If we use
np.reshape(X_train,(-1, 3, 60, 80)) #it gives (9957, 3, 60, 80)
How can I get each array with shape (batchsize, 3, 60, 80) and the final image array for training with shape(9957, batchsize, 3, 60, 80)?
You can get from i-th image until i + batchsize image as follows.
batchsize = 16
i = 0
X_batch = X_train[i: i+batchsize]
print('X_batch.shape: ', X_batch.shape) # it should be (16, 3, 60, 80)
Please change i with for loop to get each image. For example,
for i in range(0, len(X_train), batchsize):
X_batch = X_train[i: i+batchsize]
# --- Do something with X_batch ---
I have a (hopefully) quick Numpy question, and I hope you can help me.
I want to use numpy.reshape to convert (5000, 32, 32, 3) into (5000, 3072), and the only clue I got for the assignment is this:
# Reshape each image data into a 1-dim array
print (X_train.shape, X_test.shape) # Should be: (5000, 32, 32, 3) (500, 32, 32, 3)
#####################################################################
# TODO (2): #
# Reshape the image data to one dimension. #
# #
# Hint: Look at the numpy reshape function and have a look at -1 #
# option #
#####################################################################
X_train =
X_test =
#####################################################################
# END OF YOUR CODE #
#####################################################################
print (X_train.shape, X_test.shape) # Should be: (5000, 3072) (500, 3072)
I've been spending the last day scouring Google for examples, but apparently this is too trivial to warrant an ask. Help?
You can simply do:
X_train = np.reshape(X_train, (5000, -1))
X_test = np.reshape(X_test, (500, -1))
Working example:
import numpy as np
a = np.zeros((5000,32,32,3))
b = np.reshape(a, (5000, -1))
print(a.shape)
print(b.shape)
# Output
# (5000, 32, 32, 3)
# (5000, 3072)
numpy.reshape will try to fit the source array a into an array with first dimension of length 5000. The -1 tells reshape to adjust the length of the second dimension depending on the total length of the source array a.