I'm using this library to create a model to learn graphs. Here is the code (from repository):
import numpy as np
from keras_gcn.backend import keras
from keras_gcn import GraphConv
# feature matrix
input_data = np.array([[[0, 1, 2],
[2, 3, 4],
[4, 5, 6],
[7, 7, 8]]])
# adjacency matrix
input_edge = np.array([[[1, 1, 1, 0],
[1, 1, 0, 0],
[1, 0, 1, 0],
[0, 0, 0, 1]]])
labels = np.array([[[1],
[0],
[1],
[0]]])
data_layer = keras.layers.Input(shape=(None, 3), name='Input-Data')
edge_layer = keras.layers.Input(shape=(None, None), dtype='int32', name='Input-Edge')
conv_layer = GraphConv(units=4, step_num=1, kernel_initializer='ones',
bias_initializer='ones', name='GraphConv')([data_layer, edge_layer])
model = keras.models.Model(inputs=[data_layer, edge_layer], outputs=conv_layer)
model.compile(optimizer='adam', loss='mae', metrics=['mae'])
model.fit([input_data, input_edge], labels)
However, when I run the code I get the following error:
ValueError: Error when checking target: expected GraphConv to have 3 dimensions, but got array with shape (4, 1)
while the shape of labels is (1, 4, 1)
You should encode your labels using onehot-encoder, something like the following:
lables = np.array([[[0, 1],
[1, 0],
[0, 1],
[1, 0]]])
Also number of units in GraphConv layer should be equal to the number of unique labels which is 2 in your case.
I think the issue is mismatch between the shapes of your edge_layer and data_layer.
When you use the function keras.layers.Input you're giving data_layer a shape of shape=(None, 3) and then you're giving edge_layer a shape of shape=(None, None)
Match the shapes and let me know how it goes.
Related
I have a 4D array of input that I would like to normalise using MinMaxScaler. For simplicity, I give an example with the following array:
A = np.array([
[[[0, 1, 2, 3],
[3, 0, 1, 2],
[2, 3, 0, 1],
[1, 3, 2, 1],
[1, 2, 3, 0]]],
[[[9, 8, 7, 6],
[5, 4, 3, 2],
[0, 9, 8, 3],
[1, 9, 2, 3],
[1, 0, -1, 2]]],
[[[0, 7, 1, 2],
[1, 2, 1, 0],
[0, 2, 0, 7],
[-1, 3, 0, 1],
[1, 0, 1, 0]]]
])
A.shape
(3,1,5,4)
In the given example, the array contains 3 input samples, where each sample has the shape (1,5,4). Each column of the input represents 1 variable (feature), so each sample has 4 features.
I would like to normalise the input data, But MinMaxScaler expects a 2D array (n_samples, n_features) like dataframe.
How then do I use it to normalise this input data?
Vectorize the data
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
A_sq = np.squeeze(A)
print(A_sq.shape)
# (3, 5, 4)
scaler.fit(np.squeeze(A_sq).reshape(3,-1)) # reshape to (3, 20)
#MinMaxScaler()
You can use the below code to normalize 4D array.
from sklearn.preprocessing import MinMaxScaler, StandardScaler
scaler = MinMaxScaler(feature_range=(0, 1))
def norm(arr):
arrays_list=list()
objects_list=list()
for i in range(arr.shape[0]):
temp_arr=arr[i]
temp_arr=temp_arr[0]
scaler.fit(temp_arr)
temp_arr=scaler.transform(temp_arr)
objects_list.append(scaler)
arrays_list.append([temp_arr])
return objects_list,np.array(arrays_list)
pass the array to the function like
objects,array=norm(A)
it will return a list of MinMax objects and your original array with normalize values.
Output:
" If you want a scaler for each channel, you can reshape each channel of the data to be of shape (10000, 5*5). Each channel (which was previously 5x5) is now a length 25 vector, and the scaler will work. You'll have to transform your evaluation data in the same way with the scalers in channel_scalers."
Maybe this will help, not sure if this is what you're looking for exactly, but...
Python scaling with 4D data
I am working with BERT context vectors and I am trying to extract specific layer activations for specific tokens. I can extract sequential layers fine using ":" slice notation but I want specific layers given by a list (or some other method) e.g. first and fourth layers only.
# (num_target_tokens, num_tokens_in_sequence, num_bert_layers, size_of_bert_vector)
example = torch.randn([3, 12, 13, 768])
indices = torch.tensor([[0, 1], [1, 10], [2, 11]])
a = example[indices[:, 0], indices[:, 1], -4:]
b = example[indices[:, 0], indices[:, 1], 1:5]
# (num_target_tokens, num_specified_layers, size_of_bert_vector)
a.shape
>> torch.Size([3, 4, 768])
b.shape
>> torch.Size([3, 4, 768])
# Desired output shape: torch.Size([3, 4, 768])
c = example[indices[:, 0], indices[:, 1], [1, 3, 5, 7]] # Desired usage
>> shape mismatch: indexing tensors could not be broadcast together with shapes [3], [3], [4]
Is there some elegant way to achieve this with indexing or will I need to split my tensors to achieve this result.
text_tensor is a tensor in shape [None,sequence_max_length,embedding_dim] that contains embedding look-up of a batch of sequences. The sequences are padded using zeros. I need to obtain a list named text_lengths in shape [None] (None is the batch size) that contains the length of each sequence without paddings. I've tried a couple of scripts.
The nearest I've got is the code below:
text_lens = tf.math.reduce_sum(tf.cast(tf.math.not_equal(text_tensor, tf.as_tensor(numpy.zeros([embedding_dim]))), dtype=tf.int32), axis=-1)
But still calculates the lengths incorrectly. Can anyone help me with this?
If I've understood this correctly, after the sequence's original length you get 0s of size embedding_dim for the remaining indices of the first axis.
import tensorflow as tf
# batch_size = 2, first sequence length = 1, second sequence length = 3
data = [[[1, 1, 1, 1],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
[[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[0, 0, 0, 0]]]
with tf.compat.v1.Session() as sess:
tensor = tf.constant(data, dtype=tf.int32)
check = tf.reduce_all(tf.not_equal(tensor, 0), axis=-1)
lengths = tf.reduce_sum(tf.cast(check, tf.int32), axis=-1)
print(sess.run(lengths))
Output
[1 3]
Suppose I have two tensors in tensorflow, A, B (of the same shape). Suppose these are both sparse. I need to know a count of the instances where one of these tensors has a nonzero value at a given index, while the other tensor has a zero value. So, I am looking for a number of locations (i,j pairs) where one matrix has a nonzero value there and the other matrix has a zero value there. How do I do this efficiently?
I would do as follows:
import tensorflow as tf
tensor1 = tf.constant([[0, 1], [0, 2]])
tensor2 = tf.constant([[1, 0], [0, 2]])
a = tf.math.equal(tensor1, tf.zeros_like(tensor1))
b = tf.math.equal(tensor2, tf.zeros_like(tensor2))
c = tf.math.equal(a, b)
c = tf.cast(c, tf.int32)
c = tf.math.reduce_sum(c)
import tensorflow as tf
a = tf.sparse.SparseTensor(
[[0,1], [1,1]], [1,2], [2,2]
)
b = tf.sparse.SparseTensor(
[[0,0], [0,1],[1,0]], [1,2,1], [2,2]
)
res = tf.reduce_sum(
tf.cast(tf.math.logical_xor(
tf.math.not_equal(tf.sparse.to_dense(a), 0),
tf.math.not_equal(tf.sparse.to_dense(b), 0)
), 'int32')
)
This would do it. It sums the True cases according to these conditions, element-wise:
a and b have non-equal values
a is not zero
b is not zero
tf.reduce_sum(
tf.cast(
tf.logical_and(
tf.not_equal(tf.sparse.to_dense(a), tf.sparse.to_dense(b)),
tf.cast(tf.sparse.to_dense(a), tf.bool),
tf.cast(tf.sparse.to_dense(b), tf.bool)),
tf.int32))
Based on these two sparse tensors:
<tf.Tensor: shape=(3, 3), dtype=int32, numpy=
array([[1, 0, 2],
[0, 1, 0],
[0, 2, 1]])>
<tf.Tensor: shape=(3, 3), dtype=int32, numpy=
array([[2, 2, 2],
[0, 0, 0],
[0, 2, 1]])>
Complete example:
import tensorflow as tf
a = tf.SparseTensor(indices=[[0, 0], [0, 2], [1, 1], [2, 1], [2, 2]],
values=[1, 2, 1, 2, 1], dense_shape=[3, 3])
b = tf.SparseTensor(indices=[[0, 0], [0, 1], [0, 2], [2, 1], [2, 2]],
values=[2, 2, 2, 2, 1], dense_shape=[3, 3])
tf.sparse.to_dense(a)
tf.sparse.to_dense(b)
tf.reduce_sum(
tf.cast(
tf.logical_and(
tf.not_equal(tf.sparse.to_dense(a), tf.sparse.to_dense(b)),
tf.cast(tf.sparse.to_dense(a), tf.bool),
tf.cast(tf.sparse.to_dense(b), tf.bool)),
tf.int32))
I'm writing a very simple network:
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
training_data = np.array([[1, 1, 1], [2, 3, 1], [0, -1, 4], [0, 3, 0], [10, -6, 8], [-3, -12, 4]])
testing_data = np.array([6, 11, 1, 9, 10, -38])
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(units = 1, activation = tf.keras.activations.relu, input_shape = (3, )))
model.compile(optimizer = tf.keras.optimizers.RMSprop(0.001), loss = tf.keras.losses.mean_squared_error, metrics = tf.keras.metrics.mean_squared_error)
model.summary()
model.fit(training_data, testing_data, epochs = 1, verbose = 'False')
print("Traning completed.")
model.predict(np.array([1, 1, 1]))
The goal is to train the weights like : aX + bY + cZ = (output)
But I get the error
ValueError: Input 0 of layer sequential_54 is incompatible with the layer: expected axis -1 of input shape to have value 3 but received input with shape [None, 1]
I can't make scene of the dimensions, there is something I'm doing wrong! Any help?
In Keras when you specify the input shape batch size is ignored, please refer here for more details. Your declaration of input_shape = (3, ) is correct, but when you do inference you need to account for the batch size as well by adding an extra dimension for the same so instead of np.array([1, 1, 1]) you need to have np.array([[1, 1, 1]]).
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
training_data = np.array([[1, 1, 1], [2, 3, 1], [0, -1, 4], [0, 3, 0], [10, -6, 8], [-3, -12, 4]])
testing_data = np.array([6, 11, 1, 9, 10, -38])
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(units = 1, activation = tf.keras.activations.relu, input_shape = (3,)))
model.compile(optimizer = tf.keras.optimizers.RMSprop(0.001), loss = tf.keras.losses.mean_squared_error, metrics = [tf.keras.metrics.mean_squared_error])
model.summary()
model.fit(training_data, testing_data, epochs = 1, verbose = 'False')
print("Traning completed.")
model.predict(np.array([[1, 2, 1]]))
array([[0.08026636]], dtype=float32)