I'm trying to use Perceptron to reduce a tensor of size: [1, 24, 768] to another tensor with size of [1, 1, 768]. The only way I could use was to first reshape the input tensor to [1, 1, 24*768] and then pass it through linear layers. I'm wondering if there's a more elegant way of this transformation --other than using RNNs cause I do not want to use that. Are there other methods generally for the transformation that I want to make? Here is my code for doing the above operation:
lin = nn.Linear(24*768, 768)
# x is in shape of [1, 24, 768]
# out is in shape of [1, 1, 768]
x = x.view(1,1,-1)
out = lin(x)
If the broadcasting is what's bothering you, you could use a nn.Flatten to do it:
>>> m = nn.Sequential(
... nn.Flatten(),
... nn.Linear(24*768, 768))
>>> x = torch.rand(1, 24, 768)
>>> m(x).shape
torch.Size([1, 768])
If you really want the extra dimension you can unsqueeze the tensor on axis=1:
>>> m(x).unsqueeze(1).shape
torch.Size([1, 1, 768])
I have an array X_train containing 9957 images. I am making a Convolutional network.The desired shape of the array for feeding into the model is (batchsize, channel, height, width)
X_train.shape #gives (9957, 60, 80, 3)
X_train[1].shape #gives (60, 80, 3)
If we use
np.reshape(X_train,(-1, 3, 60, 80)) #it gives (9957, 3, 60, 80)
How can I get each array with shape (batchsize, 3, 60, 80) and the final image array for training with shape(9957, batchsize, 3, 60, 80)?
You can get from i-th image until i + batchsize image as follows.
batchsize = 16
i = 0
X_batch = X_train[i: i+batchsize]
print('X_batch.shape: ', X_batch.shape) # it should be (16, 3, 60, 80)
Please change i with for loop to get each image. For example,
for i in range(0, len(X_train), batchsize):
X_batch = X_train[i: i+batchsize]
# --- Do something with X_batch ---
currently I'm working on a neural network that can classify the numbers in the Street View House Number dataset (http://ufldl.stanford.edu/housenumbers/). For now, I'm just trying to do it on the second format, the one similar to the MNIST dataset.
The problem I've encountered is that the shapes of the train and test arrays of examples are (HEIGHT, WIDTH, CHANNELS, EXAMPLES) rather than (EXAMPLES, HEIGHT, WIDTH, CHANNELS).
Is there a simple way to reshape the array to what I want without using many nested loops?
I'm not sure if the object you are trying to reshape is a Tensor or numpy.ndarray.
If it is a numpy.ndarray, you can use np.transpose. For example:
import numpy as np
a = np.zeros((299, 299, 3, 50))
print(a.shape) # (299, 299, 3, 50) H x W x C x M
b = np.transpose(a, [3, 0, 1, 2])
print(b.shape) # (50, 299, 299, 3)
If it is a Tensor, You can use tf.transpose to change the order of the dimension in exactly the same way as np.transpose. For example:
import tensorflow as tf
a = tf.zeros((299, 299, 3, 50), dtype=tf.int32)
print(a.shape.as_list()) # [299, 299, 3, 50]
b = tf.transpose(a, [3, 0, 1, 2])
print(b.shape.as_list()) # [50, 299, 299, 3]
There is a 25*15 image, and i want to identify what it is by using CNN.
When training my CNN, I input a numpy named 'img' as datasets which shape is (200, 375):
sess.run(train, feed_dict={X: imgs, Y: labels}
This numpy contains 200 sample ,each of them have 375 features.
But when i reshape this numpy to a (-1, 25, 15, 1) Tensor:
X = tf.placeholder(tf.float32, [None, 375])
X = tf.reshape(X,[-1,25,15,1])
Something wrong happened:
Cannot feed value of shape (200, 375) for Tensor 'Reshape:0', which has shape '(?, 25, 15, 1)'
I don't know why it can't work, 25*15 is indeed 375.
Thank you!
You don't seem to reshape the dict variable you are feeding to the placeholder. You have to reshape your img variable as well into shape [-1, 25, 15, 1]
I am trying an Op that is not behaving as expected.
graph = tf.Graph()
with graph.as_default():
train_dataset = tf.placeholder(tf.int32, shape=[128, 2])
embeddings = tf.Variable(
tf.random_uniform([50000, 64], -1.0, 1.0))
embed = tf.nn.embedding_lookup(embeddings, train_dataset)
embed = tf.reduce_sum(embed, reduction_indices=0)
So I need to know the dimensions of the Tensor embed. I know that it can be done at the run time but it's too much work for such a simple operation. What's the easier way to do it?
I see most people confused about tf.shape(tensor) and tensor.get_shape()
Let's make it clear:
tf.shape
tf.shape is used for dynamic shape. If your tensor's shape is changable, use it.
An example: a input is an image with changable width and height, we want resize it to half of its size, then we can write something like:
new_height = tf.shape(image)[0] / 2
tensor.get_shape
tensor.get_shape is used for fixed shapes, which means the tensor's shape can be deduced in the graph.
Conclusion:
tf.shape can be used almost anywhere, but t.get_shape only for shapes can be deduced from graph.
Tensor.get_shape() from this post.
From documentation:
c = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
print(c.get_shape())
==> TensorShape([Dimension(2), Dimension(3)])
A function to access the values:
def shape(tensor):
s = tensor.get_shape()
return tuple([s[i].value for i in range(0, len(s))])
Example:
batch_size, num_feats = shape(logits)
Just print out the embed after construction graph (ops) without running:
import tensorflow as tf
...
train_dataset = tf.placeholder(tf.int32, shape=[128, 2])
embeddings = tf.Variable(
tf.random_uniform([50000, 64], -1.0, 1.0))
embed = tf.nn.embedding_lookup(embeddings, train_dataset)
print (embed)
This will show the shape of the embed tensor:
Tensor("embedding_lookup:0", shape=(128, 2, 64), dtype=float32)
Usually, it's good to check shapes of all tensors before training your models.
Let's make it simple as hell. If you want a single number for the number of dimensions like 2, 3, 4, etc., then just use tf.rank(). But, if you want the exact shape of the tensor then use tensor.get_shape()
with tf.Session() as sess:
arr = tf.random_normal(shape=(10, 32, 32, 128))
a = tf.random_gamma(shape=(3, 3, 1), alpha=0.1)
print(sess.run([tf.rank(arr), tf.rank(a)]))
print(arr.get_shape(), ", ", a.get_shape())
# for tf.rank()
[4, 3]
# for tf.get_shape()
Output: (10, 32, 32, 128) , (3, 3, 1)
The method tf.shape is a TensorFlow static method. However, there is also the method get_shape for the Tensor class. See
https://www.tensorflow.org/api_docs/python/tf/Tensor#get_shape
To create tensor in tensorflow using tf.constant()
This is to import the library
import tensorflow as tf
This will create the tensor
tensor = tf.constant([[[2,4,5], [5,6,6]], [[9,7,8], [4,8,2]], [[7,1,3], [4,8,9]]])
This will show the tensor
tensor
this will show the number of dimension
tensor.ndim
#create a tensor
tensor = tf.constant([[[1, 2, 3],
[3, 4, 5]],
[[5, 6, 7],
[8, 6, 9]],
[[2, 1, 5],
[5, 7, 8]]])
tensor
#Display result
<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy= array([[[1, 2, 3],[3, 4, 5]],
[[5, 6, 7],
[8, 6, 9]],
[[2, 1, 5],
[5, 7, 8]]], dtype=int32)>