Here I have one tensor with (1084, 1625, 3) shape.
And I need to reshape it to (none,none,none ,3).
how can i do that?
I used this code but it does not work.
image = tf.cast(img, tf.float32)
image = (image / 127.5) - 1
I don't think you can do that. I think what you're trying to do is turn a 3D tensor into a 4D tensor. I'm guessing this is the origin of your problem. You can do this to add a 4th dimension, because Tensorflow needs it:
import tensorflow as tf
tensor = tf.random.uniform((100, 100, 3), 0, 256, dtype=tf.int32)
new = tf.expand_dims(tensor, axis=0)
print(new.shape)
Out[14]: TensorShape([1, 100, 100, 3])
But then I could be wrong. If this is the case you can provide your error traceback and code.
Related
Im new to tensorflow and Im trying to feed some data with tensorflow.Dataset. Im using Cityscape dataset with 8 different classes. Here is my code:
import os
import cv2
import numpy as np
import tensorflow as tf
H = 256
W = 256
id2cat = np.array([0,0,0,0,0,0,0, 1,1,1,1, 2,2,2,2,2,2, 3,3,3,3, 4,4, 5, 6,6, 7,7,7,7,7,7,7,7,7])
def readImage(x):
x = cv2.imread(x, cv2.IMREAD_COLOR)
x = cv2.resize(x, (W, H))
x = x / 255.0
x = x.astype(np.float32)
return x
def readMask(path):
mask = cv2.imread(path, 0)
mask = cv2.resize(mask, (W, H))
mask = id2cat[mask]
return mask.astype(np.int32)
def preprocess(x, y):
def f(x, y):
image = readImage(x)
mask = readMask(y)
return image, mask
image, mask = tf.numpy_function(f, [x, y], [tf.float32, tf.int32])
mask = tf.one_hot(mask, 3, dtype=tf.int32)
image.set_shape([H, W, 3])
mask.set_shape([H, W, 3])
return image, mask
def tf_dataset(x, y, batch=8):
dataset = tf.data.Dataset.from_tensor_slices((x, y))
dataset = dataset.shuffle(buffer_size=5000)
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch)
dataset = dataset.repeat()
dataset = dataset.prefetch(2)
return dataset
def loadCityscape():
trainPath = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'datasets\\Cityscape\\train')
imagesPath = os.path.join(trainPath, 'images')
maskPath = os.path.join(trainPath, 'masks')
images = []
masks = []
print('Loading images and masks for Cityscape dataset...')
for image in os.listdir(imagesPath):
images.append(readImage(os.path.join(imagesPath, image)))
for mask in os.listdir(maskPath):
if 'label' in mask:
masks.append(readMask(os.path.join(maskPath, mask)))
print('Loaded {} images\n'.format(len(images)))
return images, masks
images, masks = loadCityscape()
dataset = tf_dataset(images, masks, batch=8)
print(dataset)
That last print(dataset) shows:
<PrefetchDataset shapes: ((None, 256, 256, 3), (None, 256, 256, 3)), types: (tf.float32, tf.int32)>
Why am I obtaining (None, 256, 256, 3) instead of (8, 256, 256, 3)? I also have some doubts about how to iterate over this dataset.
Thanks a lot.
Tensorflow is a graph based mathematical framework that abstracts for you all of those complex vectorial or matricial operations you face, particularly in machine learning.
What the developers though is that it would be unconfortable to specify every single time how many input vectors you need to pass in your model for the training, so they decided to abstract it for you.
You will not interested if your model is fed with one single or thousands samples as long as the output matches with the input dimension (but also any internal operation should match in dimensions!).
So the None size is a placeholder for a possible changing shape, that is usually the batch size of the input.
We need a placeholder because (None, 2) is a different shape with respect of just (2,), because in the first case we know we will face 2 dimensions.
Even if the None dimension is unknown when you "compile" your model, it will be evaluated only when it is strictly needed, in other words when you run it. In this way your model will be happy to run on a batch size of 64 as like as 128 samples.
For the rest a (non-scalar) Tensor behaves like a normal numpy array:
tensor1 = tf.constant([ 0, 1, 2, 3]) # shape (4, )
tensor2 = tf.constant([ [0], [1], [2], [3]]) # shape (4, 1)
for x in tensor1:
print(x) # 0, 1, 2, 3
for x in tensor2:
print(x) # Tensor([0]), Tensor([1]), Tensor([2]), Tensor([3])
The only difference is that it can be allocated into any supported device memory (CPU / Cuda GPU).
Iterating through the dataset is just like slicing it at (usually) constant sizes, where that constant is your batch size, which will fill that empty None dimension.
This line of code will be responsible of slicing your dataset into "sub-tensors" ("sub-arrays") composed by its samples:
dataset = dataset.batch(N)
# iterating over it:
for batch in dataset: # I'm taking N samples here
...
Your "runtime" shape will be (N, 256, 256, 3), but if you will try to take an element from the dataset it could still have None in the shape... That's because we can't guarantee, for example, that the dimension of the dataset is exactly divisible by the batch size, so some trailing samples of a variable shape could still be possible. You will hardly get rid off that None dimension, but in some custom methods of your model you could achieve that.
If you are still unconfortable with tensors there is the tensor.numpy() method that gives you back a numpy array, but at the cost of copying it (usually to your CPU). This is not available in every step of the process.
There are many way to define a dataset in tensorflow, I suggest to read how they think you should build an input pipeline, because it will make your life easier if you understand how much tensorflow takes your code at higher levels of abstraction.
My question is how can I reshape the tensor!
Here is one syntax tf.reshape(tensor, shape, name=None)
i do not know , where am i going wrong but i am not able to reshape
how can i do that?
thanks
axis=0
# image is your tensor
tf.expand_dims(your_image, axis)
You can use tf.expand_dims:
output = tf.expand_dims(input, 0)
Try the following code:
#assuming `img` contains the data which is in the format (256,256,3)
output = tf.reshape(img, [1, 256, 256, 3]) # 1 is batch size
currently I'm working on a neural network that can classify the numbers in the Street View House Number dataset (http://ufldl.stanford.edu/housenumbers/). For now, I'm just trying to do it on the second format, the one similar to the MNIST dataset.
The problem I've encountered is that the shapes of the train and test arrays of examples are (HEIGHT, WIDTH, CHANNELS, EXAMPLES) rather than (EXAMPLES, HEIGHT, WIDTH, CHANNELS).
Is there a simple way to reshape the array to what I want without using many nested loops?
I'm not sure if the object you are trying to reshape is a Tensor or numpy.ndarray.
If it is a numpy.ndarray, you can use np.transpose. For example:
import numpy as np
a = np.zeros((299, 299, 3, 50))
print(a.shape) # (299, 299, 3, 50) H x W x C x M
b = np.transpose(a, [3, 0, 1, 2])
print(b.shape) # (50, 299, 299, 3)
If it is a Tensor, You can use tf.transpose to change the order of the dimension in exactly the same way as np.transpose. For example:
import tensorflow as tf
a = tf.zeros((299, 299, 3, 50), dtype=tf.int32)
print(a.shape.as_list()) # [299, 299, 3, 50]
b = tf.transpose(a, [3, 0, 1, 2])
print(b.shape.as_list()) # [50, 299, 299, 3]
I need some help viewing the feature maps in a plant leaf classification program using TensorFlow.
I have a function that takes in any number of images (size 128x128x3) and convolves the images using some filter (size 3x3x32).
layer_conv1 = create_convolutional_layer(input=x,
num_input_channels=num_channels,
conv_filter_size=filter_size_conv1,
num_filters=num_filters_conv1)
print(layer_conv1)
The code outputs a tensor as printed: Tensor("Relu_182:0", shape=(?, 64, 64, 32), dtype=float32)
I am trying to display an image on the console from the tensor, and I've tried the following code (using matplotlib.pyplot):
session.run(tf.global_variables_initializer())
img = session.run(layer_conv1)
plt.imshow(img)
plt.show()
and
""
img = layer_conv1[0,:,:,:].eval(session=session)
""
""
which both don't work.
You must feed a value for placeholder tensor 'x_54' with dtype float and shape [?,128,128,3] is one of the errors that occurs.
You define your layer with
layer_conv1 = create_convolutional_layer(input=x,...)
Here, x is a placeholder defined with something like
x = tf.Placeholder(tf.float32, [None, 128, 128, 3])
When you call img = session.run(layer_conv1) you need to feed a value for x like with
img = session.run(layer_conv1, feed_dict={x: myImage})
where myImage is a numpy array of shape [1, 128, 128, 3] representing your image.
There is a 25*15 image, and i want to identify what it is by using CNN.
When training my CNN, I input a numpy named 'img' as datasets which shape is (200, 375):
sess.run(train, feed_dict={X: imgs, Y: labels}
This numpy contains 200 sample ,each of them have 375 features.
But when i reshape this numpy to a (-1, 25, 15, 1) Tensor:
X = tf.placeholder(tf.float32, [None, 375])
X = tf.reshape(X,[-1,25,15,1])
Something wrong happened:
Cannot feed value of shape (200, 375) for Tensor 'Reshape:0', which has shape '(?, 25, 15, 1)'
I don't know why it can't work, 25*15 is indeed 375.
Thank you!
You don't seem to reshape the dict variable you are feeding to the placeholder. You have to reshape your img variable as well into shape [-1, 25, 15, 1]