I'd like to combine two variable length tensors.
Since they don't match in shape I can't use tf.concat or tf.stack.
So I thought I'd flatten one and then append it to each element of the other - but I don't see how to do that.
For example,
a = [ [1,2], [3,4] ]
flat_b = [5, 6]
combine(a, flat_b) would be [ [ [1,5,6], [2,5,6] ],
[ [3,5,6], [4,5,6] ] ]
Is there a method like this?
Using tf.map_fn with tf.concat, Example code:
import tensorflow as tf
a = tf.constant([ [1,2], [3,4] ])
flat_b = [5, 6]
flat_a = tf.reshape(a, (tf.reduce_prod(a.shape).numpy(), ))[:, tf.newaxis]
print(flat_a)
c = tf.map_fn(fn=lambda t: tf.concat([t, flat_b], axis=0), elems=flat_a)
c = tf.reshape(c, (-1, a.shape[1], c.shape[1]))
print(c)
Outputs:
tf.Tensor(
[[1]
[2]
[3]
[4]], shape=(4, 1), dtype=int32)
tf.Tensor(
[[[1 5 6]
[2 5 6]]
[[3 5 6]
[4 5 6]]], shape=(2, 2, 3), dtype=int32)
Here's a somewhat simpler version of the previous answer. Rather than reshaping several times, I prefer to use tf.expand_dims and tf.stack. The latter adds a dimension so that's one less call to tf.reshape, which can be confusing.
import tensorflow as tf
a = tf.constant([[1,2], [3,4]])
b = [5, 6]
flat_a = tf.reshape(a, [-1])
c = tf.map_fn(lambda x: tf.concat([[x], b], axis=0), flat_a)
c = tf.stack(tf.split(c, num_or_size_splits=len(a)), axis=0)
<tf.Tensor: shape=(2, 2, 3), dtype=int32, numpy=
array([[[1, 5, 6],
[2, 5, 6]],
[[3, 5, 6],
[4, 5, 6]]])>
You could go through element-wise. In list form you would do something like out[i][j] = [a[i][j]]+flat_b starting from out being the same shape as a. This gets to the form you wanted. I'm not sure if there is this sort of element-wise concatenation in the tensorflow library.
Related
I am learning Tensorflow from this github
https://colab.research.google.com/github/instillai/TensorFlow-Course/blob/master/codes/ipython/1-basics/tensors.ipynb#scrollTo=TKX2U0Imcm7d
Here is an easy tutorial
import numpy as np
import tensorflow as tf
x = tf.constant([[1, 1],
[1, 1]])
y = tf.constant([[2, 4],
[6, 8]])
# Add two tensors
print(tf.add(x, y), "\n")
# Add two tensors
print(tf.matmul(x, y), "\n")
What I expect is
tf.Tensor(
[[3 5]
[7 9]], shape=(2, 2), dtype=int32)
tf.Tensor(
[[ 8 12]
[ 8 12]], shape=(2, 2), dtype=int32)
However, the results are
Tensor("Add_3:0", shape=(2, 2), dtype=int32)
Tensor("MatMul_3:0", shape=(2, 2), dtype=int32)
It does not mean that the values of the tensors are zero. Add_3:0 and MatMul_3:0 are just names of the tensors and you can only use print in Eager Execution to see the values of the tensors. In Graph mode you should use tf.print and you should see the results:
import tensorflow as tf
x = tf.constant([[1, 1],
[1, 1]])
y = tf.constant([[2, 4],
[6, 8]])
print(tf.add(x, y), "\n")
print(tf.matmul(x, y), "\n")
# Graph mode
#tf.function
def calculate():
x = tf.constant([[1, 1],
[1, 1]])
y = tf.constant([[2, 4],
[6, 8]])
tf.print(tf.add(x, y), "\n")
tf.print(tf.matmul(x, y), "\n")
return x, y
_, _ = calculate()
tf.Tensor(
[[3 5]
[7 9]], shape=(2, 2), dtype=int32)
tf.Tensor(
[[ 8 12]
[ 8 12]], shape=(2, 2), dtype=int32)
[[3 5]
[7 9]]
[[8 12]
[8 12]]
Without tf.print, you will see the your output from the function calculate:
Tensor("Add:0", shape=(2, 2), dtype=int32)
Tensor("MatMul:0", shape=(2, 2), dtype=int32)
See this guide for more information.
I have the following code:
a = tf.constant([[10],
[0],
[4]])
s = tf.sparse.from_dense(a)
b = tf.constant([[3, 4],
[1, 2],
[2, 5]])
# print(tf.multiply(s, b)) # this didn't work either though it works if s is dense
print(s.__mul__(b))
I want to multiply a with each column of b getting:
[[30, 40],
[ 0, 0],
[ 8, 20]]
But the above code gives me an error that the sparse tensor cant be broadcasted to make the shapes match.
tensorflow.python.framework.errors_impl.InvalidArgumentError: SparseDenseBinaryOpShared broadcasts dense to sparse only; got incompatible shapes: [3,1] vs. [3,2] [Op:SparseDenseCwiseMul]
How do I achieve the above efficiently without materializing the sparse tensor a into a dense one?
Not sure how efficient this is and if it is a viable option for you, but you could try using tf.TensorArray with tf.sparse.sparse_dense_matmul:
import tensorflow as tf
a = tf.constant([[10],
[0],
[4]])
s = tf.sparse.from_dense(a)
b = tf.constant([[3, 4],
[1, 2],
[2, 5]])
ta = tf.TensorArray(dtype=tf.int32, size=0, dynamic_size=True)
for i in range(len(b)):
if i == len(b):
ta = ta.write(ta.size(), tf.sparse.sparse_dense_matmul(s, b[i:])[i])
else:
ta = ta.write(ta.size(), tf.sparse.sparse_dense_matmul(s, b[i:i + 1])[i])
print(ta.stack())
tf.Tensor(
[[30 40]
[ 0 0]
[ 8 20]], shape=(3, 2), dtype=int32)
What are the differences between these three ways to multiply two matrices in tensorflow? the three ways are :
#
tf.tensordot()
tf.matmul()
I have tested them and they give the same result. but I wanted to know if there is any underlying difference.
Let us understand this with below example, I have taken two matrix a, b to perform these functions:
import tensorflow as tf
a = tf.constant([[1, 2],
[3, 4]])
b = tf.constant([[1, 1],
[1, 1]]) # or `tf.ones([2,2])`
tf.matmul(a,b) and (a # b) - both performs matrix mutiplication
print(tf.matmul(a, b), "\n") # matrix - multiplication
Output:
tf.Tensor(
[[3 3]
[7 7]], shape=(2, 2), dtype=int32)
You can see the same output here as well for same matrix:
print(a # b, "\n") # # used as matrix_multiplication operator
Output:
tf.Tensor(
[[3 3]
[7 7]], shape=(2, 2), dtype=int32)
tf.tensordot() - Tensordot (also known as tensor contraction) sums the product of elements from a and b over the indices specified by axes .
if we take axes=0 (scalar, no axes):
print(tf.tensordot(a, b, axes=0), "\n")
#One by one each element(scalar) of first matrix multiply with all element of second matrix and keeps output in separate matrix for each element multiplication.
Output:
tf.Tensor(
[[[[1 1]
[1 1]]
[[2 2]
[2 2]]]
[[[3 3]
[3 3]]
[[4 4]
[4 4]]]], shape=(2, 2, 2, 2), dtype=int32)
if we change axes=1:
print(tf.tensordot(a, b, axes=1), "\n")
# performs matrix-multiplication
Output:
tf.Tensor(
[[3 3]
[7 7]], shape=(2, 2), dtype=int32)
and for axes=2:
print(tf.tensordot(a, b, axes=2), "\n")
# performs element-wise multiplication,sums the result into scalar.
Output:
tf.Tensor(10, shape=(), dtype=int32)
You can explore more about tf.tensordot() and basic details on axes in given links.
I'm struggling with this problem in keras/tensorflow.
I'm implementing a user defined loss function and I have this problem: I have to multiply 2 matrices, obtaining a list of matrix products in the form
[column_0_matrix_1 x row_0_matrix_2], [column_1_matrix_1 x row_1_matrix_2] ecc.
Let's say I have
A = [[1 1]
[3 2]]
B = [[4 1]
[1 3]]
Then I want to have a list of products in the form
C = |[1] x [4 1]|, |[1] x [1 3]|
|[3] | |[2] |
Any idea? I tried by my self but always get back the product of the 2 starting matrices.
Any help would by appreciated. Thank you
You could split each tensor and then use tf.linalg.matmul in a list comprehension to achieve what you want
import tensorflow as tf
a = tf.constant([[1, 1], [3, 2]])
b = tf.constant([[4, 1], [1, 3]])
a_split = tf.split(a, 2, 1)
b_split = tf.split(b, 2, 0)
[tf.linalg.matmul(x, y) for x, y in zip(a_split, b_split)]
# [<tf.Tensor: shape=(2, 2), dtype=int32, numpy=
# array([[ 4, 1],
# [12, 3]], dtype=int32)>,
# <tf.Tensor: shape=(2, 2), dtype=int32, numpy=
# array([[1, 3],
# [2, 6]], dtype=int32)>]
Define x as:
>>> import tensorflow as tf
>>> x = tf.constant([1, 2, 3])
Why does this normal tensor multiplication work fine with broacasting:
>>> tf.constant([[1, 2, 3], [4, 5, 6]]) * tf.expand_dims(x, axis=0)
<tf.Tensor: shape=(2, 3), dtype=int32, numpy=
array([[ 1, 4, 9],
[ 4, 10, 18]], dtype=int32)>
while this one with a ragged tensor does not?
>>> tf.ragged.constant([[1, 2, 3], [4, 5, 6]]) * tf.expand_dims(x, axis=0)
*** tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected 'tf.Tensor(False, shape=(), dtype=bool)' to be true. Summarized data: b'Unable to broadcast: dimension size mismatch in dimension'
1
b'lengths='
3
b'dim_size='
3, 3
How can I get a 1-D tensor to broadcast over a 2-D ragged tensor? (I am using TensorFlow 2.1.)
The problem will be resolved if you add ragged_rank=0 to the Ragged Tensor, as shown below:
tf.ragged.constant([[1, 2, 3], [4, 5, 6]], ragged_rank=0) * tf.expand_dims(x, axis=0)
Complete working code is:
%tensorflow_version 2.x
import tensorflow as tf
x = tf.constant([1, 2, 3])
print(tf.ragged.constant([[1, 2, 3], [4, 5, 6]], ragged_rank=0) * tf.expand_dims(x, axis=0))
Output of the above code is:
tf.Tensor(
[[ 1 4 9]
[ 4 10 18]], shape=(2, 3), dtype=int32)
One more correction.
As per the definition of Broadcasting, Broadcasting is the process of **making** tensors with different shapes have compatible shapes for elementwise operations, there is no need to specify tf.expand_dims explicitly, Tensorflow will take care of it.
So, below code works and demonstrates the property of Broadcasting well:
%tensorflow_version 2.x
import tensorflow as tf
x = tf.constant([1, 2, 3])
print(tf.ragged.constant([[1, 2, 3], [4, 5, 6]], ragged_rank=0) * x)
Output of the above code is:
tf.Tensor(
[[ 1 4 9]
[ 4 10 18]], shape=(2, 3), dtype=int32)
For more information, please refer this link.
Hope this helps. Happy Learning!