Matrix norm in TensorFlow - python

I need to compute the Frobenius norm in order to achieve this formula using the TensorFlow framework:
where w is a matrix with 50 rows and 100 columns.
I tried to write something, but I don't understand how to fill out the axis argument.
tf.pow(
tf.norm(x, ord='fro', axis=?), 2
)
According to the TensorFlow docs I have to use a 2-tuple (or a 2-list) because it determines the axies in tensor over which to compute a matrix norm, but I simply need a plain Frobenius norm. In SciPy, for example, I can do it without specify any axis.
So, what should I use as axis to emulate the SciPy function?

So the Frobenius norm is a sum over a nxm matrix, but tf.norm allows to process several vectors and matrices in batch.
To better understand, imagine you have a rank 3 tensor:
t = [[[2], [4], [6]], [[8], [10], [12]], [[14], [16], [18]]]
It can be seen as several matrices aligned over one direction, but the function can't figure by itself which one. It could be either a batch of the following matrices:
[2, 4, 6] , [8 ,10, 12], [14, 16, 18]
or
[2 8 14], [4, 10, 16], [6, 12, 18]
So basically axis tells which directions you want to consider when doing the summation in the Frobenius norm.
In your case, any of [1,2] or [-2,-1] would do the trick.

Independent of the number of dimensions of the tensor,
tf.sqrt(tf.reduce_sum(tf.square(w)))
should do the trick.

Negative indices are supported. Example: If you are passing a tensor that can be either a matrix or a batch of matrices at runtime, pass axis=[-2,-1] instead of axis=None to make sure that matrix norms are computed.
I just tested and [-2,-1] works.

It seems to me you are better off simply calling
tf.reduce_sum(tf.multiply(x, x))
Calling norm which square-roots the above result, then pow which work for any power and therefore potentially uses an elaborate algorithm, is overkill.

Try axis=(0,1). I think, it will solve your problem!!

Frobenius norm does not work for matrix. You need to create vectors.
reshape your array to the batchsize,-1.
Use tf.norm(reshaped_data, ord= 'fro', axis = (0, 1))
Using reshape on the tensorflow eager execution may through an error. from version = 2.5 onwards use
import tensorflow.python.ops.numpy_ops.np_config as np_config np_config.enable_numpy_behavior()
for example of how I am using this :
heat_difference = gt_hm - pd_hm
heat_difference = heat_difference.reshape(batch_size, -1)
hm_loss = tf.square(tf.norm(heat_difference, ord='fro', axis=(0, 1)) / batch_size)

Related

What is the most efficient way to handle conversion from full to symmetric second order tensors using numpy?

I am processing symmetric second order tensors (of stress) using numpy. In order to transform the tensors I have to generate a fully populated tensor, do the transformation and then recover the symmetric tensor in the rotated frame.
My input is a 2D numpy array of symmetric tensors (nx6). The code below works, but I'm pretty sure there must be a more efficient and/or elegant way to manipulate the arrays but I can't seem to figure it out.
I anyone can anyone suggest an improvement I'd be very grateful? The sample input is just 2 symmetric tensors but in use this could be millions of tensors, hence the concernr with efficiency
Thanks,
Doug
# Sample symmetric input (S11, S22, S33, S12, S23, S13)
sym_tens_in=np.array([[0,9], [1,10], [2,11], [3,12], [4,13], [5,14]])
# Expand to full tensor
tens_full=np.array([[sym_tens_in[0], sym_tens_in[3], sym_tens_in[4]],
[sym_tens_in[3], sym_tens_in[1], sym_tens_in[5]],
[sym_tens_in[4], sym_tens_in[5], sym_tens_in[2]]])
# Transpose and reshape to n x 3 x 3
tens_full=np.transpose(tens_full, axes=(2, 0, 1))
# This where the work on the full tensor will go....
# Reshape for extraction of the symmetric tensor
tens_full=np.reshape(tens_full, (2,9))
# Create an array for the test ouput symmetric tensor
sym_tens_out=np.empty((2,6), dtype=np.int32)
# Extract the symmetric components
sym_tens_out[:,0]=tens_full[:,0]
sym_tens_out[:,1]=tens_full[:,4]
sym_tens_out[:,2]=tens_full[:,8]
sym_tens_out[:,3]=tens_full[:,2]
sym_tens_out[:,4]=tens_full[:,3]
sym_tens_out[:,5]=tens_full[:,5]
# Transpose....
sym_tens_out=np.transpose(sym_tens_out)
This won't be any faster, but it's more compact:
In [166]: idx=np.array([0,3,4,3,1,5,4,5,2]).reshape(3,3)
In [167]: sym_tens_in[idx].transpose(2,0,1)
Out[167]:
array([[[ 0, 3, 4],
[ 3, 1, 5],
[ 4, 5, 2]],
[[ 9, 12, 13],
[12, 10, 14],
[13, 14, 11]]])
The transpose could be done first:
sym_tens_in.T[:,idx]
Similarly the reverse mapping can be done with:
In [168]: idx1 = [0,4,8,1,2,5]
In [171]: tens_full.reshape(2,-1)[:,idx1]
Out[171]:
array([[ 0, 1, 2, 3, 4, 5],
[ 9, 10, 11, 12, 13, 14]])
with the optional transpose.
OK - Based on the answers provided here I found a really cool solution. Now, I have to say that in my original question I omitted the actual reason that I was trying to get the full tensor into nx3x3 form. Basically, I'm implementing a function to rotate 2nd order stress tensors which requires solution of σ′=R⋅σ⋅RT.
I was planning to use numpy.matmul for the matrix multiplication but to transform multiple stress tensors, matmul requires the 3x3 tensors to be in the last two indices of the nx3x3 matrix - hence the effort to get the data into nx3x3 from from the original 3x3xn form....
However, after I let go of numpy.matmul as my target solution and embraced numpy.einsum instead....... everything became much easier....
# Sample symmetric input (S11, S22, S33, S12, S23, S13)
sym_tens_in=np.array([[0,9], [1,10], [2,11], [3,12], [4,13], [5,14]])
idx=np.array([0,3,5,3,1,4,5,4,2]).reshape(3,3)
full=sym_tens_in[idx]
full_transformed=np.einsum('ij, jkn, lk->nil', rot_mat, full, rot_mat)
Thanks for the inspiration!!!!

Is the numpy documentation for calculating the 2nd order norm of a matrix across the columns slightly misleading?

I was trying to figure out how to calculate the Frobenius of a matrix in numpy. This way I can get the 2-norm of each row in the matrix x below:
My question is about the ord parameter in numpy's linalg.norm module and how the relevant part of numpy documentation describes which norm of a matrix one can calculate. I was able to get the Frobenius norm by setting ord=2, however, it says that only setting ord=None gives the Frobenius norm.
Here is my example:
x = np.array([[0, 3, 4],
[1, 6, 4]])
I found that I can the Frobenius norm with the following line of code:
x_norm = np.linalg.norm(x, ord = 2, axis=1,keepdims=True )
>>> x_norm
array([[ 5. ],
[ 7.28010989]])
My question is whether the documentation here would be considered not as helpful as possible and if this warrants a request to change the description of setting ord=2 in the aforementioned table.
You're not taking a matrix norm. Since you've passed axis=1, you're taking vector norms, and you should be looking at the vector norm column instead of the matrix norm column.
For vector norms, ord=None and ord=2 both produce a 2-norm.

How numpy dot works with broadcasting

I have two numpy arrays. When I used numpy dot function I got different results. I couldn't understand how dot function worked along with broadcasting to produce these outputs.
Can someone explain me the difference between these two.
A = np.array([[2,4,6]])
Y = np.array([[1,0,1]])
np.dot(A,Y.T) = array([8])
np.dot (Y.T, A) = array([[2, 4, 6],
[0, 0, 0],
[2, 4, 6]])
The dot function is matrix multiplication, there's no broadcasting involved.
Using np.dot(A,Y.T) is the same as A#Y.T in python 3.5+.
Matrix multiplication is not commutative (the order of arguments matters).
In the first usage, A is a row vector, Y.T is a column vector. This results in a single value.
In the second example, Y.T is a column vector, while A is a row vector. This results in a matrix.

Condition number of a matrix using numpy

[python 2.7 and numpy v1.11.1] I am looking at matrix condition numbers and am trying to compute the condition number for a matrix without using the function np.linalg.cond().
Based on numpy's documentation, the definition of a matrix's condition number is, "the norm of x times the norm of the inverse of x."
||X|| * ||X^-1||
for the matrix
a = np.matrix([[1, 1, 1],
[2, 2, 1],
[3, 3, 0]])
print np.linalg.cond(a)
1.84814479698e+16
print np.linalg.norm(a) * np.linalg.norm(np.linalg.inv(a))
2.027453660713377e+17
Where is the mistake in my computation?
Thanks!
You are trying to compute the condition using the Frobenius Norm definition. That is an optional parameter to the condition computation.
print(np.linalg.norm(a)*np.linalg.norm(np.linalg.inv(a)))
print(np.linalg.cond(a, p='fro'))
Produces
2.02745366071e+17
2.02745366071e+17
norm uses the Frobenius norm for matrix by default,when cond uses 2-norm:
In [347]: np.linalg.cond(a)
Out[347]: 38.198730775206172
In [348]:np.linalg.norm(a,2)*np.linalg.norm(np.linalg.inv(a),2)
Out[348]: 38.198730775206243
In [349]: np.linalg.norm(a)*np.linalg.norm(np.linalg.inv(a))
Out[349]: 39.29814570824248
NumPy cond() is currently buggy. There will come a time when we will fix it but for now if you are doing this for linear equation solutions you can use SciPy linalg.solve which will either produce an error for exact singularity or a warning if reciprocal condition number is below threshold and nothing if the array is invertible.

How to rotate a numpy array?

I have some numpy/scipy issue. I have a 3D array that represent an ellipsoid in a binary way [ 0 out of the ellipsoid].
The thing is I would like to rotate my shape of a certain degree. Do you think it's possible ?
Or is there an efficient way to write directly the ellipsoid equation with the rotation ?
Just a short answer. If you need more informations or you don't know how to do it, then I will edit this post and add a small example.
The right way to rotate your matrix of data points is to do a matrix multiplication. Your rotation matrix would be probably an n*n-matrix and you have to multiply it with every point. If you have your 3d-matrix you have some thing like i*j*k-points for plotting. This means for your case you have to do it i*j*k-times to find the new points. Maybe you should consider an other matrix for plotting which is just a 2D matrix and just store the plotting points and no zero values.
There are some algorithm to calculate faster the results for low valued matrix, but just google for this.
Did you understood me or do you still have some questions? Sorry for this rough overview.
Best regards
Take a look at the command numpy.shape
I used it once to transpose an array, but I don't know if it might fit your needs.
Cheers!
rotating by a non rectangular degree is tricky, because the rotated square does no longer fit in the matrix.
The simplest way would be to transpose a 2D array/matrix by:
import numpy as np
... x = np.array([[1,2,3],[4,5,6],[7,8,9]])
x
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
x.T # transpose the array
array([[1, 4, 7],
[2, 5, 8],
[3, 6, 9]])
if you require an explicit array rotation check the ndimage package from scipy library:
from scipy import ndimage, datasets
img = datasets.ascent()
img_45 = ndimage.rotate(img, 45, reshape=False)

Categories