We Keep Coding

Printing a TensorFlow object without using session

I am receiving different errors when trying to print a TensorFlow object.
import numpy as np
import tensorflow as tf
The versions of both TensorFlow and numpy are 2.6 and 1.19.5, respectively.
print("np version:", np.__version__)
print("tf version:" ,tf.version.VERSION)
print("eager is on? ", tf.executing_eagerly())
#np version: 1.19.5
#tf version: 2.6.0
#eager is on? True
Now, let me create a small array and turn it into a tf object.
arr= [0,1.2,-0.8]
arr = tf.constant(arr, dtype = tf.float32)
When I use tf.print or tf.compat.v1.print(arr), nothing happens. When I call numpy, I do receive an error.
tf.compat.v1.print(arr.numpy())
AttributeError: 'Tensor' object has no attribute 'numpy'
The only thing that has worked so far is ;
with tf.compat.v1.Session() as sess: print(arr.eval())
#[ 0. 1.2 -0.8]
However, I would like to use numpy since my goal is to print certain features of the network during the traning phase. For instance, if I want to print the learning rate, I call with tf.compat.v1.Session() as sess: print(model.optimizer.learning_rate.eval()) . Yet, it returns me another error.
'ExponentialDecay' object has no attribute 'eval'
I was able to use numpy to print everything before, however, I updated both TensorFlow and numpy packages and now am facing so many incompatibilities. The worst thing is that I don't remember which versions I was using.
I followed every step explainedd in this post
AttributeError: 'Tensor' object has no attribute 'numpy'. It did not help me.

Following code gives me an output -
import numpy as np
import tensorflow as tf
print("np version:", np.__version__)
print("tf version:" ,tf.version.VERSION)
print("eager is on? ", tf.executing_eagerly())
tf.enable_eager_execution()
arr= [0,1.2,-0.8]
arr = tf.constant(arr, dtype = tf.float32)
tf.compat.v1.print(arr.numpy())
Output: array([ 0. , 1.2, -0.8], dtype=float32)
Did you add tf.enable_eager_execution() ?

Comparing theano vs tensorflow function speed, how can I make tensorflow function faster?

OS: windows 10
procesor: core i7-6700
python 3.7 using anaconda
theano version 1.0.4 installed using: conda install theano pygpu
tensorflow version 2.1.0 installed using: pip install tensorflow
Both are running using cpu
I'm rewriting some of my code from theano to tensorflow and I find that the performance in tensorflow is not as fast as theano so I must be missing something that causes the tensorflow code to be slower.
below is sample code:
import numpy as np
import theano
import theano.tensor as T
from theano import function
import tensorflow as tf
from time import time
#define tensorflow function
#tf.function
def tf_mean(data):
return tf.math.reduce_mean(data, axis=0)
#define theano function
tdata = T.dmatrix('tdata')
tmean = T.mean(tdata, axis=0)
theano_mean = function([tdata], tmean)
if __name__ == '__main__':
np.random.seed(1234)
randomdata = np.random.random((3000,10))
#run first time to warm up
check_th = theano_mean(randomdata)
check_tf = tf_mean(randomdata)
# run each 10000 times
start = time()
for i in range(10000):
theano_mean(randomdata)
thtime = time()-start
print('theano', thtime )
start = time()
for i in range(10000):
tf_mean(randomdata)
tftime = time()-start
print('tensorflow', tftime )
print('ratio', tftime / thtime)
output:
theano 0.4887216091156006
tensorflow 2.4310362339019775
ratio 4.9742761289013275
So theano is around 5 times faster than tensorflow it seems. How can I make tensorflow code above faster at least on par with theano?

It turns out that Theano is much faster than Tensorflow when running on a CPU. Tensorflow only runs faster than Theano if you have multiple GPUs.

Upgrading from tensorflow 1.x to 2.0

I am new to tensorflow.
Have tried this simple example:
import tensorflow as tf
sess = tf.Session()
x = tf.placeholder(tf.float32)
y = tf.placeholder(tf.float32)
z = x + y
print(sess.run(z, feed_dict={x: 3.0, y: 4.5}))
and got some warnings The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. and the right answer - 7.5
After reading here, I understand that the warnings are due to upgrading from tf 1.x to 2.0, the steps described are "simple" but they don't give any example....
I have tried:
#tf.function
def f1(x1, y1):
return tf.math.add(x1, y1)
print(f1(tf.constant(3.0), tf.constant(4.5)))
Is my code correct (in the sense defined in the link)?
Now, I am getting Tensor("PartitionedCall:0", shape=(), dtype=float32) as output, how can I get the actual value?

You code is indeed correct. The warning that you get indicates that as from Tensorflow 2.0, tf.Session() won't exist in the API. Therefore, if you want you code to be compatible with Tensorflow 2.0, you should use tf.compat.v1.Session instead. So, just change this line:
sess = tf.Session()
To:
sess = tf.compat.v1.Session()
Then, even if you update Tensorflow from 1.xx to 2.xx, your code would execute in the same way. As for the code in Tensorflow 2.0:
#tf.function
def f1(x1, y1):
return tf.math.add(x1, y1)
print(f1(tf.constant(3.0), tf.constant(4.5)))
it is fine if you run it in Tensorflow 2.0. If you want to run the same code, without installing Tensorflow 2.0, you can do the following:
import tensorflow as tf
tf.enable_eager_execution()
#tf.function
def f1(x1, y1):
return tf.math.add(x1, y1)
print(f1(tf.constant(3.0), tf.constant(4.5)).numpy())
The reason for this is because the default way of executing Tensorflow operations starting from Tensorflow 2.0 is in eager mode. The way of activating eager mode in Tensorflow 1.xx, is to enable it right after the import of Tensorflow, as I am doing it in the example above.

Your code is correct as per Tensorflow 2.0. Tensorflow 2.0 is even more tightly bonded with numpy so if you want to get the result of the operation, you could use the numpy() method:
print(f1(tf.constant(3.0), tf.constant(4.5)).numpy())

module 'tensorflow._api.v2.train' has no attribute 'GradientDescentOptimizer'

I used Python 3.7.3 and installed tensorflow 2.0.0-alpha0,But there are some problems。such as
module 'tensorflow._api.v2.train' has no attribute 'GradientDescentOptimizer'
Here's all my code
import tensorflow as tf
import numpy as np
x_data=np.random.rand(1,10).astype(np.float32)
y_data=x_data*0.1+0.3
Weights = tf.Variable(tf.random.uniform([1], -1.0, 1.0))
biases = tf.Variable(tf.zeros([1]))
y=Weights*x_data+biases
loss=tf.reduce_mean(tf.square(y-y_data))
optimizer=tf.train.GradientDescentOptimizer(0.5)
train=optimizer.minimize(loss)
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
for step in range(201):
sess.run(train)
if step % 20 == 0:
print(step, sess.run(Weights), sess.run(biases))

In TensorFlow 2.0, Keras became the default high-level API, and optimizer functions migrated from tf.keras.optimizers into separate API called tf.optimizers. They inherit from Keras class Optimizer. Relevant functions from tf.train aren't included into TF 2.0. So to access GradientDescentOptimizer, call tf.optimizers.SGD

You are using Tensorflow 2.0.
The following code will be helpful:
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

This is because you are using TensorFlow version 2.
`tf.train.GradientDescentOptimizer(0.5)`
The above call is for TensorFlow version 1(ex : 1.15.0).
You can try pip install tensorflow==1.15.0 to downgrade the TensorFlow and use the code as it is.
Else use the TensorFlow version 2(what you already has) with following call.
tf.optimizers.SGD (learning_rate=0.001, lr_decay=0.0, decay_step=100, staircase=False, use_locking=False, name='SGD')

For the answer #HoyeolKim gave, it may be needed to add:
tf.disable_v2_behavior()
As it is suggested in this answer.

tensorflow tf.initializeallvariables() taking too much time even while init with few variables

Tensorflow is taking too much time to run the tf.initializeallvariables() routine for some reason. I am using cuda 8.0 with NVIDIA TitanX Pascal and latest tensorflow. I am not sure about the reason and would appreciate any help.
import tensorflow as tf
import numpy as np
cls = np.zeros((5, 10))
classes = tf.Variable(cls, dtype=tf.float32)
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.InteractiveSession(config=config)
sess.run(tf.initialize_all_variables())