We Keep Coding

Can we change tensorflow version on the fly in a python script?

I have an old python script(tf-1.15.2) that needs to be run in TensorFlow-2.2.0 (can not use tf <2.2), I have migrated most of the code to tf-2.2.0, but there are some tensorflow.contrib related methods that are used in the code. So, I would like to use the old version tf-1.15 for running those lines of code that use tensorflow.contrib related APIs.
So, now the question is I have installed tf-1.15.2 globally, I have installed tf-2.2.0 locally. But how to access the specific version of the TensorFlow at a specific point in time while the python process is running?
Example code is below
import tensorflow as tf # version: tf-2.2.0 (local package is imported)
isess = tf.compat.v1.Session()
tf.compat.v1.disable_eager_execution()
​
# Creatoin of the required placeholders
p = []
for shape in input_shapes:
p.append(tf.compat.v1.placeholder(shape=shape, dtype=input_dtype))
out = tf.einsum(equation, *p)
graph_def = isess.graph_def
# TODO
# To feed this (graph_def, feed_dict, output_tensors) to a session object of tf-1.15.2 and find the output
Now to test the unit test given in tf_einsum_op_test in tf_1.15.2 after replacing the einsum with appropriate function (trace/dot_product/...), I would like to revert back to tf-1.15.2 and check the execution.
The underlying need is to find if the tf versions can be interchanged during the execution flow of a python process. Einsum op is considered since it is not directly supported in tf-1.15.2

Up on Experimenting with subprocess API, I found that it is possible to switch between the tf versions during the python process execution through subprocess call.
# main.py
import tensorflow as tf # version: tf-2.2.0 (local package is imported)
import subprocess
import os
isess = tf.compat.v1.Session()
tf.compat.v1.disable_eager_execution()
​
# Creatoin of the required placeholders
p = []
for shape in input_shapes:
p.append(tf.compat.v1.placeholder(shape=shape, dtype=input_dtype))
out = tf.einsum(equation, *p)
graph_def = isess.graph_def
#TODO: Save the graph_def in graph.pb
#TODO: Save the feed_dict in input.npz
#TODO: Save the output_tensors
#Change the python path to the global package
os.environ['PYTHONPATH'] = '/usr/local/lib/python3.6/dist-packages'
cmd = ['python3.6','run.py']
out = subprocess.check_output(cmd) #Subprocess call
#run.py
import tensorflow as tf # version: tf-1.15.2 (global package is imported)
import numpy as np
#TODO: Load the graphdef from graph.pb
#TODO: Load the feed_dict from input.npz
#TODO: Load the output tensors
g = tf.import_graph_def(graph_def,name='')
with tf.Session(graph=g) as sess:
output = sess.run(output_tensors,feed_dict)
This works for me.

tf.global_variable_initializer() with regard to session?

My understandings on Sessions in Tensorflow still seem to be flawed even after reading the official documentation and this tutorial.
In particular, does tf.global_variable_initializer() initialize global variables with regard to a particular session, or for all the sessions in the program? Are there ways to "uninitialize" a variable in / during a session?
Can a tf.variable be used in multiple sessions? The answer seems to be yes (e.g. the following code), but then are there good cases where we want multiple sessions in a program, instead of a single one?
#!/usr/bin/env python
import tensorflow as tf
def main():
x = tf.constant(0.)
with tf.Session() as sess:
print(sess.run(x))
with tf.Session() as sess:
print(sess.run(x))
if __name__ == '__main__':
main()

In particular, does tf.global_variable_initializer() initialize global variables with regard to a particular session, or for all the sessions in the program?
With regards to a particular session. Check this out.
tf.reset_default_graph()
x = tf.Variable(tf.random.normal([1,5]))
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
first_sess_out = sess.run(x)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
second_sess_out = sess.run(x)
np.testing.assert_array_equal(first_sess_out, second_sess_out)
The assertion fails so it is per session.
Are there ways to "uninitialize" a variable in / during a session?
tf.reset_default_graph()
x = tf.Variable(tf.random.normal([1,5]))
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
first_init_out = sess.run(x)
sess.run(tf.global_variables_initializer())
second_init_out = sess.run(x)
np.testing.assert_array_equal(first_init_out, second_init_out)
Apparently there is, after running tf.global_variables_initializer() the variables got reinitialized. Thus, the assertion fails.
Can a tf.Variable be used in multiple sessions? The answer seems to be yes (e.g. the following code), but then are there good cases where we want multiple sessions in a program, instead of a single one?
Yes, it can be used as you can see on the examples above. Good cases are when you want to execute the graph multiple times in a single run.

tf.print has no result

I want to use tf.print to show tensor value, but it has no result?
This is my code and is there something wrong for that:
from __future__ import print_function
import tensorflow as tf
sess = tf.InteractiveSession()
a = tf.constant([1.0, 3.0])
tf.print(a)

From the documentation of tf.Print - that's deprecated and suggests to use tf.print:
Note that tf.print returns a no-output operator that directly prints the output. Outside of defuns or eager mode, this operator will not be executed unless it is directly specified in session.run or used as a control dependency for other operators.
This is only a concern in graph mode. Below is an example of how to ensure tf.print executes in graph mode:
sess = tf.Session()
with sess.as_default():
tensor = tf.range(10)
print_op = tf.print(tensor)
with tf.control_dependencies([print_op]):
out = tf.add(tensor, tensor)
sess.run(out)
Hence, if you enable the eager mode your code will work as you expected, if you want to continue using the static-graph mode you have to use sess.run

import tensorflow as tf
a = tf.constant([1.0, 3.0])
init_op = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init_op)
print(sess.run(a))
is what I'd do. Import tensorflow, set up your variables, set up and run the initializer for them, and then print the session evaluating the constant

Error in tensor evaluation

The below code runs fine in a tutorial but there is an error when I run it locally. Are there any installation errors or something else? Please help. This is link to that tutorial:
https://colab.research.google.com/notebooks/mlcc/tensorflow_programming_concepts.ipynb?utm_source=mlcc&utm_campaign=colab-external&utm_medium=referral&utm_content=tfprogconcepts-colab&hl=en#scrollTo=Md8ze8e9geMi
And the code:
import tensorflow as tf
#create a graph
g = tf.Graph()
#establish the graph as the default graph
with g.as_default():
x = tf.constant(8, name = "x_const")
y = tf.constant(5, name = "y_const")
my_sum = tf.add(x,y, name = "x_y_sum")
#create the session
#this runs the default graph
with tf.Session() as sess:
print(my_sum.eval())
Below is the error that occurs:
gunjan#gunjan-Inspiron-3558:~/Desktop$ python tf1.py
/home/gunjan/anaconda3/lib/python3.5/site-
packages/h5py/__init__.py:34: FutureWarning: Conversion of the second
argument of issubdtype from `float` to `np.floating` is deprecated. In
future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
2018-08-20 22:10:41.619062: I
tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports
instructions that this TensorFlow binary was not compiled to use: AVX2
FMA
Traceback (most recent call last):
File "tf1.py", line 15, in <module>
print(my_sum.eval())
File "/home/gunjan/anaconda3/lib/python3.5/site-
packages/tensorflow/python/framework/ops.py", line 680, in eval
return _eval_using_default_session(self, feed_dict, self.graph,
session)
File "/home/gunjan/anaconda3/lib/python3.5/site-
packages/tensorflow/python/framework/ops.py", line 4942, in
_eval_using_default_session
raise ValueError("Cannot use the default session to evaluate tensor: "
ValueError: Cannot use the default session to evaluate tensor: the
tensor's graph is different from the session's graph. Pass an explicit
session to `eval(session=sess)`.

The problem is that you've created one graph (g) and you're executing code in a separate graph (sess). If you don't need two graphs, you can just use sess:
x = tf.constant(8, name = "x_const")
y = tf.constant(5, name = "y_const")
my_sum = tf.add(x,y, name = "x_y_sum")
#create the session
#this runs the default graph
with tf.Session() as sess:
print(my_sum.eval())

To simply get it working you can pass the session explicitly, as suggested by the error message:
print(my_sum.eval(session=sess))
To understand why it doesn't work exactly as the tutorial specifies it, you could start by comparing the versions of Python and TensorFlow to those used in the tutorial.
import tensorflow as tf
import platform
print("Python version: ", platform.python_version())
print("TensorFlow version", tf.__version__)
For the colab environment you linked, this prints:
Python version: 2.7.14
TensorFlow version 1.10.0
EDIT
Taking another look at your code sample, it's not an issue of version compatibility. The issue is that your copy of the tutorial did not properly preserve the indentation. The second with block needs to be enclosed in the first.
# Establish the graph as the "default" graph.
with g.as_default():
# ...
# Now create a session.
# The session will run the default graph.
with tf.Session() as sess:
print(my_sum.eval())
This ensures that g is used as the default graph for the session, instead of creating a new one like MatthewScarpino points out your incorrectly-indented version does.

If you create/use a Graph object explicitly rather than using the default graph, you need to either (a) pass the graph object to your Session constructor, or (b) create the session in the graph context.
graph = tf.Graph()
with graph.as_default():
build_graph()
with tf.Session(graph=graph) as sess:
do_stuff_with(sess)
or
graph = tf.Graph()
with graph.as_default():
build_graph()
with tf.Session() as sess:
do_stuff_with(sess)

In TensorFlow, what is the difference between Session.run() and Tensor.eval()?

TensorFlow has two ways to evaluate part of graph: Session.run on a list of variables and Tensor.eval. Is there a difference between these two?

If you have a Tensor t, calling t.eval() is equivalent to calling tf.get_default_session().run(t).
You can make a session the default as follows:
t = tf.constant(42.0)
sess = tf.Session()
with sess.as_default(): # or `with sess:` to close on exit
assert sess is tf.get_default_session()
assert t.eval() == sess.run(t)
The most important difference is that you can use sess.run() to fetch the values of many tensors in the same step:
t = tf.constant(42.0)
u = tf.constant(37.0)
tu = tf.mul(t, u)
ut = tf.mul(u, t)
with sess.as_default():
tu.eval() # runs one step
ut.eval() # runs one step
sess.run([tu, ut]) # evaluates both tensors in a single step
Note that each call to eval and run will execute the whole graph from scratch. To cache the result of a computation, assign it to a tf.Variable.

The FAQ session on tensor flow has an answer to exactly the same question. I will just go ahead and leave it here:
If t is a Tensor object, t.eval() is shorthand for sess.run(t) (where sess is the current default session. The two following snippets of code are equivalent:
sess = tf.Session()
c = tf.constant(5.0)
print sess.run(c)
c = tf.constant(5.0)
with tf.Session():
print c.eval()
In the second example, the session acts as a context manager, which has the effect of installing it as the default session for the lifetime of the with block. The context manager approach can lead to more concise code for simple use cases (like unit tests); if your code deals with multiple graphs and sessions, it may be more straightforward to explicit calls to Session.run().
I'd recommend that you at least skim throughout the whole FAQ, as it might clarify a lot of things.

eval() can not handle the list object
tf.reset_default_graph()
a = tf.Variable(0.2, name="a")
b = tf.Variable(0.3, name="b")
z = tf.constant(0.0, name="z0")
for i in range(100):
z = a * tf.cos(z + i) + z * tf.sin(b - i)
grad = tf.gradients(z, [a, b])
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
print("z:", z.eval())
print("grad", grad.eval())
but Session.run() can
print("grad", sess.run(grad))
correct me if I am wrong

The most important thing to remember:
The only way to get a constant, variable (any result) from TenorFlow is the session.
Knowing this everything else is easy:
Both tf.Session.run() and tf.Tensor.eval() get results from the session where tf.Tensor.eval() is a shortcut for calling tf.get_default_session().run(t)
I would also outline the method tf.Operation.run() as in here:
After the graph has been launched in a session, an Operation can be executed by passing it to tf.Session.run(). op.run() is a shortcut for calling tf.get_default_session().run(op).

Tensorflow 2.x Compatible Answer: Converting mrry's code to Tensorflow 2.x (>= 2.0) for the benefit of the community.
!pip install tensorflow==2.1
import tensorflow as tf
tf.compat.v1.disable_eager_execution()
t = tf.constant(42.0)
sess = tf.compat.v1.Session()
with sess.as_default(): # or `with sess:` to close on exit
assert sess is tf.compat.v1.get_default_session()
assert t.eval() == sess.run(t)
#The most important difference is that you can use sess.run() to fetch the values of many tensors in the same step:
t = tf.constant(42.0)
u = tf.constant(37.0)
tu = tf.multiply(t, u)
ut = tf.multiply(u, t)
with sess.as_default():
tu.eval() # runs one step
ut.eval() # runs one step
sess.run([tu, ut]) # evaluates both tensors in a single step

In tensorflow you create graphs and pass values to that graph. Graph does all the hardwork and generate the output based on the configuration that you have made in the graph.
Now When you pass values to the graph then first you need to create a tensorflow session.
tf.Session()
Once session is initialized then you are supposed to use that session because all the variables and settings are now part of the session. So, there are two ways to pass external values to the graph so that graph accepts them. One is to call the .run() while you are using the session being executed.
Other way which is basically a shortcut to this is to use .eval(). I said shortcut because the full form of .eval() is
tf.get_default_session().run(values)
You can check that yourself.
At the place of values.eval() run tf.get_default_session().run(values). You must get the same behavior.
what eval is doing is using the default session and then executing run().