I am using some code from here: https://github.com/monikkinom/ner-lstm with tensorflow. I think the code was written for an older version of tensorflow, I am using version 1.0.0. I used tf_upgrade.py to upgrade model.py in that github repos, but I am still getting the error:
output, _, _ = contrib_rnn.bidirectional_rnn(fw_cell, bw_cell,
AttributeError: 'module' object has no attribute 'bidirectional_rnn'
this is after I changed the bidirectional_rnn call to use contrib_rnn which is:
from tensorflow.contrib.rnn.python.ops import core_rnn as contrib_rnn
The old call was
output, _, _ = tf.nn.bidirectional_rnn(fw_cell, bw_cell,
tf.unpack(tf.transpose(self.input_data, perm=[1, 0, 2])),
dtype=tf.float32, sequence_length=self.length)
which also doesn't work.
I had to change the LSTMCell, DroputWrapper, etc. to rnn.LSTMCell, but they seem to work fine. It is the bidirectional_rnn that I can't figure out how to change.
In TensorFlow 1.0, you have the choice of two bidirectional RNN functions:
tf.nn.bidirectional_dynamic_rnn()
tf.contrib.rnn.static_bidirectional_rnn()
Maybe you can try to reimplement a bidirectional RNN by simply wrapping into a single class two monodirectional RNNs with the parameter "go_backwards=True" set on one of them. Then you can also have control over the type of merge done with the outputs. Maybe taking a look at the implementation in https://github.com/fchollet/keras/blob/master/keras/layers/wrappers.py (see the class Bidirectional) could get you started.
Related
I tried to run the code from here: https://keras.io/examples/generative/cyclegan/
but when running model.fit(..) I get the following error:
ValueError: Model <__main__.CycleGan object at 0x7fec3de767c0> cannot be saved either because the
input shape is not available or because the forward pass of the model is not defined.To define a
forward pass, please override `Model.call()`. To specify an input shape, either call
`build(input_shape)` directly, or call the model on actual data using `Model()`, `Model.fit()`, or
`Model.predict()`. If you have a custom training step, please make sure to invoke the forward pass
in train step through `Model.__call__`, i.e. `model(inputs)`, as opposed to `model.call()`.
Even if I run it through the linked colab file from the author. Did anyone else already faced this issue and knows how to fix it ?
I also tried to predefine the inputsize by model.build((batch_size,256,256,3)), but still get the same error.
If i comment the callbacks, it works. I think the problem is in the model_checkpoint_callback. Without it, the code works, but then I don't save the model.
Thanks a lot in advance for every answer!
The solution is to add this line:
# Create cycle gan model
cycle_gan_model = CycleGan(
generator_G=gen_G, generator_F=gen_F, discriminator_X=disc_X, discriminator_Y=disc_Y
)
# add following line:
cycle_gan_model.compute_output_shape(input_shape=(None, 256, 256, 3))
I'm working on a project on GitHub that was made with Python 2.7 (https://github.com/AIM-Harvard/DeepCAC)
I've made most relevant changes as to update it to Python 3.7, but I'm fixed on an error message regarding simpleITK.
Error message:
TypeError: Execute() takes 2 positional arguments but 10 were given
It stems from this code:
res_filter = sitk.ResampleImageFilter()
----> img_sitk = res_filter.Execute(img_sitk, curated_size, sitk.Transform(), method, img_sitk.GetOrigin(), curated_spacing, img_sitk.GetDirection(), 0, img_sitk.GetPixelIDValue())
According to the simpleITK document on switching from 1.x to 2.x (version that is available with python 3.7) it should be done like this (https://simpleitk.readthedocs.io/en/master/migrationGuide2.0.html#filter-s-execute-method) but I can't quite grasp it.
Can someone help out?
Thanks
It looks like you're trying to use the procedural version, Resample, not the class version, ResampleImageFilter.
You can see the documentation for the Resample function here:
https://simpleitk.org/doxygen/latest/html/namespaceitk_1_1simple.html#aadbb243c10d1aedea8e955e8beda4df0
You want to the second version of Resample. So your code would look like this:
img_sitk = sitk.Resample(img_sitk, curated_size, sitk.Transform(), method, img_sitk.GetOrigin(), curated_spacing, img_sitk.GetDirection(), 0, img_sitk.GetPixelIDValue())
If you want to use the ResampleImageFilter, you would set all the parameters using the filter's various Set methods, and then just call the Execute method with the input image as the only parameter.
I have a simple LinearModel with two sparse and two real-valued features. I trained it and now I want to export it with the export_savedmodel. Referencing few sources I came up with something along the lines of:
feature_spec = create_feature_spec_for_parsing(
[
real_valued_column_1, real_valued_column_2,
sparse_column_1, sparce_column_2
]
)
input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
my_estimator.export_savedmodel('my_model/', serving_input_fn=input_receiver_fn)
where:
real_valued_column_1 = tf.contrib.layers.real_valued_column(
'avg_consumption_h')
sparse_column_1 = tf.contrib.layers.sparse_column_with_integerized_feature("sparse_1", bucket_size=24)
Unfortunately I get ValueError: A default input_alternative must be provided. on export_savedmodel. I digged in a little into the codebase of tensorflow and it seems that build_parsing_serving_input_receiver_fn always returns ServingInputReceiver but the method that extracts input_alternatives always creates them empty if serving_input_fn passed to export_savedmodel is not of the type InputFnOps.
Is build_parsing_serving_input_receiver_fn somehow deprecated, something is wrong in the process of extraction of input_alternative, or maybe I'm misunderstanding process completely and doing something wrong?
I'm using python 3.6 with tensorflow 1.2, my model is a simple tf.contrib.learn.LinearRegressor.
You can try the following
from tensorflow.contrib.learn.python.learn.utils.input_fn_utils import build_parsing_serving_input_fn
input_receiver_fn = build_parsing_serving_input_fn(feature_spec)
Here is a great question on how to find the first occurence of Nan in a tensorflow graph:
Debugging nans in the backward pass
The answer is quite helpful, here is the code from it:
train_op = ...
check_op = tf.add_check_numerics_ops()
sess = tf.Session()
sess.run([train_op, check_op]) # Runs training and checks for NaNs
Apparently, running the training and the numerical check at the same time will result in an error report as soon as Nan is encountered for the first time.
How do I integrate this into Keras ?
In the documentation, I can't find anything that looks like this.
I checked the code, too.
The update step is executed here:
https://github.com/fchollet/keras/blob/master/keras/engine/training.py
There is a function called _make_train_function where an operation to compute the loss and apply updates is created. This is later called to train the network.
I could change the code like this (always assuming that we're running on a tf backend):
check_op = tf.add_check_numerics_ops()
self.train_function = K.function(inputs,
[self.total_loss] + self.metrics_tensors + [check_op],
updates=updates, name='train_function', **self._function_kwargs)
I'm currently trying to set this up properly and not sure whether the code above actually works.
Maybe there is an easier way ?
I've been running into the exact same problem, and found an alternative to the check_add_numerics_ops() function. Instead of going that route, I use the TensorFlow Debugger to walk through my model, following the example in https://www.tensorflow.org/guide/debugger to figure out exactly where my code produces nans. This snippet should work for replacing the TensorFlow Session that Keras is using with a debugging session, allowing you to use tfdbg.
from tensorflow.python import debug as tf_debug
sess = K.get_session()
sess = tf_debug.LocalCLIDebugWrapperSession(sess)
K.set_session(sess)
I have tried to modify the CIFAR-10 example to run on the new TensorFlow distributed runtime. However, I get the following error when trying to run the program:
InvalidArgumentError: Cannot assign a device to node 'softmax_linear/biases/ExponentialMovingAverage':
Could not satisfy explicit device specification '/job:local/task:0/device:CPU:0'
I start the cluster using the following commands. On the first node I run:
bazel-bin/tensorflow/core/distributed_runtime/rpc/grpc_tensorflow_server --cluster_spec='local|10.31.101.101:7777;10.31.101.224:7778' --job_name=local --task_id=0
...and on the second node I run:
bazel-bin/tensorflow/core/distributed_runtime/rpc/grpc_tensorflow_server --cluster_spec='local|10.31.101.101:7777;10.31.101.224:7778' --job_name=local --task_id=1
For the CIFAR-10 multi-GPU code, I make the simple modifications, replacing two lines in the train() function. The following line:
with tf.Graph().as_default(), tf.device('/cpu:0'):
...is replaced with:
with tf.Graph().as_default(), tf.device('/job:local/task:0/cpu:0'):
and the following line:
with tf.device('/gpu:%d' % i):
...is replaced with:
with tf.device('/job:local/task:0/gpu:%d' % i):
In my understanding, the second substitution should take care of the model substitution. Running a simpler example, like the code below, works fine:
with tf.device('/job:local/task:0/cpu:0'):
c = tf.constant("Hello, distributed TensorFlow!")
sess.run(c)
print(c)
I can't tell from your program, but my guess is that you also have to modify the line that creates the session to specify the address of one of your worker tasks. For example, given your configuration above, you might write:
sess = tf.Session(
"grpc://10.31.101.101:7777",
config=tf.ConfigProto(
allow_soft_placement=True,
log_device_placement=FLAGS.log_device_placement))
As it happens, we've been trying to improve that error message to make it less confusing. If you update to the latest version in GitHub and run the same code, you should see an error message that explains why the device specification could not be satisfied.