We Keep Coding

Attribute Error: 'Embedding' object has no attribute 'embeddings' - TensorFlow & Keras

Okay so I have a keras model that I fully ran and then saved the weights with this line:
model.save_weights("rho_beta_true_tf", save_format="tf")
Then in another file I build just the model and then I load the weights from the model I ran above using this line:
model_build.load_weights("rho_beta_true_tf")
When I then go to call some of the attributes everything displays correctly except when I try to run this line:
model_build.stimuli.embeddings
or
model_build.stimuli.embeddings.numpy()[0]
I get an attribute error saying:
AttributeError: 'Embedding' object has no attribute 'embeddings'
This line is supposed to return a tensor and if I call any other attributes so far it works so I am not sure if it just can't find the tensors or if the problem is something else. Could someone please help me figure out how to solve this attribute Error?

Try using .get_weights():
model_build.stimuli.get_weights()

Turns out that because I had saved the weights in tf format I had to follow this step in the tensor flow documentation:
For user-defined classes which inherit from tf.keras.Model, Layer instances must be assigned to object attributes, typically in the constructor.
So then the line
build_model.stimuli.embedding(put the directory path to your custom embedding layer here)
worked!

Tensorflow Data Adapter Error: ValueError: Failed to find data adapter that can handle input

While running a sentdex tutorial script of a cryptocurrency RNN, link here
YouTube Tutorial: Cryptocurrency-predicting RNN Model,
but have encountered an error when attempting to train the model. My tensorflow version is 2.0.0 and I'm running python 3.6. When attempting to train the model I receive the following error:
File "C:\python36-64\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 734, in fit
use_multiprocessing=use_multiprocessing)
File "C:\python36-64\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 224, in fit
distribution_strategy=strategy)
File "C:\python36-64\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 497, in _process_training_inputs
adapter_cls = data_adapter.select_data_adapter(x, y)
File "C:\python36-64\lib\site-packages\tensorflow_core\python\keras\engine\data_adapter.py", line 628, in select_data_adapter
_type_name(x), _type_name(y)))
ValueError: Failed to find data adapter that can handle input: <class 'numpy.ndarray'>, (<class 'list'> containing values of types {"<class 'numpy.float64'>"})
Any advice would be greatly appreciated!

Have you checked whether your training/testing data and training/testing labels are all numpy arrays? It might be that you're mixing numpy arrays with lists.

You can avoid this error by converting your labels to arrays before calling model.fit():
train_x = np.asarray(train_x)
train_y = np.asarray(train_y)
validation_x = np.asarray(validation_x)
validation_y = np.asarray(validation_y)

If you encounter this problem while dealing with a custom generator inheriting from the keras.utils.Sequence class, you might have to make sure that you do not mix a Keras or a tensorflow - Keras-import.
This might especially happen when you have to switch to a previous tensorflow version for compatibility (like with cuDNN).
If you for example use this with a tensorflow-version > 2...
from keras.utils import Sequence
class generatorClass(Sequence):
def __init__(self, x_set, y_set, batch_size):
...
def __len__(self):
...
def __getitem__(self, idx):
return ...
... but you actually try to fit this generator in a tensorflow-version < 2, you have to make sure to import the Sequence-class from this version like:
keras = tf.compat.v1.keras
Sequence = keras.utils.Sequence
class generatorClass(Sequence):
...

I had a similar problem. In my case it was a problem that I was using a tf.keras.Sequential model but a keras generator.
Wrong:
from keras.preprocessing.sequence import TimeseriesGenerator
gen = TimeseriesGenerator(...)
Correct:
gen = tf.keras.preprocessing.sequence.TimeseriesGenerator(...)

This error occured when I updated tensorflow from 1.x to 2.x
It was solved after changing my import from
import keras
to
import tensorflow.keras as keras

For some reason I also experienced this problem when I passed my custom generator function directly to model.fit(), rather than creating an instance of it first.
I.e, given:
def batch_generator(...):
...
yield(...)
I called model.fit(batch_generator,...), rather than:
generator_instance = batch_generator(...)
model.fit(generator_instance, ...)

may be it will help someone.
First check your data type if it is numpy array & possibly ur algo required a DF.
print(X.shape, X.dtype)
print(y.shape, y.dtype)
convert your numpy array into Pandas DF
train_x = pd.DataFrame(train_x)
train_y = pd.DataFrame(train_y)

Change keras floatx (default float type) in Python

I'm looking for a way to change floatx in keras directly in python.
floatx is the default float type (float16, float32 . . .)
The config is stored in a json file at:
$HOME/.keras/keras.json
But I'm looking for a way to change the config inside my python programm without changing the config file itself.
There is a similiar question, in which somebody ask the same for changing the backend, which is also stored in keras.json.
The accepted answer involves setting the environment variable KERAS_BACKEND and reload the keras module, but I didn't find a similar environment variable for floatx.

Turns out keras.backend has function for setting and retrieving the floatx value (scroll down in the link):
keras.backend.floatx()
>>> 'float32'
keras.backend.set_floatx('float16')
keras.backend.floatx()
>>> 'float16'
Also you are not allowed to reload the keras module after using set_floatx like when changing backend, because then keras will simply reread the config file and return to its previous value:
keras.backend.floatx()
>>> 'float32'
keras.backend.set_floatx('float16')
keras.backend.floatx()
>>> 'float16'
importlib.reload(keras.backend)
keras.backend.floatx()
>>> 'float32'

Well, the floatx var should certainly be used in keras.json, as described in documentation.
The least buggy way to do it is using the file indeed and reloading the module.
Using K.set_floatx, at least for me, left parts of the models unchanged (even if sef_floatx was the very first thing I did after loading the keras model in a new python kernel)
Even though, I faced yet another bug when setting precision to float16: all my loss functions very quickly became nan. Unfortunately I had to go back to float32 (the default) to have the possibility of training.

Tensorflow Estimator: How to run an operation within model_fn

I got a very short question, which has probably a very simple answer but I just can't figure it out, although I tried for hours now.
I'm using Tensorflow Estimator and I want to access the global step within my model_fn. I've tried tf.train.get_global_step, which returns me a Tensor. I need the global_step as an integer though (or as a string)!
So I've tried to eval() (= tf.get_default_session().run(t)), but it doesn't work..
Cheers!

You can use tf.cast to cast the Tensor to int or string.
For example,
tf.cast(tf.train.get_global_step(), dtype=tf.int)
See the reference here.

One way would be to parse it from the latest checkpoint file in the model_dir.
So assuming you can pass the model_dir into the model_fn (either through the params argument of tf.estimator.Estimator(..., params={'model_dir': 'path/to/model_dir'}) or through tf.flags.FLAGS, you can then use this utility function:
import tensorflow as tf
def get_global_step_from_model_dir(model_dir):
latest_checkpoint_file = tf.train.latest_checkpoint(model_dir)
if latest_checkpoint_file is None:
return 0
else:
return int(os.path.basename(latest_checkpoint_file).split('-')[-1])

Tensorflow - tf.Layers : Error with the function .fit

I'm running the example cnn_mnist given on github that is using layers module.
I can run the program but a Warning appears telling me that one of the function is deprecated.
I couldn't find which new function needs to be used.
WARNING:tensorflow:From <ipython-input-14-ee49e8b76469>:25: calling BaseEstimator.fit (from tensorflow.contrib.learn.python.learn.estimators.estimator) with batch_size is deprecated and will be removed after 2016-12-01.
Instructions for updating:
Estimator is decoupled from Scikit Learn interface by moving into
separate class SKCompat. Arguments x, y and batch_size are only
available in the SKCompat class, Estimator will only accept input_fn.
Example conversion:
est = Estimator(...) -> est = SKCompat(Estimator(...))

It should be sufficient to modify that script like this:
# Create the Estimator
mnist_classifier = learn.SKCompat(learn.Estimator(
model_fn=cnn_model_fn, model_dir="/tmp/mnist_convnet_model"))
However, please open a Github issue to update that sample so it doesn't mislead anyone else.