I am using Keras == 1.1.0 and tensorflow-gpu == 1.12.0.
The error is called after:
input_layer = Input(shape=(2, ))
layer = Dense(self._hidden[0], activation='relu')(input_layer)
and this is the Traceback
Traceback (most recent call last):
File "D:/Documents/PycharmProjects/DDPG-master-2/main.py", line 18, in <module>
main()
File "D:/Documents/PycharmProjects/DDPG-master-2/main.py", line 14, in main
agent = Agent(state_size=world.state_size, action_size=world.action_size)
File "D:\Documents\PycharmProjects\DDPG-master-2\ddpg.py", line 50, in __init__
batch_size=batch_size, tau=tau)
File "D:\Documents\PycharmProjects\DDPG-master-2\networks\actor.py", line 68, in __init__
self._generate_model()
File "D:\Documents\PycharmProjects\DDPG-master-2\networks\actor.py", line 132, in _generate_model
layer = Dense(self._hidden[0], activation='relu')(input_layer)
File "D:\Anaconda3\lib\site-packages\keras\engine\topology.py", line 487, in __call__
self.build(input_shapes[0])
File "D:\Anaconda3\lib\site-packages\keras\layers\core.py", line 695, in build
name='{}_W'.format(self.name))
File "D:\Anaconda3\lib\site-packages\keras\initializations.py", line 59, in glorot_uniform
return uniform(shape, s, name=name)
File "D:\Anaconda3\lib\site-packages\keras\initializations.py", line 32, in uniform
return K.random_uniform_variable(shape, -scale, scale, name=name)
File "D:\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 282, in random_uniform_variable
return variable(value, dtype=dtype, name=name)
File "D:\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 152, in variable
if tf.get_default_graph() is get_session().graph:
AttributeError: 'function' object has no attribute 'graph'
Process finished with exit code 1
I previously had tensorflow-gpu == 1.9 and uninstalled it and upgraded to 1.12, as I saw that it was a common solutions for similar problems. It did not work though.
EDIT (adding some relevant code related to the Traceback):
agent = DDPG(state_size=world.state_size, action_size=world.action_size)
self._actor = Actor(tensorflow_session=tensorflow_session,
state_size=state_size, action_size=action_size,
hidden_units=actor_hidden_units,
learning_rate=actor_learning_rate,
batch_size=batch_size, tau=tau)
def _generate_model(self):
"""
Generates the model based on the hyperparameters defined in the
constructor.
:return: at tuple containing references to the model, weights,
and input later
"""
input_layer = Input(shape=(self._state_size,))
layer = Dense(self._hidden[0], activation='relu')(input_layer)
layer = Dense(self._hidden[1], activation='relu')(layer)
output_layer = Dense(self._action_size, activation='sigmoid')(layer)
model = Model(input=input_layer, output=output_layer)
return model, model.trainable_weights, input_layer
The code is related to three different classes.
I experienced the same issue. Here are the things I did to resolve it:
Make sure that there is no other files in my project named tensorflow.py
re-install tensorflow with --no-cache-dir argument pip --no-cache-dir install tensorflow and removed pip cache files.
For linux:
rm -rf ~/.cache/pip/*
For windows, delete the files in this location: %LocalAppData%\pip\Cache
I hope this helps
Here is how I solved the issue.
1. Uninstall keras with pip uninstall keras
2. Check that no other versions were installed (for example through conda)
3. Delete the cache in %LocalAppData%\pip\Cache
4. Re-install keras
Related
2023-01-25 08:21:21,659 - ERROR - Traceback (most recent call last):
File "/home/xyzUser/project/queue_handler/document_queue_listner.py", line 148, in __process_and_acknowledge
pipeline_result = self.__process_document_type(message, pipeline_input)
File "/home/xyzUser/project/queue_handler/document_queue_listner.py", line 194, in __process_document_type
pipeline_result = bill_parser_pipeline.process(pipeline_input)
File "/home/xyzUser/project/main/billparser/__init__.py", line 18, in process
bill_extractor_model = MachineGeneratedBillExtractorModel()
File "/home/xyzUser/project/main/billparser/models/qa_model.py", line 25, in __new__
cls.__model = TransformersReader(model_name_or_path=cls.__model_path, use_gpu=False)
File "/home/xyzUser/project/.env/lib/python3.8/site-packages/haystack/nodes/base.py", line 48, in wrapper_exportable_to_yaml
init_func(self, *args, **kwargs)
File "/home/xyzUser/project/.env/lib/python3.8/site-packages/haystack/nodes/reader/transformers.py", line 93, in __init__
self.model = pipeline(
File "/home/xyzUser/project/.env/lib/python3.8/site-packages/transformers/pipelines/__init__.py", line 542, in pipeline
return task_class(model=model, framework=framework, task=task, **kwargs)
File "/home/xyzUser/project/.env/lib/python3.8/site-packages/transformers/pipelines/question_answering.py", line 125, in __init__
super().__init__(
File "/home/xyzUser/project/.env/lib/python3.8/site-packages/transformers/pipelines/base.py", line 691, in __init__
self.device = device if framework == "tf" else torch.device("cpu" if device < 0 else f"cuda:{device}")
TypeError: '<' not supported between instances of 'torch.device' and 'int'
This is the error message i got after installing a requirement.txt file from my project. I think it is related to torch but also dont know how to fix it. I am new to hugging face transformers and dont know if it is a version issue.
This was a bug with the transformers package for a number of versions prior to v4.22.0, given that particular line of code does not discern between the type of the device argument could be a torch.device before comparing that with an int. Tracing through git blame, we can find that this specific change made in changeset 9d4a45509ab include the much needed if isinstance(device, torch.device): provided by line 764 in the resulting file, which will ensure this error won't happen. Checking the tags above will show that the release for v4.22.0 and after should include this particular fix. As a refresher, to update a specific package, activate the environment, and issue the following:
pip install -U transformers
Alternatively with a specific version, e.g.:
pip install -U transformers==4.22.0
I'm trying to get a graph neural network code to run on a cluster (where the code that I used, always used to work perfectly fine up to half a year ago). I have the following versions:
python 3.6.9
torch 1.6.0
torch-geometric 2.0.3
torch-scatter 2.0.5
torch-sparse 0.6.8
torchvision 0.7.0
If I run the code, I get the following error:
Traceback (most recent call last):
File "GNN_20210503_alltimes_trainsep.py", line 442, in
for train in train_dataloader:
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 363, in next
data = self._next_data()
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 403, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
return self.collate_fn(data)
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch_geometric/loader/dataloader.py", line 20, in call
self.exclude_keys)
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch_geometric/data/batch.py", line 75, in from_data_list
exclude_keys=exclude_keys,
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch_geometric/data/collate.py", line 109, in collate
out_store.batch = repeat_interleave(repeats, device=device)
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch_geometric/data/collate.py", line 205, in repeat_interleave
outs = [torch.full((n, ), i, device=device) for i, n in enumerate(repeats)]
File "/home/alkemade/.conda/envs/ML8/lib/python3.7/site-packages/torch_geometric/data/collate.py", line 205, in
outs = [torch.full((n, ), i, device=device) for i, n in enumerate(repeats)]
RuntimeError: Providing a bool or integral fill value without setting the optional dtype or out arguments is currently unsupported. In PyTorch 1.7, when dtype and out are not set a bool fill value will return a tensor of torch.bool dtype, and an integral fill value will return a tensor of torch.long dtype.
Downscaling the versions, as is suggested to others with this problem does unfortunately not work. Does anyone know how to fix this? Thank you in advance,
Best Rinske
Upgrade pytorch to 1.7.0.
This worked for me.
After installing MLFlow using one-click-mlflow I save a pytorch model using the default command that I found in the user guide. You can find the command bellow:
mlflow.pytorch.log_model(net, artifact_path="model", pickle_module=pickle)
The neural network saved is very simple, this is basically a two layer neural network with Xavier initialization and hyperbolic tangent as activation function.
class Net(T.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.hid1 = T.nn.Linear(n_features, 10)
self.hid2 = T.nn.Linear(10, 10)
self.oupt = T.nn.Linear(10, 1)
T.nn.init.xavier_uniform_(self.hid1.weight)
T.nn.init.zeros_(self.hid1.bias)
T.nn.init.xavier_uniform_(self.hid2.weight)
T.nn.init.zeros_(self.hid2.bias)
T.nn.init.xavier_uniform_(self.oupt.weight)
T.nn.init.zeros_(self.oupt.bias)
def forward(self, x):
z = T.tanh(self.hid1(x))
z = T.tanh(self.hid2(z))
z = self.oupt(z)
return z
Every things is runing fine in the Jupyter Notebook. I can log metrics and other artifact but when I save the model I got the following error message:
2021/10/13 09:21:00 WARNING mlflow.utils.requirements_utils: Found torch version (1.9.0+cu111) contains a local version label (+cu111). MLflow logged a pip requirement for this package as 'torch==1.9.0' without the local version label to make it installable from PyPI. To specify pip requirements containing local version labels, please use `conda_env` or `pip_requirements`.
2021/10/13 09:21:00 WARNING mlflow.utils.requirements_utils: Found torchvision version (0.10.0+cu111) contains a local version label (+cu111). MLflow logged a pip requirement for this package as 'torchvision==0.10.0' without the local version label to make it installable from PyPI. To specify pip requirements containing local version labels, please use `conda_env` or `pip_requirements`.
2021/10/13 09:21:01 ERROR mlflow.utils.environment: Encountered an unexpected error while inferring pip requirements (model URI: /tmp/tmpnl9dsoye/model/data, flavor: pytorch)
Traceback (most recent call last):
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/environment.py", line 212, in infer_pip_requirements
return _infer_requirements(model_uri, flavor)
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/requirements_utils.py", line 263, in _infer_requirements
modules = _capture_imported_modules(model_uri, flavor)
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/requirements_utils.py", line 221, in _capture_imported_modules
_run_command(
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/requirements_utils.py", line 173, in _run_command
raise MlflowException(msg)
mlflow.exceptions.MlflowException: Encountered an unexpected error while running ['/home/ucsky/.virtualenv/mymodel/bin/python', '/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/_capture_modules.py', '--model-path', '/tmp/tmpnl9dsoye/model/data', '--flavor', 'pytorch', '--output-file', '/tmp/tmplyj0w2fr/imported_modules.txt', '--sys-path', '["/home/ucsky/project/ofi-ds-research/incubator/ofi-pe-fr/notebook/guillaume-simon/06-modelisation-pytorch", "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/git/ext/gitdb", "/usr/lib/python39.zip", "/usr/lib/python3.9", "/usr/lib/python3.9/lib-dynload", "", "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages", "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/IPython/extensions", "/home/ucsky/.ipython", "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/gitdb/ext/smmap"]']
exit status: 1
stdout:
stderr: Traceback (most recent call last):
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/_capture_modules.py", line 125, in <module>
main()
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/_capture_modules.py", line 118, in main
importlib.import_module(f"mlflow.{flavor}")._load_pyfunc(model_path)
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/pytorch/__init__.py", line 723, in _load_pyfunc
return _PyTorchWrapper(_load_model(path, **kwargs))
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/pytorch/__init__.py", line 626, in _load_model
return torch.load(model_path, **kwargs)
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/torch/serialization.py", line 607, in load
return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/torch/serialization.py", line 882, in _load
result = unpickler.load()
File "/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/torch/serialization.py", line 875, in find_class
return super().find_class(mod_name, name)
AttributeError: Can't get attribute 'Net' on <module '__main__' from '/home/ucsky/.virtualenv/mymodel/lib/python3.9/site-packages/mlflow/utils/_capture_modules.py'>
Can somebody explain me what is wrong?
When I run the code, that error message raise.
I'm using python 2.7.5, keras 2.2.4 and tensorflow 2.0.0-beta1.
Somebody told me 'don't make file name tensorflow', but I didn't make like that.
I think it might be occurred by changed items that is in upgraded tensorflow version,
https://www.tensorflow.org/beta/guide/effective_tf2
, but the lower version also can't run with error that is "ValueError:bad marshal data".
All the processes are running on virtual environment.
Error message is
Using TensorFlow backend.
WARNING: Logging before flag parsing goes to stderr.
W0619 16:47:38.715425 139651914356544 deprecation.py:323] From /home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/tensorflow/python/compat/v2_compat.py:65: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Traceback (most recent call last):
File "./gpd_predict.py", line 165, in <module>
model = model_from_json(loaded_model_json, custom_objects={'tf':tf})
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/engine/saving.py", line 622, in model_from_json
return deserialize(config, custom_objects=custom_objects)
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/layers/__init__.py", line 168, in deserialize
printable_module_name='layer')
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 147, in deserialize_keras_object
list(custom_objects.items())))
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/engine/network.py", line 1043, in from_config
process_layer(layer_data)
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/engine/network.py", line 1029, in process_layer
custom_objects=custom_objects)
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/layers/__init__.py", line 168, in deserialize
printable_module_name='layer')
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 149, in deserialize_keras_object
return cls.from_config(config['config'])
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/engine/base_layer.py", line 1103, in from_config
return cls(**config)
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/engine/input_layer.py", line 87, in __init__
name=self.name)
File "/home/sea/Programs/generalized-phase-detection/generalized-phase-detection-master/venv/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 529, in placeholder
x = tf.placeholder(dtype, shape=shape, name=name)
AttributeError: 'module' object has no attribute 'placeholder'
I don't know if this is a problem line, but this line is only my code in the error message. (last line)
# load json and create model
json_file = open('model_pol.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json, custom_objects={'tf':tf})
How can I run the program without error code?
Tensorflow 2 is does not use placeholders by default so it is not available without using compatibility mode.
You could do
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
but, really, you probably want to migrate your code to best utelise v2. Check out the official guide here: https://www.tensorflow.org/beta/guide/migration_guide
Current Keras is not compatible with TensorFlow 2.0 (which has not been released as stable, just as a beta), so do not try to use the official keras with tf 2.0 yet (use stable tensorflow like 1.13), or use tf.keras from tensorflow 2.0 instead.
The below code snippet forcibly installs tensorflow version 1.x and provides compatibility.
%tensorflow_version 1.x
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
I have non-sudo access to a machine with NVIDIA GPUs and CUDA 7.5 installed. I installed PyTorch with CUDA 7.5 support, which seems to have worked:
>>> import torch
>>> torch.cuda.is_available()
True
To get some practice, I followed tutorial for machine translation using RNNs. When I set USE_CUDA = False and the CPUs are used, everything works quite alright. However, when want to utilize the GPUs with USE_CUDA = True I get the following error:
Traceback (most recent call last):
...
File "seq2seq.py", line 229, in train
encoder_output, encoder_hidden = encoder(input_variable[ei], encoder_hidden)
File "/.../python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "seq2seq.py", line 144, in forward
output, hidden = self.gru(embedded, hidden)
File "/.../python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/.../python2.7/site-packages/torch/nn/modules/rnn.py", line 91, in forward
output, hidden = func(input, self.all_weights, hx)
...
File "/.../python2.7/site-packages/torch/backends/cudnn/rnn.py", line 42, in init_rnn_descriptor
cudnn.DropoutDescriptor(handle, dropout_p, fn.dropout_seed)
File "/usr/lib/python2.7/ctypes/__init__.py", line 383, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python: undefined symbol: cudnnCreateDropoutDescriptor
Exception AttributeError: 'python: undefined symbol: cudnnDestroyDropoutDescriptor' in <bound method DropoutDescriptor.__del__ of <torch.backends.cudnn.DropoutDescriptor object at 0x7fe540efec10>> ignored
I've tried to use Google to search for that error but got no meaningful results. Since I'm rather a newbie with PyTorch and CUDA, I have no idea how to go on from here. The full setup is Ubuntu 14.04, Python 2.7, CUDA 7.5.
As stated in the comments: your error is with outdated CUDNN, and can be resolved by upgrading.
Install current versions of CUDA, CUDNN, and PyTorch, then you'll be fine.