according to the tutorial at this link I want to create my custom dataset and use it with tensorflow.
I have installed the tfds command and when I entering tfds new my_dataset command, I will encounter to this error :
tfds new my_dataset
Traceback (most recent call last):
File "/root/anaconda3/envs/simclr/bin/tfds", line 5, in <module>
from tensorflow_datasets.scripts.cli.main import launch_cli
File "/root/anaconda3/envs/simclr/lib/python3.7/site-packages/tensorflow_datasets/scripts/cli/main.py", line 39, in <module>
from tensorflow_datasets.scripts.cli import build
File "/root/anaconda3/envs/simclr/lib/python3.7/site-packages/tensorflow_datasets/scripts/cli/build.py", line 275, in <module>
def _search_script_path(ds_to_build: str) -> Optional[tfds.core.ReadOnlyPath]:
AttributeError: module 'tensorflow_datasets.core' has no attribute 'ReadOnlyPath'
I changed the access level to rwx, and again the error raise.
I don't know how to solve it.
this problem solved in tensorflow V2. so use tensorflow V2 or higher versions.
Related
I am following this github(https://github.com/intel/lpot/tree/master/examples/tensorflow/object_detection) for lpot and in the 5th step for downloading the dataset I am getting the below error.Unable to proceed.
HEAD is now at 7a9934df Merged commit includes the following changes: 184048729 by Zhichao Lu:
Traceback (most recent call last):
File "./object_detection/dataset_tools/create_coco_tf_record.py", line 46, in <module>
flags = tf.app.flags
AttributeError: module 'tensorflow' has no attribute 'app'
mv: cannot stat '/root/lpot/examples/tensorflow/object_detection/data/coco_testdev.record*': No such file or directory
mv: cannot stat '/root/lpot/examples/tensorflow/object_detection/data/coco_train.record*': No such file or directory
mv: cannot stat '/root/lpot/examples/tensorflow/object_detection/data/coco_val.record*': No such file or directory
Thanks in advance.
If it is tensorflow 2.0 or higher version then minor change in create_coco_tf_record.py code will resolve your issue.Replace import tensorflow as tf with import tensorflow.compat.v1 as tf
Regards,
Janani Chandran
I am trying to run the following lines:
import tensorflow as tf
physical_devices = tf.config.list_physical_devices("GPU")
for i in range(len(physical_devices)):
tf.config.experimental.set_memory_growth(physical_devices[i], True)
I am trying to run this lines too:
import tensorflow as tf
physical_devices = tf.test.gpu_device_name("GPU")
for i in range(len(physical_devices)):
tf.config.experimental.set_memory_growth(physical_devices[i], True)
But the first method errors out like this:
Traceback (most recent call last):
File "main_train.py", line 4, in <module>
physical_devices = tf.config.list_physical_devices("GPU")
AttributeError: module 'tensorflow' has no attribute 'config'
And the second method errors out like this:
Traceback (most recent call last):
File "main_train.py", line 5, in <module>
for i in range(len(tf.test.gpu_device_name("GPU"))):
AttributeError: module 'tensorflow' has no attribute 'test'
I am using Tensorflow 2.4.0 .Any ideas what I'm doing wrong?
If anyone could help that would be great:)
Thank's for every help in advance:)
I read that you already fixed it but since you didn't included it in your post here is the solution everyone else having this error: you need to change some modules from e.g. tf.config.list_physical_devices("GPU") to tf.compat.v1.config.list_physical_devices("GPU"). By adding ".compat.v1" you are ensuring backwards capability. You can use this code snippet for all other modules, that don't work.
I need to add mixed precision to my code in order to save some memory. Specifically, I have tried adding mixed precision policy near line 27 in https://github.com/nimRobotics/google-research/blob/master/ravens/train.py, below is the code excerpt
import argparse
import datetime
import os
import numpy as np
from ravens import agents
from ravens import Dataset
import tensorflow as tf
# tf.keras.mixed_precision.set_global_policy('mixed_float16')
# OR
# policy = tf.keras.mixed_precision.Policy('mixed_float16')
# mixed_precision.set_global_policy(policy)
Both the methods result in attribute error as shown below, I'm using Google Colab with TF 2.3.0
Using tf.keras.mixed_precision.set_global_policy('mixed_float16') results in
Traceback (most recent call last):
File "train.py", line 28, in <module>
tf.keras.mixed_precision.set_global_policy('mixed_float16')
AttributeError: module 'tensorflow.keras.mixed_precision' has no attribute 'set_global_policy'
Using
policy = tf.keras.mixed_precision.Policy('mixed_float16')
mixed_precision.set_global_policy(policy)
results in
Traceback (most recent call last):
File "train.py", line 29, in <module>
policy = tf.keras.mixed_precision.Policy('mixed_float16')
AttributeError: module 'tensorflow.keras.mixed_precision' has no attribute 'Policy'
Any help or hints will be highly appreciated!
For tf < 2.4 you should use mixed precision's experimental package.
tf.keras.mixed_precision.experimental.Policy(
name, loss_scale='auto'
)
For ex, in tf 2.3
policy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')
tf.keras.mixed_precision.experimental.set_policy(policy)
and in tf 2.4
tf.keras.mixed_precision.set_global_policy('mixed_float16')
From tf 2.4, this feature is no longer experimental.
I've followed the instructions in Detectron and I've configured it several times: the code compiles as it should. When it comes to run the code, I get this error:
Traceback (most recent call last):
File "tools/train_net_step.py", line 21, in <module>
import nn as mynn
File "/home/federico/PycharmProjects/Detectron.pytorch/lib/nn/__init__.py", line 2, in <module>
from .parallel import DataParallel
File "/home/federico/PycharmProjects/Detectron.pytorch/lib/nn/parallel/__init__.py", line 3, in <module>
from .data_parallel import DataParallel, data_parallel
File "/home/federico/PycharmProjects/Detectron.pytorch/lib/nn/parallel/data_parallel.py", line 4, in <module>
from .scatter_gather import scatter_kwargs, gather
File "/home/federico/PycharmProjects/Detectron.pytorch/lib/nn/parallel/scatter_gather.py", line 8, in <module>
from torch.utils.data.dataloader import numpy_type_map
ImportError: cannot import name 'numpy_type_map'
I've also tried to google it many times, but I can't find a way to solve it. What can I do? I'm using PyTorch 0.4.1 and pytorch nightly 1.0.0-dev.
EDIT: Thanks to sancelot, I managed to solve that error (PyTorch 0.4.0 did the thing). Anyway, now I've got another error:
Traceback (most recent call last):
File "tools/train_net_step.py", line 27, in <module>
from modeling.model_builder import Generalized_RCNN
File "/home/federico/PycharmProjects/Detectron.pytorch/lib/modeling/model_builder.py", line 11, in <module>
from model.roi_pooling.functions.roi_pool import RoIPoolFunction
File "/home/federico/PycharmProjects/Detectron.pytorch/lib/model/roi_pooling/functions/roi_pool.py", line 3, in <module>
from .._ext import roi_pooling
File "/home/federico/PycharmProjects/Detectron.pytorch/lib/model/roi_pooling/_ext/roi_pooling/__init__.py", line 3, in <module>
from ._roi_pooling import lib as _lib, ffi as _ffi
ImportError: /home/federico/PycharmProjects/Detectron.pytorch/lib/model/roi_pooling/_ext/roi_pooling/_roi_pooling.so: undefined symbol: PyInt_FromLong
What I can't get this time is: is this an error given by an external library? I'm using an anaconda environment previously made by my professor, who has used it for Detectron... so I can't guess why I get this.
Yeah, these are due to Pytorch version mismatch. Solution depends on what extent you are willing to go, sometimes if you are okay with hacking and just getting it running, then just copy paste the numpy_type_map from older versions:
numpy_type_map = {
'float64': torch.DoubleTensor,
'float32': torch.FloatTensor,
'float16': torch.HalfTensor,
'int64': torch.LongTensor,
'int32': torch.IntTensor,
'int16': torch.ShortTensor,
'int8': torch.CharTensor,
'uint8': torch.ByteTensor,
}
Or, up to version 1.1.0, you can try replacing the import statement,
from:
from torch.utils.data.dataloader import numpy_type_map
to:
from torch.utils.data._utils.collate import numpy_type_map
N.b. will still break in more recent versions. Again, this is a hacky quick-fix solution.
I suppose there is a version mismatch between detectron and the needed pytorch release you are using.
if you look at latest pytorch source code, there is no numpy_type_map component.
https://github.com/pytorch/pytorch/blob/master/torch/utils/data/dataloader.py
When I am trying to import the Tokenizer() function in Keras, I am getting the following error.
I am following official Keras documentation at https://keras.io/preprocessing/text/#tokenizer
import keras
tokenizer = keras.preprocessing.text.Tokenizer()
Getting error like this :
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'keras.preprocessing' has no attribute 'text'
Using Keras version '2.1.3'
Where I have done wrong?
You want
from keras.preprocessing import text
tokenizer = text.Tokenizer(...)
The reason is that keras/preprocessing/__init__.py doesn't import nor expose the text submodule