I'm using spaCy 2.0.18. I have trained a model using GPU but now I want to load this model for the predictions and run on CPU only.
I am able to load the model into memory but once I try to use it I get the following error:
import spacy
nlp = spacy.load("path_to_my_model")
# works fine up to this moment
result = nlp("Test") # throws the exception below:
Exception ignored in: <bound method Stream.__del__ of <cupy.cuda.stream.Stream object at 0x7fd288621be0>>
Traceback (most recent call last):
File "cupy/cuda/stream.pyx", line 161, in cupy.cuda.stream.Stream.__del__
AttributeError: 'Stream' object has no attribute 'ptr'
---------------------------------------------------------------------------
CUDARuntimeError Traceback (most recent call last)
<ipython-input-4-306c96b208c5> in <module>
----> 1 nlp("Yolo")
/opt/anaconda3/lib/python3.7/site-packages/spacy/language.py in __call__(self, text, disable)
344 if not hasattr(proc, '__call__'):
345 raise ValueError(Errors.E003.format(component=type(proc), name=name))
--> 346 doc = proc(doc)
347 if doc is None:
348 raise ValueError(Errors.E005.format(name=name))
nn_parser.pyx in spacy.syntax.nn_parser.Parser.__call__()
nn_parser.pyx in spacy.syntax.nn_parser.Parser.parse_batch()
/opt/anaconda3/lib/python3.7/site-packages/spacy/util.py in get_cuda_stream(require)
236
237 def get_cuda_stream(require=False):
--> 238 return CudaStream() if CudaStream is not None else None
239
240
cupy/cuda/stream.pyx in cupy.cuda.stream.Stream.__init__()
cupy/cuda/runtime.pyx in cupy.cuda.runtime.streamCreate()
cupy/cuda/runtime.pyx in cupy.cuda.runtime.streamCreate()
cupy/cuda/runtime.pyx in cupy.cuda.runtime.check_status()
CUDARuntimeError: cudaErrorNoDevice: no CUDA-capable device is detected
How to force spaCy to use CPU instead of GPU?
Related
Schreenshot of the error message
I'm trying to run Alphafold on Google runtime, and I'm getting this error:
UnfilteredStackTrace Traceback (most recent call last)
<ipython-input-5-ca4ee2dc266d> in <module>
44 processed_feature_dict = model_runner.process_features(np_example, random_seed=0)
---> 45 prediction = model_runner.predict(processed_feature_dict, random_seed=random.randrange(sys.maxsize))
46
UnfilteredStackTrace: AttributeError: 'Config' object has no attribute 'jax_experimental_name_stack'
The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.
--------------------
The above exception was the direct cause of the following exception:
AttributeError Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/haiku/_src/module.py in wrapped(self, *args, **kwargs)
406 f = functools.partial(unbound_method, self)
407 f = functools.partial(run_interceptors, f, method_name, self)
--> 408 if jax.config.jax_experimental_name_stack and module_name:
409 local_module_name = module_name.split("/")[-1]
410 f = jax.named_call(f, name=local_module_name)
AttributeError: 'Config' object has no attribute 'jax_experimental_name_stack'
How do I resolve this error? The link for Alphafold is below.
https://colab.research.google.com/github/deepmind/alphafold/blob/main/notebooks/AlphaFold.ipynb
I wanted to create a kubeflow pipeline for a MNIST model I created.I included the train function and predict function as container images in my pipeline. After creating the client instance, when I call the function create_run_from_pipeline_func to run my pipeline, It throws the following error.
AttributeError Traceback (most recent call last)
<ipython-input-24-bccaab255371> in <module>
11
12 # Submit pipeline directly from pipeline function
---> 13 run_result = client.create_run_from_pipeline_func(pipeline_func,
14 experiment_name=experiment_name,
15 run_name=run_name,
~/.local/lib/python3.8/site-packages/kfp/_client.py in create_run_from_pipeline_func(self, pipeline_func, arguments, run_name, experiment_name, pipeline_conf, namespace, mode, launcher_image, pipeline_root, enable_caching, service_account)
999 pipeline_conf=pipeline_conf)
1000
-> 1001 return self.create_run_from_pipeline_package(
1002 pipeline_file=pipeline_package_path,
1003 arguments=arguments,
~/.local/lib/python3.8/site-packages/kfp/_client.py in create_run_from_pipeline_package(self, pipeline_file, arguments, run_name, experiment_name, namespace, pipeline_root, enable_caching, service_account)
1080 experiment = self.create_experiment(
1081 name=experiment_name, namespace=namespace)
-> 1082 run_info = self.run_pipeline(
1083 experiment_id=experiment.id,
1084 job_name=run_name,
~/.local/lib/python3.8/site-packages/kfp/_client.py in run_pipeline(self, experiment_id, job_name, pipeline_package_path, params, pipeline_id, version_id, pipeline_root, enable_caching, service_account)
756 html = (
757 '<a href="%s/#/runs/details/%s" target="_blank" >Run details</a>.'
--> 758 % (self._get_url_prefix(), response.run.id))
759 IPython.display.display(IPython.display.HTML(html))
760 return response.run
AttributeError: 'NoneType' object has no attribute 'id'
the following inbuilt function should return an object but it returns a null type(response)
response = self._run_api.create_run(body=run_body)
I am running the following code:
I have specified the host inside the client instance creation.
client = kfp.Client(host='http://10.152.183.8.nip.io/')
Kubeflow version:1.21:
kfctl version: kfctl v1.0.1-0-gf3edb9b:
Kubernetes platform: Microk8s
Kubernetes version: Client-GitVersion:"v1.24.0" , Server-GitVersion:"v1.21.12-3+6937f71915b56b":
OS : Linux 18.04
When saving model in Pytorch by using:
torch.save(model, 'checkpoint.pth')
I get the following warning:
/opt/conda/lib/python3.6/site-packages/torch/serialization.py:193:
UserWarning: Couldn't retrieve source code for container of type
Network. It won't be checked for correctness upon loading. "type " +
obj.name + ". It won't be checked "
When I load it I get the following error:
state_dict = torch.load('checkpoint_state_dict.pth')
model = torch.load('checkpoint.pth')
model.load_state_dict(state_dict)
AttributeError Traceback (most recent call last)
<ipython-input-2-6a79854aef0f> in <module>()
2 state_dict = torch.load('checkpoint_state_dict.pth')
3 model = 0
----> 4 model = torch.load('checkpoint.pth')
5 model.load_state_dict(state_dict)
/opt/conda/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module)
301 f = open(f, 'rb')
302 try:
--> 303 return _load(f, map_location, pickle_module)
304 finally:
305 if new_fd:
/opt/conda/lib/python3.6/site-packages/torch/serialization.py in _load(f, map_location, pickle_module)
467 unpickler = pickle_module.Unpickler(f)
468 unpickler.persistent_load = persistent_load
--> 469 result = unpickler.load()
470
471 deserialized_storage_keys = pickle_module.load(f)
AttributeError: Can't get attribute 'Network' on <module '__main__'>
Why is not possible to save a model and reload it entirely?
Saving
torch.save({'state_dict': model.state_dict()}, 'checkpoint.pth.tar')
Loading
model = describe_model()
checkpoint = torch.load('checkpoint.pth.tar')
model.load_state_dict(checkpoint['state_dict'])
After creating ALS model object,using pyspark.
Sample code example:
from pyspark.ml.evaluation import RegressionEvaluator
from pyspark.ml.recommendation import ALS
from pyspark.sql import Row
lines = spark.read.text("data/mllib/als/sample_movielens_ratings.txt").rdd
parts = lines.map(lambda row: row.value.split("::"))
ratingsRDD = parts.map(lambda p: Row(userId=int(p[0]), movieId=int(p[1]),
rating=float(p[2]), timestamp=long(p[3])))
ratings = spark.createDataFrame(ratingsRDD)
(rating_data, test) = ratings.randomSplit([0.8, 0.2])
# Build the recommendation model using ALS on the training data
# Note we set cold start strategy to 'drop' to ensure we don't get NaN evaluation metrics
als = ALS(maxIter=5, regParam=0.01, userCol="userId", itemCol="movieId", ratingCol="rating",
coldStartStrategy="drop")
als_model = als_spec.fit(rating_data)
here i am just creating ALS model and making cloudepickel.
if we are using fit then also need to do transform?
I am trying pickel the my als_model object using the below code :
with open(os.path.join(model_path, 'als-als-model.pkl'), 'w') as out:
cloudpickle.dump(als_model, out)
I am getting error like below:
File "/usr/local/spark/python/lib/py4j-0.10.6-src.zip/py4j/protocol.py", line 324, in get_return_value
format(target_id, ".", name, value))
Py4JError: An error occurred while calling o224.__getnewargs__. Trace:
py4j.Py4JException: Method __getnewargs__([]) does not exist
#011at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
#011at
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-124-8c94f4ee0de9> in <module>()
1
----> 2 tree.fit(data_location)
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name)
152 self.latest_training_job = _TrainingJob.start_new(self, inputs)
153 if wait:
--> 154 self.latest_training_job.wait(logs=logs)
155 else:
156 raise NotImplemented('Asynchronous fit not available')
I'm using pydicom 1.0.0a1, downloaded from here, When I run the following code:
ds=pydicom.read_file('./DR/abnormal/abc.dcm',force=True)
ds.pixel_array
this error occurs:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-6-d4e81d303439> in <module>()
7 ds=pydicom.read_file('./DR/abnormal/abc.dcm',force=True)
8
----> 9 ds.pixel_array
10
/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/dataset.pyc in __getattr__(self, name)
501 if tag is None: # `name` isn't a DICOM element keyword
502 # Try the base class attribute getter (fix for issue 332)
--> 503 return super(Dataset, self).__getattribute__(name)
504 tag = Tag(tag)
505 if tag not in self: # DICOM DataElement not in the Dataset
/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/dataset.pyc in pixel_array(self)
1064 The Pixel Data (7FE0,0010) as a NumPy ndarray.
1065 """
-> 1066 return self._get_pixel_array()
1067
1068 # Format strings spec'd according to python string formatting options
/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/dataset.pyc in _get_pixel_array(self)
1042 elif self._pixel_id != id(self.PixelData):
1043 already_have = False
-> 1044 if not already_have and not self._is_uncompressed_transfer_syntax():
1045 try:
1046 # print("Pixel Data is compressed")
/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/dataset.pyc in _is_uncompressed_transfer_syntax(self)
662 """Return True if the TransferSyntaxUID is a compressed syntax."""
663 # FIXME uses file_meta here, should really only be thus for FileDataset
--> 664 return self.file_meta.TransferSyntaxUID in NotCompressedPixelTransferSyntaxes
665
666 def __ne__(self, other):
/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/dataset.pyc in __getattr__(self, name)
505 if tag not in self: # DICOM DataElement not in the Dataset
506 # Try the base class attribute getter (fix for issue 332)
--> 507 return super(Dataset, self).__getattribute__(name)
508 else:
509 return self[tag].value
AttributeError: 'Dataset' object has no attribute 'TransferSyntaxUID'
I read the google group post , and I changed the filereader.py file to the posted file, and I got this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/__init__.py", line 41, in read_file
from pydicom.dicomio import read_file
File "/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/dicomio.py", line 3, in <module>
from pydicom.filereader import read_file, read_dicomdir
File "/Applications/anaconda/lib/python2.7/site-packages/pydicom-1.0.0a1-py2.7.egg/pydicom/filereader.py", line 35, in <module>
from pydicom.datadict import dictionaryVR
ImportError: cannot import name dictionaryVR
Does anybody know how to solve this problem?
You should set the TransferSyntaxUID after reading the file before trying to get the pixel_array.
import pydicom.uid
ds=pydicom.read_file('./DR/abnormal/abc.dcm',force=True)
ds.file_meta.TransferSyntaxUID = pydicom.uid.ImplicitVRLittleEndian # or whatever is the correct transfer syntax for the file
ds.pixel_array
The correction from the post you referenced was done before some changes in the code to harmonize some naming, so the error is thrown because the current master uses dictionary_VR rather than dictionaryVR. Setting the transfer syntax in user code as above avoids that problem.