SDK version: 1.0.43
To minimize clicking and compare accuracy between PipelineRuns, I'd like to log a metric from inside a PythonScriptStep to the parent PipelineRun. I thought I could do this like:
from azureml.core import Run
run = Run.get_context()
foo = 0.80
run.parent.log("accuracy",foo)
however I get this error.
Traceback (most recent call last):
File "get_metrics.py", line 62, in <module>
run.parent.log("geo_mean", top3_runs)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/run.py", line 459, in parent
return None if parent_run_id is None else get_run(self.experiment, parent_run_id)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/run.py", line 1713, in get_run
return next(runs)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/run.py", line 297, in _rehydrate_runs
yield factory(experiment, run_dto)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/run.py", line 325, in _from_dto
return PipelineRun(experiment=experiment, run_id=run_dto.run_id)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/run.py", line 74, in __init__
service_endpoint=_service_endpoint)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_graph_context.py", line 46, in __init__
service_endpoint=service_endpoint)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_aeva_provider.py", line 118, in create_provider
service_endpoint=service_endpoint)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_aeva_provider.py", line 133, in create_service_caller
service_endpoint = _AevaWorkflowProvider.get_endpoint_url(workspace, experiment_name)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_aeva_provider.py", line 153, in get_endpoint_url
workspace_name=workspace.name, workspace_id=workspace._workspace_id)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/workspace.py", line 749, in _workspace_id
self.get_details()
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/workspace.py", line 594, in get_details
self._subscription_id)
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/_project/_commands.py", line 507, in show_workspace
AzureMachineLearningWorkspaces, subscription_id).workspaces,
File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/authentication.py", line 112, in _get_service_client
all_subscription_list, tenant_id = self._get_all_subscription_ids()
TypeError: 'NoneType' object is not iterable
Update
On further investigation, I tried just printing the parent attribute of the run with the line below and got the same Traceback
print("print run parent attribute", run.parent)
The get_properties() method the below. I'm guessing that azureml just uses the azureml.pipelinerunid property for pipeline tree hierarchy, and that the parent attribute has been left for any user-defined hierarchies.
{
"azureml.runsource": "azureml.StepRun",
"ContentSnapshotId": "45bdecd3-1c43-48da-af5c-c95823c407e0",
"StepType": "PythonScriptStep",
"ComputeTargetType": "AmlCompute",
"azureml.pipelinerunid": "e523d575-c373-46d2-a4bc-1717f5e34ec2",
"_azureml.ComputeTargetType": "batchai",
"AzureML.DerivedImageName": "azureml/azureml_dfd7f4f952ace529f986fe919909c3ec"
}
Please upgrade your SDK to the latest version. Seems like this issue was fixed sometime after 1.0.43.
Was run defined? Can you try ...
run = Run.get_context()
run.parent.log('metric1', 0.80)
I'm unable to repro on 1.0.60 (latest) or 1.0.43 (as in post)
I can do run.parent.log()
run = Run.get_context()
run.parent.log('metric1', 0.80)
in my step
and then I can do
r = Run(ws.experiments["expName"], runid)
r.get_metrics()
and can see metric on pipeline run.
Unclear if i miss something.
Related
We updated to Python 3.8.2 and are getting an error with scikit-learn:
Traceback (most recent call last):
File "manage.py", line 16, in <module>
execute_from_command_line(sys.argv)
File "/home/ubuntu/myWebApp/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/home/ubuntu/myWebApp/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/ubuntu/myWebApp/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/ubuntu/myWebApp/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/home/ubuntu/myWebApp/server_modules/rss_ml_score/management/commands/rssmlscore.py", line 22, in handle
run.build_and_predict(days=options['days'], rescore=options['rescore'])
File "/home/ubuntu/myWebApp/server_modules/rss_ml_score/utils/run.py", line 96, in build_and_predict
predict_all(filename)
File "/home/ubuntu/myWebApp/server_modules/rss_ml_score/models/predict_model.py", line 135, in predict_all
voting_predicted_hard, voting_predicted_soft = predict_from_multiple_estimator(fitted_estimators, X_predict_list,
File "/home/ubuntu/myWebApp/server_modules/rss_ml_score/models/train_model.py", line 66, in predict_from_multiple_estimator
pred_prob1 = np.asarray([clf.predict_proba(X)
File "/home/ubuntu/myWebApp/server_modules/rss_ml_score/models/train_model.py", line 66, in <listcomp>
pred_prob1 = np.asarray([clf.predict_proba(X)
File "/home/ubuntu/myWebApp/.venv/lib/python3.8/site-packages/sklearn/svm/_base.py", line 662, in _predict_proba
if self.probA_.size == 0 or self.probB_.size == 0:
File "/home/ubuntu/myWebApp/.venv/lib/python3.8/site-packages/sklearn/svm/_base.py", line 759, in probA_
return self._probA
AttributeError: 'SVC' object has no attribute '_probA'
Do I need to use another library in addition to sci-kit learn to access _probA?
Update in response to a comment:
The line of code that's throwing the error is:
pred_prob1 = np.asarray([clf.predict_proba(X)
for clf, X in zip(estimators, X_list)])
...that calls this line in _base.py:
def _predict_proba(self, X):
X = self._validate_for_predict(X)
if self.probA_.size == 0 or self.probB_.size == 0:
...which calls this line, also in _base.py:
#property
def probA_(self):
return self._probA
...which throws the error:
AttributeError: 'SVC' object has no attribute '_probA'
All this has worked fine for many months, but is not currently working, even after updating to the latest scikit-learn.
It turned out that I had to stay with the same version of sci-kit that was used to train the models we currently have (scikit-learn==0.21.2). Later versions of scikit don't work with our existing code / models. If we want to upgrade scikit, we have to retrain our models with the new version of scikit.
I also had this error and I tried so much to resolve it. At the end it got resolved by just version compatible issues. Whatever the version used in building the model, the same version should be used in load
I am trying to generate an error log html by “rebot” package of robot framework and its getting generated successfully.
But if I use the rebot function in my module then its affect default log and report html which gets generated after script execution.
[ ERROR ] Unexpected error: AttributeError: 'NoneType' object has no attribute 'encode'
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/robot/utils/application.py", line 83, in _execute
rc = self.main(arguments, **options)
File "/usr/local/lib/python3.5/dist-packages/robot/run.py", line 445, in main
result = suite.run(settings)
File "/usr/local/lib/python3.5/dist-packages/robot/running/model.py", line 248, in run
self.visit(runner)
File "/usr/local/lib/python3.5/dist-packages/robot/model/testsuite.py", line 161, in visit
visitor.visit_suite(self)
File "/usr/local/lib/python3.5/dist-packages/robot/model/visitor.py", line 87, in visit_suite
suite.tests.visit(self)
File "/usr/local/lib/python3.5/dist-packages/robot/model/itemlist.py", line 76, in visit
item.visit(visitor)
File "/usr/local/lib/python3.5/dist-packages/robot/model/testcase.py", line 74, in visit
visitor.visit_test(self)
File "/usr/local/lib/python3.5/dist-packages/robot/running/runner.py", line 159, in visit_test
self._output.end_test(ModelCombiner(test, result))
File "/usr/local/lib/python3.5/dist-packages/robot/output/output.py", line 59, in end_test
LOGGER.end_test(test)
File "/usr/local/lib/python3.5/dist-packages/robot/output/logger.py", line 183, in end_test
logger.end_test(test)
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 51, in end_test
self._writer.status(test.status, clear=True)
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 114, in status
self._clear_status()
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 124, in _clear_status
self._write_info()
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 90, in _write_info
self._stdout.write(self._last_info)
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/highlighting.py", line 51, in write
self._write(console_encode(text, stream=self.stream))
File "/usr/local/lib/python3.5/dist-packages/robot/utils/encoding.py", line 60, in console_encode
return string.encode(encoding, errors).decode(encoding)
The error message is rather clear. It looks like you're trying to encode a variable which contains a None instead of a string. You need to make sure that this variable always contains a string and handle cases where something else is inside. You can do it for example using the try ... except statement.
I'm trying to run flake8 on a docker django built like described here (tutorial page)
when building the docker image I get an error from flake8 which is run in an docker-compose file with like so
$ flake8 --ignore=E501,F401 .
multiprocessing.pool.RemoteTraceback:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 48, in
return list(map(*
File "/usr/local/lib/python3.8/site-packages/flake8/checker.py", line 666, in
return checker.run_checks()
File "/usr/local/lib/python3.8/site-packages/flake8/checker.py", line 598, in run_checks
self.run_ast_checks()
File "/usr/local/lib/python3.8/site-packages/flake8/checker.py", line 495, in run_ast_checks
checker = self.run_check(plugin, tree=ast)
File "/usr/local/lib/python3.8/site-packages/flake8/checker.py", line 426, in run_check
self.processor.keyword_arguments_for( File "/usr/local/lib/python3.8/site-packages/flake8/processor.py", line 241, in keyword_arguments_for arguments[param] = getattr(self, param)
File "/usr/local/lib/python3.8/site-packages/flake8/processor.py", line 119, in file_tokens
self._file_tokens = list(
File "/usr/local/lib/python3.8/tokenize.py", line 525, in _tokenize
pseudomatch = _compile(PseudoToken).match(line, pos)
RuntimeError: internal error in regular expression engine
The above exception was the direct cause of the following exception: Traceback (most recent call last):
File "/usr/local/bin/flake8", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/flake8/main/cli.py", line 18, in main
app.run(argv)
File "/usr/local/lib/python3.8/site-packages/flake8/main/application.py", line 393, in run
self._run(argv)
File "/usr/local/lib/python3.8/site-packages/flake8/main/application.py", line 381, in _run
self.run_checks()
File "/usr/local/lib/python3.8/site-packages/flake8/main/application.py", line 300, in run_checks
self.file_checker_manager.run()
File "/usr/local/lib/python3.8/site-packages/flake8/checker.py", line 329, in run
self.run_parallel()
File "/usr/local/lib/python3.8/site-packages/flake8/checker.py", line 293, in run_parallel
for ret in pool_map:
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 448, in <genexpr>
return (item for chunk in result for item in chunk)
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 865, in next
raise value
RuntimeError: internal error in regular expression engine
When I run flake8 with the --verbose flag, I get an error like this:
Fatal Python error: deletion of interned string failed
Python runtime state: initialized
KeyError: 'FILENAME_RE'
from the tokenizer.py
Does anyone know how to solve this?
Additional Data:
running docker-compose v1.25.4 on an raspberry 3 with buster lite.
Installed and compiled Python 3.8.2 from source with the flag --enableloadable-sqlite
Thanks for helping!
If you don't care for flake8, then just delete that line. It will work. I had the same problem.
I know this isn't an answer. I would add a comment instead if I could.
I encountered a very similar error profile when I had refactored and left a lot of data which formerly had been in the top of the project down in what became the package folder. There were 33621 files including .venv, .nox, .pytest_cache, .coverage.
File "/usr/lib/python3.8/multiprocessing/pool.py", line 448, in <genexpr>
return (item for chunk in result for item in chunk)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 868, in next
raise value
IndexError: string index out of range
If you see a similar signature (the parallel processing engine being overwhelmed in some way and throwing an exception), you may want to review your project directory structure and make sure that you did not put this kind of data in your sources directories.
I'm using DRF and Pycharm 3.4.1 and Django 1.7. When I try to test my serializer class via Pycharm django console, it gives me the following error:
Code
from items_app.serializers import ItemSerializer
s = ItemSerializer()
print(repr(s))
then cause the following error traceback:
Traceback
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/rest_framework/serializers.py", line 440, in __repr__
return unicode_to_repr(representation.serializer_repr(self, indent=1))
File "/usr/local/lib/python2.7/dist-packages/rest_framework/utils/representation.py", line 75, in serializer_repr
fields = serializer.fields
File "/usr/local/lib/python2.7/dist-packages/rest_framework/serializers.py", line 312, in fields
for key, value in self.get_fields().items():
File "/usr/local/lib/python2.7/dist-packages/rest_framework/serializers.py", line 883, in get_fields
info = model_meta.get_field_info(model)
File "/usr/local/lib/python2.7/dist-packages/rest_framework/utils/model_meta.py", line 68, in get_field_info
reverse_relations = _get_reverse_relationships(opts)
File "/usr/local/lib/python2.7/dist-packages/rest_framework/utils/model_meta.py", line 129, in _get_reverse_relationships
for relation in opts.get_all_related_objects():
File "/usr/local/lib/python2.7/dist-packages/django/db/models/options.py", line 498, in get_all_related_objects
include_proxy_eq=include_proxy_eq)]
File "/usr/local/lib/python2.7/dist-packages/django/db/models/options.py", line 510, in get_all_related_objects_with_model
self._fill_related_objects_cache()
File "/usr/local/lib/python2.7/dist-packages/django/db/models/options.py", line 533, in _fill_related_objects_cache
for klass in self.apps.get_models(include_auto_created=True):
File "/usr/local/lib/python2.7/dist-packages/django/utils/lru_cache.py", line 101, in wrapper
result = user_function(*args, **kwds)
File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 168, in get_models
self.check_models_ready()
File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 131, in check_models_ready
raise AppRegistryNotReady("Models aren't loaded yet.")
AppRegistryNotReady: Models aren't loaded yet.
but when I'm using my terminal(instead of pycharm django console), It works properly! I know there is a problem with Pycharm but I don't know how to fix it!
Loading the app registry is part of the django.setup method. If the app registry is not loaded when you start using the console, the most likely reason is that it is a plain python console instead of a fully blown Django console.
Try the following code. If that solves it, you are indeed using a plain python console.
>>> import django
>>> django.setup()
I am relatively new to Python, and at the same time, am attempting to install mod_dav_svn into my Apache web server. I am looking to get some idea of the scope of the error I'm receiving.
At the command line, I type in 'sudo yum install mod_dav_svn' and receive this output:
Loaded plugins: fastestmirror
Determining fastest mirrors
Traceback (most recent call last):
File "/usr/bin/yum", line 29, in ?
yummain.user_main(sys.argv[1:], exit_code=True)
File "/usr/share/yum-cli/yummain.py", line 229, in user_main
errcode = main(args)
File "/usr/share/yum-cli/yummain.py", line 104, in main
result, resultmsgs = base.doCommands()
File "/usr/share/yum-cli/cli.py", line 339, in doCommands
self._getTs(needTsRemove)
File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 101, in _getTs
self._getTsInfo(remove_only)
File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 112, in _getTsInfo
pkgSack = self.pkgSack
File "/usr/lib/python2.4/site-packages/yum/init.py", line 591, in
pkgSack = property(fget=lambda self: self._getSacks(),
File "/usr/lib/python2.4/site-packages/yum/init.py", line 434, in _getSacks
self.repos.populateSack(which=repos)
File "/usr/lib/python2.4/site-packages/yum/repos.py", line 223, in populateSack
self.doSetup()
File "/usr/lib/python2.4/site-packages/yum/repos.py", line 71, in doSetup
self.ayum.plugins.run('postreposetup')
File "/usr/lib/python2.4/site-packages/yum/plugins.py", line 176, in run
func(conduitcls(self, self.base, conf, **kwargs))
File "/usr/lib/yum-plugins/fastestmirror.py", line 181, in postreposetup_hook
all_urls = FastestMirror(all_urls).get_mirrorlist()
File "/usr/lib/yum-plugins/fastestmirror.py", line 333, in get_mirrorlist
self._poll_mirrors()
File "/usr/lib/yum-plugins/fastestmirror.py", line 376, in _poll_mirrors
pollThread.start()
File "/usr/lib/python2.4/threading.py", line 416, in start
_start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread
The only other question I could find with a similar error was this one: https://stackoverflow.com/search?q=can%27t+start+new+thread+python, but I'm not sure it has to do with running too many threads since I am the only one using the server, and this is one of the first python commands I have used. Could someone point me in the right direction, or towards some material that may help me troubleshoot the issue? Thanks!
Disable yum-fastermirror and you should be able to complete the install.
yum --disableplugin=fastestmirror update
Are you on a virtual machine?