We Keep Coding

`Error: MLIR Pass Manager Failed': TensorFlow on MacBook Pro M1 Max

I'm running Python 3.9.7 and TensorFlow 2.5 on my MacBook Pro with M1 Max chip, in a virtual environment managed through MiniForge. (Setup due to Jeff Heaton).
I have a custom loss function in the model. This code runs fine on my other machine, which is Ubuntu-based Pop!OS.
When I run model.fit from Keras, I occasionally get the following errors. Sometimes it runs just fine. Any idea what's causing it, why it only happens sometimes, and how to fix it?
In addition, it only produces this error from my custom loss function. Not from any Keras-native.
Thanks!
:5:10: error: invalid length (0) for size 20 | 0.00/1.00 [00:00<?, ?batch/s]
-:5:10: error: 'mps.slice' op failed to fold or infer return types
-:5:10: note: see current operation: %2 = "mps.slice"(%arg0) {axis = 0 : si32, length = 999 : i32, start = 1 : i32} : (tensor<20xf32>) -> tensor<*xf32>
/System/Volumes/Data/SWE/macOS/BuildRoots/5b2e67f8af/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/MetalPerformanceShadersGraph-2.2.1/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:1179: failed assertion `Error: MLIR pass manager failed'
model.fit(tf.convert_to_tensor(Σ),tf.zeros((T,output)),batch_size=T,epochs=1,verbose=0,callbacks=[TqdmCallback()])#,tbc])
zsh: abort python3
(tensorflow) kpmott#kevins-mbp-7 tf.olg % model.fit(tf.convert_to_tensor(Σ),tf.zeros((T,output)),batch_size=T,epochs=1,verbose=0,callbacks=[TqdmCallback()])#,tbc])
zsh: parse error near `)'
(tensorflow) kpmott#kevins-mbp-7 tf.olg % /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
model.fit(tf.convert_to_tensor(Σ),tf.zeros((T,output)),batch_size=T,epochs=1,verbose=0,callbacks=[TqdmCallback()])#,tbc])
zsh: parse error near `)'

MultiLoc2 (sub-cellular localization )error

i am trying to use MultiLoc2 and facing this problem. i am following step by step manual and in the final command i got this error.
raise Error("Configuration Error: A path is not set. Run configure script!")
util.Error: 'Configuration Error: A path is not set. Run configure script!'

Failed to create cuSolverDN instance

tensorflow version:1.12, cuda version:10.0
When I build my code, there is a error message:
2019-03-23 18:53:29.965772: I tensorflow/core/kernels/cuda_solvers.cc:159] Creating CudaSolver handles for stream 0x55c9d5a16320
2019-03-23 18:53:30.063068: F tensorflow/core/kernels/cuda_solvers.cc:94] Check failed: cusolverDnCreate(&cusolver_dn_handle) == CUSOLVER_STATUS_SUCCESS Failed to create cuSolverDN instance.
This problem occurred after I wrote code about distribution(from tensorflow-probability). Of course if I remove the code and compile again, there is nothing happened.
Is there any way to solve this problem?

U-SQL, Python, local execution, "device not found" error

I'm trying to run a U-SQL job with Python extension locally using VS 2017.
I followed these steps:
https://1drv.ms/w/s!AvdZLquGMt47g0NultCKgm38sejs
https://blogs.msdn.microsoft.com/azuredatalake/2017/02/20/enabling-u-sql-advanced-analytics-for-local-execution/
And then I tried to run this:
https://learn.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-python-extensions
It works fine if I run it in Azure, but if I try to run it locally, the error I get is: "The device is not read".
Details:
Start : 2017-08-16 14:35:13
Initialize : 2017-08-16 14:35:13
GraphParse : 2017-08-16 14:35:13
Run : 2017-08-16 14:35:13
Start 'Root' : 2017-08-16 14:35:13
End 'Root(Success)' : 2017-08-16 14:35:13
Start '1_SV1_Extract' : 2017-08-16 14:35:13
End '1_SV1_Extract(Error)' : 2017-08-16 14:35:14
Completed with 'Error' : 2017-08-16 14:35:14
Execution failed with error '1_SV1_Extract Error : '{"diagnosticCode":195887147,"severity":"Error","component":"RUNTIME","source":"User","errorId":"E_RUNTIME_USER_UNHANDLED_EXCEPTION_FROM_USER_CODE","message":"An unhandled exception from user code has been reported","description":"Unhandled exception from user code: \"The device is not ready.\r\n\"\nThe details includes more information including any inner exceptions and the stack trace where the exception was raised.","resolution":"Make sure the bug in the user code is fixed.","helpLink":"","details":"==== Caught exception System.IO.IOException\n\n at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)\r\n at System.IO.Directory.InternalCreateDirectory(String fullPath, String path, Object dirSecurityObj, Boolean checkHost)\r\n at System.IO.Directory.InternalCreateDirectoryHelper(String path, Boolean checkHost)\r\n at System.IO.Compression.ZipFileExtensions.ExtractToDirectory(ZipArchive source, String destinationDirectoryName)\r\n at System.IO.Compression.ZipFile.ExtractToDirectory(String sourceArchiveFileName, String destinationDirectoryName, Encoding entryNameEncoding)\r\n at Microsoft.MetaAnalytics.LanguageWorker.UsqlPyExecution.LocatePython(String version) in C:\\Users\\shravan\\Source\\Repos\\VSTS\\USqlExtensions\\lang\\python\\AFx\\Product\\Source\\Modules\\LanguageWorker\\LanguageWorker.Dll\\UsqlExecution.cs:line 146\r\n at Microsoft.MetaAnalytics.LanguageWorker.UsqlPyExecution.InvokeLanguage(String version, String scriptname, IList`1 infiles, IList`1 outfiles, IObserver`1 stringLogger) in C:\\Users\\shravan\\Source\\Repos\\VSTS\\USqlExtensions\\lang\\python\\AFx\\Product\\Source\\Modules\\LanguageWorker\\LanguageWorker.Dll\\UsqlExecution.cs:line 89\r\n at Microsoft.MetaAnalytics.LanguageWorker.UsqlPyExecution.Run(IRowset input, IUpdatableRow output, String script, String version) in C:\\Users\\shravan\\Source\\Repos\\VSTS\\USqlExtensions\\lang\\python\\AFx\\Product\\Source\\Modules\\LanguageWorker\\LanguageWorker.Dll\\UsqlExecution.cs:line 42\r\n at Extension.Python.Reducer.<Reduce>d__6.MoveNext() in C:\\Users\\shravan\\Source\\Repos\\VSTS\\USqlExtensions\\lang\\python\\ExtPy\\PyReducer.cs:line 56\r\n at ScopeEngine.SqlIpReducer<Extract_0_Data0,Process_1_Data0,ScopeEngine::KeyComparePolicy<Extract_0_Data0,3> >.GetNextRow(SqlIpReducer<Extract_0_Data0\\,Process_1_Data0\\,ScopeEngine::KeyComparePolicy<Extract_0_Data0\\,3> >* , Process_1_Data0* output) in c:\\users\\e\\source\\repos\\usqlapplication1\\usqlapplication1\\bin\\debug\\1b720f51a8b3caea\\script_fe316531c87f021f\\sqlmanaged.h:line 2788\r\n at std._Func_class<void>.()(_Func_class<void>* )\r\n at RunAndHandleClrExceptions(function<void __cdecl(void)>* code)","internalDiagnostics":""}
'
'
Execution failed !
I'm aware that the blog post mentions that running Python extensions locally is not officially supported, but they do make it sound like it should at least be possible somehow?
I don't get any errors if I run U-SQL scripts without using the Python extension locally.
Is there anything I'm missing? Is there any logging I could turn on to find out more? Has anyone had success running Python with U-SQL locally?

(Azure Data Lake team here)
There was a recent update in how the Python distribution is located in the Azure Data Lake Analytics service. While the change improved vertex startup times, it also broke some basic assumptions on how local execution of U-SQL scripts works.
The team is working on an alternative solution that will let a locally executing U-SQL script use an existing Python distribution that is installed on the same local machine.

SonarQube Python Plugin - scanning python code: Fail to decorate

I am trying to run a scan in the sample project (I am actually trying to scan a much larger project, but the problem is the same. I am working on the sample project because is much simpler) and it is giving me the following error:
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
Total time: 29.790s
Final Memory: 14M/379M
INFO: ------------------------------------------------------------------------
ERROR: Error during Sonar runner execution
ERROR: Unable to execute Sonar
ERROR: Caused by: Fail to decorate 'org.sonar.api.resources.File#56ffc188[key=src/__init__.py,deprecatedKey=__init__.py,path=src/__init__.py,dir=[root],filename=__init__.py,language=Python]'
ERROR: Caused by: 0
ERROR:
ERROR: To see the full stack trace of the errors, re-run SonarQube Runner with the -e switch.
ERROR: Re-run SonarQube Runner using the -X switch to enable full debug logging.
Anyone has any idea of what this means? This is the sample project right out of the box, and my sonnar runner config is firly simple:
#----- Default SonarQube server
sonar.host.url=http://localhost:9000
#----- MySQL
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8
#----- Global database settings
sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
#----- Default source code encoding
sonar.sourceEncoding=UTF-8
#----- Security (when 'sonar.forceAuthentication' is set to 'true')
sonar.login=admin
sonar.password=admin
I have enabled all pylint rules, within a new profile just with those rules. With the default profile as default (Sonar Way) the error does not happen. Curious thing is that I enabled a random rule from pylint and it worked. So maybe one (or more) rule is messing with the analysis.
Pylint integration is essential for me.
The stacktrace, although not very helpful is the following:
ERROR: Error during Sonar runner execution
org.sonar.runner.impl.RunnerException: Unable to execute Sonar
at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:91)
at org.sonar.runner.impl.BatchLauncher$1.run(BatchLauncher.java:75)
at java.security.AccessController.doPrivileged(Native Method)
at org.sonar.runner.impl.BatchLauncher.doExecute(BatchLauncher.java:69)
at org.sonar.runner.impl.BatchLauncher.execute(BatchLauncher.java:50)
at org.sonar.runner.api.EmbeddedRunner.doExecute(EmbeddedRunner.java:102)
at org.sonar.runner.api.Runner.execute(Runner.java:100)
at org.sonar.runner.Main.executeTask(Main.java:70)
at org.sonar.runner.Main.execute(Main.java:59)
at org.sonar.runner.Main.main(Main.java:53)
Caused by: org.sonar.api.utils.SonarException: Fail to decorate 'org.sonar.api.resources.File#3f3674b2[key=src/__init__.py,deprecatedKey=__init__.py,path=src/__init__.py,dir=[root],filename=__init__.py,language=Python]'
at org.sonar.batch.phases.DecoratorsExecutor.executeDecorator(DecoratorsExecutor.java:103)
at org.sonar.batch.phases.DecoratorsExecutor.decorateResource(DecoratorsExecutor.java:86)
at org.sonar.batch.phases.DecoratorsExecutor.decorateResource(DecoratorsExecutor.java:78)
at org.sonar.batch.phases.DecoratorsExecutor.decorateResource(DecoratorsExecutor.java:78)
at org.sonar.batch.phases.DecoratorsExecutor.execute(DecoratorsExecutor.java:70)
at org.sonar.batch.phases.PhaseExecutor.execute(PhaseExecutor.java:126)
at org.sonar.batch.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:222)
at org.sonar.api.platform.ComponentContainer.startComponents(ComponentContainer.java:93)
at org.sonar.api.platform.ComponentContainer.execute(ComponentContainer.java:78)
at org.sonar.batch.scan.ProjectScanContainer.scan(ProjectScanContainer.java:235)
at org.sonar.batch.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:230)
at org.sonar.batch.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:223)
at org.sonar.api.platform.ComponentContainer.startComponents(ComponentContainer.java:93)
at org.sonar.api.platform.ComponentContainer.execute(ComponentContainer.java:78)
at org.sonar.batch.scan.ScanTask.scan(ScanTask.java:65)
at org.sonar.batch.scan.ScanTask.execute(ScanTask.java:52)
at org.sonar.batch.bootstrap.TaskContainer.doAfterStart(TaskContainer.java:128)
at org.sonar.api.platform.ComponentContainer.startComponents(ComponentContainer.java:93)
at org.sonar.api.platform.ComponentContainer.execute(ComponentContainer.java:78)
at org.sonar.batch.bootstrap.BootstrapContainer.executeTask(BootstrapContainer.java:171)
at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:95)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:67)
at org.sonar.runner.batch.IsolatedLauncher.execute(IsolatedLauncher.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:87)
... 9 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
at org.sonar.plugins.core.issue.tracking.FileHashes.getHash(FileHashes.java:75)
at org.sonar.plugins.core.issue.IssueTracking.setChecksumOnNewIssues(IssueTracking.java:69)
at org.sonar.plugins.core.issue.IssueTracking.track(IssueTracking.java:54)
at org.sonar.plugins.core.issue.IssueTrackingDecorator.doDecorate(IssueTrackingDecorator.java:138)
at org.sonar.plugins.core.issue.IssueTrackingDecorator.decorate(IssueTrackingDecorator.java:112)
at org.sonar.batch.phases.DecoratorsExecutor.executeDecorator(DecoratorsExecutor.java:95)
... 36 more
Thanks for any help!

I encountered the same problem. I had to deactivate two rules to stop this error message:
Missing docstring
docstring should be defined
I also deactivate the deprecated rules but I don't know if it had any effect.