Change in running behavior of sklearn code between laptop and desktop - python

I am trying to debug a Fortran warning in some Sklearn code that runs perfectly on my laptop...but after transferring to my desktop (which is a fresh Ubuntu 15.10, fresh Pycharm, and fresh Anaconda3), I get the following error when running sklearn.cross_validation.cross_val_score:
/anaconda3/lib/python3.5/site-packages/sklearn/externals/joblib
/hashing.py:197: DeprecationWarning: Changing the shape of non-C contiguous
array by
descriptor assignment is deprecated. To maintain
the Fortran contiguity of a multidimensional Fortran
array, use 'a.T.view(...).T' instead
obj_bytes_view = obj.view(self.np.uint8)
The command I am submitting to cross_val_score is:
test_results = cross_val_score(learner(**learner_args),data,y=classes,n_jobs=n_jobs,scoring='accuracy',cv=LeaveOneOut(data.shape[0]))
Where the iterator is the sklearn cross validation object...and nothing else special is going on. What could be happening here? Am I missing some installation step?

Just for the record for people like me who found this SO post via Google, this is has been recorded as issue #6370 for scikit-learn.
As mentioned there:
This problem has been fixed in joblib master. It won't be fixed in scikit-learn until:
1) we do a new joblib release
2) we update scikit-learn master to have the new joblib release
3) if you are using released versions of scikit-learn, which I am guessing you are, you will have to wait until there is a new scikit-learn release
I was able to use the above workaround from #bordeo :
import warnings
warnings.filterwarnings("ignore")

Related

How to import and use metrics in python in the new versions of opentelemetry packages

I'm just starting in telemetry and I got stuck trying to use metrics in the new versions of opentelemetry-api and opentelemetry-sdk.
What I have found
1 - Documentation
This is a old getting started (do not try those scripts, not up to date)
https://open-telemetry.github.io/opentelemetry-python/getting-started.html
And this is the latest getting started
https://opentelemetry-python.readthedocs.io/en/latest/sdk/sdk.html
As you see, in the latest there is no information about metrics just tracing.
2 - The packages
As you see in this image in the version 1.10a0 of the opentelemetry there is the metrics module, while in the current version 1.4 there is no module metrics (see image).
The problem
To use metrics one must run pip install opentelemetry-instumentation-system-metrics by doing this the pip uninstall opentelemetry api and sdk and reinstall old version. (see image). When it happens I am able to import the metrics module, but the tracing does not work anymore.
Question
Where is the metrics module in the new version of opentelemetry?
How can I instrument metrics in the latest version of opentelemetry?
You can't as of now. There is ongoing prototype https://github.com/open-telemetry/opentelemetry-python/pull/1887 for metrics API&SDK based on specification which itself is not stable yet. There is no guaranteed timeline when metrics will be release for end user instrumentation. You may safely assume that it will take few more months to get stable release but there should be alpha-beta release much earlier.

The sklearn.* module is deprecated in version 0.22 and will be removed in version 0.24

I am migrating a piece of software from Python 2.7 to Python 3.
One problem that arises is:
The sklearn.neighbors.kde module is deprecated in version 0.22 and
will be removed in version 0.24. The corresponding classes / functions
should instead be imported from sklearn.neighbors. Anything that
cannot be imported from sklearn.neighbors is now part of the private
API.
I am not sure which line causes this, and not sure if it is an error or a warning, and what are the implications.
On python 2.7 everything works fine.
How do I get rid of this?
It will work until you update your scikit/sklearn version. Then the Kernel Density package will not run any more. You still have time to search for similar modules if you want to update your version.
But you can also set up different environments with different versions. Thus, if you need this module just start an environment and don't upgrade your sklearn version in this environment.
It's just a warning, for now -- until you upgrade sklearn to version 0.24. Then your code will need to be modified before it works. It's giving you a heads-up about this, so you can fix your code ahead of time. The modifications described below should work with your current version; you don't need to wait to upgrade before changing your code (at least, that's how these deprecation warnings usually work).
The corresponding classes / functions should instead be imported from sklearn.neighbors.
If I read this message correctly, it's saying that if you're using a function like sklearn.neighbours.kde.some_function() in your code now, you need to change it to sklearn.neighbours.some_function().
Anything that cannot be imported from sklearn.neighbors is now part of the private API.
This seems to be saying that there may be some functions that will no longer be available to you, even using the modification above.

Error in prediction using sknn.mlp

I use Anaconda on a Windows 10 laptop with Python 2.7 and Spark 2.1. Built a deep learning model using Sknn.mlp package. I have completed the model. When I try to predict using the predict function, it throws an error. I run the same code on my Mac and it works just fine. Wondering what is wrong with my windows packages.
'NoneType' object is not callable
I verified input data. It is numpy.array and it does not have null value. Its dimension is same as training one and all attributed are the same. Not sure what it can be.
I don't work with Python on Windows, so this answer will be very vague, but maybe it will guide you in the right direction. Sometimes there are cross-platform errors due to one module still not being updated for the OS, frequently when another related module gets an update. I recall something happened to me with a django application which required somebody more familiar with Windows to fix it for me.
Maybe you could try with an environment using older versions of your modules until you find the culprit.
I finally solved the problem on windows. Here is the solution in case you face it.
The Theano package was faulty. I installed the latest version from github and then it threw another error as below:
RuntimeError: To use MKL 2018 with Theano you MUST set "MKL_THREADING_LAYER=GNU" in your environment.
In order to solve this, I created a variable named MKL_Threading_Layer under user environment variable and passed GNU. Reset the kernel and it was working.
Hope it helps!

Exit code 139 Python [duplicate]

I'm trying to execute a Python script, but I am getting the following error:
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
I'm using python 3.5.2 on a Linux Mint 18.1 Serena OS
Can someone tell me why this happens, and how can I solve?
The SIGSEGV signal indicates a "segmentation violation" or a "segfault". More or less, this equates to a read or write of a memory address that's not mapped in the process.
This indicates a bug in your program. In a Python program, this is either a bug in the interpreter or in an extension module being used (and the latter is the most common cause).
To fix the problem, you have several options. One option is to produce a minimal, self-contained, complete example which replicates the problem and then submit it as a bug report to the maintainers of the extension module it uses.
Another option is to try to track down the cause yourself. gdb is a valuable tool in such an endeavor, as is a debug build of Python and all of the extension modules in use.
After you have gdb installed, you can use it to run your Python program:
gdb --args python <more args if you want>
And then use gdb commands to track down the problem. If you use run then your program will run until it would have crashed and you will have a chance to inspect the state using other gdb commands.
Another possible cause (which I encountered today) is that you're trying to read/write a file which is open. In this case, simply closing the file and rerunning the script solved the issue.
After some times I discovered that I was running a new TensorFlow version that gives error on older computers. I solved the problem downgrading the TensorFlow version to 1.4
When I encounter this problem, I realize there are some memory issues. I rebooted PC and solved it.
This can also be the case if your C-program (e.g. using cpython is trying to access a variable out-of-bound
ctypedef struct ReturnRows:
double[10] your_value
cdef ReturnRows s_ReturnRows # Allocate memory for the struct
s_ReturnRows.your_value = [0] * 12
will fail with
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
For me, I was using the OpenCV library to apply SIFT.
In my code, I replaced cv2.SIFT() to cv2.SIFT_create() and the problem is gone.
Deleted the python interpreter and the 'venv' folder solve my error.
I got this error in PHP, while running PHPUnit. The reason was a circular dependency.
I received the same error when trying to connect to an Oracle DB using the pyodbc module:
connection = pyodbc.connect()
The error occurred on the following occasions:
The DB connection has been opened multiple times in the same python
file
While in debug mode a breakpoint has been reached
while the connection to the DB being open
The error message could be avoided with the following approaches:
Open the DB only once and reuse the connection at all needed places
Properly close the DB connection after using it
Hope, that will help anyone!
11 : SIGSEGV - This signal is arises when an memory segement is illegally accessed.
There is a module name signal in python through which you can handle this kind of OS signals.
If you want to ignore this SIGSEGV signal, you can do this:
signal.signal(signal.SIGSEGV, signal.SIG_IGN)
However, ignoring the signal can cause some inappropriate behaviours to your code, so it is better to handle the SIGSEGV signal with your defined handler like this:
def SIGSEGV_signal_arises(signalNum, stack):
print(f"{signalNum} : SIGSEGV arises")
# Your code
signal.signal(signal.SIGSEGV, SIGSEGV_signal_arises)
I encountered this problem when I was trying to run my code on an external GPU which was disconnected. I set os.environ['PYOPENCL_CTX']=2 where GPU 2 was not connected. So I just needed to change the code to os.environ['PYOPENCL_CTX'] = 1.
For me these three lines of code already reproduced the error, no matter how much free memory was available:
import numpy as np
from sklearn.cluster import KMeans
X = np.array([[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]])
kmeans = KMeans(n_clusters=1, random_state=0).fit(X)
I could solve the issue by removing an reinstalling the scikit-learn package. A very similar solution to this.
This can also occur if trying to compound threads using concurrent.futures. For example, calling .map inside another .map call.
This can be solved by removing one of the .map calls.
I had the same issue working with kmeans from scikit-learn.
Upgrading from scikit-learn 1.0 to 1.0.2 solved it for me.
This issue is often caused by incompatible libraries in your environment. In my case, it was the pyspark library.
In my case, reverting my most recent conda installs fixed the situation.
I got this error when importing monai. It was solved after I created a new conda environment. Possible reasons I could imagine were either that there were some conflict between different packages, or maybe that my environment name was the same as the package name I wanted to import (monai).
found on other page.
interpreter: python 3.8
cv2.CascadeClassifier(cv2.data.haarcascades + "haarcascade_frontalface_default.xml")
this solved issue for me.
i was getting SIGSEGV with 2.7, upgraded my python to 3.8 then got different error with OpenCV. and found answer on OpenCV 4.0.0 SystemError: <class 'cv2.CascadeClassifier'> returned a result with an error set.
but eventually one line of code fixed it.

theano NotImplementedError

I am running some theano code making use of tensor.advanced_subtensor
I am getting the following error :
NotImplementedError: Could not import inplace_increment, so some advanced indexing features are disabled. They will be available if you update NumPy to version 1.8 or later, or to the latest development version. You may need to clear the cache (theano-cache clear) afterwards.
I have the latest version of theano (0.6.0.dev-60b5ccc2bcabb1010714376764daf8a50722cee9) and numpy (1.8.0). Why am I still getting this error? How can I resolve this error? How do I clear theano cache?
The theano cache is usually in ~/.theano/ if you are using *ix.
You need to clear Theano cache. Cache live in ~/.theano/ folder.
Follow below steps to clear it manually.
import theano
print (theano.config.compiledir)
# and then delete directory returned from above.
If you don not want to delete manually then use below command.
theano-cache purge

Categories