Path confusion using Conda, PIP, and Jupyter notebook at the same time - python

I've been using conda and jupyter notebook for a while now and recently I need some packages that are only available for install via pip. My understanding was that they are completely compatible and transparent with each other but apparently not.
I did a pip install pydicom and when I tried to do an import pydicom in my Jupyter Notebook, it says No module named pydicom.
In command prompt (using a Windows machine here) I ran conda list and I see pydicom is listed and marked as < pip >.
In my Jupyter notebook, this is my sys.executable output:
C:\Users\USERNAME\AppData\Local\Continuum\Anaconda2\python.exe
Is that normal/correct???
Disclaimer: I'm thoroughly confused by how the paths, environment, and dependencies work in conda and pip. I can't find a good tutorial or explanation how conda/pip work behind the scenes. Any pointers will be greatly appreciated!

While the package is named pydicom, you need to import dicom:
import dicom
You can also in install pydicom via conda. You need to provide the channel 'conda-forge`:
conda install -c conda-forge pydicom
You can find more packages a https://anaconda.org/. Just enter the package name in the search window. If the package is available you will see the channel you can with the -c option.

Related

Jupyter unable to use Vartests that’s found in my site-packages

I have Anaconda installed in my system. I have used Jupyter notebook to install vartests package using “!pip install vartests”. In my site-packages folder inside Anaconda/lib, I am able to find a directory for vartests, which means the package has been installed. But when I try to import it in the same jupyter notebook, it says import error.
I’ve tried using “!{sys.executable} -m pip install vartests” but it doesn’t help. I’ve followed things written in “Installing Python Packages from Jupyter Notebook” by Jake VanderPlas as well.
One thing, my “!type -a python” doesn’t give me anything. Is there a way out from this?

How to install NetCDF4 module in Spyder?

I don't know why this is causing me so much headache.
I know how to use pip, and have the latest version, but still when running a script in Spyder that requires netCDF4 (import netCDF4) Spyder always returns:
import netCDF4
ModuleNotFoundError: No module named 'netCDF4'"
I opened cmd, pip install netCDF4, confirmed it installed OK. Shouldn't this be enough?
I manually copied a downloaded version of netCDF4, moved it to my Python site packages, and then in Spyder manually went into PYTHONPath Manager -> Add Path --> Added the folder location 'netCDF4-1.6.0'. I thought this would definitely be enough?
Then INSIDE Spyder's python console, bot-right of screen, I tried 'pip install netCDF4' and it returns:
Note: you may need to restart the kernel to use updated packages.
C:\...\Spyder\Python\python.exe: No module named pip
So is my issue #3? Spyder's version of Python doesn't have pip and doesn't link to the netCDF4 module? I thought my step #2 would resolve this?
Any help on how to install modules in Spyder would be appreciated, thank you!
Download Miniconda and install all the packages you want in a new environment. For example,
conda create -n myenv -c conda-forge python=3.9 netcdf4 [others ...]
Then follow the Spyder documentation for adding a conda environment.
Credit to Michael Delgado (above)

Installing Rasterio

I tried installing Rasterio library using command- pip install rasterio and also used conda commands in anaconda cmd, both are didn't worked and it's taking too long to install so I aborted the process. Let me know in detail how to install raserio library for python?
I tried with installing GDAL and raterio whl file by using pip commands. It got installed, but when I tried to import module- it's giving me an error saying No module found. Please help to fix this.
Thanks in advance
I just had the similar problem, and tl:dr the issue for me was multiple environments, like Ameya said. Here's a longer version of Ameya's spot-on answer with steps to diagnose and fix.
Without realizing it, I had two environments going on: my jupyter notebook was running python 3.10 and my global python was running 3.9, and each was looking for site-packages in different locations (/opt/homebrew/Cellar/jupyterlab/3.4.7/libexec/lib/python3.10/site-packages vs /Users//Library/Python/3.9/lib/site-packages).
This happened because I had trouble with getting python and specifically jupyterlab running on Monterey and the only fix for me was using homebrew to manage the packages. Anything I installed with brew from the command line, went into /opt/homebrew/Cellar... etc and could be seen by my jupyter notebook. Anything I used pip install to get from within an open notebook also went onto this path that my notebook could see. But anything I used pip install from the command line to get, went to the path of the global environment's site packages. Then, my jupyter notebook couldn't see them.
You don't mention that you are using jupyter notebook but perhaps something analogous could happen between multiple environments for you.
You can check if this is the case for you by doing the following:
start python from the command line
import sys
run sys.path
start jupyter notebook, or start python from your other environment
same thing, import sys, run sys.path
Are they the same? If not, probably pip is putting your rasterio in other python env site-packages.
To fix, you can either pip install from within your preferred environment, or copy and paste the site packages corresponding to rasterio from one site-packages location to the other.

ModuleNotFoundError: No module named 'sklearn'

I want to import sklearn but there is no module apparently:
ModuleNotFoundError: No module named 'sklearn'
I am using Anaconda and Python 3.6.1; I have checked everywhere but still can't find answers.
When I use the command:
conda install scikit-learn should this not just work?
Where does anaconda install the package?
I was checking the frameworks in my python library and there was nothing about sklearn only numpy and scipy.
Please help, I am new to using python packages especially via anaconda.
You can just use pip for installing packages, even when you are using anaconda:
pip install -U scikit-learn scipy matplotlib
This should work for installing the package.
And for Python 3.x just use pip3:
pip3 install -U scikit-learn scipy matplotlib
Will leave below two options that may help one solve the problem:
Using conda
Using pip
One might want to consider the notes at the end, specially before resorting to the 2nd option.
Option 1
If one wants to install it in the root and one follows the requirements - (Python (>= 2.7 or >= 3.4), NumPy (>= 1.8.2), SciPy (>= 0.13.3).) - the following should solve the problem
conda install scikit-learn
Alternatively, as mentioned here, one can specify the channel as follows
conda install -c anaconda scikit-learn
Let's say that one is working in the environment with the name ML.
Then the following should solve one's problem:
conda install -n ML scikit-learn
# or
conda install -n ML -c anaconda scikit-learn
Option 2
If the above doesn't work, on Anaconda Prompt one can also use pip (here's how to pip install scikit-learn), so the following may help
pip install scikit-learn
However, consider the last note below before proceeding.
Notes:
When using Anaconda, one needs to be aware of the environment that one is working.
Then, in Anaconda Prompt, one needs to run the following
conda $command -n $ENVIRONMENT_NAME $IDE/package/module
$command - Command that one intends to use (consult documentation for general commands)
$ENVIRONMENT NAME - The name of one's environment (if one is working in the root,
conda $command $IDE/package/module is enough)
$IDE/package/module - The name of the IDE or package or module
If one needs to install/update packages, the logic is the same as mentioned in the introduction. If you need more information on Anaconda Packages, check the documentation.
What is the flag -c.
pip doesn't manage dependencies the same way conda does and can, potentially, damage one's installation.
If you are using Ubuntu 18.04 or higher with python3.xxx then try this command
$ sudo apt install python3-sklearn
then try your command. hope it will work
I did the following:
import sys
!{sys.executable} -m pip install sklearn
I've tried a lot of things but finally, including uninstall with the automated tools. So, I've uninstalled manually scikit-learn.
sudo rm -R /home/ubuntu/.local/lib/python3.6/site-packages/sklearn
sudo rm -R /home/ubuntu/.local/lib/python3.6/site-packages/scikit_learn-0.20.0-py3.6.egg-info
And re-install using pip
sudo pip3.6 install -U scikit-learn
Hope that can help someone else!
This happened to me, I tried all the possible solutions with no luck!
Finaly I realized that the problem was with Jupyter notebook environment, not with sklearn!
I solved the problem by re-installing Jupyter at the same environment as sklearn
the command is: conda install -c anaconda ipython. Done...
The other name of sklearn in anaconda is scikit-learn. simply open your anaconda navigator, go to the environments, select your environment, for example tensorflow or whatever you want to work with, search for scikit_learn in the list of uninstalled packages, apply it and then you can import sklearn in your jupyter.
SOLVED:
The above did not help. Then I simply installed sklearn from within Jypyter-lab, even though sklearn 0.0 shows in 'pip list':
!pip install sklearn
import sklearn
What I learned later is that pip installs, in my case, packages in a different folder than Jupyter. This can be seen by executing:
import sys
print(sys.path)
Once from within Jupyter_lab notebook, and once from the command line using 'py notebook.py'.
In my case Jupyter list of paths where subfolders of 'anaconda' whereas Python list where subfolders of c:\users[username]...
On Windows, I had python 3+ version. pip version - 22.3.1
I had installed:
pip install sklearn
But, it seems it is deprecated with scikit-learn.
So, I did:
pip install scikit-learn
And, it worked!!!
Cause
Conda and pip install scikit-learn under ~/anaconda3/envs/$ENV/lib/python3.7/site-packages, however Jupyter notebook looks for the package under ~/anaconda3/lib/python3.7/site-packages.
Therefore, even when the environment is specified to conda, it does not work.
conda install -n $ENV scikit-learn # Does not work
Solution
pip 3 install the package under ~/anaconda3/lib/python3.7/site-packages.
Verify
After pip3, in a Jupyter notebook.
import sklearn
sklearn.__file__
~/anaconda3/lib/python3.7/site-packages/sklearn/init.py'
I had the same problem.
The issue is when we work on multiple anaconda environments, not all packages are installed in all environments.
you can check your conda environment by writing the following code in anaconda prompt:
conda env list
then you can check the packages installed in each environment :
conda list -n NAME_OF_THE_ENVIRONMENT
for me, the environment that I was working with , was missing sklearn, although the package was installed in the other environments.
therefore, I just simply installed sklearn package in that particular environment
conda install -n NAME_OF_THE_ENVIRONMENT scikit-learn
and the issue was resolved
install these ==>> pip install -U scikit-learn scipy matplotlib
if still getting the same error then ,
make sure that your imoprted statment should be correct. i made the mistike while writing ensemble so ,(check spelling)
its
should be >>> from sklearn.ensemble import RandomForestClassifier
I had the same issue as the author, and ran into the issue with and without Anaconda and regardless of Python version. Everyone's environment is different, but after resolving it for myself I think that in some cases it may be due to having multiple version of Python installed. Each installed Python version has its own \Lib\site-packages\ folder which can contain a unique set of modules for that Python version, and where the IDE looks into folder path that doesn't have scikit-learn in it.
One way to try solve the issue: you might clear your system of all other Python versions and their cached/temp files/system variables, and then only have one version of Python installed anywhere. Then install the dependencies Numpy and Scipy, and finally Scikit-learn.
More detailed steps:
Uninstall all Python versions and their launchers (e.g. from Control Panel in Windows) except the one version you want to keep. Delete any old Python version folders in the Python directory --uninstalling doesn't remove all files.
Remove other Python versions from your OS' Environment Variables (both under the system and user variables sections)
Clear temporary files. For example, for Windows, delete all AppData Temp cache files (in C:\Users\YourUserName\AppData\Local\Temp). In addition, you could also do a Windows disk cleanup for other temporary files, and then reboot.
If your IDE supports it, create a new virtual environment in Settings, then set your only installed Python version as the interpreter.
In your IDE, install the dependencies Scipy and Numpy from the module list first, then install Scikit-Learn.
As some others have suggested, the key is making sure your environment is set up correctly where everything points to the correct library folder on your computer where the Sklearn package is located. There are a few ways this can be resolved. My approach was more drastic, but it turns out that I had a very messy Python setup on my system so I had to start fresh.
Using Anaconda-navigator UI environment
When running Anaconda-navigator:
Choose the 'Environments' tab on the left and create a new environment (e.g. ML - see Gonçalo Peres answer above, I made one called 'CourseraML').
Set Python version 3.7 (for Coursera course Applied Machine Learning in Python). Also include R.
Then find modules to install using the 'not installed' drop-down menu item. Search for each module needed in the search bar and select. sklearn is part of scikit-learn. Select it and install (it should find all relevant dependencies). Modules needed for Applied ML course: seaborn, numpy, scikit-learn, pandas, matplotlib
You'll need to restart Jupyter Notebook and reopen your file.
Command line version of above:
conda install -n CourseraML seaborn scikit-learn pandas numpy matplotlib graphviz
Causes
-your jupyter notebook might be importing the sklearn and other libraries from the
another the location(path) than the libraries from conda or pip.
MY Problem
In my case, My jupyter notebook was importing the libraries for snap manager. Since, I install jupyter using the snap instead of other ways.
You can check where other libraries are being imported in jupyter using code:
import cv2 as cv
print(cv.__file__)
Solution
So , I uninstall jupyter notebook and then install notebook using conda.
sudo snap remove jupyter
conda install -c conda-forge notebook

Keras in `pip list` and anaconda package list but cannot import

I am using anaconda 4.4.0 with Python 3.6.2. First, I pip installed Keras, it shows on pip list and conda list and also in environment>root>packages in Anaconda. But when I use import keras ,in Anaconda or in Terminal, I get
Traceback (most recent call last):
...
ModuleNotFoundError: No module named 'keras'
I tried installing Keras with anaconda also, and I get the same ModuleNotFoundError.
I have also tried uninstalling it, then download the master branch from github and install using python setup.py install. Again the package is in the lists but I cannot import it.
UPDATE:
I have uninstalled packages several times and installed using Keras package in pypi, Now I am able to import the packages in the python in terminal. But still I cannot import in jupyter notebook.
It may worth mentioning that I am working with MacOS Sierra 10.
If you are able to import in shell but not in the notebook, according to gnestor's answer to this question, your python may use different executables. Run:
import sys
sys.executable
in shell and in notebook. If the directories are different then you must change the kernelspec of the notebook. This might have happened by installing Tensorflow or other packages that change the environment and kernelspec. You can find the kernelspec directory with this command:
from jupyter_core.paths import jupyter_data_dir
print(jupyter_data_dir())
In that directory you will find a JSON file. Open it and change the path to your working python directory. (working python directory in which python)
use python -m pip install keras
Then, use python shell to check for installation.
try this code:
conda install -c deeplearn keras
Basically, the full answer you can find here:
Install Python package: "Package missing in current win-64 channels"
However, it seems to me you are using different python in your terminal.
Check this command in the terminal:
which python
It should return something like this:
/Users/***/anaconda3/bin/python
Try running:
conda install jupyter
conda install tensorflow
pip install Keras
with your conda environment activated. Don't ask me why - Anaconda's behavior baffles me. (I've created a .yml script with these yet I still seem to need to do this step)

Categories