I am at a Python boot camp this weekend but I have not been able to even use Python on my computer because of this issue. All my instructors are stumped too.
The issue is that I get the ModuleNotFoundError on Jupyter with multiple different packages, including Pandas and Requests (but oddly enough, BeautifulSoup and CSV work fine.)
Here is how I start a new Jupyter file:
Create a new directory
Install jupyter and pandas with this command: pipenv install jupyter pandas
Activate virtual environment: pipenv shell
Launch Jupyter: jupyter notebook
Create new Python 3 notebook
At this point, I try a command like import pandas as pd and get back the ModuleNotFoundError.
I am using Python version 3.6.5.
Attempts to fix this that have failed:
double-checked that pandas is installed in my virtual environment with pip graph
created completely new directory
pipenv install jupyter pandas --skip-lock
Uninstalled everything system-wide with these commands:
pip freeze > requirements.txt
pip uninstall -r requirements.txt -y
Updated pandas
Used virtualenv instead of pipenv
virtualenv first-python-notebook
cd first-python-notebook
cd Scripts
activate
cd ..
pip install jupyter pandas
I tested that pandas could be imported when I used python in the command shell (yes) -- still didn't work on Jupyter.
My instructor thinks the issue is that system-wide packages are interfering with virtual ones but we have been working for hours and cannot figure out how to fix this.
Any help would be greatly appreciated. Please include detailed instructions as I am a beginner.
If you're getting 'ModuleNotFoundError: No module named xxyyzz' in jupyter, but the module can be imported by running python via the pipenv shell (pipenv run python -c "import xxyyzz; print(xxyyzz.__version__)":
it's probably jupyter's python path isn't set properly in the kernel config file: ..\jupyter\kernels\<myProjectName>\kernel.json
the kernel needs to be created within the pipenv shell to pick up the right path
With a fresh pipenv install:
pip install pipenv
cd <project directory>
export PIPENV_VENV_IN_PROJECT=1 # creates .venv in project directory
pipenv --python=/path/to/python --site-packages # use python executable for your system or environment
pipenv shell # work in project's virtual environment
python -m ipykernel install --user --name=<myProjectName> # create jupyter kernel for project
exit # exit project's virtual environment
pipenv run jupyter notebook # start jupyter from project directory
in jupyter, choose the kernel "myProjectName"
this post provides additional explanations
Why don't you try to install ipykernel with Anaconda virtual env?
It'll will be more easy to handle.
If you haven't previously used Anaconda before, just go to the official website
https://www.anaconda.com/download/ and download the newest version for your OS.
Then, follow these steps.
Execute Anaconda prompt.
Type 'conda create -y -n $ENVIRONMENT_NAME ipykernel'
Type 'conda activate $ENVIRONMENT_NAME'
Type 'conda install -y $PACKAGES_TO_BE_INSTALLED'
Type 'python -m ipykernel install --user --name $NAME --display-name $IPYKERNEL_NAME'
This ipykernel name will be presented on your list of kernels in jupyter notebook.
You can findout the list of kernels installed by typing jupyter kernelspec list.
Hope this helps!
Thanks for the advice. However, I was advised specifically not to install Anaconda -- can't quite remember the reason but I think it's because, basically, if I ever decided I wanted to use something else then it would be a real headache to switch. I'm happy to hear your reasoning if you disagree with that.
I ended up solving the issue by uninstalling every package both within the virtual environment and the larger computer system, then re-installing it in both places. It worked, but I'm sort of confused as to what the point of a virtual environment is, if I still had to install everything twice.
Related
I could use some clarification regarding anaconda envs and the installed packages. I have just began using environments other than the base to keep my installations clean.
I just deleted my anaconda3 folder, installed it from scratch, made a new environment and tried to run a jupyter-notebook.
When reading excel via pandas I get the error Missing optional dependency 'xlrd'
prompting me to install xlrd via conda or pip.
So I open the terminal, activate the env I’m working in and install it using conda. I close the JN from the terminal, reopen it and run the same code. The problem persists.
I repeat the procedure, this time indicating the name of the env at the end. Problem persists.
I checked which python in the terminal, it is indeed the anaconda3/envs/newenv/bin/python
I do the same (in the new env) using pip. I use the command .../anaconda3/envs/newenv/bin/python -m pip install xlrd. I get the message Requirement already satisfied: xlrd in ./anaconda3/envs/newenv/lib/python3.7/site-packages (1.2.0).
I even tried installing the package inside the notebook using !conda install xlrd, still I get the same error.
Finally, I open Jupyter via the base env, and the package works there perfectly.
I have no idea why I can’t install this under the newenv, which was the point of having local envs after all.
Thanks heaps for your help!
You need to install the kernel in Jupyter to be able to use it.
jupyter kernelspec list
That command will give you the list of kernels you have. I am assuming it only shows you Python3
You will now need to install a kernel. Remember to do this while inside your virtual environment
python3 -m pip install ipykernel
python3 -m ipykernel install --user --name <your-new-kernel-name>
You should now see this in Jupyter notebook. Select the kernel in Jupyter and you should be good to go.
Another thing you may want to try is to install Jupyter while inside your virtual environment. While inside your virtual environment, you could do:
python3 -m pip install jupyter lab
and then while still inside your virtual environment, run jupyter after checking which jupyter. It should solve your problem as well.
Using Conda (4.8) on pyhthon 3.7, on Win10. I have scikit learn installed using conda conda install scikit-learn . Tried a few things: also installed it in the env conda install -n my_env scikit-learn. Also tried installing conda install -c anaconda ipython - nothing has worked.
I can list it:
scikit-learn 0.22 py37h6288b17_0
But in juypter notebook get error
from sklearn.datasets import fetch_lfw_pairs (tried couple other commands too)
ModuleNotFoundError: No module named 'sklearn'
But If I use Anaconda UI Navigator to launch notebook everything works fine
Update
I tried this command line option did not work for me, despite plenty effort and help & support from the community (as below). Meanwhile the Jupyter notebook can also be launched from the Anaconda UI itself. That always was working for me - no configuration or setup needed (none). I have not found any limitations etc with this yet (and you do get the exact same notebook). For advanced/unique use cases where you may need to fine tune your configuration cmd line could be helpful, I am not there.
Likely, you are loading the wrong kernel when you start your notebook.
Here is a barebones way to set up the environment:
conda create -n testenv python=3.7 -y
conda activate testenv
conda install scikit-learn
conda install ipython
conda install notebook
python -m ipykernel install --user --name testenv
When you click on new in the browser you will have an additional option next to python3, namely the kernel you just registered. I just tested this with anaconda 4.7 and I could import sklearn.
edit:
The code in the answer creates a new python environment. Then, it installs ipython and jupyter notebook in that environment and makes sure that this environment can be used with jupyter notebook (i.e. registering the ipykernel).
Now of course besides scikit learn, no other libraries have been installed within that specific environment.
So, if you want to use more libraries, you have to go to the command line, activate the environment, and install the libraries you want to use:
conda activate testenv
conda install scipy numpy matplotlib
To then run jupyter notebook from the environment, after you have installed all the things you want (and after having closed the command prompt or having deactivated the environment), you can do
conda activate testenv
jupyter notebook
in the command prompt.
Jupyterlab would usually use the environment inside which you launch it. For example:
If you activate my_env first and then do jupyter lab from terminal, it should detect the environment.
Incase it fails, go to Kernel -> Change Kernel and select the kernel you want to use.
Note: While creating a new kernel, I always use the display-name parameter which helps. You can do something like:
python -m ipykernel install --user --name my_env --display-name "Python (my_env)"
Hope this helps.
To fix this problem, you need to manually install this package in Anaconda.
How do I install? Open your Anaconda Prompt and run the below command:
conda install -c conda-forge scikit-learn
Then restart Jupyter Notebook and import this package.
I think the problem is that the environment is not activated. Try conda activate my_env first, and then type jupyter notebook.
The first thing you can do is:
import sys
print(sys.path)
Check if /path/to/anaconda/envs/my_env/lib/python3.7/site-packages exists in the path.
I find it useful to print the current sys.path so that I know where it is looking at.
conda info --envs
when testEnv exists:
conda activate testEnv
conda list scikit-learn
I followed a guide about setting up a pyenv workflow, it includes an IPython Startup Script that modifies the PYTHONPATH when using Jupyter. Now I don't want to use pyenv, but even after uninstalling everything in this guide. This undesirable script behavior remains.
I followed this guide https://medium.com/#henriquebastos/the-definitive-guide-to-setup-my-python-workspace-628d68552e14.
It has a separate virtual environment for Jupyter that the other virtual environments use. It includes an IPython Startup Script that runs and loads the virtualenv's site-packages into the Jupyter Notebook environment.
I no longer want to use pyenv. I am using MacOSX. I used Homebrew to install pyenv, pyenv-virtualenv, pyenv-virtualenvwrapper and so I removed all of those using 'brew uninstall'. I removed any trace of pyenv, jupyter, ipython, and python from /usr/local/bin, /usr/local/lib, and /usr/local/share, and ~/.
I then installed python using homebrew again, and ran pip3 install jupyter. I ran jupyter notebook, and it is still trying to run the kernel from this non-existent location '/Users/dhemming/.pyenv/versions/jupyter3/bin/python'.
I really don't think I understand that script or the Python Path and I'm really hoping someone can explain what is going on.
When following the guide I did this:
create jupyter3 virtual env (along with other virtual environments):
pyenv virtualenv 3.6.0 jupyter3
Install jupyter
pyenv activate jupyter3
pip install jupyter
python -m ipykernel install --user
pyenv deactivate
Set pyenv global:
pyenv global 3.6.0 2.7.13 jupyter3 ipython2 tools3 tools2
Check which jupyter:
~$ pyenv which jupyter
/Users/dhemming/.pyenv/versions/jupyter3/bin/jupyter
Install the IPython Script:
ipython profile create
curl -L http://hbn.link/hb-ipython-startup-script > ~/.ipython/profile_default/startup/00-venv-sitepackages.py
Here is the script that that curl command fetches:
import os
import sys
from warnings import warn
virtualenv = os.environ.get('VIRTUAL_ENV')
if virtualenv:
version = os.listdir(os.path.join(virtualenv, 'lib'))[0]
site_packages = os.path.join(virtualenv, 'lib', version, 'site-packages')
lib_dynload = os.path.join(virtualenv, 'lib', version, 'lib-dynload')
if not (os.path.exists(site_packages) and os.path.exists(lib_dynload)):
msg = 'Virtualenv site-packages discovery went wrong for %r' % repr([site_packages, lib_dynload])
warn(msg)
sys.path.insert(0, site_packages)
sys.path.insert(1, lib_dynload)
Upon completing the guide everything worked fine and whenever I created a new virtual environment using pyenv it used the jupyter3 virtual environment to run the notebook loaded with the new virtual environments libraries etc.
Then I no longer wanted this setup. So I remove every trace of python and anything related to python and I install a fresh python using Homebrew and do a pip3 install jupyter. After that I run jupyter notebook and I still get this:
Failed to run the command:
['/Users/dhemming/.pyenv/versions/jupyter3/bin/python', '-m', 'ipykernel_launcher', '-f', '/Users/dhemming/Library/Jupyter/runtime/kernel-1d721ce4-1619-498d-9f0b-62b98b12d0ac.json']
PATH='/usr/local/bin:/Users/dhemming/.rbenv/shims:/usr/local/opt/openssl/bin:/Users/dhemming/.nvm/versions/node/v8.11.2/bin:/Users/dhemming/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin'
with kwargs:
{'stdin': -1, 'stdout': None, 'stderr': None, 'cwd': '/Users/dhemming/workspace/tmp/tmp_01', 'start_new_session': True}
Where is that '/Users/dhemming/.pyenv/versions/jupyter3/bin/python' coming from?
I ended up deleting ~/Library/Jupyter and that fixed the problem. However I'm sure that there is a less extreme method of fixing this problem if someone knows.
I installed Anaconda3 so I can create environments and install different packages in each environment. But I fail to understand the difference between the Python in
/usr/bin/python
and
/opt/anaconda3/bin/python
I can seem to access Python 3.6.5 Anaconda from both, why is that? And, what is the difference between both?
Furthermore, I would like to install packages to a single Python environment only.
When you are running python in the terminal, it is looking up your default path to your the python command. In this case, anaconda probably put a line in your shell profile specify the path to the anaconda version, which is why you are seeing it in the interpreter when you run python from either directory.
Secondly, you can set up a conda environment to download app specific dependencies without interfering with your default set up by
conda create --name myenv
source activate myenv
conda install packagename
This will install it in the myenv environment only. To deactivate the environment just run
source deactivate
Here is the documentation on that https://conda.io/docs/user-guide/tasks/manage-environments.html
Judging by your path, you are using Linux which comes with python installed. So /usr/bin/python is default and you have installed the other one later.
For the environments use https://conda.io/docs/user-guide/tasks/manage-environments.html to activate the desired environment, then you can pip install or conda install the packages and it will be places safely only in that environment. Note that spyder icon runs the root environment by default and you have to run it from terminal after activating one of the environments.
Edit:
I'm not sure why you want to use cd to change the python version. I suggest use aliases. I guess you are just changing the path but running the same version of the python anyway. Take a look at this question:
Two versions of python on linux. how to make 2.7 the default
I wanted to create a new virtual environment to install new packages. Following worked for me:
Commands are executed in Jupyter Notebook (OS: Ubuntu 16.04 LTS)
Upgrade pip:
!pip install --upgrade pip
Install virtual environment:
!pip install virtualenv
Select version of Python you want to use in new environment:
I wanted to create an environment with Python version 3. Naming it as Python3_xyz:
!virtualenv -p python3 Python3_xyz
After execution, this will create a folder with the same name in the current working directory (i.e. the location where Jupyter notebook is present)
Create a new option with the name of the created environment
And finally, run the following command:
!python -m ipykernel install --user --name=Python3_xyz
This will create a new option with the name Python3_xyz in the menu from where we create a new notebook.
NOTE: One can run above commands from the terminal as well just don't use '!' before the commands.
This question is bit dated, but since I faced a similar issue, what worked for me might help someone!
I did pip install requests from within my conda environment, but failed to import requests even after trying out everything.
What worked for me: run python -m pip install requests or python3 -m pip install requests within you environment. This installed requests successfully for me.
I have installed a created a virtualenv machinelearn and installed a few python modules (pandas, scipy and sklearn) in that environment.
When I run jupyter notebook, I can import pandas and scipy in my notebooks - however, when I try to import sklearn, I get the following error message:
import sklearn
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-8fd979e02004> in <module>()
----> 1 import sklearn
ImportError: No module named 'sklearn'
I am able to import all modules, at the command line - so I know they have been successfully installed:
(machinelearn) me#yourbox:~/path/to/machinelearn$ python -c "import pandas, scipy, sklearn"
(machinelearn) me#yourbox:~/path/to/machinelearn$
How can I import sklearn in my jupyter notebook running in a virtualenv?
You probably have not installed jupyter / IPython in your virtualenv. Try the following:
python -c "import IPython"
and check that the jupyter command found in your $PATH is the one from the bin folder of your venv:
which jupyter
For windows users in a powershell console, you can use the following to check that the jupyter command in your $env:Path is the one from the Scripts folder of you venv:
get-command jupyter
Edit: if this is the problem, just run python -m pip install jupyter in your venv.
Edit 2: actually you might also need:
python -m ipykernel install --user --name=my-virtualenv-name
and then switch the kernel named "my-virtualenv-name" in the jupyter user interface.
Edit 3: maybe the --user flag in the last command is a bad idea:
python -m ipykernel install --name=my-virtualenv-name
Another approach to take is to have one global jupyter installation, but to point to different kernels to run as the backend.
That approach is outlined here in their docs:
http://help.pythonanywhere.com/pages/IPythonNotebookVirtualenvs
Copying below in case the link breaks:
You can use a virtualenv for your IPython notebook. Follow the following steps:
Install the ipython kernel module into your virtualenv
workon my-virtualenv-name # activate your virtualenv, if you haven't already
pip install ipykernel
Now run the kernel "self-install" script:
python -m ipykernel install --user --name=my-virtualenv-name
Replacing the --name parameter as appropriate.
You should now be able to see your kernel in the IPython notebook menu: Kernel -> Change kernel and be able so switch to it (you may need to refresh the page before it appears in the list). IPython will remember which kernel to use for that notebook from then on.
To use Jupyter notebook with virtual environment (using virtualenvwrapper) plus packages installed in that environment, follow steps below:
create a virtual environment
mkvirtualenv --no-site-packages --python=/your/python/path your_env_name
Activate the virtual environment
workon your_env_name
Install Jupyter and other packages
pip install jupyter, numpy
Add a new kernel to your Jupyter config
ipython kernel install --user --name=your_env_name
Done. You may now use Jupyter notebook under the virtual environment.
jupyter-notebook
Disclaimer: the question has been answered but is hidden in one of the replies. I googled and took sometime to find the right answer. So I just summarize it so someone having the same issue can easily follow.
Assuming that jupyter is installed on your machine, not on the virtual environtment.
Using a virtual environment with Jupyter notebook
VENV_NAME = "YOUR VIRTUAL ENV NAME"
1) virtualenv VENV_NAME
2) source venv/bin/activate
3) Add this package if not present: pip3 install ipykernel
4) Then execute this command: ipython kernel install --user --name=VENV_NAME
5) Now open up the Jupyter Notebook and in change kernel select VENV_NAME
6) To install a new package perform pip3 install <PACKAGE NAME> in your terminal and repeat step 4.
Hope it helps!
Solution without adding a new kernel globally!!
create a new virtual environment by
python3 -m virtualenv envname
Activate your enviroment and install jupyter in it by
pip install jupyter
One thing you have to make sure before installing jupyter is that you don't have following packages already installed in it.
ipykernel
ipython
ipython-genutils
ipywidgets
jupyter
jupyter-client
jupyter-console
jupyter-core
If you've previously installed them then first uninstall them by pip uninstall.
Install your desired packages in activated virtualenv and launch jupyter in it and voila!
Creation of virtualenv with python3 -m venv command
I had the same problem as yours.
In my case I had created the virtualenv with the command
python3 -m venv ./my_virtual_env --system-site-packages
The problem was I could not install jupyter inside the virtual environment as it was already in the system-site-package (when you try to install it, it tells you "Requirement already satisfied").
To install jupyter, (and in a first instance pip, that does not get installed neither in your virtual environment with this command) but still have access to system-site-package you can run :
python3 -m venv ./my_virtual_env
Activate you virtual environment, run pip3 install jupyter (and pip3 install pip) and then turn on the option include-system-site-packages in the file ./my_virtual_env/pyvenv.cfg.
After deactivation and reactivation of you environment, you will have access to system site-packages.
Creation of virtualenv with virtualenv command
Given this answer you can prevent the access to system site-packages by creating a file ./my_virtual_env/lib/python3.4/no-global-site-packages.txt,
and get the access back by removing it.
You can still install jupyter inside your virtual-environment if you have created your virtual env using:
python -m venv --system-site-packages path/to/my-venv
Simply do this:
activate-your-env
pip install -I jupyter
And you are now ready to go
jupyter notebook