I could use some clarification regarding anaconda envs and the installed packages. I have just began using environments other than the base to keep my installations clean.
I just deleted my anaconda3 folder, installed it from scratch, made a new environment and tried to run a jupyter-notebook.
When reading excel via pandas I get the error Missing optional dependency 'xlrd'
prompting me to install xlrd via conda or pip.
So I open the terminal, activate the env I’m working in and install it using conda. I close the JN from the terminal, reopen it and run the same code. The problem persists.
I repeat the procedure, this time indicating the name of the env at the end. Problem persists.
I checked which python in the terminal, it is indeed the anaconda3/envs/newenv/bin/python
I do the same (in the new env) using pip. I use the command .../anaconda3/envs/newenv/bin/python -m pip install xlrd. I get the message Requirement already satisfied: xlrd in ./anaconda3/envs/newenv/lib/python3.7/site-packages (1.2.0).
I even tried installing the package inside the notebook using !conda install xlrd, still I get the same error.
Finally, I open Jupyter via the base env, and the package works there perfectly.
I have no idea why I can’t install this under the newenv, which was the point of having local envs after all.
Thanks heaps for your help!
You need to install the kernel in Jupyter to be able to use it.
jupyter kernelspec list
That command will give you the list of kernels you have. I am assuming it only shows you Python3
You will now need to install a kernel. Remember to do this while inside your virtual environment
python3 -m pip install ipykernel
python3 -m ipykernel install --user --name <your-new-kernel-name>
You should now see this in Jupyter notebook. Select the kernel in Jupyter and you should be good to go.
Another thing you may want to try is to install Jupyter while inside your virtual environment. While inside your virtual environment, you could do:
python3 -m pip install jupyter lab
and then while still inside your virtual environment, run jupyter after checking which jupyter. It should solve your problem as well.
Using Conda (4.8) on pyhthon 3.7, on Win10. I have scikit learn installed using conda conda install scikit-learn . Tried a few things: also installed it in the env conda install -n my_env scikit-learn. Also tried installing conda install -c anaconda ipython - nothing has worked.
I can list it:
scikit-learn 0.22 py37h6288b17_0
But in juypter notebook get error
from sklearn.datasets import fetch_lfw_pairs (tried couple other commands too)
ModuleNotFoundError: No module named 'sklearn'
But If I use Anaconda UI Navigator to launch notebook everything works fine
Update
I tried this command line option did not work for me, despite plenty effort and help & support from the community (as below). Meanwhile the Jupyter notebook can also be launched from the Anaconda UI itself. That always was working for me - no configuration or setup needed (none). I have not found any limitations etc with this yet (and you do get the exact same notebook). For advanced/unique use cases where you may need to fine tune your configuration cmd line could be helpful, I am not there.
Likely, you are loading the wrong kernel when you start your notebook.
Here is a barebones way to set up the environment:
conda create -n testenv python=3.7 -y
conda activate testenv
conda install scikit-learn
conda install ipython
conda install notebook
python -m ipykernel install --user --name testenv
When you click on new in the browser you will have an additional option next to python3, namely the kernel you just registered. I just tested this with anaconda 4.7 and I could import sklearn.
edit:
The code in the answer creates a new python environment. Then, it installs ipython and jupyter notebook in that environment and makes sure that this environment can be used with jupyter notebook (i.e. registering the ipykernel).
Now of course besides scikit learn, no other libraries have been installed within that specific environment.
So, if you want to use more libraries, you have to go to the command line, activate the environment, and install the libraries you want to use:
conda activate testenv
conda install scipy numpy matplotlib
To then run jupyter notebook from the environment, after you have installed all the things you want (and after having closed the command prompt or having deactivated the environment), you can do
conda activate testenv
jupyter notebook
in the command prompt.
Jupyterlab would usually use the environment inside which you launch it. For example:
If you activate my_env first and then do jupyter lab from terminal, it should detect the environment.
Incase it fails, go to Kernel -> Change Kernel and select the kernel you want to use.
Note: While creating a new kernel, I always use the display-name parameter which helps. You can do something like:
python -m ipykernel install --user --name my_env --display-name "Python (my_env)"
Hope this helps.
To fix this problem, you need to manually install this package in Anaconda.
How do I install? Open your Anaconda Prompt and run the below command:
conda install -c conda-forge scikit-learn
Then restart Jupyter Notebook and import this package.
I think the problem is that the environment is not activated. Try conda activate my_env first, and then type jupyter notebook.
The first thing you can do is:
import sys
print(sys.path)
Check if /path/to/anaconda/envs/my_env/lib/python3.7/site-packages exists in the path.
I find it useful to print the current sys.path so that I know where it is looking at.
conda info --envs
when testEnv exists:
conda activate testEnv
conda list scikit-learn
I am at a Python boot camp this weekend but I have not been able to even use Python on my computer because of this issue. All my instructors are stumped too.
The issue is that I get the ModuleNotFoundError on Jupyter with multiple different packages, including Pandas and Requests (but oddly enough, BeautifulSoup and CSV work fine.)
Here is how I start a new Jupyter file:
Create a new directory
Install jupyter and pandas with this command: pipenv install jupyter pandas
Activate virtual environment: pipenv shell
Launch Jupyter: jupyter notebook
Create new Python 3 notebook
At this point, I try a command like import pandas as pd and get back the ModuleNotFoundError.
I am using Python version 3.6.5.
Attempts to fix this that have failed:
double-checked that pandas is installed in my virtual environment with pip graph
created completely new directory
pipenv install jupyter pandas --skip-lock
Uninstalled everything system-wide with these commands:
pip freeze > requirements.txt
pip uninstall -r requirements.txt -y
Updated pandas
Used virtualenv instead of pipenv
virtualenv first-python-notebook
cd first-python-notebook
cd Scripts
activate
cd ..
pip install jupyter pandas
I tested that pandas could be imported when I used python in the command shell (yes) -- still didn't work on Jupyter.
My instructor thinks the issue is that system-wide packages are interfering with virtual ones but we have been working for hours and cannot figure out how to fix this.
Any help would be greatly appreciated. Please include detailed instructions as I am a beginner.
If you're getting 'ModuleNotFoundError: No module named xxyyzz' in jupyter, but the module can be imported by running python via the pipenv shell (pipenv run python -c "import xxyyzz; print(xxyyzz.__version__)":
it's probably jupyter's python path isn't set properly in the kernel config file: ..\jupyter\kernels\<myProjectName>\kernel.json
the kernel needs to be created within the pipenv shell to pick up the right path
With a fresh pipenv install:
pip install pipenv
cd <project directory>
export PIPENV_VENV_IN_PROJECT=1 # creates .venv in project directory
pipenv --python=/path/to/python --site-packages # use python executable for your system or environment
pipenv shell # work in project's virtual environment
python -m ipykernel install --user --name=<myProjectName> # create jupyter kernel for project
exit # exit project's virtual environment
pipenv run jupyter notebook # start jupyter from project directory
in jupyter, choose the kernel "myProjectName"
this post provides additional explanations
Why don't you try to install ipykernel with Anaconda virtual env?
It'll will be more easy to handle.
If you haven't previously used Anaconda before, just go to the official website
https://www.anaconda.com/download/ and download the newest version for your OS.
Then, follow these steps.
Execute Anaconda prompt.
Type 'conda create -y -n $ENVIRONMENT_NAME ipykernel'
Type 'conda activate $ENVIRONMENT_NAME'
Type 'conda install -y $PACKAGES_TO_BE_INSTALLED'
Type 'python -m ipykernel install --user --name $NAME --display-name $IPYKERNEL_NAME'
This ipykernel name will be presented on your list of kernels in jupyter notebook.
You can findout the list of kernels installed by typing jupyter kernelspec list.
Hope this helps!
Thanks for the advice. However, I was advised specifically not to install Anaconda -- can't quite remember the reason but I think it's because, basically, if I ever decided I wanted to use something else then it would be a real headache to switch. I'm happy to hear your reasoning if you disagree with that.
I ended up solving the issue by uninstalling every package both within the virtual environment and the larger computer system, then re-installing it in both places. It worked, but I'm sort of confused as to what the point of a virtual environment is, if I still had to install everything twice.
TL;DR
Would like to run Jupyter notebooks with different python setups. Python packages always install globally. Don't understand why.\TL;DR
I would like to run Jupyter notebooks with different python setups using venv. See here for the official documentation.
Python 3.6 is already installed on my system
$ python --version
Python 3.6.1 :: Continuum Analytics, Inc.
Using the following commands I have created two virtual environments:
$ python3 -m venv --without-pip Documents/venv/test01
$ python3 -m venv --without-pip Documents/venv/test02
Following this guide I tried setting up different kernels for each notebook with
$ source activate test01
(Documents/venv/test02) $ python -m ipykernel install --user --name test01 --display-name test01
However, the second command failed with
/Users/dominik/Documents/venv/test02/bin/python: No module named ipykernel
So, I deactivated my venv and ran the same command outside the venv which succeeded
$ source deactivate test01
$ python -m ipykernel install --user --name test01 --display-name test01
$ python -m ipykernel install --user --name test02 --display-name test02
Inside my Jupyter notebook I can see the different kernels now:
new kernels available
Now I'm creating a new notebook using test01 kernel. Inside the notebook, I try to add a module which is not available in Python by default:
$ import mord
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-2-a00d777a7e47> in <module>()
----> 1 import mord
ModuleNotFoundError: No module named 'mord'
This is as expected. I then go about to installing that package into my test01 environment using pip
$ source activate Documents/venv/test01
(Documents/venv/test01) $ pip install mord
After restarting my test01 kernel the import error disappears - as expected. However - and now this is my question - when I import mord package on a test02 notebook, there is no import error either. Why is that?
I would expect that mord package was only installed for test01. However, it seems to be installed globally.
Looking at the venv folders it shows that nothing was added specifically to those projects
venv folders
The pyvenv.cfg file seems also unchanged.
home = /Users/dominik/anaconda/bin
include-system-site-packages = false
version = 3.6.1
Can anyone give me some pointers what I am doing wrong?
because you created virtualenv with --without-pip flag, there is no pip executable in the virtual environment, you used system pip to install package.
basically, virtual environment not involved in your setup, even though there are two kernel specs, they all created with virtualenv deactivated.
setup steps recommend: create virtualenv without --without-pip option; install ipykernel for each of your virtual environments, which means, install while virtualenv is activated; create kernel spec from corresponding virtualenv.
After georgexsh had pointed me in the right direction regarding the --without-pip flag, this is how I solved it in my case:
Create two virtual environments
$ python -m venv --without-pip Documents/venv/test01
$ python -m venv --without-pip Documents/venv/test02
I am still using the --without-pip flag, because the command otherwise produces an error message
The following steps are executed for each of the environments test01, test02:
Activate virtual environment and install pip manually
$ source Documents/venv/test01/bin/activate
(test01) $ curl https://bootstrap.pypa.io/get-pip.py | python
Install and run Jupyter inside the virtual environment
(test01) $ pip install jupyter
(test01) $ jupyter notebook
After having completed the above steps for both environments, I tested installing a package only in one of the environments
Activate the environment you want to install to first (if not activated already)
$ source Documents/venv/test01/bin/activate
(test01) $ pip install numpy
numpy package is now only available to the Jupyter version installed in test01 environment.
If found that installing a dedicated kernel per each environment was optional (for my purpose). Anyway, here are the steps to run in an activated environment:
(test01) $ pip install ipykernel
(test01) $ python -m ipykernel install --user --name test01 --display-name test01
Now within Jupyter you can also create notebooks with different kernel types.
I installed Jupyter notebooks in Ubuntu 14.04 via Anaconda earlier, and just now I installed TensorFlow. I would like TensorFlow to work regardless of whether I am working in a notebook or simply scripting. In my attempt to achieve this, I ended up installing TensorFlow twice, once using Anaconda, and once using pip. The Anaconda install works, but I need to preface any call to python with "source activate tensorflow". And the pip install works nicely, if start python the standard way (in the terminal) then tensorflow loads just fine.
My question is: how can I also have it work in the Jupyter notebooks?
This leads me to a more general question: it seems that my python kernel in Jupyter/Anaconda is separate from the python kernel (or environment? not sure about the terminology here) used system wide. It would be nice if these coincided, so that if I install a new python library, it becomes accessible to all the varied ways I have of running python.
Update
TensorFlow website supports five installations.
To my understanding, using Pip installation directly would be fine to import TensorFlow in Jupyter Notebook (as long as Jupyter Notebook was installed and there were no other issues) b/z it didn't create any virtual environments.
Using virtualenv install and conda install would need to install jupyter into the newly created TensorFlow environment to allow TensorFlow to work in Jupyter Notebook (see the following original post section for more details).
I believe docker install may require some port setup in the VirtualBox to make TensorFlow work in Jupyter Notebook (see this post).
For installing from sources, it also depends on which environment the source code is built and installed into. If it's installed into a freshly created virtual environment or an virtual environment which didn't have Jupyter Notebook installed, it would also need to install Jupyter Notebook into the virtual environment to use Tensorflow in Jupyter Notebook.
Original Post
To use tensorflow in Ipython and/or Jupyter(Ipython) Notebook, you'll need to install Ipython and Jupyter (after installing tensorflow) under the tensorflow activated environment.
Before install Ipython and Jupyter under tensorflow environment, if you do the following commands in terminal:
username$ source activate tensorflow
(tensorflow)username$ which ipython
(tensorflow)username$ /Users/username/anaconda/bin/ipython
(tensorflow)username$ which jupyter
(tensorflow)username$ /Users/username/anaconda/bin/jupyter
(tensorflow)username$ which python
(tensorflow)username$ /User/username//anaconda/envs/tensorflow/bin/python
This is telling you that when you open python from terminal, it is using the one installed in the "environments" where tensorflow is installed. Therefore you can actually import tensorflow successfully. However, if you are trying to run ipython and/or jupyter notebook, these are not installed under the "environments" equipped with tensorflow, hence it has to go back to use the regular environment which has no tensorflow module, hence you get an import error.
You can verify this by listing out the items under envs/tensorflow/bin directory:
(tensorflow) username$ ls /User/username/anaconda/envs/tensorflow/bin/
You will see that there are no "ipython" and/or "jupyer" listing out.
To use tensorflow with Ipython and/or Jupyter notebook, simply install them into the tensorflow environment:
(tensorflow) username$ conda install ipython
(tensorflow) username$ pip install jupyter #(use pip3 for python3)
After installing them, there should be a "jupyer" and a "ipython" show up in the envs/tensorflow/bin/ directory.
Notes:
Before trying to import tensorflow module in jupyter notebook, try close the notebook. And "source deactivate tensorflow" first, and then reactivate it ("source activate tensorflow") to make sure things are "on the same page". Then reopen the notebook and try import tensorflow. It should be import successfully (worked on mine at least).
i used these following which in virtualenv.
pip3 install --ignore-installed ipython
pip3 install --ignore-installed jupyter
This re-installs both ipython and jupyter notebook in my tensorflow virtual environment. You can verify it after installation by which ipython and which jupyter. The bin will be under the virtual env.
NOTE I am using python 3.*
I have another solution that you don't need to source activate tensorflow before using jupyter notebook every time.
Partion 1
Firstly, you should ensure you have installed jupyter in your virtualenv. If you have installed, you can skip this section (Use which jupyter to check). If you not, you could run source activate tensorflow, and then install jupyter in your virtualenv by conda install jupyter. (You can use pip too.)
Partion 2
1.From within your virtualenv, run
username$ source activate tensorflow
(tensorflow)username$ ipython kernelspec install-self --user
This will create a kernelspec for your virtualenv and tell you where it is:
(tensorflow)username$ [InstallNativeKernelSpec] Installed kernelspec pythonX in /home/username/.local/share/jupyter/kernels/pythonX
Where pythonX will match the version of Python in your virtualenv.
2.Copy the new kernelspec somewhere useful. Choose a kernel_name for your new kernel that is not python2 or python3 or one you've used before and then:
(tensorflow)username$ mkdir -p ~/.ipython/kernels
(tensorflow)username$ mv ~/.local/share/jupyter/kernels/pythonX ~/.ipython/kernels/<kernel_name>
3.If you want to change the name of the kernel that IPython shows you, you need to edit ~/.ipython/kernels/<kernel_name>/kernel.json and change the JSON key called display_name to be a name that you like.
4.You should now be able to see your kernel in the IPython notebook menu: Kernel -> Change kernel and be able so switch to it (you may need to refresh the page before it appears in the list). IPython will remember which kernel to use for that notebook from then on.
Reference.
Here is what I did to enable tensorflow in Anaconda -> Jupyter.
Install Tensorflow using the instructions provided at
Go to /Users/username/anaconda/env and ensure Tensorflow is installed
Open the Anaconda navigator and go to "Environments" (located in the left navigation)
Select "All" in teh first drop down and search for Tensorflow
If its not enabled, enabled it in the checkbox and confirm the process that follows.
Now open a new Jupyter notebook and tensorflow should work
Your Anaconda install probably went to different directory than your Python install
For instance on my machine I can find location here
yaroslavvb-macbookpro:~ yaroslavvb$ which ipython
/Users/yaroslavvb/anaconda/bin/ipython
When you type python, it tries to find it in PATH going in left-to-right order. So you may have another version of python in a folder before Anaconda folder, and it'll use that. To fix, you can do export PATH=.... to change the path, and put Anaconda directory in front, so that it takes python from there instead of the default, ie
export PATH=/Users/yaroslavvb/anaconda/bin:$PATH
I installed PIP with Conda conda install pip instead of apt-get install python-pip python-dev.
Then installed tensorflow
use Pip Installation:
# Ubuntu/Linux 64-bit, CPU only, Python 2.7
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
# Ubuntu/Linux 64-bit, GPU enabled, Python 2.7
# Requires CUDA toolkit 7.5 and CuDNN v4. For other versions, see "Install from sources" below.
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
...
pip install --upgrade $TF_BINARY_URL
Then it will work in jupyter notebook.
The accepted answer (by Zhongyu Kuang) has just helped me out. Here I've create an environment.yml file that enables me to make this conda / tensorflow installation process repeatable.
Step 1 - create a Conda environment.yml File
environment.yml looks like this:
name: hello-tensorflow
dependencies:
- python=3.6
- jupyter
- ipython
- pip:
- https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.1.0-cp36-cp36m-linux_x86_64.whl
Note:
Simply replace the name to whatever you want. (mine is hello-tensorflow)
Simply replace the python version to whatever you want. (mine is 3.6)
Simply replace the tensorflow pip install URL to whatever you want (mine is the Tensorflow URL where Python 3.6 with GPU support)
Step 2 - create the Conda environment
With the environment.yml being in the current path you are on, this command creates the environment hello-tensorflow (or whatever you have renamed it to):
conda env create -f environment.yml
Step 3: source activate
Activate the newly created environment:
source activate hello-tensorflow
Step 4 - which python / jupyter / ipython
which python...
(hello-tensorflow) $ which python
/home/johnny/anaconda3/envs/hello-tensorflow/bin/python
which jupyter...
(hello-tensorflow) $ which jupyter
/home/johnny/anaconda3/envs/hello-tensorflow/bin/jupyter
which ipython...
(hello-tensorflow) $ which ipython
/home/johnny/anaconda3/envs/hello-tensorflow/bin/ipython
Step 5
You should now be able to import tensorflow from python, jupyter (console / qtconsole / notebook, etc.) and ipython.
I think your question is very similar with the question post here. Windows 7 jupyter notebook executing tensorflow. As Yaroslav mentioned, you can try
conda install -c http://conda.anaconda.org/jjhelmus tensorflow .
I had a similar issue when using a custom Ubuntu 16 image. The problem was related to an existing version of numpy that was already installed on my system.
I initially tried
sudo pip3 install tensorflow
This resulted in the following exception:
Exception:
Traceback (most recent call last):
File "/anaconda/envs/py35/lib/python3.5/shutil.py", line 538, in move
os.rename(src, real_dst)
PermissionError: [Errno 13] Permission denied: '/anaconda/envs/py35/lib/python3.5/site-packages/numpy' -> '/tmp/pip-co73r3hm-uninstall/anaconda/envs/py35/lib/python3.5/site-packages/numpy'
The docs advise that if you encounter any issues with this command to try the following:
sudo pip3 install --upgrade \
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.1-cp35-cp35m-linux_x86_64.whl
However, my system was unable to locate pip3
sudo: pip3 command not found
The ulitmate solution was to create a symlink for pip3
sudo ln -s /anaconda/envs/py35/bin/pip3.5 /usr/local/bin/pip3
Finally, the following command worked without trouble
sudo /usr/local/bin/pip3 install --upgrade \
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.1-cp35-cp35m-linux_x86_64.whl
I verified the installation in the terminal and also verified a successful import in my Jupyter Notebook
import tensorflow as tf
I wonder if it is not enough to simply launch ipython from tensorflow environnement. That is
1) first activate tensorflow virtualenv with:
source ~/tensorflow/bin/activate
2) launch ipython under tensorflow environnement
(tensorflow)$ ipython notebook --ip=xxx.xxx.xxx.xxx
I found the solution from someone else's post. It is simple and works well!
http://help.pythonanywhere.com/pages/IPythonNotebookVirtualenvs
Just install the following in the Command Prompt and change kernel to Python 3 in Jupyter Notebook. It will import tensorflow successfully.
pip install tornado==4.5.3
pip install ipykernel==4.8.2
(Orginial post: https://github.com/tensorflow/tensorflow/issues/11851)
Jupyter Lab: ModuleNotFound tensorflow
For a future version of me or a colleague that runs into this issue:
conda install -c conda-forge jupyter jupyterlab keras tensorflow
Turns out jupyterlab is a plugin for jupyter.
So even if you are in an environment that has jupyter but not jupyterlab as well, if you try to run:
jupyter lab
then jupyter will look in the (base) environment for the jupyterlab plugin.
Then your imports in jupyter lab will be relative to that plugin and not your conda environment.
pip install tensorflow
This worked for me in my conda virtual environment.
I was trying to use conda install tensorflow in a conda virtual environment where jupyter notebooks was already installed, resulting in many conflicts and failure. But pip install worked fine.
conda info --envs
conda create --name py3-TF2.0 python=3
Proceed ([y]/n)? y
conda activate py3-TF2.0
conda install tensorflow
pip install --upgrade tensorflow
conda install -c anaconda ipykernel
python -m ipykernel install --user --name=py3-TF2.0
it is the best way but if it is required you should upgrade numpy with scipy as well
You can try "conda install tensorflow". This will install TensorFlow in your Anaconda directory.
Your local pip directory might not be shared with the Anaconda directory.
Thanks!
Open an Anaconda Prompt screen: (base) C:\Users\YOU>conda create -n tf tensorflow
After the environment is created type: conda activate tf
Prompt moves to (tf) environment, that is: (tf) C:\Users\YOU>
then install Jupyter Notebook in this (tf) environment:
conda install -c conda-forge jupyterlab - jupyter notebook
Still in (tf) environment, that is type
(tf) C:\Users\YOU>jupyter notebook
The notebook screen starts!!
A New notebook then can import tensorflow
FROM THEN ON
To open a session
click Anaconda prompt,
type conda activate tf
the prompt moves to tf environment
(tf) C:\Users\YOU>
then type (tf) C:\Users\YOU>jupyter notebook