Trouble with TensorFlow in Jupyter Notebook - python

I installed Jupyter notebooks in Ubuntu 14.04 via Anaconda earlier, and just now I installed TensorFlow. I would like TensorFlow to work regardless of whether I am working in a notebook or simply scripting. In my attempt to achieve this, I ended up installing TensorFlow twice, once using Anaconda, and once using pip. The Anaconda install works, but I need to preface any call to python with "source activate tensorflow". And the pip install works nicely, if start python the standard way (in the terminal) then tensorflow loads just fine.
My question is: how can I also have it work in the Jupyter notebooks?
This leads me to a more general question: it seems that my python kernel in Jupyter/Anaconda is separate from the python kernel (or environment? not sure about the terminology here) used system wide. It would be nice if these coincided, so that if I install a new python library, it becomes accessible to all the varied ways I have of running python.

Update
TensorFlow website supports five installations.
To my understanding, using Pip installation directly would be fine to import TensorFlow in Jupyter Notebook (as long as Jupyter Notebook was installed and there were no other issues) b/z it didn't create any virtual environments.
Using virtualenv install and conda install would need to install jupyter into the newly created TensorFlow environment to allow TensorFlow to work in Jupyter Notebook (see the following original post section for more details).
I believe docker install may require some port setup in the VirtualBox to make TensorFlow work in Jupyter Notebook (see this post).
For installing from sources, it also depends on which environment the source code is built and installed into. If it's installed into a freshly created virtual environment or an virtual environment which didn't have Jupyter Notebook installed, it would also need to install Jupyter Notebook into the virtual environment to use Tensorflow in Jupyter Notebook.
Original Post
To use tensorflow in Ipython and/or Jupyter(Ipython) Notebook, you'll need to install Ipython and Jupyter (after installing tensorflow) under the tensorflow activated environment.
Before install Ipython and Jupyter under tensorflow environment, if you do the following commands in terminal:
username$ source activate tensorflow
(tensorflow)username$ which ipython
(tensorflow)username$ /Users/username/anaconda/bin/ipython
(tensorflow)username$ which jupyter
(tensorflow)username$ /Users/username/anaconda/bin/jupyter
(tensorflow)username$ which python
(tensorflow)username$ /User/username//anaconda/envs/tensorflow/bin/python
This is telling you that when you open python from terminal, it is using the one installed in the "environments" where tensorflow is installed. Therefore you can actually import tensorflow successfully. However, if you are trying to run ipython and/or jupyter notebook, these are not installed under the "environments" equipped with tensorflow, hence it has to go back to use the regular environment which has no tensorflow module, hence you get an import error.
You can verify this by listing out the items under envs/tensorflow/bin directory:
(tensorflow) username$ ls /User/username/anaconda/envs/tensorflow/bin/
You will see that there are no "ipython" and/or "jupyer" listing out.
To use tensorflow with Ipython and/or Jupyter notebook, simply install them into the tensorflow environment:
(tensorflow) username$ conda install ipython
(tensorflow) username$ pip install jupyter #(use pip3 for python3)
After installing them, there should be a "jupyer" and a "ipython" show up in the envs/tensorflow/bin/ directory.
Notes:
Before trying to import tensorflow module in jupyter notebook, try close the notebook. And "source deactivate tensorflow" first, and then reactivate it ("source activate tensorflow") to make sure things are "on the same page". Then reopen the notebook and try import tensorflow. It should be import successfully (worked on mine at least).

i used these following which in virtualenv.
pip3 install --ignore-installed ipython
pip3 install --ignore-installed jupyter
This re-installs both ipython and jupyter notebook in my tensorflow virtual environment. You can verify it after installation by which ipython and which jupyter. The bin will be under the virtual env.
NOTE I am using python 3.*

I have another solution that you don't need to source activate tensorflow before using jupyter notebook every time.
Partion 1
Firstly, you should ensure you have installed jupyter in your virtualenv. If you have installed, you can skip this section (Use which jupyter to check). If you not, you could run source activate tensorflow, and then install jupyter in your virtualenv by conda install jupyter. (You can use pip too.)
Partion 2
1.From within your virtualenv, run
username$ source activate tensorflow
(tensorflow)username$ ipython kernelspec install-self --user
This will create a kernelspec for your virtualenv and tell you where it is:
(tensorflow)username$ [InstallNativeKernelSpec] Installed kernelspec pythonX in /home/username/.local/share/jupyter/kernels/pythonX
Where pythonX will match the version of Python in your virtualenv.
2.Copy the new kernelspec somewhere useful. Choose a kernel_name for your new kernel that is not python2 or python3 or one you've used before and then:
(tensorflow)username$ mkdir -p ~/.ipython/kernels
(tensorflow)username$ mv ~/.local/share/jupyter/kernels/pythonX ~/.ipython/kernels/<kernel_name>
3.If you want to change the name of the kernel that IPython shows you, you need to edit ~/.ipython/kernels/<kernel_name>/kernel.json and change the JSON key called display_name to be a name that you like.
4.You should now be able to see your kernel in the IPython notebook menu: Kernel -> Change kernel and be able so switch to it (you may need to refresh the page before it appears in the list). IPython will remember which kernel to use for that notebook from then on.
Reference.

Here is what I did to enable tensorflow in Anaconda -> Jupyter.
Install Tensorflow using the instructions provided at
Go to /Users/username/anaconda/env and ensure Tensorflow is installed
Open the Anaconda navigator and go to "Environments" (located in the left navigation)
Select "All" in teh first drop down and search for Tensorflow
If its not enabled, enabled it in the checkbox and confirm the process that follows.
Now open a new Jupyter notebook and tensorflow should work

Your Anaconda install probably went to different directory than your Python install
For instance on my machine I can find location here
yaroslavvb-macbookpro:~ yaroslavvb$ which ipython
/Users/yaroslavvb/anaconda/bin/ipython
When you type python, it tries to find it in PATH going in left-to-right order. So you may have another version of python in a folder before Anaconda folder, and it'll use that. To fix, you can do export PATH=.... to change the path, and put Anaconda directory in front, so that it takes python from there instead of the default, ie
export PATH=/Users/yaroslavvb/anaconda/bin:$PATH

I installed PIP with Conda conda install pip instead of apt-get install python-pip python-dev.
Then installed tensorflow
use Pip Installation:
# Ubuntu/Linux 64-bit, CPU only, Python 2.7
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
# Ubuntu/Linux 64-bit, GPU enabled, Python 2.7
# Requires CUDA toolkit 7.5 and CuDNN v4. For other versions, see "Install from sources" below.
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
...
pip install --upgrade $TF_BINARY_URL
Then it will work in jupyter notebook.

The accepted answer (by Zhongyu Kuang) has just helped me out. Here I've create an environment.yml file that enables me to make this conda / tensorflow installation process repeatable.
Step 1 - create a Conda environment.yml File
environment.yml looks like this:
name: hello-tensorflow
dependencies:
- python=3.6
- jupyter
- ipython
- pip:
- https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.1.0-cp36-cp36m-linux_x86_64.whl
Note:
Simply replace the name to whatever you want. (mine is hello-tensorflow)
Simply replace the python version to whatever you want. (mine is 3.6)
Simply replace the tensorflow pip install URL to whatever you want (mine is the Tensorflow URL where Python 3.6 with GPU support)
Step 2 - create the Conda environment
With the environment.yml being in the current path you are on, this command creates the environment hello-tensorflow (or whatever you have renamed it to):
conda env create -f environment.yml
Step 3: source activate
Activate the newly created environment:
source activate hello-tensorflow
Step 4 - which python / jupyter / ipython
which python...
(hello-tensorflow) $ which python
/home/johnny/anaconda3/envs/hello-tensorflow/bin/python
which jupyter...
(hello-tensorflow) $ which jupyter
/home/johnny/anaconda3/envs/hello-tensorflow/bin/jupyter
which ipython...
(hello-tensorflow) $ which ipython
/home/johnny/anaconda3/envs/hello-tensorflow/bin/ipython
Step 5
You should now be able to import tensorflow from python, jupyter (console / qtconsole / notebook, etc.) and ipython.

I think your question is very similar with the question post here. Windows 7 jupyter notebook executing tensorflow. As Yaroslav mentioned, you can try
conda install -c http://conda.anaconda.org/jjhelmus tensorflow .

I had a similar issue when using a custom Ubuntu 16 image. The problem was related to an existing version of numpy that was already installed on my system.
I initially tried
sudo pip3 install tensorflow
This resulted in the following exception:
Exception:
Traceback (most recent call last):
File "/anaconda/envs/py35/lib/python3.5/shutil.py", line 538, in move
os.rename(src, real_dst)
PermissionError: [Errno 13] Permission denied: '/anaconda/envs/py35/lib/python3.5/site-packages/numpy' -> '/tmp/pip-co73r3hm-uninstall/anaconda/envs/py35/lib/python3.5/site-packages/numpy'
The docs advise that if you encounter any issues with this command to try the following:
sudo pip3 install --upgrade \
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.1-cp35-cp35m-linux_x86_64.whl
However, my system was unable to locate pip3
sudo: pip3 command not found
The ulitmate solution was to create a symlink for pip3
sudo ln -s /anaconda/envs/py35/bin/pip3.5 /usr/local/bin/pip3
Finally, the following command worked without trouble
sudo /usr/local/bin/pip3 install --upgrade \
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.1-cp35-cp35m-linux_x86_64.whl
I verified the installation in the terminal and also verified a successful import in my Jupyter Notebook
import tensorflow as tf

I wonder if it is not enough to simply launch ipython from tensorflow environnement. That is
1) first activate tensorflow virtualenv with:
source ~/tensorflow/bin/activate
2) launch ipython under tensorflow environnement
(tensorflow)$ ipython notebook --ip=xxx.xxx.xxx.xxx

I found the solution from someone else's post. It is simple and works well!
http://help.pythonanywhere.com/pages/IPythonNotebookVirtualenvs
Just install the following in the Command Prompt and change kernel to Python 3 in Jupyter Notebook. It will import tensorflow successfully.
pip install tornado==4.5.3
pip install ipykernel==4.8.2
(Orginial post: https://github.com/tensorflow/tensorflow/issues/11851)

Jupyter Lab: ModuleNotFound tensorflow
For a future version of me or a colleague that runs into this issue:
conda install -c conda-forge jupyter jupyterlab keras tensorflow
Turns out jupyterlab is a plugin for jupyter.
So even if you are in an environment that has jupyter but not jupyterlab as well, if you try to run:
jupyter lab
then jupyter will look in the (base) environment for the jupyterlab plugin.
Then your imports in jupyter lab will be relative to that plugin and not your conda environment.

pip install tensorflow
This worked for me in my conda virtual environment.
I was trying to use conda install tensorflow in a conda virtual environment where jupyter notebooks was already installed, resulting in many conflicts and failure. But pip install worked fine.

conda info --envs
conda create --name py3-TF2.0 python=3
Proceed ([y]/n)? y
conda activate py3-TF2.0
conda install tensorflow
pip install --upgrade tensorflow
conda install -c anaconda ipykernel
python -m ipykernel install --user --name=py3-TF2.0
it is the best way but if it is required you should upgrade numpy with scipy as well

You can try "conda install tensorflow". This will install TensorFlow in your Anaconda directory.
Your local pip directory might not be shared with the Anaconda directory.
Thanks!

Open an Anaconda Prompt screen: (base) C:\Users\YOU>conda create -n tf tensorflow
After the environment is created type: conda activate tf
Prompt moves to (tf) environment, that is: (tf) C:\Users\YOU>
then install Jupyter Notebook in this (tf) environment:
conda install -c conda-forge jupyterlab - jupyter notebook
Still in (tf) environment, that is type
(tf) C:\Users\YOU>jupyter notebook
The notebook screen starts!!
A New notebook then can import tensorflow
FROM THEN ON
To open a session
click Anaconda prompt,
type conda activate tf
the prompt moves to tf environment
(tf) C:\Users\YOU>
then type (tf) C:\Users\YOU>jupyter notebook

Related

Cannot Open Jupyter Notebook From Python 3.5 Environment [duplicate]

I installed Anaconda (with Python 2.7), and installed Tensorflow in an environment called tensorflow. I can import Tensorflow successfully in that environment.
The problem is that Jupyter Notebook does not recognize the new environment I just created. No matter I start Jupyter Notebook from the GUI Navigator or from the command line within the tensorflow env, there is only one kernel in the menu called Python [Root], and Tensorflow cannot be imported. Of course, I clicked on that option multiple times, saved file, re-opened, but these did not help.
Strangely, I can see the two environments when I open the Conda tab on the front page of Jupyter. But when I open the Files tab, and try to new a notebook, I still end up with only one kernel.
I looked at this question:
Link Conda environment with Jupyter Notebook
But there isn't such a directory as ~/Library/Jupyter/kernels on my computer! This Jupyter directory only has one sub-directory called runtime.
I am really confused. Are Conda environments supposed to become kernels automatically? (I followed https://ipython.readthedocs.io/en/stable/install/kernel_install.html to manually set up the kernels, but was told that ipykernel was not found.)
I don't think the other answers are working any more, as conda stopped automatically setting environments up as jupyter kernels. You need to manually add kernels for each environment in the following way:
source activate myenv
python -m ipykernel install --user --name myenv --display-name "Python (myenv)"
As documented here:http://ipython.readthedocs.io/en/stable/install/kernel_install.html#kernels-for-different-environments
Also see this issue.
Addendum:
You should be able to install the nb_conda_kernels package with conda install nb_conda_kernels to add all environments automatically, see https://github.com/Anaconda-Platform/nb_conda_kernels
If your environments are not showing up, make sure you have installed
nb_conda_kernels in the environment with Jupyter
ipykernel and ipywidgets in the Python environment you want to access (note that ipywidgets is to enable some Juptyer functionality, not environment visibility, see related docs).
Anaconda's documentation states that
nb_conda_kernels should be installed in the environment from which
you run Jupyter Notebook or JupyterLab. This might be your base conda
environment, but it need not be. For instance, if the environment
notebook_env contains the notebook package, then you would run
conda install -n notebook_env nb_conda_kernels
Any other environments you wish to access in your notebooks must have
an appropriate kernel package installed. For instance, to access a
Python environment, it must have the ipykernel package; e.g.
conda install -n python_env ipykernel
To utilize an R environment, it must have the r-irkernel package; e.g.
conda install -n r_env r-irkernel
For other languages, their corresponding kernels must be installed.
In addition to Python, by installing the appropriatel *kernel package, Jupyter can access kernels from a ton of other languages including R, Julia, Scala/Spark, JavaScript, bash, Octave, and even MATLAB.
Note that at the time originally posting this, there was a possible cause from nb_conda not yet supporting Python 3.6 environments.
If other solutions fail to get Jupyter to recognize other conda environments, you can always install and run jupyter from within a specific environment. You may not be able to see or switch to other environments from within Jupyter though.
$ conda create -n py36_test -y python=3.6 jupyter
$ source activate py36_test
(py36_test) $ which jupyter
/home/schowell/anaconda3/envs/py36_test/bin/jupyter
(py36_test) $ jupyter notebook
Notice that I am running Python 3.6.1 in this notebook:
Note that if you do this with many environments, the added storage space from installing Jupyter into every environment may be undesirable (depending on your system).
The annoying thing is that in your tensorflow environment, you can run jupyter notebook without installing jupyter in that environment. Just run
(tensorflow) $ conda install jupyter
and the tensorflow environment should now be visible in Jupyter Notebooks started in any of your conda environments as something like Python [conda env:tensorflow].
I had to run all the commands mentioned in the top 3 answers to get this working:
conda install jupyter
conda install nb_conda
conda install ipykernel
python -m ipykernel install --user --name mykernel
Just run conda install ipykernel in your new environment, only then you will get a kernel with this env. This works even if you have different versions installed in each envs and it doesn't install jupyter notebook again. You can start youe notebook from any env you will be able to see newly added kernels.
Summary (tldr)
If you want the 'python3' kernel to always run the Python installation from the environment where it is launched, delete the User 'python3' kernel, which is taking precedence over whatever the current environment is with:
jupyter kernelspec remove python3
Full Solution
I am going to post an alternative and simpler solution for the following case:
You have created a conda environment
This environment has jupyter installed (which also installs ipykernel)
When you run the command jupyter notebook and create a new notebook by clicking 'python3' in the 'New' dropdown menu, that notebook executes python from the base environment and not from the current environment.
You would like it so that launching a new notebook with 'python3' within any environment executes the Python version from that environment and NOT the base
I am going to use the name 'test_env' for the environment for the rest of the solution. Also, note that 'python3' is the name of the kernel.
The currently top-voted answer does work, but there is an alternative. It says to do the following:
python -m ipykernel install --user --name test_env --display-name "Python (test_env)"
This will give you the option of using the test_env environment regardless of what environment you launch jupyter notebook from. But, launching a notebook with 'python3' will still use the Python installation from the base environment.
What likely is happening is that there is a user python3 kernel that exists. Run the command jupyter kernelspec list to list all of your environments. For instance, if you have a mac you will be returned the following (my user name is Ted).
python3 /Users/Ted/Library/Jupyter/kernels/python3
What Jupyter is doing here is searching through three different paths looking for kernels. It goes from User, to Env, to System. See this document for more details on the paths it searches for each operating system.
The two kernels above are both in the User path, meaning they will be available regardless of the environment that you launch a jupyter notebook from. This also means that if there is another 'python3' kernel at the environment level, then you will never be able to access it.
To me, it makes more sense that choosing the 'python3' kernel from the environment you launched the notebook from should execute Python from that environment.
You can check to see if you have another 'python3' environment by looking in the Env search path for your OS (see the link to the docs above). For me (on my mac), I issued the following command:
ls /Users/Ted/anaconda3/envs/test_env/share/jupyter/kernels
And I indeed had a 'python3' kernel listed there.
Thanks to this GitHub issue comment (look at the first response), you can remove the User 'python3' environment with the following command:
jupyter kernelspec remove python3
Now when you run jupyter kernelspec list, assuming the test_env is still active, you will get the following:
python3 /Users/Ted/anaconda3/envs/test_env/share/jupyter/kernels/python3
Notice that this path is located within the test_env directory. If you create a new environment, install jupyter, activate it, and list the kernels, you will get another 'python3' kernel located in its environment path.
The User 'python3' kernel was taking precedence over any of the Env 'python3' kernels. By removing it, the active environment 'python3' kernel was exposed and able to be chosen every time. This eliminates the need to manually create kernels. It also makes more sense in terms of software development where one would want to isolate themselves into a single environment. Running a kernel that is different from the host environment doesn't seem natural.
It also seems that this User 'python3' is not installed for everyone by default, so not everyone is confronted by this issue.
To add a conda environment to Jupyter:
In Anaconda Prompt :
run conda activate <env name>
run conda install -c anaconda ipykernel
run python -m ipykernel install --user --name=<env name>
** tested on conda 4.8.3 4.11.0
$ conda install nb_conda_kernels
(in the conda environment where you run jupyter notebook) will make all conda envs available automatically. For access to other environments, the respective kernels must be installed. Here's the ref.
This worked for me in windows 10 and latest solution :
1) Go inside that conda environment ( activate your_env_name )
2) conda install -n your_env_name ipykernel
3) python -m ipykernel install --user --name build_central --display-name "your_env_name"
(NOTE : Include the quotes around "your_env_name", in step 3)
The nb_conda_kernels package is the best way to use jupyter with conda. With minimal dependencies and configuration, it allows you to use other conda environments from a jupyter notebook running in a different environment. Quoting its documentation:
Installation
This package is designed to be managed solely using conda. It should be installed in the environment from which you run Jupyter Notebook or JupyterLab. This might be your base conda environment, but it need not be. For instance, if the environment notebook_env contains the notebook package, then you would run
conda install -n notebook_env nb_conda_kernels
Any other environments you wish to access in your notebooks must have an appropriate kernel package installed. For instance, to access a Python environment, it must have the ipykernel package; e.g.
conda install -n python_env ipykernel
To utilize an R environment, it
must have the r-irkernel package; e.g.
conda install -n r_env r-irkernel
For other languages, their corresponding kernels must be installed.
Then all you need to do is start the jupyter notebook server:
conda activate notebook_env # only needed if you are not using the base environment for the server
# conda install jupyter # in case you have not installed it already
jupyter
Despite the plethora of answers and #merv's efforts to improve them, it still hard to find a good one. I made this one CW, so please vote it to the top or improve it!
This is an old thread, but running this in Anaconda prompt, in my environment of interest, worked for me:
ipython kernel install --name "myenvname" --user
We have struggle a lot with this issue, and here's what works for us. If you use the conda-forge channel, it's important to make sure you are using updated packages from conda-forge, even in your Miniconda root environment.
So install Miniconda, and then do:
conda config --add channels conda-forge --force
conda update --all -y
conda install nb_conda_kernels -y
conda env create -f custom_env.yml -q --force
jupyter notebook
and your custom environment will show up in Jupyter as an available kernel, as long as ipykernel was listed for installation in your custom_env.yml file, like this example:
name: bqplot
channels:
- conda-forge
- defaults
dependencies:
- python>=3.6
- bqplot
- ipykernel
Just to prove it working with a bunch of custom environments, here's a screen grab from Windows:
I ran into this same problem where my new conda environment, myenv, couldn't be selected as a kernel or a new notebook. And running jupter notebook from within the env gave the same result.
My solution, and what I learned about how Jupyter notebooks recognizes conda-envs and kernels:
Installing jupyter and ipython to myenv with conda:
conda install -n myenv ipython jupyter
After that, running jupter notebook outside any env listed myenv as a kernel along with my previous environments.
Python [conda env:old]
Python [conda env:myenv]
Running the notebook once I activated the environment:
source activate myenv
jupyter notebook
hides all my other environment-kernels and only shows my language kernels:
python 2
python 3
R
This has been so frustrating, My problem was that within a newly constructed conda python36 environment, jupyter refused to load “seaborn” - even though seaborn was installed within that environment. It seemed to be able to import plenty of other files from the same environment — for example numpy and pandas but just not seaborn. I tried many of the fixes suggested here and on other threads without success. Until I realised that Jupyter was not running kernel python from within that environment but running the system python as kernel. Even though a decent looking kernel and kernel.json were already present in the environment. It was only after reading this part of the ipython documentation:
https://ipython.readthedocs.io/en/latest/install/kernel_install.html#kernels-for-different-environments
and using these commands:
source activate other-env
python -m ipykernel install --user --name other-env --display-name "Python (other-env)"
I was able to get everything going nicely. (I didn’t actually use the —user variable).
One thing I have not yet figured is how to set the default python to be the "Python (other-env)" one. At present an existing .ipynb file opened from the Home screen will use the system python. I have to use the Kernel menu “Change kernel” to select the environment python.
I had similar issue and I found a solution that is working for Mac, Windows and Linux. It takes few key ingredients that are in the answer above:
To be able to see conda env in Jupyter notebook, you need:
the following package in you base env:
conda install nb_conda
the following package in each env you create:
conda install ipykernel
check the configurationn of jupyter_notebook_config.py
first check if you have a jupyter_notebook_config.py in one of the location given by jupyter --paths
if it doesn't exist, create it by running jupyter notebook --generate-config
add or be sure you have the following: c.NotebookApp.kernel_spec_manager_class='nb_conda_kernels.manager.CondaKernelSpecManager'
The env you can see in your terminal:
On Jupyter Lab you can see the same env as above both the Notebook and Console:
And you can choose your env when have a notebook open:
The safe way is to create a specific env from which you will run your example of envjupyter lab command. Activate your env. Then add jupyter lab extension example jupyter lab extension. Then you can run jupyter lab
While #coolscitist's answer worked for me, there is also a way that does not clutter your kernel environment with the complete jupyter package+deps.
It is described in the ipython docs and is (I suspect) only necessary if you run the notebook server in a non-base environment.
conda activate name_of_your_kernel_env
conda install ipykernel
python -m ipykernel install --prefix=/home/your_username/.conda/envs/name_of_your_jupyter_server_env --name 'name_of_your_kernel_env'
You can check if it works using
conda activate name_of_your_jupyter_server_env
jupyter kernelspec list
First you need to activate your environment .
pip install ipykernel
Next you can add your virtual environment to Jupyter by typing:
python -m ipykernel install --name = my_env
Follow the instructions in the iPython documentation for adding different conda environments to the list of kernels to choose from in Jupyter Notebook. In summary, after installing ipykernel, you must activate each conda environment one by one in a terminal and run the command python -m ipykernel install --user --name myenv --display-name "Python (myenv)", where myenv is the environment (kernel) you want to add.
Possible Channel-Specific Issue
I had this issue (again) and it turned out I installed from the conda-forge channel; removing it and reinstalling from anaconda channel instead fixed it for me.
Update: I again had the same problem with a new env, this time I did install nb_conda_kernels from anaconda channel, but my jupyter_client was from the conda-forge channel. Uninstalling nb_conda_kernels and reinstalling updated that to a higher-priority channel.
So make sure you've installed from the correct channels :)
I encountered this problem when using vscode server.
In the conda environment named "base", I installed the 1.2.0 version of opennmt-py, but I want to run jupyter notebook in the conda environment "opennmt2", which contains code using opennmt-py 2.0.
I solved the problem by reinstalling jupyter in conda(opennmt2).
conda install jupyter
After reinstalling, executing jupyter notebook in the opennmt2 environment will execute the newly installed jupyter
where jupyter
/root/miniconda3/envs/opennmt2/bin/jupyter
/root/miniconda3/bin/jupyter
For conda 4.5.12, what works for me is (my virtual env is called nwt)
conda create --name nwt python=3
after that I need to activate the virtual environment and install the ipykernel
activate nwt
pip install ipykernel
then what works for me is:
python -m ipykernel install --user --name env_name --display-name "name of your choosing."
As an example, I am using 'nwt' as the display name for the virtual env. And after running the commands above. Run 'jupyter notebook" in Anaconda Prompt again. What I get is:
Using only environment variables:
python -m ipykernel install --user --name $(basename $VIRTUAL_ENV)
I just wanted to add to the previous answers: in case installing nb_conda_kernels, ipywidgets and ipekernel dosen't work, make sure your version of Jupyter is up to date. My envs suddenly stopped showing up after a period of everything working fine, and it resumed working after I simply updated jupyter through the anaconda navigator.
In my case, using Windows 10 and conda 4.6.11, by running the commands
conda install nb_conda
conda install -c conda-forge nb_conda_kernels
from the terminal while having the environment active didn't do the job after I opened Jupyter from the same command line using conda jupyter notebook.
The solution was apparently to opened Jupyter from the Anaconda Navigator by going to my environment in Environments: Open Anaconda Navigator, select the environment in Environments, press on the "play" button on the chosen environment, and select 'open with Jupyter Notebook'.
Environments in Anaconda Navigator to run Jupyter from the selected environment

ModuleNotFoundError: 'sklearn' in Jupyter notebook

Using Conda (4.8) on pyhthon 3.7, on Win10. I have scikit learn installed using conda conda install scikit-learn . Tried a few things: also installed it in the env conda install -n my_env scikit-learn. Also tried installing conda install -c anaconda ipython - nothing has worked.
I can list it:
scikit-learn 0.22 py37h6288b17_0
But in juypter notebook get error
from sklearn.datasets import fetch_lfw_pairs (tried couple other commands too)
ModuleNotFoundError: No module named 'sklearn'
But If I use Anaconda UI Navigator to launch notebook everything works fine
Update
I tried this command line option did not work for me, despite plenty effort and help & support from the community (as below). Meanwhile the Jupyter notebook can also be launched from the Anaconda UI itself. That always was working for me - no configuration or setup needed (none). I have not found any limitations etc with this yet (and you do get the exact same notebook). For advanced/unique use cases where you may need to fine tune your configuration cmd line could be helpful, I am not there.
Likely, you are loading the wrong kernel when you start your notebook.
Here is a barebones way to set up the environment:
conda create -n testenv python=3.7 -y
conda activate testenv
conda install scikit-learn
conda install ipython
conda install notebook
python -m ipykernel install --user --name testenv
When you click on new in the browser you will have an additional option next to python3, namely the kernel you just registered. I just tested this with anaconda 4.7 and I could import sklearn.
edit:
The code in the answer creates a new python environment. Then, it installs ipython and jupyter notebook in that environment and makes sure that this environment can be used with jupyter notebook (i.e. registering the ipykernel).
Now of course besides scikit learn, no other libraries have been installed within that specific environment.
So, if you want to use more libraries, you have to go to the command line, activate the environment, and install the libraries you want to use:
conda activate testenv
conda install scipy numpy matplotlib
To then run jupyter notebook from the environment, after you have installed all the things you want (and after having closed the command prompt or having deactivated the environment), you can do
conda activate testenv
jupyter notebook
in the command prompt.
Jupyterlab would usually use the environment inside which you launch it. For example:
If you activate my_env first and then do jupyter lab from terminal, it should detect the environment.
Incase it fails, go to Kernel -> Change Kernel and select the kernel you want to use.
Note: While creating a new kernel, I always use the display-name parameter which helps. You can do something like:
python -m ipykernel install --user --name my_env --display-name "Python (my_env)"
Hope this helps.
To fix this problem, you need to manually install this package in Anaconda.
How do I install? Open your Anaconda Prompt and run the below command:
conda install -c conda-forge scikit-learn
Then restart Jupyter Notebook and import this package.
I think the problem is that the environment is not activated. Try conda activate my_env first, and then type jupyter notebook.
The first thing you can do is:
import sys
print(sys.path)
Check if /path/to/anaconda/envs/my_env/lib/python3.7/site-packages exists in the path.
I find it useful to print the current sys.path so that I know where it is looking at.
conda info --envs
when testEnv exists:
conda activate testEnv
conda list scikit-learn

Can't run Python code in Visual Studio Code with Jupyter - "Jupyter kernel cannot be started from 'Python 3.6.8 64-bit"

I've installed Jupyter extension in the latest Visual Studio:
Visual Studio 1.3.01 64
Jupyter 1.1.4
As I am using tensorflow I need Python 3 64bit.
When I try to run simple code I get:
Jupyter kernel cannot be started from 'Python 3.6.8 64-bit ('tensorflow64': virtualenv)'. Using closest match Python 3.7.0 32-bit instead.
Code:
#%%
import tensorflow as tf
session = tf.Session()
hello = tf.constant("Hello from Milan.")
print(session.run(hello))
a = tf.constant(20)
b = tf.constant(22)
print('a + b = {0}'.format(session.run(a + b)))
All works fine if I Run code not using Jupyter from VS Code.
Message from Jupyter is not the best description of the issue, missing ipykernel package.
Fix was to install additional python package 'ipykernel' into virtual environment with Python 3.64 bit.
pip install ipykernel
Additional info:
https://github.com/Microsoft/vscode-python/issues/3579
conda create --name test_env
conda activate test_env
conda install -c anaconda ipykernel
python -m ipykernel install --user --name=test_env
Or this You can run this it might work
ipython kernel install --user --name=ENVNAME
Simple installation is the best. While learning Python et-all I had installed many things in many different ways. It was a mess. Removed everything and did a fresh install only I used pyenv and pipenv. However pipenv does not work with the latest version which you can install using brew. Solution revert back to version 2018.10.13 using pip install. Then I discovered that pipenv was no longer recommended so, I removed it. Built a test project and could not get jupyter to run. Spent all day trying everything.
Solution was, remove the old version of pipenv and install the latest using brew. All is well.
However, I do like the features of pyenv like setting global etc.
You have to select and save the .ipynb file with VSCode indicating a kernel that is reachable from your Jupyter installation:
In my case installing anaconda and call following comments solves it:
conda create -n tf tensorflow
conda activate tf

Jupyter can't find keras' module

I have installed Tensorflow and Keras by Anaconda (on Windows 10), I have created an environment where I am using Python 3.5.2 (the original one in Anaconda was Python 3.6).
When I try to execute import keras as ks, I get ModuleNotFoundError: No module named 'keras'.
I have tried to solve this issue by sys.path.append(C:\\Users\\ ... \\Anaconda3\\python.exe)
with both notebook and console, but I continue to get the same error.
How could I solve this issue?
Please try the following:
Run these in the jupyter notebook cell:
import sys
sys.path
sys.executable
It may not be pointing to your virtual environment but to the root
The fix is to install the jupyter notebook from inside your virtual environment
$ . your_env/bin/activate
(your_env)$ python -m pip install jupyter
Now you can import tensorflow or keras
Jupyter uses iPython under the hood, for python. So when you install Jupyter, it will also install iPython. There was one issue when I installed keras and Jupyter: I already have iPython installed in my root Anaconda environment. This is the output after I install Jupyter and keras:
In [2]: import sys; sys.path
Out[2]:
['/home/user/anaconda3/bin',
'/home/user/anaconda3/lib/python36.zip',
'/home/user/anaconda3/lib/python3.6',
'/home/user/.ipython']
Notice that even though I am inside my conda environment, it still looks for libraries in my root conda environment. And of course keras isn't there.
The step to fix is simply re-activate my environment, with:
source deactivate && source activate [my_env]
Then I am using a correct ipython:
Out[2]:
['/home/user/anaconda3/envs/ml3/bin',
'/home/user/anaconda3/envs/ml3/lib/python36.zip',
'/home/user/anaconda3/envs/ml3/lib/python3.6',
'/home/user/.ipython']
(Not an answer but some troubleshooting hints)
sys.path is not the path to your Python executable, but the path to the libraries.
Check where Keras is installed and check your sys.path. How exactly did you install Keras?
Try to execute the same command from the Python interpreter. Do you have the same issue?
How did you install Jupiter, is the sys.path visible from there the same as sys.path visible from your Python interpreter?
Do Jupiter and Keras use the same version of Python?
You might try to uninstall Jupiter and install it again, and hope that the new installation picks up the packages which are already installed. What could happen is that you have more than one Python installation and different libraries being installed to the different places. sys.path, when requested from different environments, might give you a hint if that's true.
The kernel in console and jupyter are not necessarily the same, and the problem might be that you are not on python 3.5.
python --version
should tell you what is running in the console, and in jupyter you should see it as a choice on starting a new notebook. For me, the information in
Using both Python 2.x and Python 3.x in IPython Notebook
was very helpful.
I have realized that I had two different Jupyter's directories, so I have manually deleted one on them. Finally, I have reinstalled Anaconda. Now Keras works properly.
If you are a windows/mac user who are working on Jupyter notebook “pip install keras” doesn't help you .Try the below steps.It was solved for me
1. In command prompt navigate to the “site packages” directory of your anaconda installed.
2. Now use “conda install tensorflow” and after “conda install keras”
3. Re-start your Jupyter notebook and run the packages.
Acually, I did this command pip install keras and sudo -H pip3 install keras and pip3 install keras. None of them worked. I added the following command and everything worked like a charm:
pip install Keras. Yes a capital 'K'
Here's how I solved this problem.
First, the diagnosis. When I run which python in a terminal window on my Mac (the same terminal I used to launch jupyter, I get /Users/myusername/.conda/envs/myenvname/bin/python, but when I run the same command from a terminal within Jupyter, I get /usr/bin/python. So Jupyter isn't using the correct python executable; the version it's using doesn't have any of my packages installed.
But which jupyter returns /usr/bin/jupyter; it's using a version of jupyter that isn't coming from within my conda environment. I ran conda install jupyter and now which jupyter returns /Users/myusername/.conda/envs/myenvname/bin/jupyter (for some reason I had to restart the terminal window for this to take effect.) Then if I relaunch jupyter notebook, the notebook is using the correct version of Python and I have access to all my installed conda packages. 👍
I had a similar problem.
I added the Conda environment as a new kernel.
First, install ipykernel:
conda install ipykernel
Next, create the kernet:
python -m ipykernel install --user --name tf-gpu --display-name "TensorFlow-GPU"
Now, when you run your notebook, change the kernel to the new one, to "TensorFlow-GPU" in this example.

Running Jupyter notebook in a virtualenv: installed sklearn module not available

I have installed a created a virtualenv machinelearn and installed a few python modules (pandas, scipy and sklearn) in that environment.
When I run jupyter notebook, I can import pandas and scipy in my notebooks - however, when I try to import sklearn, I get the following error message:
import sklearn
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-8fd979e02004> in <module>()
----> 1 import sklearn
ImportError: No module named 'sklearn'
I am able to import all modules, at the command line - so I know they have been successfully installed:
(machinelearn) me#yourbox:~/path/to/machinelearn$ python -c "import pandas, scipy, sklearn"
(machinelearn) me#yourbox:~/path/to/machinelearn$
How can I import sklearn in my jupyter notebook running in a virtualenv?
You probably have not installed jupyter / IPython in your virtualenv. Try the following:
python -c "import IPython"
and check that the jupyter command found in your $PATH is the one from the bin folder of your venv:
which jupyter
For windows users in a powershell console, you can use the following to check that the jupyter command in your $env:Path is the one from the Scripts folder of you venv:
get-command jupyter
Edit: if this is the problem, just run python -m pip install jupyter in your venv.
Edit 2: actually you might also need:
python -m ipykernel install --user --name=my-virtualenv-name
and then switch the kernel named "my-virtualenv-name" in the jupyter user interface.
Edit 3: maybe the --user flag in the last command is a bad idea:
python -m ipykernel install --name=my-virtualenv-name
Another approach to take is to have one global jupyter installation, but to point to different kernels to run as the backend.
That approach is outlined here in their docs:
http://help.pythonanywhere.com/pages/IPythonNotebookVirtualenvs
Copying below in case the link breaks:
You can use a virtualenv for your IPython notebook. Follow the following steps:
Install the ipython kernel module into your virtualenv
workon my-virtualenv-name # activate your virtualenv, if you haven't already
pip install ipykernel
Now run the kernel "self-install" script:
python -m ipykernel install --user --name=my-virtualenv-name
Replacing the --name parameter as appropriate.
You should now be able to see your kernel in the IPython notebook menu: Kernel -> Change kernel and be able so switch to it (you may need to refresh the page before it appears in the list). IPython will remember which kernel to use for that notebook from then on.
To use Jupyter notebook with virtual environment (using virtualenvwrapper) plus packages installed in that environment, follow steps below:
create a virtual environment
mkvirtualenv --no-site-packages --python=/your/python/path your_env_name
Activate the virtual environment
workon your_env_name
Install Jupyter and other packages
pip install jupyter, numpy
Add a new kernel to your Jupyter config
ipython kernel install --user --name=your_env_name
Done. You may now use Jupyter notebook under the virtual environment.
jupyter-notebook
Disclaimer: the question has been answered but is hidden in one of the replies. I googled and took sometime to find the right answer. So I just summarize it so someone having the same issue can easily follow.
Assuming that jupyter is installed on your machine, not on the virtual environtment.
Using a virtual environment with Jupyter notebook
VENV_NAME = "YOUR VIRTUAL ENV NAME"
1) virtualenv VENV_NAME
2) source venv/bin/activate
3) Add this package if not present: pip3 install ipykernel
4) Then execute this command: ipython kernel install --user --name=VENV_NAME
5) Now open up the Jupyter Notebook and in change kernel select VENV_NAME
6) To install a new package perform pip3 install <PACKAGE NAME> in your terminal and repeat step 4.
Hope it helps!
Solution without adding a new kernel globally!!
create a new virtual environment by
python3 -m virtualenv envname
Activate your enviroment and install jupyter in it by
pip install jupyter
One thing you have to make sure before installing jupyter is that you don't have following packages already installed in it.
ipykernel
ipython
ipython-genutils
ipywidgets
jupyter
jupyter-client
jupyter-console
jupyter-core
If you've previously installed them then first uninstall them by pip uninstall.
Install your desired packages in activated virtualenv and launch jupyter in it and voila!
Creation of virtualenv with python3 -m venv command
I had the same problem as yours.
In my case I had created the virtualenv with the command
python3 -m venv ./my_virtual_env --system-site-packages
The problem was I could not install jupyter inside the virtual environment as it was already in the system-site-package (when you try to install it, it tells you "Requirement already satisfied").
To install jupyter, (and in a first instance pip, that does not get installed neither in your virtual environment with this command) but still have access to system-site-package you can run :
python3 -m venv ./my_virtual_env
Activate you virtual environment, run pip3 install jupyter (and pip3 install pip) and then turn on the option include-system-site-packages in the file ./my_virtual_env/pyvenv.cfg.
After deactivation and reactivation of you environment, you will have access to system site-packages.
Creation of virtualenv with virtualenv command
Given this answer you can prevent the access to system site-packages by creating a file ./my_virtual_env/lib/python3.4/no-global-site-packages.txt,
and get the access back by removing it.
You can still install jupyter inside your virtual-environment if you have created your virtual env using:
python -m venv --system-site-packages path/to/my-venv
Simply do this:
activate-your-env
pip install -I jupyter
And you are now ready to go
jupyter notebook

Categories