We Keep Coding

Jupyter kernel is not linked to conda environment in Jupyter Lab

I know similar questions have been asked before but previous answers do not help.
The problem:
Although, I installed a kernel from an active conda environment, the conda environment uses the wrong python interpreter. I tried the following:
# 1. Activate my conda environment snowflakes
$ conda activate /opt/miniconda3/envs/snowflakes
# 2. Install another kernel that is connected to snowflakes after env is activated
$ python -m ipykernel install --user --name snowflakes --display-name snowflakes_2
# 3. Run jupyter-lab
$ jupyter-lab
# 4. Check path in jupyter notebook
sys.path
>>['/Users/user/Documents/Code/Python /PyCharm_Test',
'/Library/Frameworks/Python.framework/Versions/3.10/lib/python310.zip',
'/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10',
'/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload',
'',
'/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages']
# 5. The path should look like this:
>> ['/Users/user/Documents/Code/Python /PyCharm_Test/src',
'/Users/user/Documents/Code/Python /PyCharm_Test',
'/opt/miniconda3/envs/snowflakes/lib/python310.zip',
'/opt/miniconda3/envs/snowflakes/lib/python3.10',
'/opt/miniconda3/envs/snowflakes/lib/python3.10/lib-dynload',
'/opt/miniconda3/envs/snowflakes/lib/python3.10/site-packages']
I tried to reinstall ipykernel and jupyter-lab several times. Further, I tried to install kernels in various forms and I tied to start jupyter-lab from anaconda navigator. All without help.

I found a solution:
#1 install nb_conda_kernels in base environment and in conda environment of choice
#2 Run the following code in the activated conda environment
$ conda install --channel=conda-forge nb_conda_kernels
#3 Open jupyter-lab
$ jupyter-lab
Before I created Kernels that were still linked to base python. Now, I have a kernel for all miniconda3 conda environments. I don't know what went wrong from the start but it works now.

Jupyter kernel doesn't use Poetry environment

I'm trying to install a Jupyter kernel for my poetry environment, but it seems like the kernel gets my base conda environment. Here's what I'm trying:
poetry env list
>ENV_NAME-HASH-py3.9 (Activated)
poetry run which python
>/Users/myusername/Library/Caches/pypoetry/virtualenvs/ENV_NAME-HASH-py3.9/bin/python
poetry run ipython kernel install --name=ENV_NAME
>Installed kernelspec ENV_NAME in /Users/myusername/Library/Jupyter/kernels/ENV_NAME
Then if I open a Jupyter with this kernel I don't get the libraries that should be installed. Checking the Python version I get:
!which python
/Users/myusername/opt/anaconda3/bin/python
Any help appreaciated!

did you try to activate your environement first ? (using source {path to env}/bin/activate ? if so, can you try installing the ipykernel directly : poetry run python -m ipykernel install --user --name myname

Python packages not installed separately for different venv environments

TL;DR
Would like to run Jupyter notebooks with different python setups. Python packages always install globally. Don't understand why.\TL;DR
I would like to run Jupyter notebooks with different python setups using venv. See here for the official documentation.
Python 3.6 is already installed on my system
$ python --version
Python 3.6.1 :: Continuum Analytics, Inc.
Using the following commands I have created two virtual environments:
$ python3 -m venv --without-pip Documents/venv/test01
$ python3 -m venv --without-pip Documents/venv/test02
Following this guide I tried setting up different kernels for each notebook with
$ source activate test01
(Documents/venv/test02) $ python -m ipykernel install --user --name test01 --display-name test01
However, the second command failed with
/Users/dominik/Documents/venv/test02/bin/python: No module named ipykernel
So, I deactivated my venv and ran the same command outside the venv which succeeded
$ source deactivate test01
$ python -m ipykernel install --user --name test01 --display-name test01
$ python -m ipykernel install --user --name test02 --display-name test02
Inside my Jupyter notebook I can see the different kernels now:
new kernels available
Now I'm creating a new notebook using test01 kernel. Inside the notebook, I try to add a module which is not available in Python by default:
$ import mord
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-2-a00d777a7e47> in <module>()
----> 1 import mord
ModuleNotFoundError: No module named 'mord'
This is as expected. I then go about to installing that package into my test01 environment using pip
$ source activate Documents/venv/test01
(Documents/venv/test01) $ pip install mord
After restarting my test01 kernel the import error disappears - as expected. However - and now this is my question - when I import mord package on a test02 notebook, there is no import error either. Why is that?
I would expect that mord package was only installed for test01. However, it seems to be installed globally.
Looking at the venv folders it shows that nothing was added specifically to those projects
venv folders
The pyvenv.cfg file seems also unchanged.
home = /Users/dominik/anaconda/bin
include-system-site-packages = false
version = 3.6.1
Can anyone give me some pointers what I am doing wrong?

because you created virtualenv with --without-pip flag, there is no pip executable in the virtual environment, you used system pip to install package.
basically, virtual environment not involved in your setup, even though there are two kernel specs, they all created with virtualenv deactivated.
setup steps recommend: create virtualenv without --without-pip option; install ipykernel for each of your virtual environments, which means, install while virtualenv is activated; create kernel spec from corresponding virtualenv.

After georgexsh had pointed me in the right direction regarding the --without-pip flag, this is how I solved it in my case:
Create two virtual environments
$ python -m venv --without-pip Documents/venv/test01
$ python -m venv --without-pip Documents/venv/test02
I am still using the --without-pip flag, because the command otherwise produces an error message
The following steps are executed for each of the environments test01, test02:
Activate virtual environment and install pip manually
$ source Documents/venv/test01/bin/activate
(test01) $ curl https://bootstrap.pypa.io/get-pip.py | python
Install and run Jupyter inside the virtual environment
(test01) $ pip install jupyter
(test01) $ jupyter notebook
After having completed the above steps for both environments, I tested installing a package only in one of the environments
Activate the environment you want to install to first (if not activated already)
$ source Documents/venv/test01/bin/activate
(test01) $ pip install numpy
numpy package is now only available to the Jupyter version installed in test01 environment.
If found that installing a dedicated kernel per each environment was optional (for my purpose). Anyway, here are the steps to run in an activated environment:
(test01) $ pip install ipykernel
(test01) $ python -m ipykernel install --user --name test01 --display-name test01
Now within Jupyter you can also create notebooks with different kernel types.

Running Jupyter notebook in a virtualenv: installed sklearn module not available

I have installed a created a virtualenv machinelearn and installed a few python modules (pandas, scipy and sklearn) in that environment.
When I run jupyter notebook, I can import pandas and scipy in my notebooks - however, when I try to import sklearn, I get the following error message:
import sklearn
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-8fd979e02004> in <module>()
----> 1 import sklearn
ImportError: No module named 'sklearn'
I am able to import all modules, at the command line - so I know they have been successfully installed:
(machinelearn) me#yourbox:~/path/to/machinelearn$ python -c "import pandas, scipy, sklearn"
(machinelearn) me#yourbox:~/path/to/machinelearn$
How can I import sklearn in my jupyter notebook running in a virtualenv?

You probably have not installed jupyter / IPython in your virtualenv. Try the following:
python -c "import IPython"
and check that the jupyter command found in your $PATH is the one from the bin folder of your venv:
which jupyter
For windows users in a powershell console, you can use the following to check that the jupyter command in your $env:Path is the one from the Scripts folder of you venv:
get-command jupyter
Edit: if this is the problem, just run python -m pip install jupyter in your venv.
Edit 2: actually you might also need:
python -m ipykernel install --user --name=my-virtualenv-name
and then switch the kernel named "my-virtualenv-name" in the jupyter user interface.
Edit 3: maybe the --user flag in the last command is a bad idea:
python -m ipykernel install --name=my-virtualenv-name

Another approach to take is to have one global jupyter installation, but to point to different kernels to run as the backend.
That approach is outlined here in their docs:
http://help.pythonanywhere.com/pages/IPythonNotebookVirtualenvs
Copying below in case the link breaks:
You can use a virtualenv for your IPython notebook. Follow the following steps:
Install the ipython kernel module into your virtualenv
workon my-virtualenv-name # activate your virtualenv, if you haven't already
pip install ipykernel
Now run the kernel "self-install" script:
python -m ipykernel install --user --name=my-virtualenv-name
Replacing the --name parameter as appropriate.
You should now be able to see your kernel in the IPython notebook menu: Kernel -> Change kernel and be able so switch to it (you may need to refresh the page before it appears in the list). IPython will remember which kernel to use for that notebook from then on.

To use Jupyter notebook with virtual environment (using virtualenvwrapper) plus packages installed in that environment, follow steps below:
create a virtual environment
mkvirtualenv --no-site-packages --python=/your/python/path your_env_name
Activate the virtual environment
workon your_env_name
Install Jupyter and other packages
pip install jupyter, numpy
Add a new kernel to your Jupyter config
ipython kernel install --user --name=your_env_name
Done. You may now use Jupyter notebook under the virtual environment.
jupyter-notebook
Disclaimer: the question has been answered but is hidden in one of the replies. I googled and took sometime to find the right answer. So I just summarize it so someone having the same issue can easily follow.

Assuming that jupyter is installed on your machine, not on the virtual environtment.
Using a virtual environment with Jupyter notebook
VENV_NAME = "YOUR VIRTUAL ENV NAME"
1) virtualenv VENV_NAME
2) source venv/bin/activate
3) Add this package if not present: pip3 install ipykernel
4) Then execute this command: ipython kernel install --user --name=VENV_NAME
5) Now open up the Jupyter Notebook and in change kernel select VENV_NAME
6) To install a new package perform pip3 install <PACKAGE NAME> in your terminal and repeat step 4.
Hope it helps!

Solution without adding a new kernel globally!!
create a new virtual environment by
python3 -m virtualenv envname
Activate your enviroment and install jupyter in it by
pip install jupyter
One thing you have to make sure before installing jupyter is that you don't have following packages already installed in it.
ipykernel
ipython
ipython-genutils
ipywidgets
jupyter
jupyter-client
jupyter-console
jupyter-core
If you've previously installed them then first uninstall them by pip uninstall.
Install your desired packages in activated virtualenv and launch jupyter in it and voila!

Creation of virtualenv with python3 -m venv command
I had the same problem as yours.
In my case I had created the virtualenv with the command
python3 -m venv ./my_virtual_env --system-site-packages
The problem was I could not install jupyter inside the virtual environment as it was already in the system-site-package (when you try to install it, it tells you "Requirement already satisfied").
To install jupyter, (and in a first instance pip, that does not get installed neither in your virtual environment with this command) but still have access to system-site-package you can run :
python3 -m venv ./my_virtual_env
Activate you virtual environment, run pip3 install jupyter (and pip3 install pip) and then turn on the option include-system-site-packages in the file ./my_virtual_env/pyvenv.cfg.
After deactivation and reactivation of you environment, you will have access to system site-packages.
Creation of virtualenv with virtualenv command
Given this answer you can prevent the access to system site-packages by creating a file ./my_virtual_env/lib/python3.4/no-global-site-packages.txt,
and get the access back by removing it.

You can still install jupyter inside your virtual-environment if you have created your virtual env using:
python -m venv --system-site-packages path/to/my-venv
Simply do this:
activate-your-env
pip install -I jupyter
And you are now ready to go
jupyter notebook

Running Jupyter with Python 2 [duplicate]

I use IPython notebooks and would like to be able to select to create a 2.x or 3.x python notebook in IPython.
I initially had Anaconda. With Anaconda a global environment variable had to be changed to select what version of python you want and then IPython could be started. This is not what I was looking for so I uninstalled Anaconda and now have set up my own installation using MacPorts and PiP. It seems that I still have to use
port select --set python <python version>
to toggle between python 2.x and 3.x. which is no better than the anaconda solution.
Is there a way to select what version of python you want to use after you start an IPython notebook, preferably with my current MacPorts build?

The idea here is to install multiple ipython kernels. Here are instructions for anaconda. If you are not using anaconda, I recently added instructions using pure virtualenvs.
Anaconda >= 4.1.0
Since version 4.1.0, anaconda includes a special package nb_conda_kernels that detects conda environments with notebook kernels and automatically registers them. This makes using a new python version as easy as creating new conda environments:
conda create -n py27 python=2.7 ipykernel
conda create -n py36 python=3.6 ipykernel
After a restart of jupyter notebook, the new kernels are available over the graphical interface. Please note that new packages have to be explicitly installed into the new environments. The Managing environments section in conda's docs provides further information.
Manually registering kernels
Users who do not want to use nb_conda_kernels or still use older versions of anaconda can use the following steps to manually register ipython kernels.
configure the python2.7 environment:
conda create -n py27 python=2.7
conda activate py27
conda install notebook ipykernel
ipython kernel install --user
configure the python3.6 environment:
conda create -n py36 python=3.6
conda activate py36
conda install notebook ipykernel
ipython kernel install --user
After that you should be able to choose between python2
and python3 when creating a new notebook in the interface.
Additionally you can pass the --name and --display-name options to ipython kernel install if you want to change the names of your kernels. See ipython kernel install --help for more informations.

If you’re running Jupyter on Python 3, you can set up a Python 2 kernel like this:
python2 -m pip install ipykernel
python2 -m ipykernel install --user
http://ipython.readthedocs.io/en/stable/install/kernel_install.html

These instructions explain how to install a python2 and python3 kernel in separate virtual environments for non-anaconda users. If you are using anaconda, please find my other answer for a solution directly tailored to anaconda.
I assume that you already have jupyter notebook installed.
First make sure that you have a python2 and a python3 interpreter with pip available.
On ubuntu you would install these by:
sudo apt-get install python-dev python3-dev python-pip python3-pip
Next prepare and register the kernel environments
python -m pip install virtualenv --user
# configure python2 kernel
python -m virtualenv -p python2 ~/py2_kernel
source ~/py2_kernel/bin/activate
python -m pip install ipykernel
ipython kernel install --name py2 --user
deactivate
# configure python3 kernel
python -m virtualenv -p python3 ~/py3_kernel
source ~/py3_kernel/bin/activate
python -m pip install ipykernel
ipython kernel install --name py3 --user
deactivate
To make things easier, you may want to add shell aliases for the activation command to your shell config file. Depending on the system and shell you use, this can be e.g. ~/.bashrc, ~/.bash_profile or ~/.zshrc
alias kernel2='source ~/py2_kernel/bin/activate'
alias kernel3='source ~/py3_kernel/bin/activate'
After restarting your shell, you can now install new packages after activating the environment you want to use.
kernel2
python -m pip install <pkg-name>
deactivate
or
kernel3
python -m pip install <pkg-name>
deactivate

With a current version of the Notebook/Jupyter, you can create a Python3 kernel. After starting a new notebook application from the command line with Python 2 you should see an entry „Python 3“ in the dropdown menu „New“. This gives you a notebook that uses Python 3. So you can have two notebooks side-by-side with different Python versions.
The Details
Create this directory: mkdir -p ~/.ipython/kernels/python3
Create this file ~/.ipython/kernels/python3/kernel.json with this content:
{
"display_name": "IPython (Python 3)",
"language": "python",
"argv": [
"python3",
"-c", "from IPython.kernel.zmq.kernelapp import main; main()",
"-f", "{connection_file}"
],
"codemirror_mode": {
"version": 2,
"name": "ipython"
}
}
Restart the notebook server.
Select „Python 3“ from the dropdown menu „New“
Work with a Python 3 Notebook
Select „Python 2“ from the dropdown menu „New“
Work with a Python 2 Notebook

A solution is available that allows me to keep my MacPorts installation by configuring the Ipython kernelspec.
Requirements:
MacPorts is installed in the usual /opt directory
python 2.7 is installed through macports
python 3.4 is installed through macports
Ipython is installed for python 2.7
Ipython is installed for python 3.4
For python 2.x:
$ cd /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin
$ sudo ./ipython kernelspec install-self
For python 3.x:
$ cd /opt/local/Library/Frameworks/Python.framework/Versions/3.4/bin
$ sudo ./ipython kernelspec install-self
Now you can open an Ipython notebook and then choose a python 2.x or a python 3.x notebook.

From my Linux installation I did:
sudo ipython2 kernelspec install-self
And now my python 2 is back on the list.
Reference:
http://ipython.readthedocs.org/en/latest/install/kernel_install.html
UPDATE:
The method above is now deprecated and will be dropped in the future. The new method should be:
sudo ipython2 kernel install

Following are the steps to add the python2 kernel to jupyter notebook::
open a terminal and create a new python 2 environment: conda create -n py27 python=2.7
activate the environment: Linux source activate py27 or windows activate py27
install the kernel in the env: conda install notebook ipykernel
install the kernel for outside the env: ipython kernel install --user
close the env: source deactivate
Although a late answer hope someone finds it useful :p

Use sudo pip3 install jupyter for installing jupyter for python3 and sudo pip install jupyter for installing jupyter notebook for python2. Then, you can call ipython kernel install command to enable both types of notebook to choose from in jupyter notebook.

I looked at this excellent info and then wondered, since
i have python2, python3 and IPython all installed,
i have PyCharm installed,
PyCharm uses IPython for its Python Console,
if PyCharm would use
IPython-py2 when Menu>File>Settings>Project>Project Interpreter == py2 AND
IPython-py3 when Menu>File>Settings>Project>Project Interpreter == py3
ANSWER: Yes!
P.S. i have Python Launcher for Windows installed as well.

Under Windows 7 I had anaconda and anaconda3 installed.
I went into \Users\me\anaconda\Scripts and executed
sudo .\ipython kernelspec install-self
then I went into \Users\me\anaconda3\Scripts and executed
sudo .\ipython kernel install
(I got jupyter kernelspec install-self is DEPRECATED as of 4.0. You probably want 'ipython kernel install' to install the IPython kernelspec.)
After starting jupyter notebook (in anaconda3) I got a neat dropdown menu in the upper right corner under "New" letting me choose between Python 2 odr Python 3 kernels.

If you are running anaconda in virtual environment.
And when you create a new notebook but i's not showing to select the virtual environment kernel.
Then you have to set it into the ipykernel using the following command
$ pip install --user ipykernel
$ python -m ipykernel install --user --name=test2