In Jupyter Notebook, ipywidgets work fine, however they seem to not work in Jupyter Lab (which is supposedly better than Notebook).
I followed these directions.
Step 1: Installed Node.js (https://nodejs.org/en/)
Step 2: Installed requirements on Python 3 with conda:
conda install -c conda-forge ipywidgets
jupyter labextension install #jupyter-widgets/jupyterlab-manager
Unfortunately, basic widgets do not work inside the jupyter lab notebook:
JupyterLab now prefers a model where arbitrary javascript is no longer allowed to be embedded in a cell's output, which is how many interactive Jupyter Notebook modules used to work. They now ask that modules with interactivity create a JupyterLab extension.
ipywidgets provides #jupyter-widgets/jupyterlab-manager extension which satisfies this requirement.
When using ipywidgets 7.6 or newer in JupyterLab 3.0 or newer you do not need to do anything: it is installed by default. You can check if this is installed by running:
jupyter labextension list
which should include a line like:
#jupyter-widgets/jupyterlab-manager v3.0.0 enabled OK (python, jupyterlab_widgets)
If you are using JupyterLab 1 or 2 (or old version of ipywidgets) you need to install this extension manually by running this on your command line (which assumes you already have NodeJS installed):
jupyter labextension install #jupyter-widgets/jupyterlab-manager
I had the same pbm, and tried this solution (hope it can help others):
The jupyter labextension install #jupyter-widgets/jupyterlab-manager gave this kind of error in my case:
> /Users/user/.nvm/versions/node/v8.7.0/bin/npm pack #jupyter-widgets/jupyterlab-manager
jupyter-widgets-jupyterlab-manager-0.35.0.tgz
Errored, use --debug for full output:
ValueError:
"#jupyter-widgets/jupyterlab-manager#0.35.0" is not compatible with the current JupyterLab
Conflicting Dependencies:
JupyterLab Extension Package
>=0.15.4-0 <0.16.0-0 >=0.16.0-0 <0.17.0-0 #jupyterlab/application
>=1.1.4-0 <2.0.0-0 >=2.0.0-0 <3.0.0-0 #jupyterlab/services
>=0.15.4-0 <0.16.0-0 >=0.16.0-0 <0.17.0-0 #jupyterlab/rendermime
>=0.15.4-0 <0.16.0-0 >=0.16.0-0 <0.17.0-0 #jupyterlab/notebook
Then, what I did is to use a previous version 0.34 instead of 0.35:
jupyter labextension install #jupyter-widgets/jupyterlab-manager#0.34
In fact, according to this, sometime teams get time to consider the last version.
UP (according to comments):
You can check jupyter lab --version and find match on its version compatibility.
And it works now !
Had the same issue, and what worked for me today was running the 'clean' command, as mentioned here: https://ipywidgets.readthedocs.io/en/latest/user_install.html#installing-the-jupyterlab-extension
So:
jupyter lab clean
jupyter labextension install #jupyter-widgets/jupyterlab-manager
And that got it working right for me just now.
Note: most of the above answers are outdated (as of july 19 2021). It should be a lot more seamless with the latest versions of these packages. However, in 2021 I was having an issue rendering panel widgets in jupyterlab hosted on jupyterhub and came across this post. I tried a few of these answers at first but none of them worked. After digging into some of the libraries, I found the following:
jupyter labextension install #jupyter-widgets/jupyterlab-manager is no longer required as of ipywidgets==7.6 and jupyterlab>=3.0. source. If you're on the latest version of jupyterlab, you should just need to install ipywidgets and the extension will be enabled automatically as long as widget extension authors also follow some steps.
The main change in this release is that installing ipywidgets 7.6.0 will now automatically enable ipywidgets support in JupyterLab 3.0—a user has no extra JupyterLab installation step and no rebuild of JupyterLab, nor do they need Node.js installed. Simply install the python ipywidgets package with pip (pip install ipywidgets==7.6.0) or conda/mamba (conda install -c conda-forge ipywidgets=7.6.0) and ipywidgets will automatically work in classic Jupyter Notebook and in JupyterLab 3.0.
panel has also followed suit and bundled the necessary extensions in the pyviz_comms package, so theoretically everything should have been included for my case. source
In the classic Jupyter notebook environment and JupyterLab, first make sure to load the pn.extension(). Panel objects will then render themselves if they are the last item in a notebook cell. For versions of jupyterlab>=3.0 the necessary extension is automatically bundled in the pyviz_comms package, which must be >=2.0.
The answer to my problem turned out to be that I was installing panel after building the jupyterlab server through jupyterhub. (e.g. for a specific notebook, a user was running !pip install panel). This is where I get a little fuzzy since I'm not sure why this doesn't work, given that ipywidgets is already installed and panel install includes the pre-built jupyterlab extension. However, I was able to fix my issue by instead using an a jupyterlab image to spawn from jupyterhub which had ipywidgets>=7.6 and panel>=0.11.3 (and therefore pyviz_comms>=2.0) pre-installed. After this, panel widgets in jupyterlab on jupyterhub now work. Hope this helps anybody having a similar issue.
I have the same issue as #jtlz2 that none of the above suggestion works for me except #hainm's very specific version combination for jupyter-lab, ipywidgests, and jupyter-widgets/jupyterlab-manager.
Following the initial version numbers listed in this post (https://github.com/jupyter-widgets/ipywidgets/issues/2488#issuecomment-509719214), I tried to find the most updated version combination that works. I list them below, such that one can try it if there is really nothing else working for him/her.
pythonversion=3.8.0
labversion=2.1.5
labmanagerversion=2.0
ipywidgetsversion=7.5.1
nodejsversion=10.13.0
conda create -n lab python=$pythonversion -y
source activate lab
conda install nodejs=$nodejsversion -c conda-forge -y
conda install ipywidgets=$ipywidgetsversion -c conda-forge -y
conda install jupyterlab=$labversion -y -c conda-forge
jupyter-labextension install #jupyter-widgets/jupyterlab-manager#$labmanagerversion
It seems that the version of nodejs plays a key role. Holing everything else equal, if I update nodejsversion to 12.x+ or the latest 14.x, this combination as well as #hainm's combination both fail to make ipywidgets behave normally in Jupyterlab.
Other than the one I listed in the above code cell, below 6 combinations also work for me.
(pythonversion ,labversion ,labmanagerversion ,ipywidgets ,nodejsversion)
(3.7 , 0.34 , 0.37 , 7.4.2 , 10.13)
(3.7 , 1.0 , 1.0 , 7.4.2 , 10.13)
(3.7 , 2.0 , 2.0 , 7.4.2 , 10.13)
(3.8 , 2.0 , 2.0 , 7.4.2 , 10.13)
(3.8 , 2.0 , 2.0 , 7.5.1 , 10.13)
(3.8 , 2.1.5 , 2.0 , 7.5.1 , 10.13)
I was getting a Permission Denied error, so adding sudo to the accepted command helped: sudo jupyter labextension install #jupyter-widgets/jupyterlab-manager.
According to ipywidgets.readthedocs.io documentation (Installing the JupyterLab Extension), for "JupyterLab" do the following steps in "Anaconda Prompt".
Step 1
conda install -c conda-forge nodejs
Step 2
jupyter labextension install #jupyter-widgets/jupyterlab-manager
None of the other answers worked to me. It all seems to be down to version compatibility. Finally got it working - see below.
All credit to #hainm on github (original link: https://github.com/jupyter-widgets/ipywidgets/issues/2488#issuecomment-509719214)
My SO answer on this: https://stackoverflow.com/a/60059786/1021819
Here is a straight copy of that answer:
Leveraging https://github.com/jupyter-widgets/ipywidgets/issues/2488#issuecomment-509719214, in a jupyterlab terminal - running on jupyterhub - execute:
pythonversion=3.7
labversion=0.34.12
labmanagerversion=0.37.4
ipywidgetsversion=7.4.2
conda install ipywidgets=$ipywidgetsversion -c conda-forge -y --override-channels -c main
conda install jupyterlab=$labversion -y -c conda-forge --override-channels -c main
jupyter-labextension install #jupyter-widgets/jupyterlab-manager#$labmanagerversion
At this point a jupyter lab clean; jupyter lab build might be of interest.
Then in a .ipynb notebook running in the same jupyterlab window, hit the restart kernel button.
IMPORTANT: Don't forget to also REFRESH the browser page - or all efforts will have been in vain . :\
Then execute the example:
from ipywidgets import interact
#interact(x=(0, 100, 10))
def p(x=50):
pass
I never thought I would live to see the day but - hey presto - the widget finally appears!
The sad things are that the setup is extremely sensitive to the installation of other extensions and the combination of compatible versions is very specific.
Besides everything else, make sure that your ipywidgets are installed in the same conda environment as jupyterlab.
Here is my story (ipywidgets 7.7.0, jupyterlab 3.2.5 installed via conda inside docker):
Running dockerized jupyterlab in base conda environment
Created new conda environment, install ipykernel, make it visible to jupyterlab, install ipywidgets
Switch to new kernel, now jupyterlab recognized ipywidgets as installed, but widgets are not working (only displays text widget description), which is confusing
Solution:
Install ipywidgets in the same conda environment as jupyterlab
Disclaimer: My answer may not be applicable to the original question, since the author did not use separate conda environment. But it can be useful for others facing similar problem.
If you're on linux and you'd rather avoid conda entirely, and use virtual envs (venvs) to keep python happy, AND you happen to be using an 'older'/LTS Debian based OS, which may not have upto date nodejs: Ie, Ubuntu 16.04 LTS, which doesn't have a node but rather nodejs (node belongs to another package, and the 'legacy nodejs' version is too old), then read on.
This is a little more complicated to setup, but much easier to maintain long-term than conda is. (you can always just mk a new venv for a new project, without breaking your old projects).
Main points are:
use PPA's to get fresh versions of the things you need
Use virtualenvwrapper so you can:
use up-to-date python3
avoid messing up your 'pip install''s
avoid getting the system package manager confused
also easily work with people with different versions of python
follow python Best Practise
easily have different venvs which might have old or incompatible python and pip packages.
Use Nodejs binary distributions
need node.js version "10.x" for jupyterlab widgets
allows using the system's package manager to keep nodejs fresh
will be maintained and available as long as LTS's are
So, all actual steps (these were tested to work on Linux Mint 18.3 Sylvia, which is basically compatible with ubuntu xenial aka Ubuntu 16.04 LTS. Differences will arise mostly in nodejs, read the readme in the github link above to solve for other OS):
Get an admin to do (or do yourself if you can sudo):
sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install -y python3.8 python3.8-dev python3.8-distutils python3-pip python3-venv
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
sudo apt-get install -y nodejs
Then, as your own user, you can complete the rest of the steps:
pip3 install --user virtualenv virtualenvwrapper
mkdir ~/.envs
You'll then want to add the following to the end of your .bashrc :
export PATH=~/.local/bin:$PATH
export WORKON_HOME=~/.envs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source ~/.local/bin/virtualenvwrapper.sh
At this point, make a new shell, and you will be able to run the rest of the setup, actually installing jupyterlab:
mkvirtualenv -p python3.8 jupenv
pip install jupyter matplotlib pandas ipympl tqdm
jupyter labextension install #jupyter-widgets/jupyterlab-manager
Now you're done.
To open/use jupyter, you want (because of the venv I've called jupenv above, you can name it as you like in that mkvirtualenv line):
workon jupenv
jupyter lab
Otherwise, I had no end of hell trying to get nodejs to work with outdated ubuntu packages. Sometimes it would work, for a few restarts, and then fail. Other times it would just keep giving me the same missing widgets, or sometimes little lines of junk js code.
Virtualenvs are well worth using, especially when you start using python seriously, and working with others who may use different versions / different sets of pip packages. VirtualEnvWrapper makes this pretty painless.
The basic point is that everything you 'pip install', even jupyter, ends up being kept cleanly separate (and separate from the system packages), which keeps everything working very nicely.
There are some basic DO's and DON'T's:
DON'T run pip install ... lines without being in a venv
DON'T use pip3 in place of pip within a venv.
DO just use python and not python3 to run within.
DON'T use conda...!
DO know that you can have all your virtualenvs updated at once with:
allvirtualenv pip install -U pip
As for the nodejs binary distribution packages: These are highly recommended where they support your particular OS. They'll be very up-to-date and should present the minimum of trouble.
Please check steps to make it work in Jupyter 3.0 docs
Please check steps to make it work in Jupyter 1 0r 2 docs
After this restart or build the lab to see the changes
Related
When I try to export my Jupyter Notebook in pdf format in VSCode like this:
then I got this error:
and jupyter output panel says:
so i tried to install MikTeX and update the required packages, but still I can't export Jupyter Notebooks in PDF format by VSCode!
how can I fix this problem?
Note That I know i can do it by convert it to HTML and then with ctrl+p try to save it as pdf! but I want to convert it to pdf in straight way!
Since I'm using conda venvs, I did these steps:
Activate conda venv using: conda activate <NAME_OF_VENV> in Anaconda prompt.
Install nbconvert using conda install -c anaconda nbconvert
Now it's all okay, and I can export Jupyter notebooks in HTML and PDF format both.
Update
Also, you should consider that nbconvert is compatible with Python 3.7-3.9 based on the official doc.
You can try following URL. Hope it will solve your issue
https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_export-your-jupyter-notebook
I just tried in Linux(Ubuntu 20.04) and it worked for me
You can follow this steps:
sudo apt-get install texlive-xetex texlive-fonts-recommended texlive-latex-recommended
Active env where you have jupyter installed
Execute this command: jupyter nbconvert --to pdf your_file.ipynb
Output:
While this question has been answered twice neither one of the answers seemed to address the question on OP's OS. I wanted to setup the same thing for a more streamlined workflow on Windows Jupyter development.
Unfortunately, I have been unabled to get the export as PDF button to work due to not knowing how to get VS Code to search the correct locations for the various dependencies; however, I have found a work around that allows one to export as PDF on Windows through WSL (Windows Subsystem for Linux).
WSL is a tool that I have grown to love using during all my development both professionally and within school, so I am sure you can find great use for it outside of PDF export if you do not already have it installed.
This method primarily uses the command line method for exporting using jupyter nbconvert --to pdf <filename>
This can also be used in Windows powershell; however, I have had little luck setting up the dependencies in powershell.
In WSL you can run this command and use the friendly neighbourhood Linux command line to install all the required dependencies.
Here is the documentation on how to setup nbconvert for Linux which I also found to work on WSL (https://nbconvert.readthedocs.io/en/latest/install.html)
If there are any other dependencies missing it is typically fairly easy to figure out how to install these through the command line.
If you are using arch linux or any child distro of arch,
❯ sudo pacman -S texlive-latexextra texlive-core
That fixed it for me
Using anaconda for virtual environments on linux (Ubuntu 21.10), I needed to install these packages within the venv:
conda activate <NAME_OF_VENV>
pip install notebook
conda install nbconvert
conda install pandoc
conda deactivate
I already had texlive installed, but I was missing the xetex packages:
sudo apt install texlive-xetex
These were all required for me, as stated by the error messages given by VSCode
I am using a Debian distribution installed in Windows Subsystem Linux 2 with a Conda environment with a 3.7 Python version. I found the following error in the jupyter output while exporting a jupyter file:
! LaTeX Error: File `ucharcat.sty' not found
I solved the issue just by upgrading the Debian distribution from Debian 9 (strecht) to bullseye and re-installing latex in the upgraded distribution.
I found that the Windows Store installs Debian 9 (still in 2022) by default and therefore the LaTex version installed with the apt tool package dates from 2016. Perhaps there is a compatibility issue with Python 3.7 and the nbconvert package used to export to pdf files.
The code I used to
Check my Linux, and LaTex versions
Upgrade the Debian WSL distro
Uninstall and re-install LaTex
can be found in the following link
I hope this answer helps.
I met the same problem... But I fixed it by simply using pip install jupyter_contrib_nbextensions according to the output panel (and I already have texlive installed in my win environment b
I am trying to install the jupyterlab plotly extension with this command (according to https://plotly.com/python/getting-started/):
jupyter labextension install jupyterlab-plotly#4.14.3
I get this error:
An error occured.
ValueError: Please install Node.js and npm before continuing installation. You may be able to install Node.js from your package manager, from conda, or directly from the Node.js website (https://nodejs.org).
See the log file for details: /tmp/jupyterlab-debug-epx8b4n6.log
I didn't install Node.js on system level, but in a virtual environment using pip. Pip list shows both nodejs 0.1.1 and npm 0.1.1 . I am also using ipywidgets in jupyterlab, which requires nodejs and it is working fine.
So I have two questions:
How to use plotlywidgets with pip nodejs in a virtual environment?
What's the difference between pip nodejs and system level Node.js
No, you have not installed node.js. You installed some kind of Python bindings for node (python-nodejs, with its repository archived by the author) which itself require an actual nodejs. It is dangerous to install stuff from PyPI without checking what you are installing. It could have been a malicious code - you shouldn't just type a name after pip install and hope that it installs what you think. It's the same for your npm installation (package comes from the same author); both were not updated in the last 6 years and may contain some vulnerabilities so I would uninstall those quickly ;)
It can be seen immediately from the version number that something is wrong because the current nodejs versions are generally >10, (with exact version depending on your JupyterLab version, i.e. either 10 or 12; 14 might work too).
First install nodejs latest version
conda install nodejs -c conda-forge --repodata-fn=repodata.json
Then install jupyterlab extension:
jupyter labextension install jupyterlab-plotly#4.14.3
Then RESTART JUPYTER LAB
(Using latest miniconda + VS Code on 64-bit Windows10:) After clean reinstalling VS Code, I can no longer launch jupyter notebook from within it. When I tried to create a new jupyter file for the first time, the Python extension installed ipykernel in my virtual environment "da38" (my main working environment). Then it stays on Connecting to IPython kernel: Connecting to kernel for an unusually long time, and stops with the error message Unable to start session for kernel Python 3.8.5 64-bit ('da38':conda) (images pasted below). I also removed and re-created da38 environment just in case. Used jupyter many times before with no issues until this new VS Code install today, and an identical setup is working on my other computer. Any help is appreciated. Thanks!
I had the latest version and below command worked for me
python -m pip install 'traitlets==4.3.3' --force-reinstall
This issue also occurs on my computer. The solution is to restore the version number of a dependency package "traitlets" of ipython kernel to 4.3.3.
You could try to use "conda list" to view the version of the module traitlets in the current conda environment. If it shows version 5.0, it is recommended that you use version 4.3.3.
Reference: Unable to start session for kernel Python.
I had this same problem. What worked for me is updating ipykernel.
$ pip install ipykernel --upgrade
I also did a few other upgrades just in case.
$ pip install traitlets --upgrade
$ pip install notebook --upgrade
I did not role back to an older version like 'traitlets==4.3.3'.
If anyone facing same issues now, Can check below steps which helped me to solve this
Update and Restart VS code
Install latest Jupyter Extension from Extensions.
We need to update "ipykernel" and restart VSC:
pip install ipykernel --upgrade
The following code does not render in Jupyter lab:
%matplotlib widget
import plotly.express as px
import numpy as np
import pandas as pd
df = pd.DataFrame(np.random.randint(0,100,size=(5, 4)), columns=list('ABCD'))
px.bar(df, x='A', y='B')
I have tried to install all the dependencies and extensions mentioned in here
https://plot.ly/python/getting-started/#jupyterlab-support-python-35
but also the steps in here
https://github.com/matplotlib/jupyter-matplotlib
Nothing worked
Here is my set up:
jupyter lab --version
1.0.2
python --version
Python 3.6.1 :: Continuum Analytics, Inc.
conda list jupyterlab
# packages in environment at C:\Users\***\Anaconda3:
#
# Name Version Build Channel
jupyterlab 1.0.2 py36hf63ae98_0
jupyterlab_launcher 0.13.1 py36_0
jupyterlab_server 1.0.0 py_0
conda list nodejs
# packages in environment at C:\Users\***\Anaconda3:
#
# Name Version Build Channel
nodejs 0.1.1 pypi_0 pypi
conda list plotly
# packages in environment at C:\Users\***\Anaconda3:
#
# Name Version Build Channel
plotly 4.1.0 pypi_0 pypi
plotly-express 0.4.1 pypi_0 pypi
EDIT:
jupyter-labextension list
JupyterLab v1.0.2
Known labextensions:
app dir: C:\Users\***\Anaconda3\share\jupyter\lab
#jupyter-widgets/jupyterlab-manager v1.0.2 enabled ok
#jupyterlab/git v0.8.0 enabled ok
#jupyterlab/plotly-extension v1.0.0 enabled ok
jupyter-matplotlib v0.4.2 enabled ok
jupyterlab-chart-editor v1.2.0 enabled ok
jupyterlab-plotly v1.1.0 enabled ok
plotlywidget v1.1.0 enabled ok
EDIT: these instructions and more are now in our official Troubleshooting Guide!
It could be that remnants of previous installations or attempts at installation are causing issues. I recommend either starting with a clean install or uninstalling all Plotly modules (from both pip and conda!) and plotly-related jlab extensions, and then following the instructions here: https://plot.ly/python/getting-started/
Uninstalling the module is a matter of
conda uninstall plotly
pip uninstall plotly
And then reinstalling with one or the other but not both, according to the instructions linked above.
Uninstalling JupyterLab extensions is performed with
jupyter labextension uninstall #jupyterlab/plotly-extension
jupyter labextension uninstall jupyterlab-plotly
jupyter labextension uninstall plotlywidget
Following the official plotly.py repo https://github.com/plotly/plotly.py, for correct rendering of plotly in JupyterLab there's a need to installing the special extension by command
jupyter labextension install jupyterlab-plotly#4.14.3
I ran into the same problem but with a different cause and requiring a different solution. Just thought I'd share it for anyone encountering the same issue.
I'm running jupyterlab in a Docker container which did not yet have nodejs or npm installed.
I was unable to install the required extension via:
jupyter labextension install jupyterlab-plotly
Because it gave me this error:
ValueError: Please install nodejs and npm before continuing installation. nodejs may be installed using conda or directly from the nodejs website.
Conda was not available on the container and when installing node and npm via the jupyterlab terminal (through pip or apt-get) I got the same error, or a version mismatch (when using apt-get the nodejs version I got was too old).
The following steps helped me solve the problem.
Install nvm in the docker container when building the container, thus in the Dockerfile:
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
Mind the version number, you might want to change that to whatever is the latest stable version
Make the nvm command available by loading some included init scripts:
SHELL ["bash", "-lc"] <-- Only necessary if your container is not using bash as shell already
RUN export NVM_DIR="$HOME/.nvm"
RUN [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
RUN [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
Install a specific nodejs version via nvm:
RUN nvm install 14.17.0
Mind the version number again, change to whatever version you need.
Install the jupyter extension:
RUN jupyter labextension install jupyterlab-plotly
Restart the kernel and happy plotting ;)
You might also consider installing conda and then nodejs via conda if that makes sense for your use case. I have not tested if that works though.
Try installing jupyterlab dash, it worked for me!
you can do it through the jupyterlab menu or by following these instructions.
https://github.com/plotly/jupyter-dash
It looks like you should upgrade your plotly as well, because plotly express is now part of plotly i.e.
import plotly.express as px
For anyone still struggling to make this work – these steps worked for me:
If you have jupyter lab running – shut down your session.
Uninstall plotly and all other related packages like #nicolaskruchten recommends here.
Make a fresh install of plotly and jupyter-dash with EITHER conda or pip, but do not mix between both package managers.
Execute jupyter lab from terminal again. Now you should see a window Build recommended – jupyterlab-dash needs to be included in build. Confirm this.
After the successful build close the jupyter-lab session and initiate again.
I'm trying to use Jupyter notebook on a fresh install of Ubuntu 19.04. I've run through the usual installation steps, but Jupyter does not work. When I try to open a notebook, I get
500 : Internal Server Error
When I look in the terminal, I see that Jupyter has encountered the error
ImportError: cannot import name ConverterMapping
I've searched, and found questions/answers for other causes of the internal server error in Jupyter, but I cannot find a way to fix the ConverterMapping issue. I have tried uninstalling (pip uninstall ...) and reinstalling (pip install --user ...) all Jypyter-related packages, but it had no effect.
Update:
After routine system updates, the problem has returned. This indicates to me that deleting the .local directory of pip installed packages (and then reinstalling) is not necessarily a long-term fix, since it seems like automatic updates can cause the problem to re-emerge. It's also possible that some other package (installed via apt), while not explicitly relating to python, has altered the python environment in some way that breaks Jupyter notebooks.
Any ideas or suggestions would be greatly appreciated.
(answering my own question)
In my experience, these problems are usually caused by a compatibility issues between python packages installed via apt, and those installed via pip.
Unfortunately, I could not reconstruct the lengthy series of steps taken to install the jupyter/scipy/etc environment. Some packages must have been pulled in via apt and others via pip, leading to version conflicts.
This approach fixed the problem initially:
Following this answer, I ran python -c "import site; print(site.USER_BASE)" to find where pip installs --user packages
For me, this was ~/.local
I moved ~/.local to ~/.local_disabled to eliminate all locally installed packages
At this point, Jupyter worked again, (it must have been installed with the system python via apt).
I've added only a minimal subset of user packages, and will keep an eye out for incompatibilities.
I imagine avoiding pulling in Jupyter/ipython via apt might be a better way to avoid this problem in the future.
After subsequent system updates, the issue returned
This seems to confirm that the problem arises due to a mis-match between python packages installed via apt and those installed via pip. This time, to fix the issue, I:
Uninstalled python packages that were installed via apt (but not part of the core system installed with Ubuntu). This mainly included jupyter, and many of its dependencies.
Reinstalled Jupyter using pip install jupyter --user.
This is working (for now).
( Please feel free to edit/comment/extend this answer. )
The safest solution to this is to create a virtual environment and run your jupyter from it. This clearly separates all the operations of the pip and apt. It worked for me. I use Ubuntu
To create a virtual environment, run :
1)sudo pip install --upgrade virtualenv (install virtualenv)
2)virtualenv xyz --python=python3.7 (xyz is the name of the new virtual environment)
3) cd into the directory of xyz
4) source ./bin/activate
5)pip install jupyter
I had a similar experience and found Mrule's answer. The deleting/renaming .local worked for me too but his long term solution didn't sadly. So I dug a little deep to find why the first solution worked.
Turns out which jupyter returned $HOME/.local/bin/jupyter when .local was not deleted and /usr/bin/jupyter when I did. So the problem was in the pip packages (the ones installed in .local by the pip install --user jupyter option).
I had previously did sudo pip uninstall jupyter several times before, but that doesn't remove the packages in .local (see here). You have to do pip uninstall jupyter without sudo to uninstall those in the home directory (or manually delete them). I uninstalled any other jupyter related packages in my system by finding them using pip list | grep jupyter and apt list | grep jupyter.
And then finally did a fresh install of jupyter via sudo apt install jupyter. And now it seems to be working.
Bottom line is that the packages installed via pip system wide and in the home directory (.local) and that installed via apt were conflicting somehow (I couldn't find exactly why).