I followed a guide about setting up a pyenv workflow, it includes an IPython Startup Script that modifies the PYTHONPATH when using Jupyter. Now I don't want to use pyenv, but even after uninstalling everything in this guide. This undesirable script behavior remains.
I followed this guide https://medium.com/#henriquebastos/the-definitive-guide-to-setup-my-python-workspace-628d68552e14.
It has a separate virtual environment for Jupyter that the other virtual environments use. It includes an IPython Startup Script that runs and loads the virtualenv's site-packages into the Jupyter Notebook environment.
I no longer want to use pyenv. I am using MacOSX. I used Homebrew to install pyenv, pyenv-virtualenv, pyenv-virtualenvwrapper and so I removed all of those using 'brew uninstall'. I removed any trace of pyenv, jupyter, ipython, and python from /usr/local/bin, /usr/local/lib, and /usr/local/share, and ~/.
I then installed python using homebrew again, and ran pip3 install jupyter. I ran jupyter notebook, and it is still trying to run the kernel from this non-existent location '/Users/dhemming/.pyenv/versions/jupyter3/bin/python'.
I really don't think I understand that script or the Python Path and I'm really hoping someone can explain what is going on.
When following the guide I did this:
create jupyter3 virtual env (along with other virtual environments):
pyenv virtualenv 3.6.0 jupyter3
Install jupyter
pyenv activate jupyter3
pip install jupyter
python -m ipykernel install --user
pyenv deactivate
Set pyenv global:
pyenv global 3.6.0 2.7.13 jupyter3 ipython2 tools3 tools2
Check which jupyter:
~$ pyenv which jupyter
/Users/dhemming/.pyenv/versions/jupyter3/bin/jupyter
Install the IPython Script:
ipython profile create
curl -L http://hbn.link/hb-ipython-startup-script > ~/.ipython/profile_default/startup/00-venv-sitepackages.py
Here is the script that that curl command fetches:
import os
import sys
from warnings import warn
virtualenv = os.environ.get('VIRTUAL_ENV')
if virtualenv:
version = os.listdir(os.path.join(virtualenv, 'lib'))[0]
site_packages = os.path.join(virtualenv, 'lib', version, 'site-packages')
lib_dynload = os.path.join(virtualenv, 'lib', version, 'lib-dynload')
if not (os.path.exists(site_packages) and os.path.exists(lib_dynload)):
msg = 'Virtualenv site-packages discovery went wrong for %r' % repr([site_packages, lib_dynload])
warn(msg)
sys.path.insert(0, site_packages)
sys.path.insert(1, lib_dynload)
Upon completing the guide everything worked fine and whenever I created a new virtual environment using pyenv it used the jupyter3 virtual environment to run the notebook loaded with the new virtual environments libraries etc.
Then I no longer wanted this setup. So I remove every trace of python and anything related to python and I install a fresh python using Homebrew and do a pip3 install jupyter. After that I run jupyter notebook and I still get this:
Failed to run the command:
['/Users/dhemming/.pyenv/versions/jupyter3/bin/python', '-m', 'ipykernel_launcher', '-f', '/Users/dhemming/Library/Jupyter/runtime/kernel-1d721ce4-1619-498d-9f0b-62b98b12d0ac.json']
PATH='/usr/local/bin:/Users/dhemming/.rbenv/shims:/usr/local/opt/openssl/bin:/Users/dhemming/.nvm/versions/node/v8.11.2/bin:/Users/dhemming/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin'
with kwargs:
{'stdin': -1, 'stdout': None, 'stderr': None, 'cwd': '/Users/dhemming/workspace/tmp/tmp_01', 'start_new_session': True}
Where is that '/Users/dhemming/.pyenv/versions/jupyter3/bin/python' coming from?
I ended up deleting ~/Library/Jupyter and that fixed the problem. However I'm sure that there is a less extreme method of fixing this problem if someone knows.
Related
Update update: json_lines is not supported by python versions < 3 - my issue had pretty much nothing to do with environments. I am now using 3.9.1 and all is gucci.
Update: After using which python in my jupyter notebook and in my Terminal, I see that they are both using the same environment. As such I am still at a loss as to why my notebook cannot find json_lines.
I have two python environments on my computer, a default one and one I have for running my jupyter notebook on. I am trying to install the library, json_lines to the latter environment. I am not used the Anaconda environment manager.
On my Mac's Terminal I used the general pip install command pip install json-lines, but when I try to execute the following line of Python import json_lines in my notebook, I still receive the following error ImportError: No module named json_lines.
As I suspect I am not installing to the correct environment, I tried installing the library from inside my notebook with the following, import sys; !{sys.executable} -m pip install jsonlines.
However, this has not changed my dilemma.
Is there some way I can specify from my Terminal which environment to install to? or is it likely I am encountering a different issue to what I suspect?
The package for json_lines in pip in json-lines. Hence you could install it as:
$ pip install json-lines
It may be appropriate to use an isolated python environment for your particular project if you want to use particular conda libraries but without the whole package. In this instance, you would be able to use virtualenv. This will allow you to create an isolated python environment.
$ pip3 install virtualenv
You can call virtualenv to create a virtual python environment with the working name e.g. myvenv.
$ virtualenv myvenv
From here, you can set your terminal to use this python version. If you are on *nix:
$ which python
/usr/bin/python
$ source myvenv/bin/activate
(myvenv)$ which python
/.../myvenv/bin/python
This article can help you out.
https://janakiev.com/blog/jupyter-virtual-envs/
You need to create a virtualenv which will be used by your notebooks.
I am at a Python boot camp this weekend but I have not been able to even use Python on my computer because of this issue. All my instructors are stumped too.
The issue is that I get the ModuleNotFoundError on Jupyter with multiple different packages, including Pandas and Requests (but oddly enough, BeautifulSoup and CSV work fine.)
Here is how I start a new Jupyter file:
Create a new directory
Install jupyter and pandas with this command: pipenv install jupyter pandas
Activate virtual environment: pipenv shell
Launch Jupyter: jupyter notebook
Create new Python 3 notebook
At this point, I try a command like import pandas as pd and get back the ModuleNotFoundError.
I am using Python version 3.6.5.
Attempts to fix this that have failed:
double-checked that pandas is installed in my virtual environment with pip graph
created completely new directory
pipenv install jupyter pandas --skip-lock
Uninstalled everything system-wide with these commands:
pip freeze > requirements.txt
pip uninstall -r requirements.txt -y
Updated pandas
Used virtualenv instead of pipenv
virtualenv first-python-notebook
cd first-python-notebook
cd Scripts
activate
cd ..
pip install jupyter pandas
I tested that pandas could be imported when I used python in the command shell (yes) -- still didn't work on Jupyter.
My instructor thinks the issue is that system-wide packages are interfering with virtual ones but we have been working for hours and cannot figure out how to fix this.
Any help would be greatly appreciated. Please include detailed instructions as I am a beginner.
If you're getting 'ModuleNotFoundError: No module named xxyyzz' in jupyter, but the module can be imported by running python via the pipenv shell (pipenv run python -c "import xxyyzz; print(xxyyzz.__version__)":
it's probably jupyter's python path isn't set properly in the kernel config file: ..\jupyter\kernels\<myProjectName>\kernel.json
the kernel needs to be created within the pipenv shell to pick up the right path
With a fresh pipenv install:
pip install pipenv
cd <project directory>
export PIPENV_VENV_IN_PROJECT=1 # creates .venv in project directory
pipenv --python=/path/to/python --site-packages # use python executable for your system or environment
pipenv shell # work in project's virtual environment
python -m ipykernel install --user --name=<myProjectName> # create jupyter kernel for project
exit # exit project's virtual environment
pipenv run jupyter notebook # start jupyter from project directory
in jupyter, choose the kernel "myProjectName"
this post provides additional explanations
Why don't you try to install ipykernel with Anaconda virtual env?
It'll will be more easy to handle.
If you haven't previously used Anaconda before, just go to the official website
https://www.anaconda.com/download/ and download the newest version for your OS.
Then, follow these steps.
Execute Anaconda prompt.
Type 'conda create -y -n $ENVIRONMENT_NAME ipykernel'
Type 'conda activate $ENVIRONMENT_NAME'
Type 'conda install -y $PACKAGES_TO_BE_INSTALLED'
Type 'python -m ipykernel install --user --name $NAME --display-name $IPYKERNEL_NAME'
This ipykernel name will be presented on your list of kernels in jupyter notebook.
You can findout the list of kernels installed by typing jupyter kernelspec list.
Hope this helps!
Thanks for the advice. However, I was advised specifically not to install Anaconda -- can't quite remember the reason but I think it's because, basically, if I ever decided I wanted to use something else then it would be a real headache to switch. I'm happy to hear your reasoning if you disagree with that.
I ended up solving the issue by uninstalling every package both within the virtual environment and the larger computer system, then re-installing it in both places. It worked, but I'm sort of confused as to what the point of a virtual environment is, if I still had to install everything twice.
I installed Anaconda3 so I can create environments and install different packages in each environment. But I fail to understand the difference between the Python in
/usr/bin/python
and
/opt/anaconda3/bin/python
I can seem to access Python 3.6.5 Anaconda from both, why is that? And, what is the difference between both?
Furthermore, I would like to install packages to a single Python environment only.
When you are running python in the terminal, it is looking up your default path to your the python command. In this case, anaconda probably put a line in your shell profile specify the path to the anaconda version, which is why you are seeing it in the interpreter when you run python from either directory.
Secondly, you can set up a conda environment to download app specific dependencies without interfering with your default set up by
conda create --name myenv
source activate myenv
conda install packagename
This will install it in the myenv environment only. To deactivate the environment just run
source deactivate
Here is the documentation on that https://conda.io/docs/user-guide/tasks/manage-environments.html
Judging by your path, you are using Linux which comes with python installed. So /usr/bin/python is default and you have installed the other one later.
For the environments use https://conda.io/docs/user-guide/tasks/manage-environments.html to activate the desired environment, then you can pip install or conda install the packages and it will be places safely only in that environment. Note that spyder icon runs the root environment by default and you have to run it from terminal after activating one of the environments.
Edit:
I'm not sure why you want to use cd to change the python version. I suggest use aliases. I guess you are just changing the path but running the same version of the python anyway. Take a look at this question:
Two versions of python on linux. how to make 2.7 the default
I wanted to create a new virtual environment to install new packages. Following worked for me:
Commands are executed in Jupyter Notebook (OS: Ubuntu 16.04 LTS)
Upgrade pip:
!pip install --upgrade pip
Install virtual environment:
!pip install virtualenv
Select version of Python you want to use in new environment:
I wanted to create an environment with Python version 3. Naming it as Python3_xyz:
!virtualenv -p python3 Python3_xyz
After execution, this will create a folder with the same name in the current working directory (i.e. the location where Jupyter notebook is present)
Create a new option with the name of the created environment
And finally, run the following command:
!python -m ipykernel install --user --name=Python3_xyz
This will create a new option with the name Python3_xyz in the menu from where we create a new notebook.
NOTE: One can run above commands from the terminal as well just don't use '!' before the commands.
This question is bit dated, but since I faced a similar issue, what worked for me might help someone!
I did pip install requests from within my conda environment, but failed to import requests even after trying out everything.
What worked for me: run python -m pip install requests or python3 -m pip install requests within you environment. This installed requests successfully for me.
Hi I have a curious (and maybe trivial who knows?) problem.
For the requirements of a course on coursera I have created a python 2.7 environment in my anaconda distribution and installed DATOs' GraphLab Create modules.
While the simple "import graphlab" functions properly from the python command line or the ipython console it fails in a jupyter notebook.
checking installed modules with pip.get_installed_distributions() yields 148 modules in ipython console and only 42 in the jupyter notebook!
So, what is at work here? :-)
Thanks for any hint,
Alain
As you said you have created an environment with Python 2.7, you thus need to register this environment with the notebook.
Activate the environement:
$ source activate myenv
Then install the kernelspec for this environment:
(myenv)$ python -m ipykernel install --name myenv
You will need to install ipykernel in this env if it is not installed.
Now when starting a notebook, you can create a new notebook using this env, and/or change the currently used env, using the Kernel menu.
I uninstalled Anaconda 2.3 with Python 3.4.3 and then installed the same Anaconda version with Python 2.7.10.
When I open a notebook via $ ipython notebook "Example Notebook.ipynb" it tries to use the python3 kernel as opposed to opening with the installed python2. Of course I get the error python3 kernel not found.
How can I get ipython notebooks to open with the python2 kernel? I've tried to uninstall ipython and ipython notebook, then delete .ipython and .jupyter from my user directory in case there were any defaults set in these folders, then reinstalled both. Still get the same problem.
Any help would be appreciated
You can install several python versions alongside each other. Just create another environment (replace "all my packages" with the names of the packages).
conda create --name mypy_27 python = 2.7
or
conda create --name mypy_34 python = 3.4
afterwards you can activate the environments by typing
source activate mypy_34
if you then do
conda install "all your packages"
you install the desired packages in the active environment.
You can do much more.