I'm using virtualenvwrapper-win to manage environments, I created one, pip-installed a bunch of packages but when I try to import them from jupyter notebook or lab it only looks in the main directory and not in the environments. I tried using the conda environments and had the same problem, so I suspect the problem is in the jupyter configuration, which I haven't meddled with much but might have messed up at some point.
sys.executable returns 'c:\\anaconda3\\python.exe', but I suspect it should be using the one in the virtual environment right?
Related
I located a really weird behavior of Jupyter and caused me to reinstall it but the problem persisted.
I have a Windows machine where I have installed anaconda (the latest version). There are two environments:
(base) Which contains nothing special maybe seaborn pandas etc. and is on python version 3.8.12.
(tf-gpu) Which contains tensorflow-gpu and is on python version 3.9.7.
Mind you, these two environments are created directly from a fresh installation of anaconda, and I didn't touch anything except installing tensorflow-gpu.
When I launch Jupyter through Anaconda, it launches normally on the home folder "C:\User\user" for both environments i.e. correct packages installed and correct version of python on both. I can also launch an instance of Powershell through Anaconda or Windows, (always in home dir), and launch Jupyter both environments behave as they should (packages, python version). So far so good...
The problem starts when I want to launch Jupyter from a directory other than home dir. I keep all my project files in a separate partition (D:), thus I navigate to that directory through a cmd/powershell (launched through Anaconda or Win) and type "jupyter notebook". The notebooks open on that director and:
When the base env is selected the notebook reports a correct name "base"(os.environ['CONDA_DEFAULT_ENV']) and python version 3.8.12 (sys.version_info). Everything behave as base env was setup.
When the tf-gpu env is selected the notebook reports a correct name "tf-gpu" but I get a python version 3.8.12, which is completely wrong! It should be 3.9.7. I can't import tensorflow since I get an error. Essentially I am in the base env without knowing it.
I then check in the same cli window (tf-gpu) that launched jupyter and python --version reports 3.9.7 and of course can import tf run it etc.
Jupyter just reports the environment name but in reality it uses the base environment even though it launched from said env. How can this change?
Why does Jupyter works normally when launched from the user directory?
If this is not solved easily, what is a workaround? Jupyter doesn't recognize symbolic links, and thus I can not navigate to another directory if it is not a subdirectory of home.
Finally, I also tried adding the env as kernel through python -m ipykernel install --user --name tf-gpu --display-name "Tensorflow GPU (tf-gpu)", but it doesn't seem to change a thing.
Update 1:
Of course I activate the tf-gpu through conda activate tf-gpu and check the python version and tensorflow (correct results), the problem is when launching jupyter notebook. The notebook reports the tf-gpu env but wrong python version and tf is missing.
Update 2:
After some searching around it seems that jupyter notebook has much trouble selecting a python version if multiple versions of python are installed in the system (in my case 2, 1 on each environment environments). Which is ridiculous if you think that that's why we are using environments for...
The reason you are having this problem I’m guessing is because your launched jupyter notebook from your conda base environment. You will see a (base) in your comment prompt.
To remedy this, When you use the command prompt to launch, do conda activate tf-gpu before you launch Jupyter notebook, now check if your versions are correct.
What I did and worked was run ipython3 notebook in the environment which launched the notebook server and everything was working properly correct environment name and correct python version and packages (tensorflow is working as it should). This must have updated some configuration files since I can now run jupyter notebook from a different folder and the gpu-tfu env works correctly.
Please remember that these problems were occurring only if I was launching jupyter outside the home directory (C:\User\user) on a different partition. Also keep in mind that this is a Windows installation and more complicated things may be happening with user paths, environments, etc than in linux.
I have been programming in Python using Spyder in combination with virtual environments for a while now. Before I started using virtual envs, I installed Spyder using pip. I am not very familiar with paths and environments, but this seemed the way to go when I repetitively ran into package compatibility issues. However, it has been bugging me that every time I want to switch environment, I need to change the working directory to the folder of the environment and I need to change the Python interpreter manually within Spyder.
When I was looking for a solution, I found that it is possible to install Spyder within a virtual environment. I did this and found out that opening Spyder from a virtual environment has the exact same working directory and interpreter as the standalone version (the one I started with). This means that I still have to change the interpreter and working directory manually... Am I doing something wrong or is the only way to switch environments by changing it in the preferences tab within Spyder?
Background info: Using Windows, Spyder 4.2.1, venv as described on https://docs.python.org/3/tutorial/venv.html
I am attempting to import cartopy into a notebook but am running into issues. If I do a "conda install cartopy" in my base environment, I get the frozen/flexible solve issue (https://github.com/conda/conda/issues/9367). When I open a new environment and do a "conda install cartopy", it seems like everything works. "conda list" shows version 0.18.0 in that environment. Then I open a Jupyter Notebook from within that environment and try to import it into the notebook but originally got the response "no module named cartopy". I tried fiddling with my environments and settings based on feedback on this page (In which conda environment is Jupyter executing?), but now the error is "no module named numpy"! Can someone please help me understand why the notebook isn't seeing these modules? Thank you.
Are you certain that numpy is installed in this new environment?
Given that you are using the Anaconda distribution of Python, you should be able to view/configure your environment and it's installed modules using the Anaconda Navigator. Here you can see a full list of all installed modules, and via the terminal/console, launch a Jupyter Notebook from not only within a specific environment, but within a specific directory!
On the left hand side, you can choose the environment that you want to use to start Jupyter Notebook, and on the right you can view all of the installed modules in that environment. Make sure that jupyter, numpy, and cartopy are all listed as installed packages.
To make sure that I am launching Jupyter from my desired environment, I always launch it directly from the terminal. I "Open Terminal" with my environment, "cd" into the directory that I want to use, and then use the jupyter notebook or jupyter lab command.
When I try import command for pandas or numpy in Jupyter notebook, I get a 'ModuleNotFoundError' (see below).
I have only recently installed Jupyter Notebooks (using the Anaconda installer). It seemed to work fine initially, but creating kernels for Python2 ad Python3 have created a problem.
import numpy runs fine if I put it in a separate .py file and run from the terminal window - no error messages.
---------------------------------------------------------------------------
`ModuleNotFoundError Traceback (most recent call last)
<ipython-input-3-5a0bd626bb1d> in <module>()
----> 1 import numpy
ModuleNotFoundError: No module named 'numpy'`
This question is almost two years old, but there are so many different potential problems, related to conda environments and multiple ipython kernels, that it's worth answering.
There might be several different issues here. The first question is whether or not the needed package is installed in both environments? Considering, that import numpy works for you when you start Python interpreter from the console, it is installed in the base environment, but what about the others? You can check installed packages in other environments with conda list -n ENV_NAME.
If the package is missing, it can be installed to the target environment with conda install -n ENV_NAME PACKAGE_NAME.
Next question is how the Jupyter Notebook is started? Looking at your paths, you are on Windows. Thus, there might be shortcuts in the Start menu created by Anaconda, or you might run Jupyter from the command prompt.
If you use the shortcuts, the conda environment with Jupyter should be activated automatically, and all the packages in that environment should be available. But if you attempt to run Jupyter from the command prompt, you have to activate the environment yourself, before starting Jupyter:
activate `ENV_NAME`
jupyter notebook
To simplify environment activation on Windows, you can create .bat/.cmd start files, which will activate the relevant environment and run Python interpreter or Jupyter in the appropriate context. Here you can find an example.
Finally, to complicate matters, you might have multiple local Jupyter installations, each in its own environment, containing its own local ipython kernel, or there might be a single Jupyter in one environment, connected to ipython kernels in other environments.
In the former case, activating the relevant environment before running Jupyter should be sufficient. In the latter case, there are several ways of adding kernel specs to Jupyter, but the easiest is using nb_conda_kernels package. With it, Jupyter should find ipython kernels in other conda environments dynamically.
Don't have much expertise in programming. Only picked up Python last summer.
I have installed both Atom and Conda on my computer. Recently, I've used Atom to edit my scripts in Python, then run the scripts via Command Line. As per standard practice, I created Virtual Environments where I installed packages I needed to run different Python scripts.
I now want to use Atom as an IDE, and so have installed the Script package on Atom so I can run my scripts in Atom itself. However, when I tried running a Python script that required the Python numpy package, I got this:
ImportError: No module named 'numpy'
This error is obviously going to appear for other packages that haven't already been installed in the root environment (I think?).
So now, my question is how do I activate the needed Virtual Environment in Atom? In other applications like Jupyter and Spyder, I would activate the Virtual Environment I needed then open the Application via Command Line, but I can't do that with Atom.
(If possible, is there a way to use Virtual Environments created by Conda)
Thanks
One way is to start atom from the activated virtual environment. In this case, executing programs/scripts uses the configured python interpreter and imports the installed in the virtual environment.
EDIT:
It's been long though, it might be useful for people redirected to this question.
By installing atom-python-virtualenv you can create, change or deactivate virtual environments with atom editor.
The hydrogen package for Atom works well for this.
You will also need to install Jupyter into your conda environment and Hydrogen should pick up the IPython kernel automatically.