I have a conda environment with Tensorflow-gpu installed for python 3.6.6. I am running Ubuntu linux 16.04 LTS. When I activate the environment and start a python shell, I can import Tensorflow just fine. But when I start up a Jupyter notebook or Jupyter lab session, I can't import Tensorflow.
Now there is a pre-existing SE post on this issue. However, that post is 2 years old, so there are changes in Jupyter since then. Second, I tried the fixed mentioned in that post, and none of them have worked. So I was hoping there might be some progress or some new ways to fix this.
So let me provide a little more detail on the issue.
First, when I activated a python 3.6.6 environment in anaconda and did conda list in a regular Tilix terminal session, I can see that Tensorflow-gpu is listed.
When I start ipython in a standard Tilix terminal and type import tensorflow as tf, that works fine.
Next, I started a jupyter lab session and opened a terminal from the jupyter launcher. In the terminal I started an ipython session and tried to import tensorflow. That does not work.
I opened a new jupyter notebook and tried to import Tensorflow then I "ModuleNotFoundError: No module named 'tensorflow'" error.
If I just run a python file with tensorflow code from the regular tilix terminal, then it will run fine and find Tensorflow. So only the notebook seems to be the big problem.
Things that I have tried:
I did try to implement a few of the fixes suggested in the older SE post, as well as the closed github issue on this problem.
I tried to reinstall tensorflow
I did a conda install of ipython, jupyter because some folks suggested installing those into the environment directly.
I looked at the jupyter kernelspec list. Some folks suggested making a change to this, but it was not clear what to change.
I installed the conda install -c anaconda-nb-extensions nb_conda in the environment too, but still no luck.
Anyone else have a suggestion about how to fix this.
Related
I am using anaconda with VSCode. I import tensorflow after I downloaded it and made sure I was in the correct environment, but everytime I run the program in the terminal in both VSCode and the regular terminal I am getting this error saying no module name tensorflow. it will work in jupyter notebook perfectly without running into the error. I have tried launching VSCode from the anaconda navigator as well nothing is working. What am I doing wrong?
I am using python 3.9.7 with the recommended version from google of tensorflow 2.6
see if your python interpreter path is correct. And u can edit if u click on this at the left bottom corner of vs code.
Click on this and vs code will show u all python interpreters it found all if your path is not there u can select it manually
Have you checked which python environment you are using in the terminal?
get-command python
pip --version
pip show tensorflow
And which python interpreter you have selected in the Jupyter NoteBook？
After installing VS code in your system, You need to install python extension for VS code. Then click on "Create new Jupyter notebook" and select the appropriate Python interpreter(virtual environment) as in Anaconda to run the TensorFlow code.
Please check this similar issue for your reference.
I located a really weird behavior of Jupyter and caused me to reinstall it but the problem persisted.
I have a Windows machine where I have installed anaconda (the latest version). There are two environments:
(base) Which contains nothing special maybe seaborn pandas etc. and is on python version 3.8.12.
(tf-gpu) Which contains tensorflow-gpu and is on python version 3.9.7.
Mind you, these two environments are created directly from a fresh installation of anaconda, and I didn't touch anything except installing tensorflow-gpu.
When I launch Jupyter through Anaconda, it launches normally on the home folder "C:\User\user" for both environments i.e. correct packages installed and correct version of python on both. I can also launch an instance of Powershell through Anaconda or Windows, (always in home dir), and launch Jupyter both environments behave as they should (packages, python version). So far so good...
The problem starts when I want to launch Jupyter from a directory other than home dir. I keep all my project files in a separate partition (D:), thus I navigate to that directory through a cmd/powershell (launched through Anaconda or Win) and type "jupyter notebook". The notebooks open on that director and:
When the base env is selected the notebook reports a correct name "base"(os.environ['CONDA_DEFAULT_ENV']) and python version 3.8.12 (sys.version_info). Everything behave as base env was setup.
When the tf-gpu env is selected the notebook reports a correct name "tf-gpu" but I get a python version 3.8.12, which is completely wrong! It should be 3.9.7. I can't import tensorflow since I get an error. Essentially I am in the base env without knowing it.
I then check in the same cli window (tf-gpu) that launched jupyter and python --version reports 3.9.7 and of course can import tf run it etc.
Jupyter just reports the environment name but in reality it uses the base environment even though it launched from said env. How can this change?
Why does Jupyter works normally when launched from the user directory?
If this is not solved easily, what is a workaround? Jupyter doesn't recognize symbolic links, and thus I can not navigate to another directory if it is not a subdirectory of home.
Finally, I also tried adding the env as kernel through python -m ipykernel install --user --name tf-gpu --display-name "Tensorflow GPU (tf-gpu)", but it doesn't seem to change a thing.
Update 1:
Of course I activate the tf-gpu through conda activate tf-gpu and check the python version and tensorflow (correct results), the problem is when launching jupyter notebook. The notebook reports the tf-gpu env but wrong python version and tf is missing.
Update 2:
After some searching around it seems that jupyter notebook has much trouble selecting a python version if multiple versions of python are installed in the system (in my case 2, 1 on each environment environments). Which is ridiculous if you think that that's why we are using environments for...
The reason you are having this problem I’m guessing is because your launched jupyter notebook from your conda base environment. You will see a (base) in your comment prompt.
To remedy this, When you use the command prompt to launch, do conda activate tf-gpu before you launch Jupyter notebook, now check if your versions are correct.
What I did and worked was run ipython3 notebook in the environment which launched the notebook server and everything was working properly correct environment name and correct python version and packages (tensorflow is working as it should). This must have updated some configuration files since I can now run jupyter notebook from a different folder and the gpu-tfu env works correctly.
Please remember that these problems were occurring only if I was launching jupyter outside the home directory (C:\User\user) on a different partition. Also keep in mind that this is a Windows installation and more complicated things may be happening with user paths, environments, etc than in linux.
I downloaded Anaconda Navigator months ago and have been using it to launch Jupyter Notebooks. Recently, I was trying to import Graphviz to visualize a decision tree and had a ton of trouble getting the package to work. I tried following various command line instructions that I read on forums, but somewhere in following some of those instructions, I must have completely messed up my conda environment. I am no longer able to open Anaconda. I tried uninstalling it and reinstalling (followed the directions for uninstalling it from the Anaconda documentation page). Everytime I try to reinstall it, it says the installation has failed or that it already exists on my computer.
I gave up on Anaconda and went ahead installing Jupyter Notebooks on its own. This worked just fine and I'm able to launch a kernel, but every time I try to execute code, it says the kernel has died and will automatically restarted.
I have gone down many rabbit holes...any help would be greatly appreciated!
In the past, I downloaded an open-source package through pip when I was just using it and it worked fine but I have recently started to help with the development of the package so I cloned the github repository onto my laptop and am now having troubles. I checked out and pulled the changes to a new branch in the github repository to work with some changes someone else made but my Jupyter Notebook was still using the old version of the package as I couldn't access the new methods in the notebook I was working in. If I open the code through 'open .' when in the directory, I can see that it is the proper code that I need to run but Jupyter notebook was not running that code. I then ran 'pip freeze' in my terminal and saw a package was still installed there so I ran 'sudo pip uninstall ' and uninstalled the package from pip. Although now, when I try to import the package into my Jupyter Notebook I get the error where there is:
No module named <package name>
Does anyone know why this is happening and how I can fix it? I would like to be able to use the version of the code of which branch I am in currently.
EDIT:
The package library is stored in /Users/myName/pkg. I am trying to access this through the 'import ' statement in Python. I am running this import statement through a Jupyter notebook in Jupyter lab. Jupyter lab is installed under python3.7/anaconda3/bin/jupyter-lab. I used to have another version of this open-source library installed which I installed using pip, but I recently uninstalled it as I thought that was causing the problem.
I also ran
sys.executable
and it returned '/anaconda3/bin/python'
EDIT 2:
I tried following the solution from this post but when I activated my new environment created, my entire jupyter lab local-host page just kept refreshing constantly and would not load anything. Although when I do not activate that environment I can run my old jupyter lab fine.
EDIT 3:
SOLUTION:
The solution I linked above worked for me, although I also had to update my Jupyter Lab in the new conda environment to stop my page from being infinitely refreshed.
My knowledge with Jupyter and python is limited. You can sense it after reading this question. I have two set of servers: one set of servers running the spark-standalone cluster and one server running the Jupyter notebook.
I have installed Jupyter in my server twice with commands:
pip2 install jupyter
pip3 install jupyter
After installing I set the following environments:
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS="notebook --notebook-dir=./ --ip=* --no-browser"
Then I used Jupyter to connect to the spark-standalone cluster with the following command:
./bin/pyspark --master spark://192.XXX.XXX.XXX:7077
I can login in the Jupyter homepage in my laptop and in the right corner the message indicates that I am using Python2. However, I cannot execute Python2 codes while Python3 codes work.
I tried the following command
export PYSPARK_PYTHON=python2
and
import os
os.environ["PYSPARK_PYTHON"]="/usr/local/bin/python2"
but with no luck.
Is this because I am using a Jupyter notebook created by pip3? If it is, how can I switch to the Jupyter notebook created by pip2?
Thanks in advance if you have any suggestions. If you need more information to locate the cause, please feel free to let me know.
I put this problem aside these days and picked it again today. To determine whether spark or jupyter was responsible for this problem, I launched the Jupyter Notebook without spark: jupyter notebook. The same problem arose again so I knew there was something wrong with my Jupyter Notebook.
I referred to this document: Installing the IPython kernel. After executing the following commands I could run Python2 codes.
python2 -m pip install ipykernel
python2 -m ipykernel install --user
I have tried these two commands before posting the questions but with no luck. I guess maybe I made some mistakes when coping/pasting? Anyway, the problem is now solved.