When started, Jupyter notebook encounters a problem with module import
import findspark
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-ff073c74b5db> in <module>
----> 1 import findspark
ModuleNotFoundError: No module named 'findspark'
Conda list shows that module is here
filelock 3.0.8 py37_0
findspark 1.3.0 py_1 conda-forge
flask 1.0.2 py37_1
Python version
(myenv) mm#mm-HP-EliteBook-8560p:~$ python -V
Python 3.6.8
It seems that my installation is not clean.
Three Python lines from .bash_profile
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
Why do I get import error?
I'd suggest a slightly different route.
Download spark on your local. Let's say you've unzipped in /Users/me/spark-2.4.0-bin-hadoop2.7 location.
Assuming you're on mac, update your ~/.bash_profile to contain these entries:
export SPARK_HOME=/Users/me/spark-2.4.0-bin-hadoop2.7
export PYTHONPATH=${SPARK_HOME}/python:$PYTHONPATH
export PYTHONPATH=${SPARK_HOME}/python/lib/py4j-0.10.7-src.zip:$PYTHONPATH
export PYSPARK_PYTHON=<path to your python location>
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS=notebook
PATH=$PATH:$SPARK_HOME/bin
Execute a source ~/.bash_profile.
From your bash shell, just run pyspark and it'll open the jupyter notebook. Now your notebook will be tied to this spark installation.
If you're using linux, I think the only change is in the syntax for appending stuffs to path, and instead of changing bash_profile you probably need to change bashrc file.
Make sure you are using the correct virtualenv.
Create a fresh virtualenv for your work (eg. using 3.7.4 as an example here. Use a version you have installed):
pyenv virtualenv 3.7.4 myenv
You can see which python versions you have installed with:
pyenv versions
And which versions are available for installation with:
pyenv install -l
You can either activate the virtualenv shell with:
pyenv shell myenv
With the virtualenv active, you should see the virtualenv name before your prompt. Something like "(myenv)~$: "
Now install all the python packages as you normally would. Make sure you are in the right virutalenv before you run your packages. You can also set the PYENV_VERSION environment variable to specify the virtualenv to use. Something like:
PYENV_VERSION=myenv python -m pip install findspark
Then
PYENV_VERSION=myenv python -m pip show findspark
Should give you something like:
Name: findspark
Version: 1.3.0
Summary: Find pyspark to make it importable.
Home-page: https://github.com/minrk/findspark
Author: Min RK
Author-email: benjaminrk#gmail.com
License: BSD (3-clause)
Location: /home/tzhuang/.pyenv/versions/3.7.4/envs/myenv/lib/python3.7/site-packages
Requires:
Required-by:
Related
I'm using virtualenvwrapper to manage a project where I'm just running this for now
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
with beam.Pipeline(options=PipelineOptions()) as p:
pass
This is returning the following error
Traceback (most recent call last):
File "path/to/pipeline.py", line 1, in <module>
import apache_beam as beam
ModuleNotFoundError: No module named 'apache_beam'
I have run pip install apache-beam. Running pip list returns
➜ pip list
Package Version
------------------------------ ---------
apache-beam 2.23.0
avro-python3 1.9.2.1
...
I'll add the following outputs as suggested in a similar question.
pip freeze
➜ pip freeze
apache-beam==2.23.0
avro-python3==1.9.2.1
...
pip -V
➜ pip -V
pip 20.2.2 from /Users/miguel/.virtualenvs/myenv/lib/python3.7/site-packages/pip (python 3.7)
python -V
➜ python -V
Python 3.7.3
which python
➜ which python
/Users/miguel/.virtualenvs/myenv/bin/python
which pip
➜ which pip
/Users/miguel/.virtualenvs/myenv/bin/pip
I don't know if this is relevant but I'm using VSCode and I have selected my python interpreter according to VSCode instructions here. Additionally, I installed python following the instructions here.
Any idea why this is happening?
First test if your installed Python modules work outside of an ide like VSCode.
We can do this by opening the terminal/command prompt and activating our virtual enviornment. In this case, as you're using virtualenvwrapper you need to use the command:
workon myenv
Documentation for the virtualenvwrapper can be found here. Once activated, we can open the Python interpreter in our terminal by using the command:
python
Once running, try and import apache_beam as before using:
import apache_beam as beam
If this works we now know it's a virtual environment set up issue using VSCode. For setting up Virtual Environments for Python in VSCode, use the official documentation Using Python environments in VS Code. This should allow you to specify the virtual environment.
If this fails to work, another option is to create a new virtual environment in VSCode and install your modules. Installing modules to a environment in VSCode can be found in the VSCode instructions you linked to in your post.
I tried to re-create your issue as above using the following:
Python 3.7.7
VSCode 1.48.0
apache-beam 2.23.0
My virtual environment was named 'stack'. Once I had created it the only module I installed was apache-beam using:
pip install apache-beam
When creating the project I made the directory and launched via terminal using:
- mkdir hello
- cd hello
- code .
And then added the Python interpreter via the command:
Python: Select Interpreter
One additional step I needed (as I usually use PyCharm for Python) was to install:
Shell Command: Install 'code' command in PATH** command.
To allow the launch of code via terminal. Below is a screenshot of the project after following these steps and running the same commands as posted in your question for reference.
My .vscode/settings.json is as follows:
{
"python.pythonPath": "/Users/robertyoung/envs/stack/bin/python"
}
I'm getting started with virtual environments, and I'm having some sort of path issue.
First I create a virtual environment:
python -m venv conda-env
.\conda-env\Scripts\activate
(conda-env) shows up in the prompt.
Now I install a package:
pip install netcdf4
pip list
Package Version
---------- -------
cftime 1.1.3
netCDF4 1.5.3
numpy 1.19.0
pip 19.2.3
setuptools 41.2.0
Everything looks good. Now if i start a console and try to load the module, things go haywire:
jupyter console
In [1]: import netCDF4
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-9588a3d4fb24> in <module>
----> 1 import netCDF4
ModuleNotFoundError: No module named 'netCDF4'
If I run the console from the site-packages directory however, the following runs with no errors.
cd .\conda-env\Lib\site-packages\
jupyter console
import netCDF4
edit1:
If I install jupyter in my virtual environment, that fixes the problem.
pip install jupyter
I don't get it. Do I need to do this for every virtual environment?
edit2:
If I use python directly, instead of a jupyter console I don't get this problem. The code below works where test.py has the line import netCDF4.
python -m venv venv
./venv/Scripts/activate
pip install netCDF4
python test.py
edit 3
On a mac, pip install jupyter doesn't fix the problem. I'm very confused.
I set the PYTHONPATH environment variable in the virtual environment to include the virtual environment site-packages path, for ex.
# Setting the PYTHONPATH
export PYTHONPATH=/Users/leo/dev/lib/python3.9/site-packages
This should pick up the packages you installed in your virtual environment with pip install.
In my "base" environment, installing pandas and then importing it using the Spyder IDE works just fine.
However, this is no longer the case when working under a virtual environment.
In my terminal, I created a Python 2.7 environment named bodhi:
conda create -n bodhi python=2.7
I activated it:conda activate bodhi
I then installed spyder: sudo -i apt-get install spyder
And finally I installed pandas: conda install pandas
Typing conda list confirms that pandas 0.24.2 was installed properly.
After opening Sypder, typing import pandas in the IPython console returns the following ImportError: No module named pandas.
What am I doing wrong?
I am fairly new to virtual environments so I apologize if the question seems obvious!
Your conda env isn't the same as your kernel. Open an IPython console Ctrl-T and do your installs and environment management there as described here.
You can also work with your kernel from the console using some of the commands described here:
https://stackoverflow.com/a/28840041/10553976
Importantly: path/to/python -m ipykernel install <options>
I am trying to import Keras lib code to execute CRF using the import command below but an error raises as titled. Please share the solution for this.
The command used to execute is
from keras_contrib.layers import CRF
Traceback (most recent call last):
File "", line 1, in
from keras_contrib.layers import CRF
ImportError: No module named 'keras_contrib'
A simple
(sudo) pip install git+https://www.github.com/keras-team/keras-contrib.git
as mentioned in the installation instructions did the trick for me.
This error means that Python is unable to find the module in one of the directories defined by Python path. The module is either not installed or is installed in another directory.
If not installed, then see https://github.com/keras-team/keras-contrib for installation instructions.
If installed but not found, you will most likely need to add the directory where it is installed to your Python path. You can find out what your current Python path is by inspecting the variable sys.path (such as python -c 'import sys; print sys.path'). You may need to add another directory to your path by setting the environment variable PYTHONPATH before running your script, but there are other options. See for example PYTHONPATH vs. sys.path for some insight.
After struggling for a while, I was so willing to make myself clear of this issue, so I searched for a while, and just figured out and tested.
When you create a new conda env by specifying the python version, it will use the conda_root_python version. And if you didn't install the pip package, and try to use pip under your created conda env, it will only run the conda_root_pip and install the package in the root site_packages.
I know three ways to install python packages only in your created conda env.
For a better explanation, we create a conda env with same python version of conda root environment.
conda create -n myenv python
I. One of the officials advise, install package with conda command for specified conda environment,
conda install -n myenv tensorflow
II. Another one of official advise, get into your specified environment and run conda install
source activate myenv
conda install tensorflow
in above two ways you don't need to install extra packages like pip and other pip related packages.
III. For people who really want to pip, just because get used of that.
install pip package(just as above two ways did).
conda install -n myenv pip
or
source active myenv
conda install pip
then comes the pip install when you are in your environment
pip install tensorflow
--------new edit above 15.April.2018--------------
Just to make it more clear.
If you are working under anaconda environment, you should also install all the modules and IDE you need in that environment.
Here I just put one example of anaconda env flows:
conda create --name=my_conda_env python=2.7 #create an environment
activate my_conda_env #get into that env
pip install numpy #install packages you need
...
pip install keras_contrib
pip install spyder #install IDE
Getting Started with conda
---------
Try install in root
activate root
pip install keras_conrib
go back to your tensorflow
start your spyder and try again
Maybe this is your issue
Module installed on Conda, but gives error on importing in Spyder (Python IDE)
----------------- above new answer
It seems you are under conda environment, env-name is "tensorflow", so try to start python and try import again. To make it clear
make sure you have (tensorflow) in front of C:\Users>
type python to start python
import keras_contrib to see if you have keras_contrib in anaconda env (tensorflow) due your comment, it should be
from keras_conrib.layers import CRF (crf or CRF? just try)
If you installed keras_contrib in env "tensorflow", should also start python and do your jobs in the same env, for a new env, you have to install it again.
Here is something for newbie just like me after playing with python for a while and still not familiar with anaconda, I hope you didn't come up with that. As follows:
I used to think in my anaconda env is already in python(actually not yet), so I just type
from keras_contrib.layers import CRF when I saw (tensorflow)C:/Users> which is actually wrong
The right way as described up is get into python(step 2.) or ipython or jupyter just for test if you get the package.
--------------------- below is old answer
I think you confused keras with keras_contrib.
They are two different modules.
try pip install keras_contrib or use other ways to install keras_contrib.
If you are trying to install the tensorflow-keras version or even the keras version using git cloning and setup.py installing and getting the above error then you might want to add the path of the keras-contrib folder to system path using -
import sys
sys.path.append('<remaining_path>/keras_contrib')
Simply run:
conda install git+https://www.github.com/keras-team/keras-contrib.git
If you use Tensorflow, you may get over this error by replacing
from keras_contrib.layers import CRF
with
from tensorflow_addons.layers import CRF
If you're still interested in Keras itself, use the following line in Jupyter
!pip install git+https://www.github.com/keras-team/keras-contrib.git
Just note that you may face other errors while using seperate Keras, so I suggest that you use Keras that is supported in Tensorflow.
Win 10, x64, Python 3.6.6 & Python 2.7.12 (I tried the code in both), IPython 6.1.0
So far I am only able to run VTK projects using Visual Studio in C++ but would like the ease of using an interpreted language for development. So I installed the latest VTK via the Anaconda command prompt, conda install -c anaconda vtk
No problems everything updated & superseded as necessary. So I tried out a trial program from the Python examples only to get...
ModuleNotFoundError: No module named 'vtk'
I added C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\pkgs\vtk-8.1.0-py36he6bbf13_201\Lib\site-packages\vtk to my PYTHONPATH in Spyder but I still get the same error.
Any idea why IPython cant see the vtk library?
EDIT: VTK shows up as an installed library when I type conda list at the command line
The following worked for me. Make sure to use the Anaconda Prompt.
# Create a new conda environment mypy3env and install vtk.
conda create -n "mypy3env" python=3
conda activate "mypy3env"
conda install -c conda-forge vtk
# Verify that python3 from Anaconda is used!
where python
python --version
# Test if vtk installation was successful
python -c "import vtk; print(vtk.vtkVersion.GetVTKVersion())"
See also the comments here to verify the Anaconda installation.
I tried to create a new environment and the vtk module works for Python version 3.6. When using a newer version of Python the vtk module cannot be loaded Exception has occurred: ModuleNotFoundError No module named 'vtkmodules.vtkCommonCore'(I don't know why).
conda create --name MyEnv python=3.6
conda activate MyEnv
pip install vtk