We Keep Coding

venv cannot find python library even though I definitely have it installed

Im runnig a windows 10 machine, and im getting started on playing with virtual enviorments. I want to run a small fastapi application
I created a venv like this:
python -m venv venv
I then activated the venv, and installed my two dependencies:
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app> pip install fastapi
and
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app> pip install unicorn
Which seemed to work fine, running pip list also seems to indicate that i have what I need:
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app> pip list
Package Version
----------------- ---------
beautifulsoup4 4.9.3
certifi 2020.6.20
cffi 1.14.3
chardet 3.0.4
crypto 1.4.1
cryptography 3.1.1
cycler 0.10.0
fastapi 0.63.0
idna 2.10
myModule 1.0.0
Naked 0.1.31
pip 21.0.1
pycparser 2.20
pycrypto 2.6.1
pydantic 1.8.1
PyYAML 5.3.1
requests 2.24.0
setuptools 49.2.1
shellescape 3.8.1
six 1.15.0
soupsieve 2.0.1
starlette 0.13.6
typing-extensions 3.7.4.3
unicorn 1.0.2
urllib3 1.25.10
wheel 0.35.1
But when I run my program it fails with the inmport of the fastapI:
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app> python .\main.py
Traceback (most recent call last):
File "C:\Users\Ask\python_Projects\garse_dockerAPI\app\main.py", line 3, in <module>
from fastapi import FastAPI
Also, running python my terminal doesnt wanna do it either:
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app> python
Python 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import fastapi
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'fastapi'
What's going on? why do i not have the dependencies i need?
EDIT:
In order to find the installation on my machine, I ran 'where python' in my normal terminal:
C:\Users\Ask>where python
C:\Users\Ask\AppData\Local\Programs\Python\Python39\python.exe
C:\Users\Ask\AppData\Local\Microsoft\WindowsApps\python.exe
Which give sme two different paths. I dont really know what to do with this information?
running where pip:
C:\Users\Ask>where pip
C:\Users\Ask\AppData\Local\Programs\Python\Python39\Scripts\pip.exe
When I run it in my venv, the commands output nothing at all:
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app> where pip
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app> where python
(venv) PS C:\Users\Ask\python_Projects\garse_dockerAPI\app>

I definitely recommend using python3 -m pip instead of pure pip. It might save you some headache. The reason: when I used pip to install new packages in a venv virtual environment, they ended up in my OS environment! But just using python3 -m pip did the trick.
To make sure that the packages are called from your virtual environment, you can use python3 -m PACKAGE_NAME instead of PACKAGE_NAME. For example, flower didn't work in my virtual environment, but python3 -m flower works with no problem.
You can directly ask python where it is executed from! Try this:
>>> import sys
>>> print(sys.executable)
/PATH/TO/VENV/bin/python
>>> print(sys.version)
'3.10.5 (main, Jun 6 2022, 18:49:26) [GCC 12.1.0]'

Why is there a conda glitch specific to VSCode terminal, but not normal terminal? [duplicate]

This question already has answers here:
PATH not updated correctly from conda activate in VSCode's terminal
(4 answers)
Closed 1 year ago.
I am on a Mac Catalina 10.15.7. Visual Studio Code Version: 1.52.1
Suddenly, my conda environments are not able to find pandas (or any other) module that are clearly installed in the activated environments in the VSCode terminal.
Below, I show you the output from the terminal in VSCode. I have already activated my conda environment, dm_assay. I show that when I try to install pandas, it says 'already installed'. I list the conda packages, and you can see pandas in the list. But when I open python repl and import pandas, I get a ModuleNotFoundError: No Module named 'pandas'. This was not happening yesterday.
I opened the normal terminal (not VScode terminal), and did the same thing, and now pandas import works. (?)
What might be wrong with VSCode terminal the prevents python from accessing available packages in the active conda env?
The python paths change with activating and deactivating conda envs in the normal terminals, but do not change in the VSCode terminal.
I have tried restarting my computer. I want to use terminal in VSCode as it is very convenient.
Both terminals are using bin/zsh shell.
(dm_assay) me#mycomp % conda install pandas
Collecting package metadata (current_repodata.json): done
Solving environment: done
# All requested packages already installed.
(dm_assay) me#mycomp % conda list
# packages in environment at /usr/local/Caskroom/miniconda/base/envs/dm_assay:
#
# Name Version Build Channel
blas 1.0 mkl
ca-certificates 2021.1.19 hecd8cb5_0
certifi 2020.12.5 py38hecd8cb5_0
colorful 0.5.4 pypi_0 pypi
intel-openmp 2019.4 233
joblib 1.0.0 pyhd3eb1b0_0
libcxx 10.0.0 1
libedit 3.1.20191231 h1de35cc_1
libffi 3.3 hb1e8313_2
libgfortran 3.0.1 h93005f0_2
llvm-openmp 10.0.0 h28b9765_0
mkl 2019.4 233
mkl-service 2.3.0 py38h9ed2024_0
mkl_fft 1.2.0 py38hc64f4ea_0
mkl_random 1.1.1 py38h959d312_0
ncurses 6.2 h0a44026_1
numpy 1.19.2 py38h456fd55_0
numpy-base 1.19.2 py38hcfb5961_0
openssl 1.1.1i h9ed2024_0
pandas 1.2.1 py38hb2f4e1b_0
pip 20.3.3 py38hecd8cb5_0
prettyprinter 0.18.0 pypi_0 pypi
pygments 2.7.4 pypi_0 pypi
python 3.8.5 h26836e1_1
python-dateutil 2.8.1 py_0
pytz 2020.5 pyhd3eb1b0_0
readline 8.0 h1de35cc_0
scikit-learn 0.23.2 py38h959d312_0
scipy 1.6.0 py38h2515648_0
setuptools 51.1.2 py38hecd8cb5_4
six 1.15.0 py38hecd8cb5_0
sqlite 3.33.0 hffcf06c_0
threadpoolctl 2.1.0 pyh5ca1d4c_0
tk 8.6.10 hb0a8c7a_0
wheel 0.36.2 pyhd3eb1b0_0
xz 5.2.5 h1de35cc_0
zlib 1.2.11 h1de35cc_3
(dm_assay) me#mycomp % python3
Python 3.8.2 (default, Nov 4 2020, 21:23:28)
[Clang 12.0.0 (clang-1200.0.32.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'
>>> quit()
(dm_assay) me#mycomp % python3 --version
Python 3.8.2
(dm_assay) me#mycomp % conda --version
conda 4.9.2
(dm_assay) me#mycomp % pip --version
pip 20.3.3 from /usr/local/Caskroom/miniconda/base/envs/dm_assay/lib/python3.8/site-packages/pip (python 3.8)
In VSCode
(dm_assay) me#mycomp % echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/Caskroom/miniconda/base/envs/dm_assay/bin:/usr/local/Caskroom/miniconda/base/condabin
(dm_assay) me#mycomp % echo $SHELL
/bin/zsh
(dm_assay) me#mycomp project % which python
/usr/bin/python
in Normal Terminal
(dm_assay) me#mycomp ~ % echo $PATH
/usr/local/Caskroom/miniconda/base/envs/dm_assay/bin:/usr/local/Caskroom/miniconda/base/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
(dm_assay) me#mycomp ~ % echo $SHELL
/bin/zsh
(dm_assay) me#mycomp ~ % which python
/usr/local/Caskroom/miniconda/base/envs/dm_assay/bin/python
I notice that in the normal terminal, the python path (output of which python command) changes when I activate and deactivate conda environments. However, in the VSCode terminal, the python path remains the same (usr/bin/python) no matter if a conda env is activated or not.
Why isn't conda working in the VSCode terminal? How do I made it act like my normal terminal?

I have experienced the same in the Windows operating system, and I figured out why that happens in windows, and I think it's the same reason in Mac also;
the problem is when you close the vscode with the opened terminal when you restart the terminal opens automatically, but at there whatever environment you have configured into that directory by using .vscode does not execute.
you can see that, because there is just the terminal without activation any env.
but if there is no terminal opened at the start, when you open the terminal it executes the environment activation command [most of the time].
NOTE:
the best way to solve this, whether or not the terminal has opened is;\
Click the little + icon I have circled by blue,
then then it created a new terminal, the definitely your environment activates on time. Also, you can close the first terminal by selecting it in dropdown and click bin icon
this same problem appears even in the python virtual environment and pip environments. \
check that out this trick in Mac, because I'm not really sure.

In VS Code, the internal terminal it uses is integrated from the system's own terminal. Therefore, if the system terminal can be used, the internal terminal of VS Code can also be used.
Usually, when VS Code uses the python environment, the modules are stored in "lib\site-packages" of the corresponding python environment.
You could use the command "pip show pandas" in the VS Code terminal to check the installation location of the module "pandas":
In addition, it is recommended that you find the installation package of the module "pandas" in this environment, and then try to delete it and reinstall it to avoid problems due to damaged or incompatible files inside the module:
If it still doesn't work, please try to restart VS Code to let it reload the terminal.

Module import error in virtualenv but pip list --local shows it

I'm wondering about the following importError which I can't get ride off. I have a local virtualenv (venv) in which I've installed multiple modules:
(venv) stam#stam:~$ pip list --local
cryptography (1.5.2)
cvxopt (1.1.9)
cycler (0.10.0)
Cython (0.25.1)
matplotlib (1.5.3)
more-itertools (2.2)
ndg-httpsclient (0.4.2)
numpy (1.11.2)
pandas (0.19.0)
paramiko (1.15.2)
Pillow (2.8.1)
pip (8.1.2)
psycopg2 (2.6)
python-apt (0.9.3.12)
python-dateutil (2.5.3)
python-debian (0.1.27)
scipy (0.18.1)
setuptools (28.6.1)
simplejson (3.8.2)
urllib3 (1.9.1)
wheel (0.24.0)
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(venv) stam#stam:~$
now starting a python interpreter and trying to import cvxopt (note it is in the list above) gives me a importError. Why is this? I'm using the right python version, I work on the venv so everything should work out of the box.
(venv) stam#stam:~$ python
Python 2.7.9 (default, Aug 13 2016, 16:41:35)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cvxopt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named cvxopt
>>>
I'm a bit lost and don't know how exactly I can resolve this issue. It seems the right python version is started, at least the one I've attached to the virtualenv.
Additional outps asked for in the comments
(venv) stam#stam:~$ which python
/usr/bin/python
(venv) stam#stam:~$ which pip
/usr/local/bin/pip

Based on the output from which it seems clear that your python and pip belong to two different instances. So all libraries you install will be installed for some other python instance.
The acute fix is to force pip to run through your specified interpreter:
python -m pip list --local
python -m pip install cvxopt

Debian No Module named numpy

I've installed Python Numpy on Debian using...
apt-get install python-numpy
But when run the Python shell I get the following...
Python 2.7.10 (default, Sep 9 2015, 20:21:51)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named numpy
When I view the contents of /usr/local/lib/python2.7/site-packages/ I noticed numpy is not list.
If I install it via pip i.e pip install numpy it works just fine, However, I want to use the apt-get method. What I'm I doing wrong?
Other:
echo $PYTHONPATH /usr/local/lib/python2.7
dpkg -l python-numpy...
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================================-============================-============================-====================================================================================================
ii python-numpy 1:1.8.2-2 amd64 Numerical Python adds a fast array facility to the Python language
Python 2.7.10
['', '/usr/local/lib/python2.7', '/usr/local/lib/python27.zip', '/usr/local/lib/python2.7/plat-linux2', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/site-packages']
which -a python...
/usr/local/bin/python
/usr/bin/python
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

As you can tell from your which result, the python you are running when just typing python is /usr/local/bin/python.
It's a python you probably installed there yourself, as Debian will never put anything in /usr/local by itself (except for empty directories).
How? Well, by running pip for instance. As a rule, you should never use pip outside of a virtualenv, because it will install stuff on your system that your package manager will not know about. And maybe break stuff, like what you see on your system.
So, if you run /usr/bin/python, it should see the numpy package you installed using your package manager.
How to fix it? Well, I would clear anything in /usr/local (beware, it will definitely break stuff that rely on things you installed locally). Then I would apt-get install python-virtualenv, and always work with a virtualenv.
$ virtualenv -p /usr/bin/python env
$ . env/bin/activate
(env)$ pip install numpy
(env)$ python
>>> import numpy
>>>
That way, packages will be installed in the env directory. You do all this as a regular user, not root. And your different projects can have different environments with different packages installed.

Python virtualenv picks up wrong library

$ virtualenv --version
13.0.3
I create a new virtualenv with Python3 without access to global site packages.
$ virtualenv --no-site-packages venv_pygments --python=/usr/local/bin/python3
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4'
New python executable in venv_pygments/bin/python3.4
Also creating executable in venv_pygments/bin/python
Installing setuptools, pip, wheel...done.
Then I use the virtualenv's Python3 interpreter and try to import pygments
$ cd venv_pygments
$ venv_pygments bin/python3
Python 3.4.3 (default, May 1 2015, 19:14:18)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.49)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygments
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Applications/QGIS.app/Contents/Resources/python/pygments/__init__.py", line 46
except TypeError, err:
^
SyntaxError: invalid syntax
pip freeze though only shows these packages
$ bin/pip freeze
wheel==0.24.0
So it seems like the virtualenv's Python3 is accessing global site packages. How can I avoid that?
It doesn't change when I install pygments for the virtualenv
$ bin/pip install pygments
Collecting pygments
Using cached Pygments-2.0.2-py3-none-any.whl
Installing collected packages: pygments
Successfully installed pygments-2.0.2
$ venv_pygments bin/python3
Python 3.4.3 (default, May 1 2015, 19:14:18)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.49)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygments
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Applications/QGIS.app/Contents/Resources/python/pygments/__init__.py", line 46
except TypeError, err:
^
SyntaxError: invalid syntax
>>>
Note: The original question was updated based on the comments.

Create isolated environment without access to global site-packages. Either enable/disable that behaviour intentionally. Recent versions disable access by default.
I assume you want an isolated environment to test Python 3.3.4.
Below I am using virtualenv provided by system python 2.7.6
$ virtualenv --version
1.11.2
$ virtualenv
You must provide a DEST_DIR
Usage: virtualenv [OPTIONS] DEST_DIR
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-v, --verbose Increase verbosity.
-q, --quiet Decrease verbosity.
-p PYTHON_EXE, --python=PYTHON_EXE
The Python interpreter to use, e.g.,
--python=python2.5 will use the python2.5 interpreter
to create the new environment. The default is the
interpreter that virtualenv was installed with
(/usr/bin/python)
--clear Clear out the non-root install and start from scratch.
--no-site-packages DEPRECATED. Retained only for backward compatibility.
Not having access to global site-packages is now the
default behavior.
--system-site-packages
Give the virtual environment access to the global
site-packages.
--always-copy Always copy files rather than symlinking.
--unzip-setuptools Unzip Setuptools when installing it.
--relocatable Make an EXISTING virtualenv environment relocatable.
This fixes up scripts and makes all .pth files
relative.
--no-setuptools Do not install setuptools (or pip) in the new
virtualenv.
--no-pip Do not install pip in the new virtualenv.
--extra-search-dir=DIR
Directory to look for setuptools/pip distributions in.
This option can be used multiple times.
--never-download DEPRECATED. Retained only for backward compatibility.
This option has no effect. Virtualenv never downloads
pip or setuptools.
--prompt=PROMPT Provides an alternative prompt prefix for this
environment.
--setuptools DEPRECATED. Retained only for backward compatibility.
This option has no effect.
--distribute DEPRECATED. Retained only for backward compatibility.
This option has no effect.
I create a virtualenv, change into the directory and use relative paths to address python interpreter and pip from inside this virtualenv.
$ virtualenv --no-site-packages venv_pygments
New python executable in venv_pygments/bin/python
Installing setuptools, pip...done.
$ cd venv_pygments/
$ bin/pip install pygments
Downloading/unpacking pygments
Downloading Pygments-2.0.2-py2-none-any.whl (672kB): 672kB downloaded
Installing collected packages: pygments
Successfully installed pygments
Cleaning up...
$ bin/python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygments
>>> pygments.__version__
'2.0.2'
While creating your virtualenv you need to keep using the option
--python=/usr/local/bin/python3
to refer to your Python 3.3.4 interpreter as you already do.
A recent default virtualenv usually has only a few packages installed. If yours has more you may have access to global site-packages.
$ bin/pip freeze
Pygments==2.0.2
argparse==1.2.1
wsgiref==0.1.2