We Keep Coding

Confused Pip install with Virtual Environments

Probably a rookie question: I created a project folder, cd'ed to it, created a virtual environment via
python -m venv .venv
and it made the VE in the right place with what looks like the right stuff. But pip-installing modules sometimes (??) fails, with the module going into the global site-packages. This is a vanilla Python 3.9 install on Linux (Raspberry Pi) and being used within Visual Studio Code. To get it right I need to use the "Pip Manager" extension. Why?? To illustrate my confusion, I understand that Python (and Pip?) uses sys.path to resolve packages (see XKCD 1897). So after activating the VE, I looked and got this? I expected to see the .venv site-packages in that path first, right? Well here is what I get:
Where am I misinformed or otherwise confused?
Info added in response to questions below
This system never had Python2 on it. Within the activated .venv there are links:
pi#rpi1:~/Documents/alpyca-device*.venv/bin $ ls -l pyt*
lrwxrwxrwx 1 pi pi 15 Dec 14 14:41 python -> /usr/bin/python
lrwxrwxrwx 1 pi pi 6 Dec 14 14:41 python3 -> python
lrwxrwxrwx 1 pi pi 6 Dec 14 14:41 python3.9 -> python
and this is the usual way it is put into the Python3 virtuals by the venv module, which is recommended now over virtualenv and other schemes.
(.venv) pi#rpi1:~/Documents/alpyca $ which python
/home/pi/Documents/alpyca/.venv/bin/python
and other requests for info:
(.venv) pi#rpi1:~/Documents/alpyca $ grep VIRTUAL_ENV= .venv/bin/activate
VIRTUAL_ENV="/home/pi/Documents/alpyca/.venv"
(.venv) pi#rpi1:~/Documents/alpyca $ .venv/bin/python
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/pi/Documents/alpyca/.venv/lib/python3.9/site-packages']
>>>
I don't know about Python 2, I have never used it.

MacOS: Multiple python versions installed and unmanageable. Cleanup wanted

I've been on macOS for 2 years now and in the past year I've begun to work a lot in Python via VScode. But lately I have been running into so many problems because I didn't set up python properly from the start. I have multiple versions and modules installed globally (I know that is bad)... But I was wondering if anyone had advice about how I can clean up the Python set up so that there is the latest version being used and all modules will be installed properly.
I used homebrew as well and that is just adding to the mess. I want to do this right so that I can stop messing with configurations every day and just be able to develop.
I'll include some basic terminal outputs but if there is more that anyone would like to see I would be happy to provide more detail. If starting from scratch is the best thing to do then I'll do it. I don't know my way around all the configuration files and pathing so I'll need some help if that is what I'll have to do.
$ which python
/usr/bin/python
$ which python3
/usr/local/bin/python3
$ python --version
Python 2.7.16
$ python3 --version
Python 3.7.7
$ python3
Python 3.7.7 (default, Mar 10 2020, 15:43:33)
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
>>> tensorflow.__file__
'/usr/local/lib/python3.7/site-packages/tensorflow/__init__.py'
VScode has the following interpreters available (not sure if that helps)
2.7.16 /usr/bin/python
2.7.16 /System/Library/Frameworks....
3.7.3 /usr/bin/python3
3.7.7 /usr/local/bin/python3
3.7.7 /usr/local/opt/python/bin/python3.7
Any help would be awesome!! I'm just tired of fighting with this and wanted to ask for help

Having multiple versions of python is not really a propblem per se.
What I recommend is :
# In $HOME/.bashrc or .zshrc
PATH=/usr/local/bin:$PATH
cd /usr/local/bin
ln -fs python3 python
# Once the first and this step done, when you type [python],
# you'll be using /usr/local/bin/python3
As the first line of your python scripts, put :
#!/usr/bin/env python
This way, you ensure your are always using the version /usr/local/bin/python3

Python 2.7 crashes when importing PyQt4.QtDeclarative or PyQt4.Qt on Ubuntu

Some time ago (months?) the program rqt_plot started crashing on startup (SIGSEGV) on my machine. I finally tracked it down a little deeper and found that the problem occurs while python is trying to import PyQt4.QtDeclarative. Unfortunately I don't remember when this started happening, and my Internet searches have turned up nothing. Any ideas what's going wrong? I suspect an incompatible package update somewhere along the way, but have no idea how to find the root cause.
Here's a simple session transcript:
$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt4 import QtDeclarative
Segmentation fault (core dumped)
$
Here's some system information:
$ uname -a
Linux [HOSTNAME] 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ echo $PYTHONPATH
$ which python
/usr/bin/python
$ ls -l /usr/bin/python
lrwxrwxrwx 1 root root 9 Jan 8 2015 /usr/bin/python -> python2.7
Follow-up:
I found later that I had the same problem (Segmentation fault) when doing import PyQt4.Qt as well.

It can be difficult to diagnose a segfault when the only error message is
Segmentation fault (core dumped)
In this case, because reinstallation of python-qt4 does not take much time at all, I would recommend you do it by running:
sudo apt-get install --reinstall python-qt4
Edit: It looks like OP encountered another segfault when doing import PyQt4.Qt. This is probably related to python-sip, which is a dependency of python-pyqt4. To get rid of the segfault, reinstall python-sip by running:
sudo apt-get install --reinstall python-sip

Two Python Version conflict in Ubuntu Oneiric 11.10 issue: ImportError: No module

I installed python 2.7.5 using make install not altinstall
Previously it was 2.7.2+
now when i run python it gives
ImportError: cannot import name MAXREPEAT
I know there are already entries related to that here. But I dont have virtualenv installed neither did i have before.
If I run python2.7
It runs perfectly.
Now when I am trying to import a library like mysqldb it says the module doesnt exist.
i installed it before installing python2.7.
What is the solution? Should I uninstall the whole python or rather how should I uninstall python2.7.5 only or should i keep that? If I keep that then how can i import those modules?
** Latest UPDATE**
me#me:/usr/local/bin$ dir python2*
python2 python2-config python2.7 python2.7-config
AND:
/usr/local/bin$ ls -l
total 1776
-rwxrwxr-x 1 root root 101 Jul 17 20:17 2to3
-r-xr-xr-x 1 root root 7223 Mar 31 20:47 config_data
-rwxr-xr-x 1 root root 299 Jul 18 20:53 easy_install
-rwxr-xr-x 1 root root 307 Jul 18 20:53 easy_install-2.7
-rwxrwxr-x 1 root root 99 Jul 17 20:17 idle
-r-xr-xr-x 1 root root 3910 Mar 31 20:47 json_pp
-r-xr-xr-x 1 root root 487 Mar 31 20:47 package-stash-conflicts
-rwxr-xr-x 1 root root 281 Jul 17 19:51 pip
-rwxr-xr-x 1 root root 289 Jul 17 19:51 pip-2.7
-rwxrwxr-x 1 root root 84 Jul 17 20:17 pydoc
lrwxrwxrwx 1 root root 7 Jul 18 23:04 python -> python2
lrwxrwxrwx 1 root root 14 Jul 18 23:04 python-config -> python2-config
lrwxrwxrwx 1 root root 9 Jul 18 23:04 python2 -> python2.7
lrwxrwxrwx 1 root root 16 Jul 18 23:04 python2-config -> python2.7-config
-rwxr-xr-x 1 root root 1737376 Jul 18 23:03 python2.7
-rwxr-xr-x 1 root root 1674 Jul 18 23:04 python2.7-config
-rwxrwxr-x 1 root root 18547 Jul 17 20:17 smtpd.py
-rwxr-xr-x 1 root root 316 Jul 18 21:10 virtualenv
-rwxr-xr-x 1 root root 324 Jul 18 21:10 virtualenv-2.7
would reinstalling this solve the problem of getting and running the default 2.7.2+ in ubuntu back?: https://launchpad.net/ubuntu/oneiric/+source/python-defaults/2.7.2-7ubuntu2
* Prior UPDATE*
Now I can run python from bash
Here is the new problem. and it is with every module that I install using pip.
existing modules from lib can be imported but newly installed ones cant be:
me:~$ sudo pip install requests
Downloading/unpacking requests
Downloading requests-1.2.3.tar.gz (348kB): 348kB downloaded
Running setup.py egg_info for package requests
Installing collected packages: requests
Running setup.py install for requests
Successfully installed requests
Cleaning up...
me:~$ python -i
Python 2.7.5 (default, Jul 17 2013, 20:16:12)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
>>>
After checking version of python it is running locally, for conflict issues:
me:~$ python
Python 2.7.5 (default, Jul 17 2013, 20:16:12)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
me:~$ sudo python
[sudo] password for me:
Python 2.7.5 (default, Jul 17 2013, 20:16:12)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
me:~$ which python
/usr/local/bin/python
me:~$ sudo which python
/usr/local/bin/python
me:~$ sudo which python2.7
/usr/local/bin/python2.7
me:~$ which python2.7
/usr/local/bin/python2.7
I think its running the same. and then..
me:/usr/local/bin$ dir python2.7
python2.7
me:/usr/local/bin$ cd /usr/bin
me:/usr/bin$ dir python2.7
python2.7
me:/bin$ cd /usr/bin
me:/usr/bin$ dir python
python
me:/usr/local/bin$ dir python
python
me:/usr/local/bin$ dir python2.7
python2.7
Also check following:
~$ which pip
/usr/local/bin/pip
Ok now I installed virtualenv
and this is what i did
(foldername)me#me:~/caller$ sudo pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in /usr/local/lib/python2.7/dist-packages
Cleaning up...
(foldername)me#me:~/caller$ python -i
Python 2.7.2+ (default, Jul 20 2012, 22:15:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
And NOW we can see that 2.7.2+ is back again! So what is happening now??
what is wrong?
Uninstalling older python from linux is not recommended. So how can i run virtualenv with 2.7.5 on it and not 2.7.2+ ?!

When installing python use the following steps
using prefix to specify the installation directory
./configure --prefix=/scratch/bin/
make
sudo make install
Then everytime u run a new Terminal u have specify
export PATH="$PATH:/scratch/bin/"
to tell where is the installation directory of Python
This way u can use any number of pythons

Have you installed it using apt-get or built from sources?
If built from sources, are you sure the installation has finished successfully? Usually in order to build Python from sources one has to do the following steps
./configure
make
sudo make install (sudo might not be required, but the make script will attempt to change files in /usr/ directory.
in your python sources directory. The last command, amongst others, copies python to the /usr/ directory. If you want to have it installed somewhere else you'd have to pass --path=XXX (if i'm not mistaken) to ./configure.

pip is a Python program. When you run pip from the command line, it uses a particular Python executable to run the program. pip will install modules in a location where that Python executable can find them.
When you run sudo pip install some_package, the OS looks at the shebang line to determine what Python executable to run.
Type which pip and look at the first line in the file to see which python executable it is linked to.
To install packages for your new python2.7.5, you could either install a new pip. Or, you could just run /usr/local/bin/python pip ..., but I think the easiest way to manage multiple python installations is to use virtualenv.
If you use virtualenv (which I'd highly recommend), then a new pip is automatically installed since it is one of its dependencies.
If you go this route, I'd recommend installing virtualenvwrapper too.
If you install pip globally, notice the line
python get-pip.py
or
python setup.py install
The python executable you use on this line sets the "link" between that version of pip and that version of python.
How to use checkinstall to remove a custom-built Python:
sudo apt-get install libsqlite3-dev libgdbm-dev liblzma-dev checkinstall
sudo apt-get build-dep python
http://www.python.org/ftp/python/2.7.5/Python-2.7.5.tar.bz2
tar xvjf Python-2.7.5.tar.bz2
cd Python-2.7.5
# inspect the options
./configure --help
./configure
make
make test
sudo checkinstall -D --fstrans=no make install
# Accept the default answers EXCEPT that you MUST change the name to something
# other than python. That way, the name of your custom package does not conflict
# with the name of Ubuntu's "python" package
Should I create a default set of package docs? [y]:
Enter a number to change any of them or press ENTER to continue: 2
Enter new name:
>> python275
Copying files to the temporary directory...OK
# wait, wait, wait...
# python2.7.5 is now installed in /usr/local/bin
# remove it with:
sudo dpkg -r python275

python location on mac osx

I'm a little confused with the python on osx. I do not know if the previous owner of the laptop has installed macpython using macport. And I remembered that osx has an builtin version of python. I tried using type -a python and the result returned
python is /usr/bin/python
python is /usr/local/bin/python
However running both python at these locations give me [GCC 4.2.1 (Apple Inc. build 5646)] on darwin. Do they both refer to the same builtin python mac provided?
I also read that installing macpython one would
A MacPython 2.5 folder in your Applications folder. In here you
find IDLE, the development environment that is a standard part of
official Python distributions...
I looked at Applications, and theres a MacPort folder with python2.6 and the mentioned stuff in it. But running IDLE, i find the same message as above.
Hmm I'm a little confused. Which is which?

This one will solve all your problems not only on Mac but
to find it on Linux also ( & every basic shell).
TL;DR (you don't have to go through all the answer - just the 1st half).
LET'S GO
Run in terminal:
which python3
On Mac you should get:
/usr/local/bin/python3
WAIT!!! It's prob a symbolic link, how do you know? Run:
ls -al /usr/local/bin/python3
and you'll get (if you've installed Python w/ Brew):
/usr/local/bin/python3 -> /usr/local/Cellar/python/3.6.4_4/bin/python3
which means that your
/usr/local/bin/python3
is actually pointing to (the real location)
/usr/local/Cellar/python/3.6.4_4/bin/python3
That's it!
Longer version (optional):
If for some reason, your
/usr/local/bin/python3
is not pointing to the place you want, which in our case:
/usr/local/Cellar/python/3.6.4_4/bin/python3
just back it up (+cool trick to add .orig suffix to file):
cp /usr/local/bin/python3{,.orig}
and run:
rm -rf /usr/local/bin/python3
now create a new symbolic link:
ln -s /usr/local/Cellar/python/3.6.4_4/bin/python3 /usr/local/bin/python3
and now your
/usr/local/bin/python3
is pointing to
/usr/local/Cellar/python/3.6.4_4/bin/python3
Check it out by running:
ls -al /usr/local/bin/python3

I found the easiest way to locate it, you can use
which python
it will show something like this:
/usr/bin/python

[GCC 4.2.1 (Apple Inc. build 5646)] is the version of GCC that the Python(s) were built with, not the version of Python itself. That information should be on the previous line. For example:
# Apple-supplied Python 2.6 in OS X 10.6
$ /usr/bin/python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
# python.org Python 2.7.2 (also built with newer gcc)
$ /usr/local/bin/python
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
Items in /usr/bin should always be or link to files supplied by Apple in OS X, unless someone has been ill-advisedly changing things there. To see exactly where the /usr/local/bin/python is linked to:
$ ls -l /usr/local/bin/python
lrwxr-xr-x 1 root wheel 68 Jul 5 10:05 /usr/local/bin/python# -> ../../../Library/Frameworks/Python.framework/Versions/2.7/bin/python
In this case, that is typical for a python.org installed Python instance or it could be one built from source.

On Mac OS X, it's in the Python framework in /System/Library/Frameworks/Python.framework/Resources.
Full path is:
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
Btw it's easy to find out where you can find a specific binary: which Python will show you the path of your Python binary (which is probably the same as I posted above).

I checked a few similar discussions and found out the best way to locate all python2/python3 builds is:
which -a python python3

On High Sierra
which python
shows the default python but if you downloaded and installed the latest version from python.org you can find it by:
which python3.6
which on my machine shows
/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6

installed with 'brew install python3', found it here

Run this in your interactive terminal
import os
os.path
It will give you the folder where python is installed

i found it here:
/Library/Frameworks/Python.framework/Versions/3.6/bin

which python3 simply result in a path in which the interpreter settles down.

run the following code in a .py file:
import sys
print(sys.version)
print(sys.executable)

I have a cook recipe for finding things in linux/macos
First update the locate db then do a
locate WHATiWANTtoSEARCH | less
do a /find to find what you are looking for.
to update your locate db in macos do this:
sudo /usr/libexec/locate.updatedb
it sometimes takes a while. Hope this helps :)