Fix multiple python installations - python

It looks like I have multiple python installations, which are making things difficult -- e.g. pip will tell me that a package has been installed through pip freeze, but when attempting to use it within a program, I get an error that it does not exist. Here are a few diagnostic tests I've run:
>>> import sys
>>> sys.executable
'/usr/local/opt/python/bin/python2.7'
>>> sys.exec_prefix
'/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7'
>>> print '\n'.join(sys.path)
/usr/local/lib/python2.7/site-packages/setuptools-15.2-py2.7.egg
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/site-packages
/Library/Python/2.7/site-packages/ipython-2.1.0-py2.7.egg
/Library/Python/2.7/site-packages/gnureadline-6.3.3-py2.7-macosx-10.9-intel.egg
/Library/Python/2.7/site-packages/pip-6.1.1-py2.7.egg
/Library/Python/2.7/site-packages
As well as checking the path for python and some common tools:
which easy_install
/usr/local/bin/easy_install
which python
/usr/local/bin/python
which pip
/usr/local/bin/pip
How can I fix it, short of doing a clean re-install of OS X?
Edited to add $PATH and .bashrc info:
My $PATH is as follows:
Users/me/workspace/google-cloud-sdk/bin:/usr/local/bin:/Users/me/workspace/google-cloud-sdk/bin:/urs/local/bin:/Users/me/workspace/google-cloud-sdk/bin:/urs/local/bin:/Users/me/workspace/google-cloud-sdk/bin:usr/local/python:usr/bin/python:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/usr/texbin: No such file or directory
-- the fact that it is returning an error makes me suspect that I've set up my .bash_profile wrong somehow. These are the only modifications I have made:
#Set architecture flags
export ARCHFLAGS="-arch x86_64"
#Ensure user-installed binaries take precedence
export PATH=/usr/local/bin:$PATH
#Load .bashrc if it exists
test -f ~/.bashrc && source ~/.bashrc
# The next line updates PATH for the Google Cloud SDK.
source '/Users/me/workspace/google-cloud-sdk/path.bash.inc'
# The next line enables bash completion for gcloud.
source '/Users/me/workspace/google-cloud-sdk/completion.bash.inc'
And this is my .bashrc:
# pip should only run if there is a virtualenv currently activated
export PIP_REQUIRE_VIRTUALENV=true
# cache pip-installed packages to avoid re-downloading
export PIP_DOWNLOAD_CACHE=$HOME/.pip/cache

Couldn't find a way to fix it, so ended up doing a clean OS install and setting up a new python dev environment.

Related

Visual Studio Code is rejecting the Tensorflow installation in a Virtual Environment

I created a virtual environment called env using
python -m venv env
.\env\Scripts\activate.bat
pip install tensorflow
I verified tensorflow is in the env\Lib\site-packages folder
Next I loaded VS Code and created a workspace, added a python file, it prompted me to install pylint,
I typed in python: select interpreter and I browsed to C:\Users\admin\env\Scripts folder
This is the command line at the beginning of the script
(env) PS C:\Users\admin\env\project> cd 'c:\Users\admin\env\project'; & 'C:\Users\admin\env\Scripts\python.exe' 'c:\Users\admin\.vscode\extensions\ms-python.python-2020.8.106424\pythonFiles\lib\python\debugpy\launcher' '54436' '--' 'c:\Users\admin\env\project\face_gan.py'
This is the error I get when debugging the python file:
ImportError: Keras requires TensorFlow 2.2 or higher. Install TensorFlow via `pip install tensorflow`
PS C:\Users\admin\env\project> & C:/Users/admin/env/Scripts/Activate.ps1
When I type in pip install tensorflow in VS Code terminal, it shows its already installed
(env) PS C:\Users\admin\env\project> pip install tensorflow
Requirement already satisfied: tensorflow in c:\users\admin\env\lib\site-packages (2.3.0)
I don't understand this, is it not running in virtual environment?
Why is it executing C:/Users/admin/env/Scripts/Activate.ps1 at the end of the debugging session, not at the beginning
Lastly, is running python from the virtual environment folder C:\Users\admin\env\Scripts the same as using the activate.bat file or the source command? Does it automatically defer to using the C:\Users\admin\env\Lib folder, or is it still trying to use the default python installation to look for Tensorflow?
What step did I miss to make it use the virtual environment correct in VS Code?
First question: executing C:/Users/admin/env/Scripts/Activate.ps1 after debugging command make no difference. It just because it's the first command of the terminal. You can run it again to make a try.
Second question: Yes, that's the same. In your case, it will add 'C:\Users\admin\env' and 'C:\Users\admin\env\lib\site-packages' path to the PYTHONPATH variable.
You can through these codes to get the PYTHONPATH(the default search path for module files) variable value:
import sys
print(sys.path)
If you import 'tensorflow' directly. you will find you can import it correctly. It's a version problem. You should downgrade the version of the packages, and you can refer to this comment to get some useful information.

"which" not able to locate packages installed with pip (WSL)

I've attempted to start writing Python in WSL, but the whole experience has been super messy. To install packages, I know I want to use pip3, but any pip3 commands result in
Traceback (most recent call last):
File "/usr/bin/pip3", line 9, in <module>
from pip import main
ImportError: cannot import name 'main'
My work-around has been to use the following command to install, which has worked:
python3 -mpip install --user somePackageName
My real confusion now comes from the fact that the which command is not locating packages installed by the above command. However, I know they are there as they appear when I run 'python3 -mpip freeze'. Is this because they are installed locally?
(Also If anyone has experience with Python and pip in WSL and could weigh in on how to go about configuring my Python environment, in order to avoid these types of issues, that would be super helpful.)
Thanks!
Try: python -m site
(perhaps replacing python by python3, etc)
You should get a short output with the following block,
From Windows:
sys.path = [
'<your current working directory>',
'C:\\Python38a2\\python38.zip',
'C:\\Python38a2\\DLLs',
'C:\\Python38a2\\lib',
'C:\\Python38a2',
'C:\\Python38a2\\lib\\site-packages',
]
From WSL:
sys.path = [
'<your current working directory>',
'/usr/lib/python37.zip',
'/usr/lib/python3.7',
'/usr/lib/python3.7/lib-dynload',
'/home/<username>/.local/lib/python3.7/site-packages',
'/usr/local/lib/python3.7/dist-packages',
'/usr/lib/python3/dist-packages',
]
That's where you'll want to look for installed packages.
The order here is the import precedence (earlier entries "trump" later ones)
For pip installed packages, you'll generally want to look in the site-packages directory.
For <package manager> installed packages you'll generally want to look in the dist-packages directory.
Lastly, which is generally used to find executables on your $PATH environment variable (and, at least for me, requires that they have the executable bit set to appear in the results).
Here, you're looking for something different, modules or packages that are importable by Python. In other words, modules or packages on your $PYTHONPATH.
These are two different types of "path"s.
Inside your interpreter, $PYTHONPATH maps to sys.path and $PATH maps to os.environ['PATH'].

How to change default install location for pip

I'm trying to install Pandas using pip, but I'm having a bit of trouble. I just ran sudo pip install pandas which successfully downloaded pandas. However, it did not get downloaded to the location that I wanted. Here's what I see when I use pip show pandas:
---
Name: pandas
Version: 0.14.0
Location: /Library/Python/2.7/site-packages/pandas-0.14.0-py2.7-macosx-10.9-intel.egg
Requires: python-dateutil, pytz, numpy
So it is installed. But I was confused when I created a new Python Project and searched under System Libs/lib/python for pandas, because it didn't show up. Some of the other packages that I've downloaded in the past did show up, however, so I tried to take a look at where those were. Running pip show numpy (which I can import with no problem) yielded:
---
Name: numpy
Version: 1.6.2
Location: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
Requires:
Which is in a completely different directory. For the sake of confirming my error, I ran pip install pyquery to see where it would be downloaded to, and got:
Name: pyquery
Version: 1.2.8
Location: /Library/Python/2.7/site-packages
Requires: lxml, cssselect
So the same place as pandas...
How do I change the default download location for pip so that these packages are downloaded to the same location that numpy is in?
Note: There were a few similar questions that I saw when searching for a solution, but I didn't see anything that mentioned permanently changing the default location.
According to pip documentation at
http://pip.readthedocs.org/en/stable/user_guide/#configuration
You will need to specify the default install location within a pip.ini file, which, also according to the website above is usually located as follows
On Unix and Mac OS X the configuration file is: $HOME/.pip/pip.conf
On Windows, the configuration file is: %HOME%\pip\pip.ini
The %HOME% is located in C:\Users\Bob on windows assuming your name is Bob
On linux the $HOME directory can be located by using cd ~
You may have to create the pip.ini file when you find your pip directory. Within your pip.ini or pip.config you will then need to put (assuming your on windows) something like
[global]
target=C:\Users\Bob\Desktop
Except that you would replace C:\Users\Bob\Desktop with whatever path you desire. If you are on Linux you would replace it with something like /usr/local/your/path
After saving the command would then be
pip install pandas
However, the program you install might assume it will be installed in a certain directory and might not work as a result of being installed elsewhere.
You can set the following environment variable:
PIP_TARGET=/path/to/pip/dir
https://pip.pypa.io/en/stable/user_guide/#environment-variables
Open Terminal and type:
pip config set global.target /Users/Bob/Library/Python/3.8/lib/python/site-packages
except instead of
/Users/Bob/Library/Python/3.8/lib/python/site-packages
you would use whatever directory you want.
Follow these steps
pip config set global.target D:\site-packages to change install path
or py -m pip config --user --editor notepad edit
[global]
target = D:\site-packages
set environment variable to use download import xxx
PIP_TARGET=site-packages
PYTHONPATH=site-packages
3.pip config unset global.target, to upgrade pip py -m pip install --upgrade pip
#Austin's answer is outdated, here for more up-to-date solution:
According to pip documentation at
https://pip.pypa.io/en/stable/topics/configuration/
You will need to specify the default install location within a configuration file, which, also according to the website above is usually located as follows
Mac OS
$HOME/Library/Application Support/pip/pip.conf if directory $HOME/Library/Application Support/pip exists else $HOME/.config/pip/pip.conf.
The legacy “per-user” configuration file is also loaded, if it exists: $HOME/.pip/pip.conf.
The $HOME folder can be located by navigating to ~/ (cmd+shift+G in Finder; cmd+shift+. to show hidden files).
Windows
%APPDATA%\pip\pip.ini
The legacy “per-user” configuration file is also loaded, if it exists: %HOME%\pip\pip.ini
The %HOME% is located in C:\Users\Bob on windows assuming your username is Bob
Unix
$HOME/.config/pip/pip.conf, which respects the XDG_CONFIG_HOME environment variable.
The legacy “per-user” configuration file is also loaded, if it exists: $HOME/.pip/pip.conf.
On linux the $HOME directory can be located by using cd ~
You may have to create the configuration file when you find your pip directory. Put something like
[global]
target = /Library/Frameworks/Python.framework/Versions/Current/lib/python3.10/site-packages/
if you are on a Mac. Except that you would replace /Library/Frameworks/Python.framework/Versions/Current/lib/python3.10/site-packages/ with whatever path you desire. If you are on Linux you would replace it with something like /usr/local/your/path
After saving the command would then be
pip install pandas
However, the program you install might assume it will be installed in a certain directory and might not work as a result of being installed elsewhere.
Please note that
pip3 install pandas
might be the solution if your packages gets installed in the Python2 folder vs Python3.

virtualenv error: "'.../bin/easy_install' not found"

using python3.3 on OS X 10.7.5
when i run
virtualenv-3.3 virtualenv/
it errors on
FileNotFoundError: [Errno 2] No such file or directory: '.../virtualenv/bin/easy_install'
after
Overwriting virtualenv/lib/python3.3/site.py with new content
Overwriting virtualenv/lib/python3.3/orig-prefix.txt with new content
Overwriting virtualenv/lib/python3.3/no-global-site-packages.txt with new content
New python executable in virtualenv/bin/python3.3
Not overwriting existing python script virtualenv/bin/python (you must use virtualenv/bin/python3.3)
Overwriting virtualenv/lib/python3.3/distutils/__init__.py with new content
File virtualenv/lib/python3.3/distutils/distutils.cfg exists with different content; not overwriting
Installing distribute...done.
Installing pip...
my system has setuptools / distribute.
why isn't virtualenv setting up easy_install?
This issue is related to having an outdated version of Pip, I think.
I got a similar error on CEntOS 6.3.
I suggest at least attempting to bootstrap a virtual environment. If it doesn't work, all you have to lose are some megabytes and a 20 minutes of your time.
What you will need:
Compiled Python3.4
Source of the latest VirtualEnv
Fortitude
Pen and paper
Something similar to the answer outlined here might work: What's the proper way to install pip, virtualenv, and distribute for Python?
In short, download the virtualenv package from source from PyPI, extract it, and cd into the directory created by the extraction.
From the documentation:
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz
$ tar xvfz virtualenv-X.X.tar.gz
$ cd virtualenv-X.X
Replace the X.X above with the version number. For example, cd virtualenv-X.X would, after replacing the Xs with the right version number, be cd virtualenv-13.1.2.
Then, use virtualenv.py from there, not just $ virtualenv, but $ ./virtualenv.py.
I hope that this would work much better, since pip with the latest virtualenv will include easy_install in the expected location.
I have tested this on a CEntOS 6.3 machine with a similar configuration. It works on there, but I cannot afford a copy of OS X 10.7.5.

How to use easy_install to install locally?

I try to install PyTables package using easy_install.
My problem is that I am not root on the system and am not allowed to write to /usr/local/lib/python2.7/dist-packages/ directory.
To solve this problem I decided to install locally. For that I have created a new directory: /home/myname/mylibs. Then I executed easy_install -d /home/myname/mylibs tables. As a result easy_install complained about the PYTHONPATH environment variable. To resolve this problem I added my new directory to the PYTHONPATH and repeated the previous command. As a result I get syte.py and syte.pyc file in my /home/myname/mylibs directory.
However, when I try import tables from Python it still does not work. What also worries me is that the new files do not have "table" in their names as well in their content.
Can anybody, please, explain me what is going on?
I had a similar problem recently trying to use easy_install to put packages in a location I had write-access to. Like you, I had to add the location to my PYTHONPATH variable to get it to install. It was then necessary to set PYTHONPATH every time I wanted to run a command using the installed package(s).
For example, to install and use the flask package I needed to do this:
$ PYTHONPATH=/tmp easy_install -d /tmp flask
$ PYTHONPATH=/tmp python -c 'import flask; print(flask.__version__)'
0.10.1
Without the PYTHONPATH variable set you'd get an error:
$ python -c 'import flask; print(flask.__version__)'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named flask
You can of course set the variable permanently by running:
$ export PYTHONPATH=/tmp
Or putting export PYTHONPATH=/tmp in your ~/.bashrc file.

Categories