How to deal with system Python and `brew link python` on MacOS? - python

I cannot link Python 3.8 over system Python 3.9.
I have few installed python versions (3.8, 3.10, etc) by brew and system Python 3.9.
P.S. I cannot uninstall system one (it does not appear in Applications).
$ python3 --version
Python 3.10.9
$ brew unlink python#$(python3 --version | grep -oE '[0-9]+\.[0-9]+'); brew link python#3.8
Unlinking /opt/homebrew/Cellar/python#3.10/3.10.9... 25 symlinks removed.
Warning: Already linked: /opt/homebrew/Cellar/python#3.8/3.8.16
To relink, run:
brew unlink python#3.8 && brew link python#3.8
$ python3 --version
Python 3.9.6
$ type python3
python3 is /usr/bin/python3
$ python3[press TAB]
python3 python3-config python3.10 python3.10-config python3.11 python3.11-config
$ python3.8 --version
zsh: command not found: python3.8
$ echo $PATH
/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
Questions:
How to set brewed Python over system one?
How to uninstall system python? (not recommended)
How to link python in addition to python3?
Why there is no python3.8 available? (solved with brew reinstall python#3.8)
UPD:
python3.8 became available after brew reinstall python#3.8.
UPD 2:
It looks like there is no way to fix this properly. Must be a bug of 3.8 installer.
Here is workaround commands:
mkdir ~/bin/
ln -s /opt/homebrew/bin/python3.8 ~/bin/python
chmod u+x,og-rwx ~/bin/python
And this ~/bin to your PATH.
#micromoses described the same in Method #4.

How to set brewed Python over system one?
All you need to do is prepend /opt/homebrew/bin to the beginning of your path. Your $PATH variable seems to be correctly set, so if it is not working I'd try restarting your terminal.
If all else fails, you can also explicitly prepend /opt/homebrew/opt/python#VERION_NUMBER/bin to the beginning of your path.
How to link python in addition to python3?
If you run brew info python, Homebrew tells you this caveat:
Unversioned symlinks python, python-config, pip etc. pointing to
python3, python3-config, pip3 etc., respectively, have been installed into
/opt/brew/opt/python#3.10/libexec/bin
So, all you need to do is prepend /opt/brew/opt/python#VERSION_NUMBER/libexec/bin to the beginning of your path. Just add this line to the end of your zshrc, bashrc, or equivalent:
export PATH="/opt/brew/opt/python#VERSION_NUMBER/libexec/bin:$PATH"
This will make all the unversioned Python utils available to you.

It is best to avoid directly overriding (modifying) symbolic or hard links pointing to the system's python version. These might be used by system functions/scripts which are (hopefully) tested against that specific python binary. Changing the version (especially downgrading, but sometimes even upgrading) can break things for the OS.
For the same reason stated above, uninstalling/deleting the system's python should also be avoided.
I struggled myself with MacOS not having python as an executable, so I'll focus my answer on that.
Looks like you already found a way to resolve that.
I can think of 4 ways to get a python executable;
Method #1 - probably not what you want
The easy way to make python available in (z)shell on mac is to add an alias:
echo "alias python=python3" >> ~/.zshrc
With this, in all new zsh sessions (or where .zshrc is reloaded), running python will run the python3 executable in your PATH. You can also set the alias's target to be a full path, so point it to the python version you like (if, for example, one is installed using homebrew). There are two problems: changing the target requires reloading .zshrc, and this will only take effect when executing python from an open shell (will not work for executable scripts/entrypoints - more on that later).
Method #2 - optimal for homebrew
Unversioned python, pip and other such symlinks are installed in libexec/bin directories of brew packages. If you are using homebrew you can add these to your path, like so:
echo 'export PATH="$PATH:/opt/homebrew/opt/python#3/libexec/bin"' >> .zprofile
As homebrew uses symlinks for package versioning, you can replace python#3 in the above example with a more explicit version (e.g. python#3.8 or python#3.10).
Method #3 - requires admin privileges
If you have admin privileges, you can add a symlink in your path to the executable, like so (change /usr/bin/python3 to the brew's python executable you like):
sudo ln -s /usr/bin/python3 /usr/local/bin/python
While hardlinks will work as well, I recommend avoiding them as there might be a chance that it will negatively affect minor version upgrades by keeping the older executable (haven't tested myself).
The upside of this method is that it will probably work for all users and all applications, as it is extremely rare for applications to not include /usr/local/bin in the execution search path.
Method #4 - intermediary executable
This (much like method #2) probably won't work for applications that are not invoked by (z)shell (possibly your favorite IDE), but it is most flexible, and does not require admin privileges. It entails adding an intermediary binary, and adding that to your PATH. I usually create a bin directory in my homedir for such occasions, so I'll use that path in the examples. First, create the executable:
mkdir ~/bin
echo '#!/bin/sh\nexec /usr/bin/python3 "$#"' > ~/bin/python
chmod +x ~/bin/python
Now add ~/bin to your shell's path:
echo 'export PATH="$PATH:$HOME/bin"' >> .zprofile
Alternatively, .zshrc or .zshenv can be used, but additional precaution has to be taken to avoid repeatedly adding ~/bin to the path in subshells:
echo '_home_bindir="$HOME/bin"\n[ "${PATH##*:}" = "$_home_bindir" ] || export PATH="$PATH:$_home_bindir"' >> ~/.zshrc
Now, every new shell (or if using .zshrc - if it is reloaded) will have python executable in its path, and you can easily edit the file to point to your favorite python version.
Notes:
Method #2 is probably the best option if using homebrew, which is the case in question. Still, as I struggled with not having a python executable on a "clean" MacOS installation (without homebrew), I've added methods #1, #3, and #4, in case I'm not the only one not using homebrew's python.
Methods #2, #3, and #4, allow for executable scripts/entrypoints. By that, I'm referring to executable scripts with shebang, most commonly using the env executable:
#!/usr/bin/env python
These 3 methods will also honor additional python arguments (e.g. python -u).
As MacOS no longer comes with a python executable (only python3), I think that method #3 is fairly safe to use (at least from a perspective of not breaking OS scripts). As for $PATH manipulation, I wouldn't recommend prepending it with user-writeable paths (like ~/bin), as this may lead to a security issue, or worse - a debugging nightmare (of course, no one would ever create executables like ~/bin/ls or ~/bin/which, right?).
As for the specific shell, this will also work for bash by changing the relevant files (.z.. to .bash..) as long as the profile/rc files are properly loaded. For other shells (e.g. tcsh) the syntax needs to be adjusted.

Related

Virtualenv uses wrong python, even though it is first in $PATH

I had a problem where python was not finding modules installed by pip while in the virtualenv.
I have narrowed it down, and found that when I call python when my virtualenv in activated, it still reaches out to /usr/bin/python instead of /home/liam/dev/.virtualenvs/noots/bin/python.
When I use which python in the virtualenv I get:
/home/liam/dev/.virtualenvs/noots/bin/python
When I look up my $PATH variable in the virtualenv I get:
bash: /home/liam/dev/.virtualenvs/noots/bin:/home/liam/bin:/home/liam/.local/bin:/home/liam/bin:/home/liam/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin: No such file or directory
and yet when I actually run python it goes to /usr/bin/python
To make things more confusing to me, if I run python3.5 it grabs python3.5 from the correct directory (i.e. /home/liam/dev/.virtualenvs/noots/bin/python3.5)
I have not touched /home/liam/dev/.virtualenvs/noots/bin/ in anyway. python and python3.5 are still both linked to python3 in that directory. Traversing to /home/liam/dev/.virtualenvs/noots/bin/ and running ./python, ./python3 or ./python3.5 all work normally.
I am using virtualenvwrapper if that makes a difference, however the problem seemed to occur recently, long after install virtualenv and virtualenvwrapper
My problem was that i recently moved my project with virtualenv to another location, due to this activate script had wrong VIRTUAL_ENV path.
$ cat path_to_your_env/bin/activate
... # some declarations
VIRTUAL_ENV="/path_to_your_env/bin/python" # <-- THIS LINE
export VIRTUAL_ENV
... # some declarations
To fix this, just update VIRTUAL_ENV in activate script.
Also you maybe need to fix first line of your bin/pip to link to real python path.
As tdelaney suggested in the comments, I ran alias and found that I had previously aliased python to /usr/bin/python3.5 in my .bashrc.
I removed that alias from my .bashrc, ran unalias python, and source ~/.bashrc and the problem was solved.
If you don't get the program that which says you should get, you need to look higher up the chain than the platform executor. Shells typically have a way to alias commands and on most unixy shells you can just enter alias to see which commands have been remapped. Then its just a matter of going to the config files for your shell and removing the alias.
Sometimes people alias python to try to sort out which python they should be using. But there are usually other, better ways. On my linux machine, for example, python3 is in the path but is a symlink to the real python I am using.
td#mintyfresh ~ $ which python3
/usr/bin/python3
td#mintyfresh ~ $ ls -l /usr/bin/python3
lrwxrwxrwx 1 root root 9 Feb 17 2016 /usr/bin/python3 -> python3.4
td#mintyfresh ~ $
This is nice because non-shell programs running python get the same one I do and virtual environments work naturally.
On Cygwin, I still have a problem even after I created symlink to point /usr/bin/python to F:\Python27\python.exe. Here, after source env/Scripts/activate, which python is still /usr/bin/python.
After a long time, I figured out a solution. Instead of using virtualenv env, you have to use virtualenv -p F:\Python27\python.exe env even though you have created a symlink.
I'm currently having the same problem. Virtualenv was created in Windows, now I'm trying to run it from WSL.
In virtualenv I renamed python.exe to python3.exe(as I have only python3 command in WSL). In $PATH my virtualenv folder is first, there is no alias for python. I receive which python3
/usr/bin/python3. In /usr/bin/python3 there is symlink `python3 -> python3.6. I suppose it doesn't matter for order resolution.
Had the exact same problem.
I ran:
virtualenv -p /venv/bin/python3 env
and got a permission denied.
so i tried:
sudo chmod 777 -R /venv/bin
which python and print(sys.executable)
were not agreeing for me. This meant that with an active virtualenv pip install <package> would install to the virtualenv, but running python would be the base install.
I eventually got around this by running
virtualenv -p \path\to\python.exe --always-copy <venvName>
I'm not sure if specifying the path to the original python is really necessary, but can't hurt. According to the man page:
--copies, --always-copy try to use copies rather than symlinks, even when symlinks are the default for the platform (default: False)
I'm using windows powershell with msys64.

Install Python 3 to /usr/bin/ on macOS

I installed python2.x and python3.x using homebrew and the executable python paths are listed below:
$ which python
/Library/Frameworks/Python.framework/Versions/2.7/bin/python
$ which python3
/Library/Frameworks/Python.framework/Versions/3.5/bin/python3
It's quite too long and not so clean to write a shebang in a python code to make it runnable on Terminal:
#!/Library/Frameworks/Python.framework/Versions/2.7/bin/python OR
#!/Library/Frameworks/Python.framework/Versions/3.5/bin/python3
I prefer
#!/usr/bin/python OR
#!/usr/bin/python3
My issue here is, how can I correcly move or reinstall python on macOS to /usr/bin such as /usr/bin/python OR /usr/bin/python3 Instead of
/Library/Frameworks/Python.framework/Versions/2.7/bin/python
/Library/Frameworks/Python.framework/Versions/3.5/bin/python3
This is NOT possible on Mac OS X El Capitan anymore as from then on System Integrity Protection prevents that. More info in Cannot create a symlink inside of /usr/bin even as sudo
Create a symbolic link in /usr/bin/
Open terminal and do:
$ sudo ln /Library/Frameworks/Python.framework/Versions/2.7/bin/python /usr/bin/python
$ sudo ln /Library/Frameworks/Python.framework/Versions/3.5/bin/python3 /usr/bin/python3
You can now do what you wanted to do.
Edit: Unfortunately as you can read from the other answers, this solution no longer works on MacOS >= El Capitan due to System Integrity Protection. (See here)
A possible alternative is to use the folder /usr/local/bin that should be accessible.
The basic question in the OP seems to not be doable because the newer Mac OSes have "System Integrity Protection" which prevents "unauthorized" changes to key directories such as /usr/bin, and sudo cannot override that.
The suggestion of using /usr/local/bin and /usr/local/Frameworks seems like it should work (I haven't tried it). However, at https://opensource.com/article/19/5/python-3-default-mac, Matthew Broberg suggests that it is likely to create problems when updating.
In that same article, Moshe Zadka recommends using pyenv to manage Python environments and using shell aliases rather than symlinks.
I was going to go with the /usr/local suggestion above, but having found this contraindication, I'm going to try Moshe's method. I'll report back here if I hit any snags.
Update: I followed the method and I still was getting pip not found and which python and python -V were not giving me the expected results. Doh! I ran:
. ~/.bash_profile
(where I had put the recommended eval "$(pyenv init -)" command) and suddenly everything was finding the paths and versions as expected.

Error Installing OpenCV with Python on OS X

I have been trying to install Open CV 3 on my mac using this tutorial but I cannot get past step three.
So after I do
brew install python
I do
nano ~/.bash_profile
And the at the bottom of the script I paste
# Homebrew
export PATH=/usr/local/bin:$PATH
After that I reload the file like this
source ~/.bash_profile
Finally I check the python like this
which python
And it prints
/usr/bin/python
instead of
/usr/local/bin/python
I have also tried edited the file in TextEdit but it has the same result.
Am I doing something wrong or is this just a bad tutorial?
Thank You in Advance!
Edit:
# Setting PATH for Python 3.5
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.5/bin:${PATH}"
export PATH
##
# Your previous /Users/UserName/.bash_profile file was backed up as /Users/UserName/.bash_profile.macports-saved_2016-07-26_at_12:50:19
##
# MacPorts Installer addition on 2016-07-26_at_12:50:19: adding an appropriate PATH variable for use with MacPorts.
export PATH="/opt/local/bin:/opt/local/sbin:$PATH"
# Finished adapting your PATH environment variable for use with MacPorts.
# Homebrew
export PATH=/usr/local/bin:$PATH
pydoc3.5
python3
python3-32
python3-config
python3.5
python3.5-32
python3.5-config
python3.5m
python3.5m-config
Is there a
/usr/local/Cellar/python/2.7.12/
directory? (Version number might differ.)
Is there a
/usr/local/bin/python
file?
If the Cellar directory is present, but the file isn't, then Homebrew decided to be careful and not put Python in /usr/local/bin/ immediately.
You could manually do
brew link python
and see if there's now a
/usr/local/bin/python
file.
In your case, it appears you have some files related to Python (they might be from a Python 3 installation, can't tell), such as 2to3. You can safely overwrite them, since Python 2 also has this.
Thus:
brew link --overwrite python
is fine.
Note:
Specific Python versions will always exist as python2.7, python3.5 etc (including the full path as necessary). Thus, even overwriting the python executable is safe (provided it's not the system one in /usr/bin): you should then simply be explicit which python executable to use.
Also, when using a tool like pip, you can make sure you're using the correct version by running it e.g. as
/usr/local/bin/pythnon2.7 -m pip <...>
or whatever python executable you want to install things for.
Okay so one brute force solution could be this one https://stackoverflow.com/a/9821036/128517
But maybe you could check the value of your $PATH after source ~/.bash_profile
typing
> echo $PATH
and see if /usr/local/bin is indeed at the beginning.
if it's not, you might need to check if there's another export before yours or maybe you need to edit .profile instead.

Switch between python 2.7 and python 3.5 on Mac OS X

I generally use Python 2.7 but recently installed Python 3.5 using Miniconda on Mac OS X. Different libraries have been installed for these two versions of python. Now, the entering either of the keywords 'python' or 'python3' in terminal invokes python 3.5, and 'python2' returns '-bash: python2: command not found'. How can I now invoke them specifically using aliases 'python2' and 'python3' respectively?
I am currently using OS X El Capitan.
IMHO, the best way to use two different Python versions on macOS is via homebrew. After installing homebrew on macOS, run the commands below on your terminal.
brew install python#2
brew install python
Now you can run Python 2.7 by invoking python2 or Python 3 by invoking python3. In addition to this, you can use virtualenv or pyenv to manage different versions of python environments.
I have never personally used miniconda but from the documentation, it looks like it is similar to using pip and virtualenv in combination.
OSX's Python binary (version 2) is located at /usr/bin/python
if you use which python it will tell you where the python command is being resolved to. Typically, what happens is third parties redefine things in /usr/local/bin (which takes precedence, by default over /usr/bin). To fix, you can either run /usr/bin/python directly to use 2.x or find the errant redefinition (probably in /usr/local/bin or somewhere else in your PATH)
I already had python3 installed(via miniconda3) and needed to install python2 alongside in that case brew install python won't install python2, so you would need
brew install python#2 .
Now alias python2 refers to python2.x from /usr/bin/python
and alias python3 refers to python3.x from /Users/ishandutta2007/miniconda3/bin/python
and alias python refers to python3 by default.
Now to use python as alias for python2, I added the following to .bashrc file
alias python='/usr/bin/python'.
To go back to python3 as default just remove this line when required.
Similar to John Wilkey's answer I would run python2 by finding which python, something like using /usr/bin/python and then creating an alias in .bash_profile:
alias python2="/usr/bin/python"
I can now run python3 by calling python and python2 by calling python2.
Here is how to set the Python version back to 2.7 if you have installed Anaconda3 (Python 3.6) on MacOS High Sierra 10.13.5.
Edit the .bash_profile file in your home directory.
vi $HOME/.bash_profile
Comment out the line with anaconda3 addition to your PATH like this:
# export PATH="/Users/YOURUSERNAME/anaconda3/bin:$PATH"
Close the shell, then open it again
You should now see 2.7 when you run python
Then, if you want 3.6 you can simply uncomment out the anaconda3 line in your .bash_profile.
Trying to unlink python will end in tears in Mac OS X.
You will get something like this
unlink: /usr/bin/python: Operation not permitted
I just follow up the answer from #John Wilkey.
My alias python used to represent python2.7 (located in /usr/bin).
However the default python_path is now preceded by /usr/local/bin for python3; hence when typing python, I didn't get either the python version.
I tried make a link in /usr/local/bin for python2:
ln -s /usr/bin/python /usr/local/bin/
It works when calling python for python2.
If you want to use Apple’s system install of Python 2.7, be aware that it doesn’t quite follow the naming standards laid out in PEP 394.
In particular, it includes the optional symlinks with suffix 2.7 that you’re told not to rely on, and does not include the recommended symlinks with suffix 2 that you’re told you should rely on.
If you want to fix this, while sticking with Apple’s Python, you can create your own symlinks:
$ cd <somewhere writable and in your PATH>
$ ln -s /usr/bin/python python2
Or aliases in your bash config:
alias python2 python2.7
And you can do likewise for Apple’s 2to3, easy_install, etc. if you need them.
You shouldn’t try to put these symlinks into /usr/bin, and definitely don’t try to rename what’s already there, or to change the distutils setup to something more PEP-compliant. Those files are all part of the OS, and can be used by other parts of the OS, and your changes can be overwritten on even a minor update from 10.13.5 to 10.13.6 or something, so leave them alone and work around them as described above.
Alternatively, you could:
Just use python2.7 instead of python2 on the command line and in your shbangs and so on.
Use virtual environments or conda environments. The global python, python3, python2, etc. don’t matter when you’re always using the activated environment’s local python.
Stop using Apple’s 2.7 and instead install a whole other 2.7 alongside it, as most of the other answers suggest. (I don’t know why so many of them are also suggesting that you install a second 3.6. That’s just going to add even more confusion, for no benefit.)

Installing Python 3.3 on mac

I used the installer from http://www.python.org/download. The install appeared successful, and it dropped the Python 3.3 folder in my Applications directory. I ran the "Update Shell Profile.command" script it contained, and it prepended /Library/Frameworks/Python.framework/Versions/3.3/ to my path. Yet the Python version in that directory appears to be 2.7.5.
/Library/Frameworks/Python.framework/Versions/3.3 ls
Headers Python Resources bin include lib share
/Library/Frameworks/Python.framework/Versions/3.3 Python --version
Python 2.7.5
Also, ls /usr/bin | grep python shows:
python
python-config
python2.5
python2.5-config
python2.6
python2.6-config
python2.7
python2.7-config
pythonw
pythonw2.5
pythonw2.6
pythonw2.7
What have I missed?
There are multiple problems here.
First, you should not be running Python, the framework's executable. Framework executables aren't meant to be run, and it's really only a coincidence that this one happens to work.
Frameworks with programs meant to be run put them in a bin directory somewhere—either outside the framework (like /usr/local/bin) or inside it (like /Library/Frameworks/Foo.framework/Versions/X.Y/bin). By default, Python 3.3 does the latter.
And the programs inside the bin directory are all lowercased, not capitalized, so there is no Python to run.
Next, on Mac, and on almost every other platform in the world besides Windows, the current working directory is not part of your PATH. So, when you type Python --version, that finds Python somewhere on the PATH. The fact that you happened to have an executable of that name in the current directory doesn't mean anything (except that it's confusing to you). If you really want to run that file (but again, you really don't), you have to write ./Python instead.
Also, there is really no good reason to cd into the framework directory in the first place. Sure, you could run the file you want, from there, with the appropriate relative pathname: bin/python3, for example, but you don't want to.
Next, likely you're using a shell you already had running before installing Python 3.3. The Update Shell Profile.command script can be used to add Python 3.3 to the PATH for all future shells, or to spawn a new shell with that PATH, but either way it will not affect any existing shells. To solve that, you just have to start a new shell.
Next:
ls /usr/bin | grep python shows:
The /usr/bin directory is only for programs that are part of the OS. Programs you install yourself go in /usr/local/bin, or somewhere else on your PATH, instead. The Python installer has an option (although it may be hidden, I can't remember…) to put launchers in /usr/local/bin. And it also has an option—which you selected—to put its framework bin directory onto your PATH. But either way, it's never going to put anything in /usr/bin.
And finally, even after installing Python 3.3, the default python will still be 2.7. python3 and python3.3 will be 3.3, but python and python2.7 will be 2.7. See PEP 394 — The "python" Command on Unix-Like Systems for the rationale. But the short version is, there's all kinds of code that depends on Python 2.7 and isn't compatible with 3.3 that may be installed on your system, and you don't want it all to stop working.
So, putting it all together:
Create a new tab or window in Terminal.app.
Type python3 --version.
You may want to consider using a virtualenv:
$ /Library/Frameworks/Python.framework/Versions/3.3/bin/python3 -m venv ~/myvenv
$ source ~/myvenv/bin/activate
(myvenv) $ curl https://bootstrap.pypa.io/get-pip.py | python
(myvenv) $ deactivate
$ source ~/myvenv/bin/activate
http://docs.python-guide.org/en/latest/dev/virtualenvs/

Categories