Use system python in homebrew - python

Is it possible to use system python in homebrew?
I have python 2.7.5 installed on my mac, but when I try to install any homebrew package with python in dependencies, it starts loading python 2.7.9.
It is important for me to use system python because of lots of installed python packages.

What version of python homebrew uses depends on two things:
Where the python alias points to.
What the python path is in the formula.
In most cases, homebrew forumlae simply use the python command, and does not specify a python path. If it does, then it is usually /usr/bin/python.
So, you can fix the problem in two ways:
Change the homebrew formulate to point to the right python interpreter of your choice.
Change where the python command points to.
The first option is problematic, since it means that you need to go into your Cellar folder, and alter things. This is not nice, and usually not required.
A better alternative is to point to the right python command. This is where pyenv comes in handy. pyenv is a command line tool that helps you manage different versions of python. Follow the instructions here:
https://github.com/yyuu/pyenv-installer
to install it.
Once that is done, change your python command to whatever version of python you'd like it to point to.

Related

General question about python interpreters (having multiple interpreters, pyenv, etc)

I'm new to coding and I was wondering if someone could explain what exactly python interpreters/environments do and the how that relates to the python versions and packages that can be used in certain projects.
I was trying to code a twitter bot with python and I use VS code as my IDE. While trying to import tweepy into my python code, I noticed I kept getting an error that it couldn't import. After some googling I realized it was because I had to set the python interpreter of the python file to the one that had tweepy installed to. So I changed it and the error with tweepy was solved. But I noticed that I have a bunch of python interpreters in seemingly different locations (screenshot of the interpreter options I'm given). The interpreter that fixed the issue was the one in the pyenv path.
I had installed pyenv some months back because I wanted my terminal (I'm on macOS) to automatically launch python3 when I typed python into the terminal. However, I don't really know what it does beyond that.
So my questions are:
Why do I have multiple python interpreters? Is there a way to get rid of the ones and just keep the one from pyenv (like clean up the ones in /usr/local/bin/python3, /opt/homebrew/bin/python or /usr/bin/python3 since I'm not using them)? or should I not do that?
What exactly does pyenv do? Is it okay that my python libraries are getting installed to pyenv by default? Or should I change it so that it's getting installed to homebrew or one of the usr/bin paths?
Sorry, this is my first time asking a question here so I might not sound that cohesive.
The interpreter is the program which executes a python source file. This is a program like any other, and you can have various versions of it at once (indeed python is quite good at being self contained, and multiple versions will live alongside one another quite happily).
Macs ship with an outdated python 2, used for something in the system. If you remove or update it, things may break. So you install your own python 3 somewhere. If you want to load the new python when you type python in a shell, that shell's $PATH needs to point to the right python. You generally set this in something like ~/.profile.
Pyenv is a tool for managing multiple python interpreters. It's usually used on a per-project basis, to test code against multiple pythons. Further to complicate things, it's often used alongside a virtual environment tool like pipenv. Using it to avoid manually fiddling with $PATH is fine, but a slightly orthogonal use case. All pyenv does is to put little scripts with the same name as python executables somewhere in your $PATH where they supercede anything else. These scripts then figure out which python should get called in this case. So pyenv isn't installing anything, it's just working out which python is going to do it.
The solution to all your problems turns out to be quite simple:
python -m pip install abc
Get into the habit of installing stuff with the python you want to use already set up in the shell, and calling pip like that. That way whatever you use it will always be installed in the right env.

IDLE and python is different, not able to install modules properly

thanks for reading this. I am using macOS High Sierra. I am not very familiar with terminal or environment variables, but am trying to learn more. From reading other threads and google, it seems like I either have multiple pythons installed, or have pythons running from different paths. However I am not able to find a solution to resolving this, either by re-pathing my IDLE or deleting it entirely.
I do have python, python launcher, and anaconda (not very sure how anaconda works, have it installed a few years back and didn't touch it) installed. I am trying to install pandas (pip install pandas), which tells me that I have it installed, but when I run it on IDLE, it says module not found. Though if i run python3 on terminal and type my code in, it works (so pandas has indeed been installed).
When i run which python on terminal, it returns
/Users/myname/anaconda3/bin/python
(when i enter into this directory from terminal, it shows that in the bin folder, I have python, python.app, python3, python3-config, python3.7, python3.7-config, python3.7m, python3.7m-config)
When i run which idle on terminal, it returns
/usr/bin/idle (im not even sure how to find this directory from the terminal)
When i run import os; print(os.path) on IDLE, it returns module 'posixpath' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py'
Would really appreciate some help to figure out how to ensure that when i install modules from terminal, it would be installed into the same python as the one IDLE is using. Also, I would like to know whether it is possible for me to work on VSCode instead of IDLE. I cant seem to find suitable extensions for data science and its related modules (like statsmodels, pandas etc). Thanks a lot!
First of all, a quick description of anaconda:
Anaconda is meant to help you manage multiple python "environments", each one potentially having its own python version and installed packages (with their own respective versions). This is really useful in cases where you would like multiple python versions for different tasks or when there is some conflict in versions of packages, required by other ones. By default, anaconda creates a "base" environment with a specific python version, IDLE and pip. Also, anaconda provides an improved way (with respect to pip) of installing and managing packages via the command conda install <package-name>.
For the rest, I will be using the word "vanilla" to refer to the python/installation that you manually set up, independent of anaconda.
Explanation of the problem:
Now, the problem arises since you also installed python independently. The details of the problem depend on how exactly you set up both python and anaconda, so I cannot tell you exactly what went wrong. Also, I am not an OSX user, so I have no idea how python is installed and what it downloads/sets alongside.
By your description however, it seems that the "vanilla" python installation did not overwrite neither your anaconda python nor anaconda's pip, but it did install IDLE and set it up to use this new python.
So right now, when you are downloading something via pip, only the python from anaconda is able to see that and not IDLE's python.
Possible solutions:
1. Quick fix:
Just run IDLE via /Users/myname/anaconda3/bin/idle3 every time. This one uses anaconda's python and should be able to see all packages installed via conda install of pip install (*). I get this is tiresome, but you don't have to delete anything. You can also set an "alias" in your ~/.bashrc file to make the command idle specifically linking you there. Let me know with a comment if you would like me to explain how to do that, as this answer will get too long and redundant.
2. Remove conda altogether (not recommended)
You can search google on how to uninstall anaconda along with everything that it has installed. What I do not know at this point is whether your "vanilla" python will become the default, whether you will need to also manually install pip again and whether there is the need to reinstall python in order for everything to work properly.
3. Remove your python "vanilla" installation and only use anaconda
Again, I do not know how python installation works in OSX, but it should be reasonably straightforward to uninstall it. The problem now is that probably you will not have a launcher for IDLE (since I am guessing anaconda doesn't provide one on OSX) but you will be able to use it via the terminal as described in 1..
4. Last resort:
If everything fails, simply uninstall both your vanilla python (which I presume will also uninstall IDLE) and anaconda which will uninstall its own python, pip and idle versions. The relevant documentation should not be difficult to follow. Then, reinstall whichever you want anew.
Finally:
When you solve your problems, any IDE you choose, being VScode (I haven't use that either), pycharm or something else, will probably be able to integrate with your installed python. There is no need to install a new python "bundle" with every IDE.
(*): Since you said that after typing pip install pandas your anaconda's python can import pandas while IDLE cannot, I am implying in my answer that pip is also the one that comes with anaconda. You can make sure this is the case by typing which pip which should point to an anaconda directory, probably /Users/myname/anaconda3/bin/pip
First: This would be a comment if I had enough reputation.
Second: I would just delete python. Everything. And reinstall it.
To repeat and summarized what has been said on various other question answers:
1a. 3rd party packages are installed for a particular python(3).exe binary.
1b. To install multiple packages to multiple binaries, see the option from python -m pip -h.
To find out which python binary is running, execute import sys; print(sys.executable).
3a. For 3rd party package xyz usually installed in some_python/Lib/site-packages, IDLE itself has nothing to do with whether import xyz works. It only matters whether xyz is installed for 'somepython' (see 1a).
3b. To run IDLE with 'somepython', run somepython -m idlelib in a terminal or console.
somepython can be a name recognized by the OS or a path to a python executable.

How to use and install all the libraries of Python3?

Using Ubuntu 16.04.
I was doing development in Python2.7, although recently moved to Python3.5, both of which come by default. The problem is, I find all the python libraries have to be reinstalled or downloaded for the next Python3. Also, the behaviour of Python seems weird because to execute the same script in python3, I have to enter in terminal
python3 script.py
which is different from other applications where I do not have to give the version number. Anyway, the questions I am trying to find answers are
To what extent are the libraries, packages (such as pip etc.) shared between 2.7 and 3.5? Or do I need double installations (and double the space) for everything now? A bit space limited in my old laptop.
Most of the installation instructions and commands I find online do not specify whether they are for 2 or 3. Given that I have two versions, how do I control/make sure they go to 3.5?
Is it advisable and possible to completely remove everything related to 2.7? Can I keep working with 3.5?
The first thing you need to know is that all official libraries and python tools for python3 got the "3" character to separate them from the previous versions. So, you need to use pip3, and not pip, python3, not python, and the packages are called python-pygame, not python-pygame.
So, to answer to you in order:
Yes, the you have to double the space needed if you decide to use both python2 and python3
Usually, if in the tool name there is 3, it is for python3, and if not it's for python2
Python2 and python3 are completly indipendent (different path, indipendent versions, etc.) so having python 2.7 installed doesn't affect python3 BUT since python 3 is the next version of python 2, it makes it obsolete (in my opinion) so if you don't have enough space for both, keeping python2 is absolutely not needed nor useful
If you want to control your python execution then you have various method or techniques:
For downloading packages according to version
You can use pip{version-name} to download the libraries. Like if you want to download library of python 2.7 then write
pip2.7 install package-name
for python 3.5 then use
pip3.5 install package-name
For execution of program:
If you want to execute the program accoridng to you choice of version then just use
python{version-name} script.py
eg:
python2 script.py
python3 script.py
or you just write down the path of your python version on the top of script. Please refer this for more details:
Why do people write #!/usr/bin/env python on the first line of a Python script?

#!/usr/bin/python and #!/usr/bin/env python, which support?

How should the shebang for a Python script look like?
Some people support #!/usr/bin/env python because it can find the Python interpreter intelligently. Others support #!/usr/bin/python, because now in most GNU/Linux distributions python is the default program.
What are the benefits of the two variants?
The Debian Python Policy states:
The preferred specification for the Python interpreter is /usr/bin/python or /usr/bin/pythonX.Y. This ensures that a Debian installation of python is used and all dependencies on additional python modules are met.
Maintainers should not override the Debian Python interpreter using /usr/bin/env python or /usr/bin/env pythonX.Y. This is not advisable as it bypasses Debian's dependency checking and makes the package vulnerable to incomplete local installations of python.
Note that Debian/Ubuntu use the alternatives system to manage which version /usr/bin/python actually points to. This has been working very nicely across a lot of python versions at least for me (and I've been using python from 2.3 to 2.7 now), with excellent transitions across updates.
Note that I've never used pip. I want automatic security upgrades, so I install all my python needs via aptitude. Using the official Debian/Ubuntu packages keep my system much cleaner than me messing around with the python installation myself.
Let me emphasize one thing. The above recommendation refers to system installation of python applications. It makes perfectly sense to have these use the system managed version of python. If you are actually playing around with your own, customized installation of python that is not managed by the operating system, using the env variant probably is the correct way of saying "use the user-preferred python", instead of hard-coding either the system python installation (which would be /usr/bin/python) or any user-custom path.
Using env python will cause your programs to behave differently if you call them from e.g. a python virtualenv.
This can be desired (e.g. you are writing a script to work only in your virtualenv). And it can be problematic (you write a tool for you, and expect it to work the same even within a virtualenv - it may suddenly fail because it is missing packages then).
My humble opinion is that you should use the env-variant. It's a POSIX component thus found in pretty much every system, while directly specifying /usr/bin/python breaks in many occasions, i.e. virtualenv setups.
I use #!/usr/bin/env python as the default install location on OS-X is NOT /usr/bin. This also applies to users who like to customize their environment -- /usr/local/bin is another common place where you might find a python distribution.
That said, it really doesn't matter too much. You can always test the script with whatever python version you want: /usr/bin/strange/path/python myscript.py. Also, when you install a script via setuptools, the shebang seems to get replaced by the sys.executable which installed that script -- I don't know about pip, but I would assume it behaves similarly.
As you note, they probably both work on linux. However, if someone has installed a newer version of python for their own use, or some requirement makes people keep a particular version in /usr/bin, the env allows the caller to set up their environment so that a different version will be called through env.
Imagine someone trying to see if python 3 works with the scripts. They'll add the python3 interpreter first in their path, but want to keep the default on the system running on 2.x. With a hardcoded path that's not possible.

Proper Unix (.profile, .bash_profile) changes for Python usage

I new to Python and to programming in general. I'm a novice, and do not work in programming, just trying to teach myself how to program as a hobby. Prior to Python, I worked with Ruby for a bit and I learned that one of the biggest challenges was actually properly setting up my computer.
Background: I'm on a Macbook with OSX 10.7.
With Ruby, you have to (or rather, you should), edit your ./profile and add PATH info. When you install and use RVM, there are additional items you need to add to your bash_profile.
Do you have to make similar changes with Python? What are the best practices as I'm installing/getting started to ensure I can install modules and packages correctly?
python works out of the box on OS X (as does ruby, for that matter). The only changes I would recommend for a beginner are:
1) Python likes to be reassured that the terminal can handle UTF-8 before it will print Unicode strings. Add export LANG=en_US.UTF-8 to .profile. (It may be that the .UTF-8 part is already present by default on Lion - I haven't checked since Snow Leopard.) Of course, this is something that will help you in debugging, but you shouldn't rely on it being set this way on other machines.
2) Install pip by doing easy_install pip (add sudo if necessary). After that, install Python packages using pip install; this way, you can easily remove them using pip uninstall.
Take a loot at Python on the Macintosh page first. Like it says, Python comes pre-installed on Mac OS X. It means that you don't have to do anything special in order to use it.
To get started, you can run a Terminal.app, type python and that will get you Python interactive shell up and running.
However, Python on OS X might be of a slightly older version. For example, OS X 10.7.3 comes with Python 2.7.1, whereas latest release version of the Python is 3.2.3. If you want to use other versions, then you will have to install them. Then it all depends on what, where and how you install. If you want to have multiple versions alongside, you may need to set some environment variables like PATH to have binaries you installed found by the bash etc. You can do it through bash ~/.profile if needed.
But until you get to that point - don't worry about it use a version shipped with OS X. Once you want a newer one - download and install it. Then, if it doesn't work out of the box or you have any other problems or concerns, feel free to ask a more specific question.

Categories