I installed brew, python3 (default and latest version) and pip3, pyenv.
TensorFlow does not support python3.7 now, so I heard that I should make a virtualenv that runs 3.6 or lower version independently.
I installed python 3.6.7 by pyenv install 3.6.7 but can't make virtualenv -p 3.6.7 (mydir) because 3.6.7 is not in the PATH (usr/local/bin).
How can I update my PATH?
You don't need the executable to be on the PATH. Assuming you want /usr/local/bin/python3.6.7 to be used in the virtual environment,
virtualenv -p /usr/local/bin/python3.6.7 mydir
Updating your PATH is easy:
PATH=/usr/local/bin:$PATH
This will only update it in your current session; you might want to add this to your shell's startup files to make it permanent. This is a common FAQ but depends on a number of factors (your shell, etc) so google for details. Here is one question with several popular variants in the answers: Setting PATH environment variable in OSX permanently
I know that this doesn't answer the question exactly, but for completeness I'd like to add an Anaconda solution. Provided that an Anaconda environment is present on the system, a new Python environment can be created using conda create -n py36 python=3.6 pip. The name py36 can be arbitrarily chosen (could also be e.g. myenv or tensorflow), the desired Python version (in this example 3.6) is specified by python=3.6.
This environment can then be activated using conda activate py36 (or whatever name you assigned in the previous step). Once the environment is active, you can install tensorflow via pip: pip install tensorflow-gpu. To deactivate the current environment and return to the default environment, use conda deactivate. In this way, you don't have to modify PATH variables.
See also this documentation page for more details on the Anaconda environment.
Related
I have been learning data science using python for about a year now. I have become quite comfortable with the syntax and model creation. I have exclusively used Google Colab just due to how convenient it is and I love the notebook style. However, one thing I do not understand is the environment stuff. Although I use Colab, I do have python and anaconda on my machine and have installed various packages using the exact following format: pip install (package name). When I open my terminal, the first line is lead with (base) and when I check the Environments tab in anaconda navigator, it appears as though I installed all of these packages into a base environment named base (root)? Is that right? If so, what would my environment's name be then? What is a base environment compared to a venv?
The reason I am asking is because if I ever decide to use an IDE in the future, I would need to set my environment to be able to run packages, correct?
Just for fun I want to try using R and its reticulate package that allows python use in R. As stated in the answer to this question, I need to set my virtual environment before I can use python in R. Would my virtual environment be base (root)?
I'm a complete noob about all of this environment stuff. Again, I just opened my terminal and typed pip install (package name) for all packages I've installed. Thanks for any help in advance.
So from your description, it sounds like your default Python installation on your computer is through Anaconda. If that's the case, base is actually going to be the name of the conda virtual environment that you're using.
Virtual environments can be tricky, so I'll walk you through what I usually do here.
First, you can always check which Python installation you're currently using by using the which command on Mac/Linux, or if you're using Windows the command will probably be where (if you're on Windows, this answer might be helpful: equivalent of 'which' in Windows.)
(base) ➜ ~ which python
/Users/steven/miniconda3/bin/python
From the above, you can see that my default Python is through Miniconda, which is just a small version of Anaconda.
This means that when you use pip to install packages, those are getting installed into this base conda environment. And, by the way, you can use the which command with pip as well, just to double-check that you're using the version of pip that's in your current environment:
(base) ➜ ~ which pip
/Users/steven/miniconda3/bin/pip
If you want to see the list of packages currently installed, you can do pip freeze, or conda env export. Both pip and conda are package managers, and if you're using an Anaconda Python installation then you can (generally) use either to install packages into your virtual environment.
(Quick side note: "virtual environments" are a general concept that can be implemented in different ways. Both conda and virtualenv are ways to use virtual environments in Python. I'm also a data scientist, and I use conda for all of my virtual environments.)
If you want to create a new virtual environment using conda, it's very straightforward. First, you can create the environment and install some packages right away, like pandas and matplotlib. Then you can activate that environment, check your version of python, and then deactivate it.
(base) ➜ ~ conda create -n my-new-environment pandas matplotlib
(base) ➜ ~ which python
/Users/steven/miniconda3/bin/python
(base) ➜ ~ conda activate my-new-environment
(my-new-environment) ➜ ~ which python
/Users/steven/miniconda3/envs/my-new-environment/bin/python
(my-new-environment) ➜ ~ conda deactivate
(base) ➜ ~ which python
/Users/steven/miniconda3/bin/python
And, if you want to see which conda virtual environments you currently have available, you can run conda env list.
Here's the documentation for conda environments, which I reference all the time: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
I hope this is helpful!
I'm using MAC OS and installed python2, then miniconda python2, and then Anaconda python3.
Now normally in terminal my python version would be python3, but if I activate a conda environment "test", then the default python will be python2.
I believe the reason is the $PATH is changed, at "base" env (when terminal started), the starting part is "/anaconda3/bin:/anaconda3/condabin", and after "conda activate env", it changed to "/anaconda3/envs/test/bin:/anaconda3/condabin" which do not have python3 in those bin folder.
Is there a method to make the conda environment using the python3 of Anaconda?
I tried to create a link (both symbolic and hard) /anaconda3/envs/test/bin/python -> /anaconda3/bin/python. By this way, anaconda python3 can be started inside conda env. Then I found pip still not reachable. I think may create a pip link file too. But I guess this is not the correct way to use a conda environment?
(base) $which python
/anaconda3/bin/python
(base) $which pip
/anaconda3/bin/pip
(base) $conda activate test
(test) $which python
/usr/bin/python
(test) $which pip
'''no result'''
I expect the created conda env to use the same python3 as in base env, and can use pip, without setup symbolic or hard link.
When you activate a conda environment, you'll use the Python version from that environment. pip will automatically install into the active conda environment, unless you do something to break it, for example by calling another pip installation than the one from the current environment.
If you want to use a different Python, then don't activate the conda environment. If you want to use a specific version of Python in an environment, then install that version into that environment. Inside an environment, you can only (reasonably) use the Python version that's installed in the environment.
By the way, it is pointless to install different versions of Anaconda or Miniconda alongside. Just install Miniconda and create conda environments for everything else. Install the package anaconda into an environment to get the packages that Anaconda brings along.
I think you have added the path of "/usr/bin/python" in $PATH by anyway. So while doing python from inside a specific conda environment you are redirected to use the "/usr/bin/python" instead of "/anaconda3/bin/python". You can check the path by
$ $PATH
For development in Python, I am using Miniconda on my Mac with macos Sierra. However, I have to use a framework that only works with the default Python (present at /usr/bin/python).
My question is: How can I install packages for this default Python? If I use pip some-package, this will automatically install the package for the root conda environment.
EDIT: As discussed in the comments, I agree it is a bad idea to mess with the system default version of Python. Instead, I would like this SDK to work in a conda environment or with Python 2.7 installed from python.org. However, none of these seem to work! How to get this working?
You are not trying to install any package, you are trying to install very specific package pynaoqi that require external SDK, that is compiled for the specific architecture. You should edit your question to reflect that.
According to what I saw on the Net, it is not a trivial task. First, make sure you have the version 2.5 of the SDK, that suits the MacOS version. Then, look at this script. As they say, in order to work the dynamic libraries should be renamed.
By the way, you need 64-bit Python 2.7.
To install packages using the system Python you can use /usr/bin/easy_install, which ships with MacOS.
You may wish to install pip from there, by running:
sudo easy_install pip
Once you have pip installed, you won't be able to use it directly if the conda pip command is shadowing it. You have multiple options here, depending on your tastes. You may choose to alias the system pip to alias systempip=/usr/bin/pip, or have /usr/bin in front of /Users/user/anaconda/ in your $PATH.
As mentioned in other answers, messing with the system Python is not recommended, things may break, and you can (most) certainly get any package to work within a conda environment.
Some packages don't work out of the box with Anaconda's Python for some people because the distribution would default to a non-framework build of Python. A common complaint came from users of matplotlib, for example, who could not get figure windows to display properly. Fortunately, conda offers a framework build of python as well, named pythonw or pythonw3. Maybe you can try to get your package to work with pythonw, if that is where your problems come from.
Try this, find your way to path/to/python/scripts:
/usr/bin/python/scripts
And run pip here.
pip some-package is installing for the root anaconda environment because it is using pip from anaconda library. Anaconda add anaconda root dir to path before /usr/bin/ . so when you use pip if finds it on anaconda root. check path of pip using which pip, this will tell you complete path of pip.
You can install it on default python using /usr/bin/python -m pip install some-package. or use /path/to/default/pip install some-package.
The common issue is that anaconda (or miniconda) will have a python executable and pip executable in its bin directory.
If you truly need to run /usr/bin/python, this leads to the annoying conclusion: you cannot put conda in your path.
This lead me to some annoying machinations in my .bash_profile. You can use the same techniques for you, but your exact path to conda may differ:
# variables for using Conda
export BASE_PATH=$PATH
export CONDA_PATH="/Users/cmerriam/l/miniconda2/bin"
export CONDA_BIN="$CONDA_PATH/conda"
export CONDA_ACTIVATE="source $CONDA_PATH/activate"
export CONDA_DEACTIVATE="source $CONDA_PATH/deactivate"
# prompt function for [conda: myenv] when it is on.
function conda_branch {
type $CONDA_BIN >/dev/null 2>&1 && $CONDA_BIN info --envs | grep \* | awk '{print $1;}' | grep -v '^root'
}
function conda_part {
echo "[conda:$(conda_branch)]" | grep -v "\[conda:\]"
}
# Conda alias set
alias c="echo \"
cls = list conda environments
con <name> = activate conda environment
coff = deactivate conda environment
conda create -n tensorflow python=3.6 anaconda tensorflow jupyter
conda remove --name tensorflow --all
echo \\\$CONDA_DEFAULT_ENV\""
alias cls="$CONDA_BIN info --envs"
alias con="$CONDA_ACTIVATE"
alias coff="$CONDA_DEACTIVATE"
Option 2:
You may find you just need to run the same version or PYTHONPATH as /usr/bin/python. Then you make a conda environment with that version:
conda create -n myenv python=2.7
or, set a PYTHONPATH
export PYTHONPATH=/usr/lib/python2.7/site-packages
Option 3:
Uninstall Conda. It is a good tool for a set of problems that is not your current set of problems.
I too have a Mac with Sierra.
Assumption: Let's say you have Anaconda. Now you would have DefaultPython(python2) and say this Anaconda is of Python3.
Secret: The way any shell/default python selected is with PATH variable set in a shell. So when you install Anaconda, installer shall try to set new path variables to your default bash shell
Solution: Have one python (2 or 3) as default. For the less used one, try using full path. If you have to use
Cheat code: create /usr/bin/python as symbolic link to the python's actual repo(save the old one). This can be altered according to your usage.
Hope this works!
I am not sure I correctly understood your problem, but if the problem is that you want to easily switch between several versions of python like system python, other versions of python, different versions of miniconda or anaconda bound to specific versions of python (that is not just virtual environments) on your mac, then the best solution is pyenv.
It automatically relinks where your current /usr/bin/python, /usr/bin/pip and other binaries like ipython currently look at depending on the context. There are several options of what context is, but I usually use system python as default and pyenv local for each project which basically puts a file with the version name in your current directory and each time you cd in the directory or its subdirecories /usr/bin/python is automatically switched to that version you selected.
You need to manage your PATH environment variable. I like to keep functions in my shell profile to turn conda "on" and "off." So, on a Mac for example:
deactconda() {
export PATH="${PATH/\/Users\/<your_usrname>\/anaconda\/bin:/}"
}
actconda() {
export PATH=$HOME/anaconda/bin:$PATH
}
Then, whenever you want to stop using conda, just type deactconda at the prompt, etc.
You can do this multiple ways, first by providing the path to the pip in your default location,
your_default_python_path/pip install package
for example if you have a python 2.7 then,
sudo /usr/local/bin/pip2.7 install networkx
This should do the trick for you. After that you can install any packages and invoke them using default python(mine is 2.7 here)
with pip version 0.8 onwards you can also specify sudo /usr/local/bin/pip-2.x install package as well
I want to keep the Mac Python as my main 'python'. The reason for that is the recommendation in Python website here. I also want to add a separate environment for my Python3 (Anaconda).
For doing that I installed the Anaconda Python to get access to Conda and then I made an environment for my Python3 using the following command:
conda create -n py36 python=3.6 anaconda
When I installed the Anaconda python it added this to my .bash_profile file to get access to all conda commands:
# added by Anaconda3 4.4.0 installer
# export PATH="/Users/omidb/anaconda/bin:$PATH"
Now my default python is anaconda python which I don't want to.
How can I have default Mac python as my main python and then when I needed Anaconda, just use source activate py36 ?
UPDATED ANSWER
After testing this, I feel it's appropriate to offer this as a simple solution for using Mac Python as the default and only using Conda Python when desired.
You need to add/move the conda path to the end of your PATH environment via export command. This should allow you to use the Mac Python as the default and only use Anaconda Python after calling source activate py36.
export PATH="$PATH:/Users/omidb/anaconda/bin"
Path Resolution
This solution assumes you have /usr/bin/ (where Mac Python is) already in your PATH. Resolution order should check that directory first assuming it's first in the PATH. Also, this setup does not require symlinks in /usr/local/bin. I am not a fan of manipulating system-level resources for solutions that can be done with user resources (directories).
Default Python Setup
After moving the Anaconda path to the end of your PATH environment variable, you can validate that which python references /usr/bin/python, the location for Mac Python. You will run Mac python by default at the command line.
Running Conda Python
As previously noted, you have to call source activate py36 when you want to use the conda virtual environment. There is no need for adding symlinks to /usr/local/bin as they are already available through ~/anaconda/bin/.
Furthermore, source activate py36 (or any other Anaconda environment), it will add the appropriate environment path for Anaconda python to the beginning of your PATH environment variable, which (referring back to Path Resolution) would be executed when run as python on the command line. You can validate this with which python after running source activate py36. conda also stores the previous path as the environment variable CONDA_PATH_BACKUP.
Deactivating Conda
After running source deactivate, the original path is restored, so you will then be back to running Mac python.
Faced the same problem and question is too old but the simplest of doing this which I found is:
1. Let's check if the default python version is pointing to Conda python
which python - If Conda installation updated to your bashrc or zshrc, it will show that path
Running Command:
conda config --set auto_activate_base false
It's will make sure that when you start the terminal, Conda doesn't get activated as base
Now if you check python --version or which python - It should be pointing to mac default python version
Now whenever you want to use conda, all conda commands are available with conda <command>
Create the virtual env using conda create --name venv and activate it using conda activate <venv_name>
Now, I am able to use different python versions I required with conda and default python version as system default
I've currently got a working installation of the Enthought Python Distribution on my machine that I don't want to necessarily disrupt, but I'd like to look at moving over to Anaconda from Continuum.
I can easily install Anaconda into the virtualenv directory I create, but I'm not sure how to tell that virtualenv to use the anaconda-version of Python. If I was telling my whole system to use it I can alter .bash_profile with something like export PATH="/DIRECTORIES/anaconda/bin:$PATH. Is there a way to do that within a virtualenv?
I just tested the Anaconde 1.6 installer from http://continuum.io/downloads
After downloading, I did:
bash Anaconda-1.6.0-Linux-x86_64.sh
If you take the defaults, you'll end up with a directory anaconda in your home directory, completely separate from your EPD or system Python installation.
To activate the anaconda installation's default environment, do the following:
source $HOME/anaconda/bin/activate ~/anaconda
All Python commands will now come from the default Anaconda environment in $HOME/anaconda, which is itself a kind of a virtual environment. You can create sub-environments with e.g. conda create -n myenv1 ipython scipy, but this is not necessary.
As a side note, you can also use pip (also in $HOME/anaconda/bin) to install PyPI packages into your Anaconda default environment (it has pip installed by default) or any of the sub-environments (in which case you should first install pip into the sub-environment using conda install -n myenv1 pip).
It is possible to install parts of Anaconda manually into an existing virtualenv, but using their installer is by far the easiest way to test and use, without affecting any of your existing Python installations.
When you create your virtualenv use the -p flag to give it the path to the Python executable you want to use:
virtualenv -p /path/to/python-anaconda-version