I have been learning data science using python for about a year now. I have become quite comfortable with the syntax and model creation. I have exclusively used Google Colab just due to how convenient it is and I love the notebook style. However, one thing I do not understand is the environment stuff. Although I use Colab, I do have python and anaconda on my machine and have installed various packages using the exact following format: pip install (package name). When I open my terminal, the first line is lead with (base) and when I check the Environments tab in anaconda navigator, it appears as though I installed all of these packages into a base environment named base (root)? Is that right? If so, what would my environment's name be then? What is a base environment compared to a venv?
The reason I am asking is because if I ever decide to use an IDE in the future, I would need to set my environment to be able to run packages, correct?
Just for fun I want to try using R and its reticulate package that allows python use in R. As stated in the answer to this question, I need to set my virtual environment before I can use python in R. Would my virtual environment be base (root)?
I'm a complete noob about all of this environment stuff. Again, I just opened my terminal and typed pip install (package name) for all packages I've installed. Thanks for any help in advance.
So from your description, it sounds like your default Python installation on your computer is through Anaconda. If that's the case, base is actually going to be the name of the conda virtual environment that you're using.
Virtual environments can be tricky, so I'll walk you through what I usually do here.
First, you can always check which Python installation you're currently using by using the which command on Mac/Linux, or if you're using Windows the command will probably be where (if you're on Windows, this answer might be helpful: equivalent of 'which' in Windows.)
(base) ➜ ~ which python
/Users/steven/miniconda3/bin/python
From the above, you can see that my default Python is through Miniconda, which is just a small version of Anaconda.
This means that when you use pip to install packages, those are getting installed into this base conda environment. And, by the way, you can use the which command with pip as well, just to double-check that you're using the version of pip that's in your current environment:
(base) ➜ ~ which pip
/Users/steven/miniconda3/bin/pip
If you want to see the list of packages currently installed, you can do pip freeze, or conda env export. Both pip and conda are package managers, and if you're using an Anaconda Python installation then you can (generally) use either to install packages into your virtual environment.
(Quick side note: "virtual environments" are a general concept that can be implemented in different ways. Both conda and virtualenv are ways to use virtual environments in Python. I'm also a data scientist, and I use conda for all of my virtual environments.)
If you want to create a new virtual environment using conda, it's very straightforward. First, you can create the environment and install some packages right away, like pandas and matplotlib. Then you can activate that environment, check your version of python, and then deactivate it.
(base) ➜ ~ conda create -n my-new-environment pandas matplotlib
(base) ➜ ~ which python
/Users/steven/miniconda3/bin/python
(base) ➜ ~ conda activate my-new-environment
(my-new-environment) ➜ ~ which python
/Users/steven/miniconda3/envs/my-new-environment/bin/python
(my-new-environment) ➜ ~ conda deactivate
(base) ➜ ~ which python
/Users/steven/miniconda3/bin/python
And, if you want to see which conda virtual environments you currently have available, you can run conda env list.
Here's the documentation for conda environments, which I reference all the time: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
I hope this is helpful!
Related
Right now I'm trying to install python (3.10) and all further installations on my new pc (windows 10) and so far everything is set up:
Python installed
Windows paths for "Python" & "Python\Scrips"
I am able to call the python and pip version and also install some packages. But after installing virtualenv and creating one the - at the moment - unfixable error appears: I am unable to install packages into the pip-path of the virtualenviroment itself. Whenever I'm trying to run any pip-command I'm getting the following error:
Unable to create process using 'C:\Users\ExampleUser\AppData\Local\Programs\Python\Python310\python.exe "C:\folder\env\Scripts\pip.exe" '
As you can see, it's always refering to the original python-path, but on the other hand it's refering to the pip-path of the virtualenv!? Don't know if it's helpful, but when typing in where python and where pip the paths inside the venv are the first one listed. I've also watched out for no blank spaces in my path...
Unfortunately no explanation out there could help me until now and I never faced this problem on my old machine - mostly the same, except some older version of python, pip and virtualenv.
Does anyone else has an idea what I am missing?
downloading Python 3 at the official website and installing it via express installation
Copy & Paste the standalone python into the /python folder and overwriting the python version
running python -m pip install --upgrade pip in cmd
Now pip and python 3 are installed in their latest version.
It's work for me
Could you use venv to create your virtual environment, instead of virtualenv (given that venv is the recommended way to create virtual environments for Python 3.3, and newer)?
If using venv is an option, this procedure may give you some idea on how to do it.
I have not done any Python development on Windows, but I think the basics would be:
python3 -m venv your-env-directory
your-env-directory\Scripts\activate.bat
If using venv is not an option, maybe you can try specifying the -v flag when running your virtualenv command to increase verbosity so you can further troubleshoot what's going on.
try upgrade pip version python -m pip install --upgrade pip
[ Sorry if this answer turns out to be more of a comment than an answer. I only have 21 reputation, so I cannot comment ]
When trying to install pip packages and run python files, is the CWD (Current Working Directory) C:\folder\env\Scripts? If so, try chaning your CWD to C:\folder. I had a similar problem and doing this fixed it.
You may need to look into a cygwin environment, and look into a chroot or jail environment to run the application without conflict.
Have you tried to use virtualenv-wrapper-win module.
It helps me a lot to manage virtual envs
Life is much easier using Anaconda 3 (it's definitely bloated compared to normal Python though), or use the minimal Miniconda (barebone install, basically just Python + a package manager). You can download it here: https://docs.conda.io/en/latest/miniconda.html#windows-installers
Then you can make a new virtual environment super easy:
conda create -n myenv pip
conda activate
If you have multiple environments you do: conda activate [environment_name]
Now you're in your new environment with pip installed. And you get drop down menus in the Windows menu to get to your new environment too, so there isn't any searching required. They just appear. Now if you want to link Jupyter Notebook or Spyder to the installation, it takes more steps since you need more packages. I used this guide which basically activates Jupyter first, then Spyder IDE. https://medium.com/#apremgeorge/using-conda-python-environments-with-spyder-ide-and-jupyter-notebooks-in-windows-4e0a905aaac5
Since you created the environment with pip added you can pip install whatever packages you need. I had to do this recently with OpenBLAS backed NumPy and SciPy (the defaults from pip, not from conda). Now Miniconda is the closest thing to basic Python installation, and comes with some nice tools to make your life easier. Hopefully this is helpful.
My anaconda command prompt now says:
(base) C:\users\user_name>
I would like to know why (base) appears in front of the prompt. This started appearing since I created a virtual environment to run Django. If I enter 'deactivate' the prompt disappears but if I close the terminal and open a new terminal the 'base' is back again.
(base) C:\users\user_name> deactivate
C:\users\user_name>
The trouble is that this prompt does not recognize pip or any anaconda commands. I can run pip on the 'base' prompt, but not on the original one. I installed a module in '(base)...' using pip but this module is recognized by my ide.
conda list anaconda
in '(base)..' gives th version of Anaconda as 4.4. without the base, it does not recognize conda. I have run pip and conda several times before this happened. I'm running Windows 10.
Try this:
conda config --set auto_activate_base false
The changeps1 only hide the command prompt, you still in this environment.
auto_activate_base can avoid entering the environment.
Although Blockchain Business's answer is correct, as of v 4.6.4
DeprecationWarning: 'source deactivate' is deprecated. Use 'conda deactivate'.
so,
conda deactivate
This actually deactivates Anaconda completely, so may not be the best solution...
To restore:
conda activate
Change command prompt (changeps1) in ~/.condarc:
changeps1: False
Reference:
how to modify conda 'source activate' ps1 behavior
Document:
https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#change-command-prompt-changeps1
"(base)" is displayed to tell you which environment you are in. "base" is the default environment.
You can create a new environment from within Anaconda Navigator. You might do this to run a different version of python, perhaps, for example, because a library you wish to use is only compatible with python 3.6 or lower.
Also if you want to install an experimental library/package and not have it break your other installed packages, you might install it in a new environment, where it is isolated from the rest of your installed libraries/packages.
You are probably best to use the Environments tab in Anaconda Navigator for the creation, management of environments and installation of libraries/packages, but you can use the Anaconda Prompt commands if you like and on some machines it is a lot faster to do so.
Be aware that although pip install and conda install commands are supposed to play better together these days, I would still recommend sticking to conda install and only using pip if conda install won’t work for a particular package. This is because installing python packages so they do not conflict with one another is apparently not so trivial - and using two different package managers could be a problem. In the old days pip was almost certain to break environments that had been set up with conda or Anaconda Navigator.
If you face the issue in Ubuntu you can try the below two steps to remove it from your command prompt.
conda config
conda config --set changeps1 False
I had the same issue, I typed the following command to remove the (base) reference:
source deactivate
In my case, I had run source ~/anaconda3/etc/profile.d/conda.sh and expected the env to get activate.
Instead it must be conda activate
type 'conda deactivate' in the working terminal.
It sounds like when you installed Anaconda, it's now seeing that as your default environment, and with Anaconda it's best that you install packages using conda rather than by using pip, because at best pip installed packages won't be recognized by Anaconda, and at worst the pip installed packages can break your Anaconda install. I went through some of this before myself, as I needed to use Anaconda for a Python course I was taking, however I use Kubuntu, so I don't have a lot of knowledge around using it on Windows.
For more info on using Anaconda, see https://conda.io/docs/user-guide/getting-started.html You can probably use Anaconda Navigator to create virtual environments and install packages too.
Showing the active environment in front of the path like (base) C:\users\user_name> is a feature, not a bug. If you pip-install a module into the base environment (bad practice warning: Use conda install this_module -c conda-forge or pip install this_module into a separate environment), you'll first need to activate the base environment before you can actually use this_module. If you don't know how to do this in your IDE (although this a base functionallity of any IDE), open a cmd shell, type conda activate and then start your IDE directly from the (base) C:\users\user_name> prompt.
*I hope this isn't a repeat. I've tried looking for clarity, but I'm having trouble.
I'm fairly new to Python and used Homebrew to avoid using the system Python on Mac. I guess I also installed anaconda at some point. (probably following some tutorial)
Now when I do which for the following, I get different bins:
which python: /Users/ryangoree/anaconda3/bin/python
which python2: /usr/local/bin/python2
which python3: /Users/ryangoree/anaconda3/bin/python3
which pip: /Users/ryangoree/anaconda3/bin/pip
So my issue is that when I'm using python2, there are modules that that I can't use. If I try to pip install them, It just tells me they're already installed since they are in the anaconda directory.
I don't know what I don't know right now, but I'm sure there is a better way to handle this. Can someone enlighten me or send me on the right path to developing with Python and managing the packages.
Thank you!
This stumped me for a while until I figured out one pain when using anaconda: just because you’re in a conda environment doesn’t mean that pip belongs to that environment. Instead you must run conda install pip for pip to be associated with that environment. Then every pip install will be tied to that environment.
You can check your PYTHONPATH to see the order in which the various python installations are interrogated.
Better still you should create each conda environment with its own python using:
conda create -n <envname> python=2.7 # python 2.7
conda create -n <envname> python=3.6 # python 3.6
which will automatically include pip for that environment.
For development in Python, I am using Miniconda on my Mac with macos Sierra. However, I have to use a framework that only works with the default Python (present at /usr/bin/python).
My question is: How can I install packages for this default Python? If I use pip some-package, this will automatically install the package for the root conda environment.
EDIT: As discussed in the comments, I agree it is a bad idea to mess with the system default version of Python. Instead, I would like this SDK to work in a conda environment or with Python 2.7 installed from python.org. However, none of these seem to work! How to get this working?
You are not trying to install any package, you are trying to install very specific package pynaoqi that require external SDK, that is compiled for the specific architecture. You should edit your question to reflect that.
According to what I saw on the Net, it is not a trivial task. First, make sure you have the version 2.5 of the SDK, that suits the MacOS version. Then, look at this script. As they say, in order to work the dynamic libraries should be renamed.
By the way, you need 64-bit Python 2.7.
To install packages using the system Python you can use /usr/bin/easy_install, which ships with MacOS.
You may wish to install pip from there, by running:
sudo easy_install pip
Once you have pip installed, you won't be able to use it directly if the conda pip command is shadowing it. You have multiple options here, depending on your tastes. You may choose to alias the system pip to alias systempip=/usr/bin/pip, or have /usr/bin in front of /Users/user/anaconda/ in your $PATH.
As mentioned in other answers, messing with the system Python is not recommended, things may break, and you can (most) certainly get any package to work within a conda environment.
Some packages don't work out of the box with Anaconda's Python for some people because the distribution would default to a non-framework build of Python. A common complaint came from users of matplotlib, for example, who could not get figure windows to display properly. Fortunately, conda offers a framework build of python as well, named pythonw or pythonw3. Maybe you can try to get your package to work with pythonw, if that is where your problems come from.
Try this, find your way to path/to/python/scripts:
/usr/bin/python/scripts
And run pip here.
pip some-package is installing for the root anaconda environment because it is using pip from anaconda library. Anaconda add anaconda root dir to path before /usr/bin/ . so when you use pip if finds it on anaconda root. check path of pip using which pip, this will tell you complete path of pip.
You can install it on default python using /usr/bin/python -m pip install some-package. or use /path/to/default/pip install some-package.
The common issue is that anaconda (or miniconda) will have a python executable and pip executable in its bin directory.
If you truly need to run /usr/bin/python, this leads to the annoying conclusion: you cannot put conda in your path.
This lead me to some annoying machinations in my .bash_profile. You can use the same techniques for you, but your exact path to conda may differ:
# variables for using Conda
export BASE_PATH=$PATH
export CONDA_PATH="/Users/cmerriam/l/miniconda2/bin"
export CONDA_BIN="$CONDA_PATH/conda"
export CONDA_ACTIVATE="source $CONDA_PATH/activate"
export CONDA_DEACTIVATE="source $CONDA_PATH/deactivate"
# prompt function for [conda: myenv] when it is on.
function conda_branch {
type $CONDA_BIN >/dev/null 2>&1 && $CONDA_BIN info --envs | grep \* | awk '{print $1;}' | grep -v '^root'
}
function conda_part {
echo "[conda:$(conda_branch)]" | grep -v "\[conda:\]"
}
# Conda alias set
alias c="echo \"
cls = list conda environments
con <name> = activate conda environment
coff = deactivate conda environment
conda create -n tensorflow python=3.6 anaconda tensorflow jupyter
conda remove --name tensorflow --all
echo \\\$CONDA_DEFAULT_ENV\""
alias cls="$CONDA_BIN info --envs"
alias con="$CONDA_ACTIVATE"
alias coff="$CONDA_DEACTIVATE"
Option 2:
You may find you just need to run the same version or PYTHONPATH as /usr/bin/python. Then you make a conda environment with that version:
conda create -n myenv python=2.7
or, set a PYTHONPATH
export PYTHONPATH=/usr/lib/python2.7/site-packages
Option 3:
Uninstall Conda. It is a good tool for a set of problems that is not your current set of problems.
I too have a Mac with Sierra.
Assumption: Let's say you have Anaconda. Now you would have DefaultPython(python2) and say this Anaconda is of Python3.
Secret: The way any shell/default python selected is with PATH variable set in a shell. So when you install Anaconda, installer shall try to set new path variables to your default bash shell
Solution: Have one python (2 or 3) as default. For the less used one, try using full path. If you have to use
Cheat code: create /usr/bin/python as symbolic link to the python's actual repo(save the old one). This can be altered according to your usage.
Hope this works!
I am not sure I correctly understood your problem, but if the problem is that you want to easily switch between several versions of python like system python, other versions of python, different versions of miniconda or anaconda bound to specific versions of python (that is not just virtual environments) on your mac, then the best solution is pyenv.
It automatically relinks where your current /usr/bin/python, /usr/bin/pip and other binaries like ipython currently look at depending on the context. There are several options of what context is, but I usually use system python as default and pyenv local for each project which basically puts a file with the version name in your current directory and each time you cd in the directory or its subdirecories /usr/bin/python is automatically switched to that version you selected.
You need to manage your PATH environment variable. I like to keep functions in my shell profile to turn conda "on" and "off." So, on a Mac for example:
deactconda() {
export PATH="${PATH/\/Users\/<your_usrname>\/anaconda\/bin:/}"
}
actconda() {
export PATH=$HOME/anaconda/bin:$PATH
}
Then, whenever you want to stop using conda, just type deactconda at the prompt, etc.
You can do this multiple ways, first by providing the path to the pip in your default location,
your_default_python_path/pip install package
for example if you have a python 2.7 then,
sudo /usr/local/bin/pip2.7 install networkx
This should do the trick for you. After that you can install any packages and invoke them using default python(mine is 2.7 here)
with pip version 0.8 onwards you can also specify sudo /usr/local/bin/pip-2.x install package as well
I've currently got a working installation of the Enthought Python Distribution on my machine that I don't want to necessarily disrupt, but I'd like to look at moving over to Anaconda from Continuum.
I can easily install Anaconda into the virtualenv directory I create, but I'm not sure how to tell that virtualenv to use the anaconda-version of Python. If I was telling my whole system to use it I can alter .bash_profile with something like export PATH="/DIRECTORIES/anaconda/bin:$PATH. Is there a way to do that within a virtualenv?
I just tested the Anaconde 1.6 installer from http://continuum.io/downloads
After downloading, I did:
bash Anaconda-1.6.0-Linux-x86_64.sh
If you take the defaults, you'll end up with a directory anaconda in your home directory, completely separate from your EPD or system Python installation.
To activate the anaconda installation's default environment, do the following:
source $HOME/anaconda/bin/activate ~/anaconda
All Python commands will now come from the default Anaconda environment in $HOME/anaconda, which is itself a kind of a virtual environment. You can create sub-environments with e.g. conda create -n myenv1 ipython scipy, but this is not necessary.
As a side note, you can also use pip (also in $HOME/anaconda/bin) to install PyPI packages into your Anaconda default environment (it has pip installed by default) or any of the sub-environments (in which case you should first install pip into the sub-environment using conda install -n myenv1 pip).
It is possible to install parts of Anaconda manually into an existing virtualenv, but using their installer is by far the easiest way to test and use, without affecting any of your existing Python installations.
When you create your virtualenv use the -p flag to give it the path to the Python executable you want to use:
virtualenv -p /path/to/python-anaconda-version