We Keep Coding

Conda init polluting environment?

I have a project set up in Pycharm, with an existing conda environment. My scripts work when run from within the console.
I would like to be able to run python -m path_to_my_script/script.py from any location, but I need conda activated. Conda recommends I do conda init but I'm worried it may change settings someplace and break things.
What does conda init do?

Strategy for Answering
Exactly what the conda init command does and its consequences are shell-specific. Instead of trying to cover all cases, let's walk through a case, noting along the way that one can replicate this analysis by substituting their shell of interest.
Case Study: conda init zsh
Let's look at zsh as the shell. This is a common shell (default for macOS 10.15+) and very close to bash. Plus, I don't already have it configured.
Probing the Command: Dry Run
Many Conda commands include some form of dry run functionality via a --dry-run, -d flag, which - combined with verbosity flags - enables seeing what this would do without doing them. For the init command, dry run alone will only tell us what files it would modify:
$ conda init -d zsh
no change /Users/mfansler/miniconda3/condabin/conda
no change /Users/mfansler/miniconda3/bin/conda
no change /Users/mfansler/miniconda3/bin/conda-env
no change /Users/mfansler/miniconda3/bin/activate
no change /Users/mfansler/miniconda3/bin/deactivate
no change /Users/mfansler/miniconda3/etc/profile.d/conda.sh
no change /Users/mfansler/miniconda3/etc/fish/conf.d/conda.fish
no change /Users/mfansler/miniconda3/shell/condabin/Conda.psm1
no change /Users/mfansler/miniconda3/shell/condabin/conda-hook.ps1
no change /Users/mfansler/miniconda3/lib/python3.7/site-packages/xontrib/conda.xsh
no change /Users/mfansler/miniconda3/etc/profile.d/conda.csh
modified /Users/mfansler/.zshrc
==> For changes to take effect, close and re-open your current shell. <==
Here we can see that it plans to target the user-level resources file for zsh, /Users/mfansler/.zshrc, but it doesn't tell us how it will modified it. Also, OMG! the UX here is awful, because it in no way reflects the fact that I used the -d flag. But don't worry: as long as the -d flag is there it won't actually change things.
Patch Preview
To see what exactly it will do, add a single verbosity flag (-v) to the command. This will give everything from the previous output, but will now shows us the diff it will use to patch (update) the .zshrc file.
$ conda init -dv zsh
/Users/mfansler/.zshrc
---
+++
## -0,0 +1,16 ##
+
+# >>> conda initialize >>>
+# !! Contents within this block are managed by 'conda init' !!
+__conda_setup="$('/Users/mfansler/miniconda3/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
+if [ $? -eq 0 ]; then
+ eval "$__conda_setup"
+else
+ if [ -f "/Users/mfansler/miniconda3/etc/profile.d/conda.sh" ]; then
+ . "/Users/mfansler/miniconda3/etc/profile.d/conda.sh"
+ else
+ export PATH="/Users/mfansler/miniconda3/bin:$PATH"
+ fi
+fi
+unset __conda_setup
+# <<< conda initialize <<<
+
# ...the rest is exactly as above
That is, the plan of action is to add these 16 lines to the .zshrc file. In this case, I don't have an existing .zshrc file, so it plans to add it at line 1. If the file had already existed, it would append these lines.
Interpreting the Shell Code
Let's overview this code, before focusing on the details. Essentially, this is a redundant sequence of attempts to set up some shell functionality. They are ordered from most to least functional.
What Conda Hopes To Do
The code
__conda_setup="$('/Users/mfansler/miniconda3/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
gets something from conda itself, storing the result to a string, and then evaluates that string if the command had a clean exit ($? -eq 0). The neat engineering here is that the subprocess (technically python -m conda) passes back a result that can be run within this current process (zsh), allowing it to define shell functions.
I'll dig deeper into what is going on here in a second.
Fallback 1: Hardcoded Shell Functions
If that strange internal command fails, the devs included a hardcoded version of some essential shell functions (specifically conda activate). This is in:
miniconda3/etc/profile.d/conda.sh
and they simply check the file exists and source it. Let's hit that last option, then we'll swing back to look at the functionality.
Fallback 2: The Last Resort
The absolute last resort is to literally violate the standing recommendation since Conda v4.4, which is to simply put the base environment's bin directory on PATH. In this case, there is no conda activate functionality; this only ensures that Conda is on your PATH.
Details: Shell Functionality
Coming back to the intended case, we can inspect exactly what it would evaluate by simply getting that string result:
$ conda shell.zsh hook
__add_sys_prefix_to_path() {
# In dev-mode CONDA_EXE is python.exe and on Windows
# it is in a different relative location to condabin.
if [ -n "${_CE_CONDA}" ] && [ -n "${WINDIR+x}" ]; then
SYSP=$(\dirname "${CONDA_EXE}")
else
SYSP=$(\dirname "${CONDA_EXE}")
SYSP=$(\dirname "${SYSP}")
fi
if [ -n "${WINDIR+x}" ]; then
PATH="${SYSP}/bin:${PATH}"
PATH="${SYSP}/Scripts:${PATH}"
PATH="${SYSP}/Library/bin:${PATH}"
PATH="${SYSP}/Library/usr/bin:${PATH}"
PATH="${SYSP}/Library/mingw-w64/bin:${PATH}"
PATH="${SYSP}:${PATH}"
else
PATH="${SYSP}/bin:${PATH}"
fi
\export PATH
}
__conda_exe() (
__add_sys_prefix_to_path
"$CONDA_EXE" $_CE_M $_CE_CONDA "$#"
)
__conda_hashr() {
if [ -n "${ZSH_VERSION:+x}" ]; then
\rehash
elif [ -n "${POSH_VERSION:+x}" ]; then
: # pass
else
\hash -r
fi
}
__conda_activate() {
if [ -n "${CONDA_PS1_BACKUP:+x}" ]; then
# Handle transition from shell activated with conda <= 4.3 to a subsequent activation
# after conda updated to >= 4.4. See issue #6173.
PS1="$CONDA_PS1_BACKUP"
\unset CONDA_PS1_BACKUP
fi
\local ask_conda
ask_conda="$(PS1="${PS1:-}" __conda_exe shell.posix "$#")" || \return
\eval "$ask_conda"
__conda_hashr
}
__conda_reactivate() {
\local ask_conda
ask_conda="$(PS1="${PS1:-}" __conda_exe shell.posix reactivate)" || \return
\eval "$ask_conda"
__conda_hashr
}
conda() {
\local cmd="${1-__missing__}"
case "$cmd" in
activate|deactivate)
__conda_activate "$#"
;;
install|update|upgrade|remove|uninstall)
__conda_exe "$#" || \return
__conda_reactivate
;;
*)
__conda_exe "$#"
;;
esac
}
if [ -z "${CONDA_SHLVL+x}" ]; then
\export CONDA_SHLVL=0
# In dev-mode CONDA_EXE is python.exe and on Windows
# it is in a different relative location to condabin.
if [ -n "${_CE_CONDA:+x}" ] && [ -n "${WINDIR+x}" ]; then
PATH="$(\dirname "$CONDA_EXE")/condabin${PATH:+":${PATH}"}"
else
PATH="$(\dirname "$(\dirname "$CONDA_EXE")")/condabin${PATH:+":${PATH}"}"
fi
\export PATH
# We're not allowing PS1 to be unbound. It must at least be set.
# However, we're not exporting it, which can cause problems when starting a second shell
# via a first shell (i.e. starting zsh from bash).
if [ -z "${PS1+x}" ]; then
PS1=
fi
fi
conda activate base
I'm not going to walk through all this, but the main part is that instead of directly putting bin on PATH, it defines a shell function called conda and this serves as a wrapper for the condabin/conda entrypoint. This also defines a new functionality conda activate, which uses a shell function, __conda_activate(), behind the scenes. At the final step, it then activates the base environment.
Why do it this way?
This is engineered like this in order to be responsive to the configuration settings. Configuration options like auto_activate_base and change_ps1 affect how Conda manipulates the shell, and so that changes what functionality Conda includes in its shell functions.
Does Conda "Pollute the Environment"?
Not really. The main behavioral things like auto-activation and prompt modification can be disabled through configuration settings, so that conda init ultimately just adds the conda activate function to the shell, enabling clean switching between environments without ever having to manually manipulate PATH.

Your caution may be warranted. The conda init command adds Anaconda to the path on Linux or Mac (not recommended on Windows). Anaconda FAQ
You also need to prep for conda init by running source <path to conda>/bin/activate first.
You can run your script in your desired conda environment by specifying the python environment in the shebang statement in the first line of your code.
You can get that value by activating your desired environment and executing which python
(base) -> conda activate py39
(py39) -> which python
/home/user/anaconda3/envs/py39/bin/python
The shebang would be:
#!/home/user/anaconda3/env/py39/bin/python
On Mac and Linux, once you add the shebang you can chmod the file to be executable (e.g. chmod 700 myscript.py) and run from the command line directly. (I'm not a Windows user, so ymmv.)
(base) -> <path-to-script>/myscript.py
(Which now runs in shebang virtual environment instead of base.)

Cant start my virtual environment of flask through cygwin

Cant start my virtual environment of flask through cygwin :
my system :(windows 10)
Method 1:
$ env/Scripts/activate
Method 2:
$ env/Scripts/activate.bat
Method 3:
$ source ./env/Scripts/activate
on my cygwin cli, but it gives the following error all the time :
-bash: $'\r': command not found
-bash: ./env/Scripts/activate: line 4: syntax error near unexpected token `$'{\r''
'bash: ./env/Scripts/activate: line 4: `deactivate () {
Note : The first two methods of activating virtual environment works nicely on VSCode but not on cygwin.
My activate file :
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
# reset old environment variables
if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then
PATH="${_OLD_VIRTUAL_PATH:-}"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then
PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
hash -r 2> /dev/null
fi
if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then
PS1="${_OLD_VIRTUAL_PS1:-}"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1:-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="D:\projects\websites\googleHostFlaskApp\env"
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/Scripts:$PATH"
export PATH
# unset PYTHONHOME if set
# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
# could use `if (set -u; : $PYTHONHOME) ;` in bash
if [ -n "${PYTHONHOME:-}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
_OLD_VIRTUAL_PS1="${PS1:-}"
PS1="(flask) ${PS1:-}"
export PS1
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
hash -r 2> /dev/null
fi

As #Doug suggested , changing CR-LF to LF did worked.
The error was due to difference in line endings.
Unix systems (Linux and Mac) default to the LF (line feed) character for line breaks.
Windows on the other hand is “special” and defaults to CR/LF (carriage return AND line feed).
If you are using vscode like me: Then there is CRLF written at the bottom of vscode
By clicking there you can change activate file from CRLFto LF and it will remove the error you are facing.

Why am I stuck with the system default Python 2 interpreter after activating a Python 3 Conda environment?

I created a new Conda environment on an x86-64 Linux mainframe, using the command
conda create --name myenv --file somefile.txt --python=3.8.
I double checked my Python version in this environment using conda list, which returns
...
python 3.8.3 hcff3b4d_0
...
However, after activating this environment, Python 3 scripts doesn't run, and running which python reveals that the environment defaults to using the default system Python 2 interpreter:
$ which python
/usr/bin/python
My Efforts So Far
First, I added the line export PATH=$PATH:/home/miniconda3/envs/myenv/bin to my ~/.bashrc and ~/.profile files, to no effect. which python still returns /usr/bin/python.
I then checked my alias file to see if python is aliased to Python 2. But there is no entry in the alias file about python.
For your reference
my ~/.bashrc looks like this:
$ cat ~/.bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
# User specific aliases and functions
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/miniconda3/etc/profile.d/conda.sh" ]; then
. "/home/miniconda3/etc/profile.d/conda.sh"
else
export PATH="/home/miniconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
alias julia="/home/julia-1.4.0/bin/julia"
export PATH=$PATH:/home/miniconda3/envs/myenv/bin
$ cat ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
$ cat ~/.profile
export PATH=$PATH:/home/miniconda3/envs/myenv/bin

I think you need to add your env path to the front of the PATH variable so it will find python there first. The OS will look for a file or application in the PATH list, and use the first match that it finds.
Change
export PATH=$PATH:/home/qingyanz/miniconda3/envs/myenv/bin
to this:
export PATH=/home/qingyanz/miniconda3/envs/myenv/bin:$PATH
As an example, here are the settings on my conda environment:
(ds_tensorflow) C:\Users\me>which python
/c/Users/me/miniconda3/envs/ds_tensorflow/python
(ds_tensorflow) C:\Users\me>env | grep PATH
...
PATH=/c/Users/me/miniconda3/envs/ds_tensorflow:/mingw-w64/bin:/usr/bin:/bin:...

tmux recognised Conda env but still use the default python

I have trouble using anaconda environment in Tmux. I am on OSX.
I first
tmux
Then activate the Conda env:
conda activate myenv
At this point when I conda info shows the correct active environment and correct location. But which python still points to the default /usr/bin/python.
In the .bash_profile I have these lines added by anaconda when installed. So I guess I need to add something to let the terminal find the right python path?
# added by Anaconda3 5.3.1 installer
# >>> conda init >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$(CONDA_REPORT_ERRORS=false '/Users/jiajunyang/anaconda3/bin/conda' shell.bash hook 2> /dev/null)"
if [ $? -eq 0 ]; then
\eval "$__conda_setup"
else
if [ -f "/Users/username/anaconda3/etc/profile.d/conda.sh" ]; then
. "/Users/username/anaconda3/etc/profile.d/conda.sh"
CONDA_CHANGEPS1=false conda activate base
else
\export PATH="/Users/username/anaconda3/bin:$PATH"
fi
fi
Thank you for your suggestion.

Below works for me on macOS 10.15 with zsh:
Add below line to your ~/.tmux.conf file
set -g default-command "/bin/zsh"
If you're using a shell other than zsh, I assume changing the path to the shell you're using will also work.

same here, mac 10.14, tmux 3.0a
solution is:
source /Users/mac/opt/anaconda3/etc/profile.d/conda.sh
[[ -z $TMUX ]] ∨ conda deactivate; conda activate base
add 2 lines in your .zshrc. Then
source ~/.zshrc
everything works fine now!!

I had the same issue. Finally I find that tmux will always call the profile for your shell, not just the rc. So if you are using bash like I do, it will call /etc/profile, which will have a call to path_helper.
In order to fix this, change /etc/profile to:
if [[ -z $TMUX ]] && [ -x /usr/libexec/path_helper ]; then
eval `/usr/libexec/path_helper -s`
fi
If you are using bash, also change any export PATH=$PATH:/foo in .bashrc to
if [[ -z $TMUX ]]; then
export PATH=$PATH:/foo
fi
then you restart the terminal(for example Iterm). Everything should be good!

Can I automatically change my PYTHONPATH when activating/deactivating a virtualenv?

I would like to have a different PYTHONPATH from my usual in a particular virtualenv. How do I set this up automatically? I realize that it's possible to hack the bin/activate file, is there a better/more standard way?

This django-users post is probably going to help you a lot. It suggests using virtualenvwrapper to wrap virtualenv, to use the add2virtualenv command. Using this, when the environment is active, you can just call:
add2virtualenv directory1 directory2 ...
to add the directories to your pythonpath for the current environment.
It handles autonomously the PATH changes on environment switches. No black magic required. Et voila!

Here is the hacked version of bin/activate for reference. It sets the PYTHONPATH correctly, but unsetting does not work.
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
if [ -n "$_OLD_VIRTUAL_PATH" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "$BASH" -o -n "$ZSH_VERSION" ] ; then
hash -r
fi
if [ -n "$_OLD_VIRTUAL_PS1" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
if [ -n "$_OLD_PYTHONPATH" ] ; then
PYTHONPATH="$_OLD_PYTHONPATH"
export PYTHONPATH
unset _OLD_PYTHONPATH
fi
unset VIRTUAL_ENV
if [ ! "$1" = "nondestructive" ] ; then
# Self destruct!
unset deactivate
fi
}
# unset irrelavent variables
deactivate nondestructive
VIRTUAL_ENV="env_location" # Anonymized
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
_OLD_VIRTUAL_PS1="$PS1"
if [ "`basename \"$VIRTUAL_ENV\"`" = "__" ] ; then
# special case for Aspen magic directories
# see http://www.zetadev.com/software/aspen/
PS1="[`basename \`dirname \"$VIRTUAL_ENV\"\``] $PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`)$PS1"
fi
export PS1
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "$BASH" -o -n "$ZSH_VERSION" ] ; then
hash -r
fi
_OLD_PYTHONPATH="$PYTHONPATH"
PYTHONPATH="new_pythonpath" #Anonymized
export PYTHONPATH