We Keep Coding

Stuck when setting up to use anaconda with VS Code and Integrated Git terminal

I want to learn Data Science and so have used some really popular Python modules likes Pandas, Matplotlib, Numpy, etc. So I clean installed Anaconda and am now using it as my default Python interpreter and also using Conda for installing packages and making virtual environments. I use VS Code as my daily text editor. But I have run into some issues when using the integrated Git terminal in VS Code with the Anaconda Python interpreter.
There are a couple of issues that I am facing. One of the first issues that I see is when I am using CMD to run Python. If I type and enter python in cmd, the Python interpreter provided by anaconda comes up. But I also get a warning:
Warning:
This Python interpreter is in a conda environment, but the environment has not been activated. Libraries may fail to load. To activate this environment please see https://conda.io/activation
I didn't expect to get this output. Anyway, there's another problem in VS code. But first I would like to mention that I have checked "Add to PATH" when installing Anaconda so no issues there. Now, when I open a new Terminal in VS Code, automatically C:/Users/User/Anaconda3/Scripts/activate is run and then conda activate base is run. But when conda activate base is run, automatically, as mentioned, I get a CommandNotFoundError. It states Your shell has not been properly configured to use 'conda activate'.
If using 'conda activate' from a batch script, change your
invocation to 'CALL conda.bat activate'
And then I am told to initialize my shell, so I did conda init bash but still no luck. And this brings me to talk about .bash_profile. I think it has to do something with this bash profile. Anyway, this is what is in my bash profile
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
eval "$('/C/Users/User/Anaconda3/Scripts/conda.exe' 'shell.bash' 'hook')"
# <<< conda initialize <<<
Just a summary of the problem:
Unexpected warning in CMD when running Anaconda Python interpreter
Automatically run Anaconda Scripts and conda activate base when opening new Terminal in VS Code
Conda init bash not helping
P.S I have tried using conda activate [env_name] in CMD and also in Git Bash and they work without any issues. In other words, Anaconda and Conda work perfectly outside of VS Code terminal.

I have figured out the answer myself and would like to share it here. First of all at the time of writing the question I was using Git Bash as my Terminal in VS Code (am still using it). So the issue was that when I ran the command conda init bash in Git Bash or the VS Code Terminal, Conda just basically put the command used for activating Conda environments in the .bash_profile since it is sourced during logging into Bash. But the integrated Terminal in VS Code is a subshell of a Git Bash session. That is why .bash_profile is NOT sourced in VS Code since .bash_profile is only sourced during the main Bash session. The .bashrc file is the file that is sourced when creating a Terminal session in VS Code. So what you actually need to do is take the code that is put into .bash_profile by conda init bash and paste it into your .bashrc file and make the .bash_profile source that .bashrc file automatically.
So, to sum up, using conda init bash will put the conda command in the .bash_profile and it is usually sourced by Git Bash, but VS Code Git Bash terminal will use .bashrc.
So you can just cut and paste the code from .bash_profile to .bashrc (as already mentioned) or if you want, just simply follow this: put this code in your .bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
And in your .bashrc, put this code:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
eval "$('{path_to_your_conda.exe}' 'shell.bash' 'hook')"
# <<< conda initialize <<<
# You can get conda.exe path by using `which conda` in Git Bash
Now, when you open up a Git Bash session in VS Code Terminal, you can use conda activate env_name to activate any environments you have.
Everything is now supposed to work as expected in VS Code terminal but I would like to further elaborate about something. If you want, you can skip the conda init bash process (NOT recommended, just read on for additional knowledge but please follow the above steps only). This is a feature that was introduced in conda 4.4.0. Till then the way of activating conda environments was by using source activate but that command was NOT cross-platform, meaning that the command could not be used in OSes like Windows.
So they made this change by introducing commands like: conda activate env_name so that conda environments become much easier to use despite the OS platform.
conda activate also has other advantages. This is directly from their release docs:
conda activate: The logic and mechanisms underlying environment activation have been reworked. With conda 4.4, conda activate and conda deactivate are now the preferred commands for activating and deactivating environments. You'll find they are much more snappy than the source activate and source deactivate commands from previous conda versions. The conda activate command also has advantages of (1) being universal across all OSes, shells, and platforms, and (2) not having path collisions with scripts from other packages like Python virtualenv's activate script.
I used this question as a reference. Check it out to learn more.
Having said that, using source activate env_name will still work if you are using Git Bash, even in VS Code Git Bash terminal. source activate env_name requires no prior set up or config. But it is highly recommended that you only use conda init to set everything up and then use conda activate env_name.
[NOTE]: Locating and modifying the said .bashrc and .bash_profile on Windows is usually not as simple as it is on Linux. But can be done fairly easily like this:
It goes without saying but, you should have Git Bash installed. Having Git Bash installed should, as far as I know, automatically create .bashrc or .bash_profile or maybe both. These files are called "dotfiles" (since they start with a dot) and these are by default hidden on most OSes and definitely on Windows. If they were auto-created by Git Bash on your system, it is most likely that they are placed in your home directory. Home directory on Windows is C:\Users\<you>\. With that said, follow this:
Open Git Bash and go to your home directory with: cd. Just type this and you will be in your home directory
Enter this command: ls -a and you will see all your files, even hidden ones. Look for .bash_profile and .bashrc. Both should be present. If they are, you are ready to follow the above instructions. But if one is not there or if both are missing create them using: touch .bashrc && touch .bash_profile. You should now see these files when you again type: ls -a
That's it. Now that you have your .bashrc and .bash_profile, you can follow the above instructions. Also, to access these two files quicker, open them like this with VS Code: code ~/.bashrc or code ~/.bash_profile. Now, modify these two files as per the instructions.
In the question, I have also talked about VS Code activating Conda environments automatically. There's no issue with VS Code doing that since this is the default behavior.
I misinterpreted that as something that's an issue. But if anyone was looking to stop VS Code from automatically doing that, I would recommend trying to set this in the user settings:
"python.terminal.activateEnvironment": false

EDIT: A better solution than using source activate to get conda activate commands to work in the git bash terminal in VS Code:
Run conda init in the Git Bash Terminal in VS Code
Type in bash -l in VS Code's Git Bash terminal to launch your configured shell as a login shell
You should now be able to run conda activate commands per normal!
More info: bash -l runs your ~/.profile/~/.bash_profile/~/.zprofile scripts where the conda executable is actually referenced (but in which Git Bash as a integrated terminal does not run by default and refers to). Hence, git bash does not know where to search for conda when running conda activate commands and per Arafat's explanation above, running conda init changes the git bash PATHs in this .bash_profile file, but is ineffectual as the git bash terminal in VS Code doesn't actually refer to this file! Further info in VS Code's official docs.
Supplementing the explanation of the accepted answer, I've posted a solution that worked for me here that might possibly help others (changing user settings did not solve the issue for me). Link could also point to other working solutions if the below or accepted answer above doesn't work.
NOTE: Please read Arafat's answer before attempting the source activate method below to understand why it's not normally recommended. That said leaving it up as it still solves the problem.
Here's what worked for me using the Git Bash terminal in VS Code on
windows in succinct steps:
source activate env-name - You should see your line appended by the (base) tag now.
After calling on source activate, I've found following conda activate commands to work: i.e. conda activate env2-name
What didn't work for Git Bash (as a VS Code terminal) for me: activate
env-name and conda activate env-name.

A year later I am still running into this issue. The following is a streamlined and updated approach based on Arafat's answer.
Install Git Bash
Configure Git Bash to be used in VSC (see How do I use Bash on Windows from the Visual Studio Code integrated terminal?)
Open the git bash Terminal from VSC
If conda activate is run successfully, skip the rest
run
conda init bash
Check for the exiting bash dot files:
ls -al ~/.bash*
Likely only one of '.bashrc' and '.bash_profile' exist
Check the existing dot file for conda initialization code
e.g.
cat ~/.bash_profile
This included in my case '>>> conda initialize >>> ...' code
(But, and this is the source of the problem, it is not executed when the terminal is opened. To check which of the files is executed simple add 'echo hello-X' to each of them.)
To fix the problem, we must create the missing dot file and make it execute the OTHER previously existing one:
tee -a ~/.bashrc << END
if [ -f ~/.bash_profile ]; then
source ~/.bash_profile
fi
END
Reopen the Terminal in VSC

activate a python virtualenv from makefile [duplicate]

At the top of my makefile I have this line:
SHELL := /bin/sh
which is needed for most of the commands. However, I would like to also have a make command to activate my virtual env, which is on a different path.
Here is the code that I wrote for it:
activate:
source ~/.envs/$(APP)/bin/activate; \
The problem with this is, that this just prints out what is written here, and it doesn't get executed. I read that it might have something todo with only bash knowing about source, but I can't figure out how to temporarily switch modes within the activate command.
How would I have to write this method, so that it activates my virtualenv?

It does get executed.
Virtualenv works by modifying your current process's environment (that's why you have to "source" it). However, one process cannot modify the environment of the other process. So, to run your recipe make invokes a shell and passes it your virtualenv command, it works, then the shell exits, and your virtualenv is gone.
In short, there's no easy way to do this in a makefile. The simplest thing to do is create a script that first sources the virtualenv then runs make, and run that instead of running make.

Create a file called "make-venv" like this:
#!/bin/bash
source ./.venv/bin/activate
$2
Then add this to the first line of your Makefile
SHELL=./make-venv
Now, make-venv activates virtualenv before every command runs. Probably inefficient, but functional.

You can do it by using set, which allows you to set or unset values of shell options and positional parameters:
set -a && . venv/bin/activate && set +a

How does one enter a Python virtualenv when executing a bashscript?

If one defines which version of python to use in a bash script, it would be
export PYTHON = "/path/python/python-3.5.1/bin/python"
But for Python virtualenv's, one executes these commands in the command line
cd /path/pathto/virtualenv
source activate
cd another_directory
How does one "enter" a Python virtualenv in a bash script? What is the standard approach here?

We have to distinguish two cases here:
You want to use/call python (or python-based tools) in your bash script, but python or those tools should be taken from and run in a virtualenv
You want a script that, amongst other things, lets the shell from which you call it enter the virtualenv, so that you can interactively call python (or python-based tools) inside the virtualenv
Case 1: Using a virtualenv inside a script
How does one "enter" a Python virtualenv in a bash script?
Just like on the interactive bash command line:
source /path/to/the/virtual_env/bin/activate
What is the standard approach here?
The standard approach is not to enter the virtualenv in a bash script. Instead, call python and/or the python-based commands you want to use by their full path. To make this easier and less repetitive, you can use aliases and variables.
Case 2: Activating a virtualenv in an interactive bash session by calling a script
There already is such a script. It's called activate and it's located in the bin directory of the virtualenv. You have to source it rather than calling it like a normal command. Only then will it run in the same session instead of in a subshell, and thus only then can it make modifications to the session that won't be lost due to the subshell terminating at the end of the script.
So just do:
source /path/to/the/virtual_env/bin/activate
in your interactive shell session.
But what if you want to do more than the activate script does? You can put
source /path/to/the/virtual_env/bin/activate
into a shell script. But, due to the reason mentioned above, it won't have much effect when you call your script normally. Instead, source your script to use it from an interactive session.
Thus:
Content of my_activate.sh
#!/bin/bash
# Do something
# ...
# then
source /path/to/the/virtual_env/bin/activate
# Do more stuff
# ...
and in your interactive session
source my_activate.sh

I recommend using virtualenvwrapper. It provides some useful tools for managing your virtual environments.
pip install --user virtualenvwrapper
When you create the virtual environment, you specify which version of python should be used in the environment.
mkvirtualenv -p /usr/local/bin/python2.6 myproject.2.6
mkvirtualenv -p /usr/local/bin/python3.3 myproject.3.3
Then, "enter" the environment with the workon command.
workon myproject.2.6

Here are few steps to follow, one thing you can do is
export PYTHON = "/path/pathto/virtualenv/python"
Use this path in bashrc to use. Or you can do something like:-
vim ~/.bashrc
Go to end and set
alias python=/path/pathto/virtualenv/python
source ~/.bashrc

how to use python to do a series of commands in terminal

I put the following in my python_go.py
import os
os.system("cd some_dir") # This is the directory storing an existing virtual environment
os.system(". activate") #because I want to activate the virtual environment
os.system("cd another_dir") #this the directory I can start my work
I hope I can run the python_go.py, it can do the work mentioned above.
But when I run it, seems it can only do the first step, the rest of it, e.g. . activate seems not working.
Can someone tell me how to do it? Thank you!!

Most probably you don't have to change to some_dir to source activate so saving these lines
. some_dir/activate
cd another_dir
as, let's say go.sh and doing
. go.sh
has the same effect

If you're relying to os.system(". activate") to work if it's in the directory some_dir that won't work because the current directory won't persist across calls to os.system().
You're going to be better off calling a shell script that aggregates all three of the commands you want to do and execute that once from the python script.
Otherwise you want to set up the environment for the parent python process using os.chdir() before calling os.system on the activate call. Also, the os.system(". activate") call won't do what you want because the "dot space" notation will load information into a shell that's going to go away when the os.system call finishes.
Edited (to your followup comment):
Your shell script should look like this (do_activate.sh):
cd some_dir
. activate
cd another_dir
and the python code like this:
os.system("db_activate.sh").
Keep in mind that whatever environment variables were saved by ". activate" won't persist after the os.system call.

Your code does nothing. os.system() starts a new shell for each command i.e., all os.system() calls have no positive effect: cd and . activate may have effect only on the current shell (and possibly its children).
If all you want is to activate a virtualenv in the current shell then you should use a shell command:
$ . some_dir/activate && cd another_dir
Note: the command has effect only on the current (running) shell (and its descendants).
virtualenvwrapper provides several hooks that allows to execute commands before/after activating a virtualenv e.g., you could put cd another_dir into $VIRTUAL_ENV/bin/postactivate then it is enough to run:
$ workon <virtualenv-name>
to activate virtualenv-name virtualenv and run all the hooks (cd another_dir in this case).

You probably want to install virtualenvwrapper which does what you want:
workon envname will source the file and activate the virtualenv.
You can then do setvirtualenvproject in the desired directory and you'll automatically go to the directory where the project is located. You only need to do this command once as it'll then happen automatically from then on.

running a virtualenv python under Xcode

So far I've been able to work with Python 3 and Xcode 5.1.1 by editing the scheme, pointing the Run executable to python3 and then adding some arguments as in the following picture:
But now I would like to work with isolated Python environments through virtualenv. I can do this through the terminal, as I just have to type workon pydev where pydev is my Python environment. But I have no idea of how to modify my Xcode configuration so that I can run within a Python environment. Is there an environmental variable or something I can set to make this happen?
UPDATE
So I have a partial answer but I still can't make it work on Xcode. I created a simple bash script:
#!/usr/bin/env bash
source ~/.bash_profile
workon pydev
python3 src/__main__.py
The source is needed because otherwise the workon command doesn't work (another file provided by the virtualenv package is sourced inside .bash_profile. I can run the script in the Terminal, but the problem now is that Xcode doesn't like it:
Xcode cannot run using the selected destination.
Choose a destination with a supported architecture in order to run on this system.

I solved it, so you have to add the following pre-action to Xcode.
The source is needed because otherwise the workon command doesn't work (another file provided by the virtualenv package is sourced inside .bash_profile.