I want to use a makefile to build my project's environment using a makefile and anaconda/miniconda, so I should be able to clone the repo and simply run make myproject
myproject: build
build:
#printf "\nBuilding Python Environment\n"
#conda env create --quiet --force --file environment.yml
#source /home/vagrant/miniconda/bin/activate myproject
If I try this, however, I get the following error
make: source: Command not found
make: *** [source] Error 127
I have searched for a solution, but [this question/answer(How to source a script in a Makefile?) suggests that I cannot use source from within a makefile.
This answer, however, proposes a solution (and received several upvotes) but this doesn't work for me either
( \
source /home/vagrant/miniconda/bin/activate myproject; \
)
/bin/sh: 2: source: not found
make: *** [source] Error 127
I also tried moving the source activate step to a separate bash script, and executing that script from the makefile. That doesn't work, and I assume for the a similar reason, i.e. I am running source from within a shell.
I should add that if I run source activate myproject from the terminal, it works correctly.
I had a similar problem; I wanted to create, or update, a conda environment from a Makefile to be sure my own scripts could use the python from that conda environment.
By default make uses sh to execute commands, and sh doesn't know source (also see this SO answer). I simply set the SHELL to bash and ended up with (relevant part only):
SHELL=/bin/bash
CONDAROOT = /my/path/to/miniconda2
.
.
install: sometarget
source $(CONDAROOT)/bin/activate && conda env create -p conda -f environment.yml && source deactivate
Hope it helps
You should use this, it's functional for me at moment.
report.ipynb : merged.ipynb
( bash -c "source ${HOME}/anaconda3/bin/activate py27; which -a python; \
jupyter nbconvert \
--to notebook \
--ExecutePreprocessor.kernel_name=python2 \
--ExecutePreprocessor.timeout=3000 \
--execute merged.ipynb \
--output=$< $<" )
I had the same problem. Essentially the only solution is stated by 9000. I have a setup shell script inside which I setup the conda environment (source activate python2), then I call the make command. I experimented with setting up the environment from inside Makefile and no success.
I have this line in my makefile:
installpy :
./setuppython2.sh && python setup.py install
The error messages is:
make
./setuppython2.sh && python setup.py install
running install
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/test-easy-install-29183.write-test'
Essentially, I was able to set up my conda environment to use my local conda that I have write access. But this is not picked up by the make process. I don't understand why the environment set up in my shell script using 'source' is not visible in the make process; the source command is supposed to change the current shell. I just want to share this so that other people don't wast time trying to do this. I know autotoools has a way of working with python. But the make program is probably limited in this respect.
My current solution is a shell script:
cat py2make.sh
#!/bin/sh
# the prefix should be change to the target
# of installation or pwd of the build system
PREFIX=/some/path
CONDA_HOME=$PREFIX/anaconda3
PATH=$CONDA_HOME/bin:$PATH
unset PYTHONPATH
export PREFIX CONDA_HOME PATH
source activate python2
make
This seems to work well for me.
There were a solution for similar situation but it does not seems to work for me:
My modified Makefile segment:
installpy :
( source activate python2; python setup.py install )
Error message after invoking make:
make
( source activate python2; python setup.py install )
/bin/sh: line 0: source: activate: file not found
make: *** [installpy] Error 1
Not sure where am I wrong. If anyone has a better solution please share it.
I've noticed when I get traceback errors in any project, the file path in the trace shows one specific project I no longer use. For example:
File "/Users/MyComputer/Developer/SampleApp/venv/lib/python2.7/site-packages/flask/app.py"...
I'm assuming the site-packages are not suppose to be from SampleApp, correct? I do use virtual environments in all my projects. When I run echo $PATH while in an activated virtual environment I get:
/Users/MyComputer/Developer/SampleApp/venv/bin: #<----- Shouldn't point to a specific project!
/Library/Frameworks/Python.framework/Versions/3.5/bin:
/Users/MyComputer/bin/:
/usr/local/bin:
/usr/bin:
/bin:
/usr/sbin:
/sbin:
/usr/local/git/bin:
/Users/MyComputer/.rvm/bin:
/Applications/Postgres.app/Contents/Versions/9.4/bin
When I run echo $PATH in a deactivated virtual environment I get the above without /Users/MyComputer/Developer/SampleApp/venv/bin.
Here is what is in my ./bashrc:
export PATH=$PATH:$HOME/usr/local/bin
export PATH=$PATH:$HOME/usr/bin/env
Here's what is in my .bash_profile:
[[ -s "$HOME/.profile" ]] && source "$HOME/.profile" # Load the default .profile
# Postgres.app
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/latest/bin
# Setting PATH for Python 3.5
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.5/bin:${PATH}"
export PATH
Why is the activated venv path different from a deactivated env? Thank you in advance! :)
I tried to change the default behaviour of cd with virtualenvwrapper via the instructions here: http://virtualenvwrapper.readthedocs.org/en/latest/tips.html#changing-the-default-behavior-of-cd
and placing the code in my .virtualenvs folder and postactivate and postdeactivate files.
postactivate:
#!/bin/bash
# This hook is sourced after every virtualenv is activated.
cd () {
if (( $# == 0 ))
then
builtin cd $VIRTUAL_ENV
else
builtin cd "$#"
fi
}
cd
post deactivate:
#!/bin/bash
# This hook is sourced after every virtualenv is deactivated.
cd () {
builtin cd "$#"
}
However it doesn't seem to work properly and now when I use workon project it doesn't automatically cd to the project folder listed in the .project file (which can be made with the mkproject command.
(Note if relevant I'm using zshell & prezto)
The recipe you posted isn't supposed to do what you're expecting. What it actually does is that whenever you type cd without any path in the terminal it navigates to the virtualenv root instead of the home folder.
I'd recommend you to set up virtualenvwrapper projects so that you can separate your codebase from the virtualenv (use a requirements file instead for portability!). I.e. to add to your shell file
PROJECT_HOME='path/to/your/projects/folder'
So that mkproject will create a path/to/your/projects/folder/[PROJECT_NAME] folder for you and workon will automatically cd into it.
However, if you don't want to use projects, you should change your postactivate script like this in order to achieve what you want:
cd $VIRTUAL_ENV
I am trying to install requirements for each project in a list automatically into its own virtualenv. I have gotten to the point of making the virtualenv correctly, but I cannot get it to activate and install requirements into only that virtualenv:
#!/usr/bin/env python
import subprocess, sys, time, os
HOMEPATH = os.path.expanduser('~')
CWD = os.getcwd()
d = {'cwd': ''}
if len(sys.argv) == 2:
projects = sys.argv[1:]
def call_sp(command, **arg_list):
p = subprocess.Popen(command, shell=True, **arg_list)
p.communicate()
def my_makedirs(path):
if not path.startswith('/home/cchilders'):
path = os.path.join(HOMEPATH, path)
try: os.makedirs(path)
except: pass
for project in projects:
path = os.path.join(CWD, project)
my_makedirs(path)
git_string = 'git clone git#bitbucket.org:codyc54321/{}.git {}'.format(project, d['cwd'])
call_sp(git_string)
d = {'executable': 'bash'}
call_sp("""source /usr/local/bin/virtualenvwrapper.sh && mkvirtualenv --no-site-packages {}""".format(project), **d)
# call_sp("""source /usr/local/bin/virtualenvwrapper.sh && workon {}""".format(project), **d)
# below, the dot (.) means the same as 'source'. the dot doesn't error, calling source does
call_sp('. /home/cchilders/.virtualenvs/{}/bin/activate'.format(project))
d = {'cwd': path}
call_sp("pip install -r requirements.txt", **d)
It works up to
call_sp("""source /usr/local/bin/virtualenvwrapper.sh && mkvirtualenv --no-site-packages {}""".format(project), **d)
but when the script ends, I am not active in the venv and the venv does not have any packages from requirements. Both efforts to source the venv (the one commented out and live) both fail.
The answer that helped me get the mkvirtualenv to work is subprocess.Popen: mkvirtualenv not found.
I also noticed I have a need to do more than just pip install, in one case I need to run 'python setup.py mycommand' which automates setup for each project. How can run commands as if a virtualenv is activated and also install dependencies to arbitrary venvs in a python script?
The only way I've found around this is turning the virtualenv on by hand, then calling my python script by hand. I was surprised, turning it on by bash worked, but calling the python script bombed (maybe because it's a different process than the bash one)
Thank you
This is because each call_sp call creates a new shell, so after the first call to call_sp ends all the settings created by sourcing of virtualenvwrapper are gone. You have to combine all your commands into the single call_sp chain. Otherwise you can just start shell using 'Popen' and feed commands to it using communicate.
If you go with the later you need to be careful with synchronizing and detecting when installation of requirements ends. Pip can take a long time downloading and installing packages with complex dependencies.
This is the way I have done this kind of bootstrapping for virtual environments. Let the script take care of it's own env and just run the script. Running this app.py will setup its VE and modules if missing.
./requirements.txt file
flask
./app.py script
#!/bin/bash
""":"
VENV=$(realpath -s $(dirname $0)/ve)
PYTHON=$VENV/bin/python
if [ ! -f "$PYTHON" ]; then
echo "installing env app"
python3 -m venv $VENV
${VENV}/bin/pip install -r $(dirname $0)/requirements.txt
fi
exec $PYTHON $0 $#
"""
import flask
print("I am Python with flask", flask)
No matter what dir we are in, app.py bootstrapps though the bash script header, installing a ve if python does not exist, running pip, and whatever else you need. Then exec $PYTHON $0 $# is a slick way to swap out bash process for the python process keeping the same pid.
When python takes over, it skips over the bash part because that script is in triple quotes string. So the first line python executes is import flask (well it discards the bash script string 1st). Another cool thing is the pid of the bash process is the same as the pid of the python process. So any daemon utility that babysits this will still see the pid it started.
The last trick in this is that bash needs one extra quote to balance its string """:" at the top. Python does not care about that extra quote
I hope you see the pattern. To upgrade modules in requirements.txt, just rm the ve and run the app again. Simple.
I have been searching and tried various alternatives without success and spent several days on it now - driving me mad.
Running on Red Hat Linux with Python 2.5.2
Began using most recent Virtualenv but could not activate it, I found somewhere suggesting needed earlier version so I have used Virtualenv 1.6.4 as that should work with Python 2.6.
It seems to install the virtual environment ok
[necrailk#server6 ~]$ python virtualenv-1.6.4/virtualenv.py virtual
New python executable in virtual/bin/python
Installing setuptools............done.
Installing pip...............done.
Environment looks ok
[necrailk#server6 ~]$ cd virtual
[necrailk#server6 ~/virtual]$ dir
bin include lib
Trying to activate
[necrailk#server6 ~/virtual]$ . bin/activate
/bin/.: Permission denied.
Checked chmod
[necrailk#server6 ~/virtual]$ cd bin
[necrailk#server6 bin]$ ls -l
total 3160
-rw-r--r-- 1 necrailk biz12 2130 Jan 30 11:38 activate
-rw-r--r-- 1 necrailk biz12 1050 Jan 30 11:38 activate.csh
-rw-r--r-- 1 necrailk biz12 2869 Jan 30 11:38 activate.fish
-rw-r--r-
Problem, so I changed it
[necrailk#server6 bin]$ ls -l
total 3160
-rwxr--r-- 1 necrailk biz12 2130 Jan 30 11:38 activate
-rw-r--r-- 1 necrailk biz12 1050 Jan 30 11:38 activate.csh
-rw-r--r-- 1 necrailk biz12 2869 Jan 30 11:38 activate.fish
-rw-r--r-- 1 necrailk biz12 1005 Jan 30 11:38 activate_this.py
-rwxr-xr-x 1 necrailk biz
Try activate again
[necrailk#server6 ~/virtual]$ . bin/activate
/bin/.: Permission denied.
Still no joy...
Here is my workflow after creating a folder and cd'ing into it:
$ virtualenv venv --distribute
New python executable in venv/bin/python
Installing distribute.........done.
Installing pip................done.
$ source venv/bin/activate
(venv)$ python
You forgot to do source bin/activate where source is a executable name.
Struck me first few times as well, easy to think that manual is telling "execute this from root of the environment folder".
No need to make activate executable via chmod.
You can do
source ./python_env/bin/activate
or just go to the directory
cd /python_env/bin/
and then
source ./activate
Good Luck.
Go to the project directory. In my case microblog is the flask project directory and under microblog directory there should be app and venv folders. then run the below command, This is one worked for me in Ubuntu.
source venv/bin/activate
Cd to the environment path, go to the bin folder.
At this point when you use ls command, you should see the "activate" file.
now type
source activate
$ mkdir <YOURPROJECT>
Create a new project
$ cd <YOURPROJECT>
Change directory to that project
$ virtualenv <NEWVIRTUALENV>
Creating new virtualenv
$ source <NEWVIRTUALENV>/bin/activate
Activating that new virtualenv
run this code it will get activated if you on a windows machine
source venv/Scripts/activate
run this code it will get activated if you on a linux/mac machine
. venv/bin/activate
The problem there is the /bin/. command. That's really weird, since . should always be a link to the directory it's in. (Honestly, unless . is a strange alias or function, I don't even see how it's possible.) It's also a little unusual that your shell doesn't have a . builtin for source.
One quick fix would be to just run the virtualenv in a different shell. (An obvious second advantage being that instead of having to deactivate you can just exit.)
/bin/bash --rcfile bin/activate
If your shell supports it, you may also have the nonstandard source command, which should do the same thing as ., but may not exist. (All said, you should try to figure out why your environment is strange or it will cause you pain again in the future.)
By the way, you didn't need to chmod +x those files. Files only need to be executable if you want to execute them directly. In this case you're trying to launch them from ., so they don't need it.
instead of ./activate
use source activate
For Windows You can perform as:
TO create the virtual env as: virtualenv envName –python=python.exe (if not create environment variable)
To activate the virtual env : > \path\to\envName\Scripts\activate
To deactivate the virtual env : > \path\to\env\Scripts\deactivate
It fine works on the new python version .
Windows 10
In Windows these directories are created :
To activate Virtual Environment in Windows 10.
down\scripts\activate
\scripts directory contain activate file.
Linux Ubuntu
In Ubuntu these directories are created :
To activate Virtual Environment in Linux Ubuntu.
source ./bin/activate
/bin directory contain activate file.
Virtual Environment copied from Windows to Linux Ubuntu vice versa
If Virtual environment folder copied from Windows to Linux Ubuntu then according to directories:
source ./down/Scripts/activate
I would recommend virtualenvwrapper as well. It works wonders for me and how I always have problems with activating. http://virtualenvwrapper.readthedocs.org/en/latest/
Create your own Python virtual environment called <Your Env _name >:.
I have given it VE.
git clone https://github.com/pypa/virtualenv.git
python virtualenv.py VE
To activate your new virtual environment, run (notice it's not ./ here):
. VE/bin/activate
Sample output (note prompt changed):
(VE)c34299#a200dblr$
Once your virtual environment is set, you can remove the Virtualenv repo.
On Mac, change shell to BASH (keep note that virtual env works only in bash shell )
[user#host tools]$. venv/bin/activate
.: Command not found.
[user#host tools]$source venv/bin/activate
Badly placed ()'s.
[user#host tools]$bash
bash-3.2$ source venv/bin/activate
(venv) bash-3.2$
Bingo , it worked. See prompt changed.
On Ubuntu:
user#local_host:~/tools$ source toolsenv/bin/activate
(toolsenv) user#local_host~/tools$
Note : prompt changed
I had trouble getting running source /bin/activate then I realized I was using tcsh as my terminal shell instead of bash. once I switched I was able to activate venv.
Probably a little late to post my answer here but still I'll post, it might benefit someone though,
I had faced the same problem,
The main reason being that I created the virtualenv as a "root" user
But later was trying to activate it using another user.
chmod won't work as you're not the owner of the file, hence the alternative is to use chown (to change the ownership)
For e.g. :
If you have your virtualenv created at /home/abc/ENV
Then CD to /home/abc
and run the command : chown -Rv [user-to-whom-you want-change-ownership] [folder/filename whose ownership needs to be changed]
In this example the commands would be : chown -Rv abc ENV
After the ownership is successfully changed you can simply run source /ENV/bin/./activate and your should be able to activate the virtualenv correctly.
1- open powershell and navigate to your application folder
2- enter your virtualenv folder ex : cd .\venv\Scripts\
3- active virtualenv by type .\activate