Is there a difference between running script using virtualenv interpreter (without virtualenv activation) and running it in activated virtualenv?
venv/bin/python some_script.py
vs
source venv/bin/activate
python some_script.py
Running source bin/activate will set the PATH variable to point to your environment bin directory which is useful if you have other command line scripts/binaries installed (this can happen with certain python packages that add shell commands), it will also unset/set PYTHONHOME.
So, if bin/python works for you then you're fine but if some of the packages you're using start behaving strangely (or wrong one gets imported) it's probably because Python is getting the wrong PYTHONHOME or because a certain script is not found in PATH.
If you directly run a script or the python interpreter from the virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or /path/to/ENV/bin/python-script.py) then sys.path will automatically be set to use the Python libraries associated with the virtualenv. But, unlike the activation scripts, the environment variables PATH and VIRTUAL_ENV will not be modified. This means that if your Python script uses e.g. subprocess to run another Python script (e.g. via a #!/usr/bin/env python shebang line) the second script may not be executed with the same Python binary as the first nor have the same libraries available to it. To avoid this happening your first script will need to modify the environment variables in the same manner as the activation scripts, before the second script is executed.
source: https://virtualenv.pypa.io/en/16.7.9/userguide.html#activate-script
Yes. Virtualenv creates an interpreter in its own right. Just do this,
which python
For each interpreter, virtualenv and your normal interpreter and see what happens. They will show you two different links to python interpreter. Here's my example:
quazinafiulislam#Nafiuls-Mac: ~/Code/Python/PyTestingZone
$ which python [7:49:26]
/Users/quazinafiulislam/.pyenv/shims/python
quazinafiulislam#Nafiuls-Mac: ~/Code/Python/PyTestingZone
$ source .venv/bin/activate [7:49:29]
(.venv)
quazinafiulislam#Nafiuls-Mac: ~/Code/Python/PyTestingZone
$ which python [7:49:35]
/Users/quazinafiulislam/Code/Python/PyTestingZone/.venv/bin/python
Related
What is the difference between running a script by first sourcing:
source /venv/bin/activate
python script.py
and running the script with the actual python exe?
/venv/bin/python script.py
do these two commands always do the same thing? The problem that I am seeing is that if script.py calls other python scripts, and the settings and packages that are in venv don't work.
It depends on how does script.py runs other scripts. If it uses sys.executable, that is the current python, /venv/bin/python then both commands are equivalent.
If script.py runs other scripts using shell (os.system, subprocess, etc) then the first one is the preferred form because it sets $PATH for all subprocesses so that all python scripts use the same virtual environment.
And the final note. If some script(s) being run from script.py have fixed shebang like #!/usr/bin/python those scripts will not be run in your virtual environment regardless of the 1st or 2nd way you run script.py.
Using Python 3.7.0 on Mac. Trying to use venv module that was added post python 3.4.
I setup my virtual env using python3 venv -m path/to/my/dir - my question is do I need to activate this virtual env to use?
The documentation seem to imply I don't need to?
You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.
If I don't have to activate, what is the benefit of prepending venv to binary directory? Wouldn't this have to happen regardless for a venv to work?
Activating the virtualenv gives you convenience. It is never required.
Even for scripts that are configured to run with #!/usr/bin/env python, (which looks up the python executable on your path), you could manually update the PATH environment variable:
$ PATH="/path/to/venv/bin" some_script
Activating makes the PATH update stick until you deactivate again, and that can be convenient.
For example, I regularly have several virtualenvs in use at any one time. Some of them are there only to install some command-line tools I have symlinked into my ~/bin/ directory, another is home to a Jupyter notebook, and 2 more are used to quickly verify code snippets in different Python versions with access to 3rd-party libraries. I don't activate any of those.
When you don’t activate a virtualenv, all that happens is that your terminal PATH variable is not updated to put the bin directory of the virtualenv first, so when you enter python or pip or other script without any path into the terminal, the shell will find a different Python binary to run. You can always use any of the commands in the virtualenv bin/ directory by giving the full path to that command.
I'm using PythonAnywhere. I've used virtualenvwrapper to create an environment where I have the required packages.
When in bash I do:
activate virtualenv
python my_subfolder/my_script.py
and it works.
However, I want to run this scheduled using PythonAnywhere's feature for scheduling. But when it runs, it reverts (expectedly) to the default Python interpreter.
I'm not so good with paths and navigating, so I tried shebanging in various ways. I know for certain that my venv is in this path:
https://www.pythonanywhere.com/user/my_username/files/home/my_username/.virtualenvs/my_venv/bin/python
I'm omitting the domain of course, but trying to append any of the following parts of the URL to #! like so never works:
#!user/my_username/files/home/my_username/.virtualenvs/my_venv/bin/python
Please note that the script is in a subfolder my_subfolder/my_script.py. I've also chmod u+x the Python script. Very confused.
If one defines which version of python to use in a bash script, it would be
export PYTHON = "/path/python/python-3.5.1/bin/python"
But for Python virtualenv's, one executes these commands in the command line
cd /path/pathto/virtualenv
source activate
cd another_directory
How does one "enter" a Python virtualenv in a bash script? What is the standard approach here?
We have to distinguish two cases here:
You want to use/call python (or python-based tools) in your bash script, but python or those tools should be taken from and run in a virtualenv
You want a script that, amongst other things, lets the shell from which you call it enter the virtualenv, so that you can interactively call python (or python-based tools) inside the virtualenv
Case 1: Using a virtualenv inside a script
How does one "enter" a Python virtualenv in a bash script?
Just like on the interactive bash command line:
source /path/to/the/virtual_env/bin/activate
What is the standard approach here?
The standard approach is not to enter the virtualenv in a bash script. Instead, call python and/or the python-based commands you want to use by their full path. To make this easier and less repetitive, you can use aliases and variables.
Case 2: Activating a virtualenv in an interactive bash session by calling a script
There already is such a script. It's called activate and it's located in the bin directory of the virtualenv. You have to source it rather than calling it like a normal command. Only then will it run in the same session instead of in a subshell, and thus only then can it make modifications to the session that won't be lost due to the subshell terminating at the end of the script.
So just do:
source /path/to/the/virtual_env/bin/activate
in your interactive shell session.
But what if you want to do more than the activate script does? You can put
source /path/to/the/virtual_env/bin/activate
into a shell script. But, due to the reason mentioned above, it won't have much effect when you call your script normally. Instead, source your script to use it from an interactive session.
Thus:
Content of my_activate.sh
#!/bin/bash
# Do something
# ...
# then
source /path/to/the/virtual_env/bin/activate
# Do more stuff
# ...
and in your interactive session
source my_activate.sh
I recommend using virtualenvwrapper. It provides some useful tools for managing your virtual environments.
pip install --user virtualenvwrapper
When you create the virtual environment, you specify which version of python should be used in the environment.
mkvirtualenv -p /usr/local/bin/python2.6 myproject.2.6
mkvirtualenv -p /usr/local/bin/python3.3 myproject.3.3
Then, "enter" the environment with the workon command.
workon myproject.2.6
Here are few steps to follow, one thing you can do is
export PYTHON = "/path/pathto/virtualenv/python"
Use this path in bashrc to use. Or you can do something like:-
vim ~/.bashrc
Go to end and set
alias python=/path/pathto/virtualenv/python
source ~/.bashrc
I am trying to get a django project up and running, and I have django running, but I am having trouble with python's virtualenv.
here is my error (on terminal open this shows up)
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named virtualenvwrapper
virtualenvwrapper.sh: There was a problem running the initialization hooks.
If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenvwrapper has been installed for
VIRTUALENVWRAPPER_PYTHON=/Library/Frameworks/Python.framework/Versions/2.7/bin/python and that PATH is
set properly.
Here is what My .bash profile looks like:
# Setting PATH for Python 3.4
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.4/bin:${PATH}"
export PATH
# Setting PATH for Python 2.7
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
export WORKON_HOME=$HOME/.virtualenvs
source /Library/Frameworks/Python.framework/Versions/3.4/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=$WORKON_HOME
if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi
I am pretty new to command line stuff and virtual environments, so I might not know some "obvious" things I am doing wrong. Thanks.
First some basics of the command-line shell:
PATH is an environment variable that contains a list of filesystem directories. When you type a command such as ls, python or virtualenvwrapper.sh your shell will search each directory starting from the first one listed. To see your current PATH type:
$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
To see what a given command is going to resolve to the location of the program that is going to be run, use the which command:
$ which ls
/bin/ls
Now in your example you are first adding a Python 3.4 location to your PATH and then a Python 2.7 location. The latter location is going to be first on your PATH. So all your Python related commands are first going to try and run Python 2.7, if a command isn't found there, it next searches in your Python 3.4 installation.
You appear to have installed virtualenvwrapper for Python 3.4, but when you run virtualenvwrapper.sh from the line:
source /Library/Frameworks/Python.framework/Versions/3.4/bin/virtualenvwrapper.sh
You are explicitly running the virtualenvwrapper.sh installed for Python 3.4. This runs a python command where your Python 2.7 is run, which does not appear to have virtualenvwrapper installed, as shown by this error message:
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named virtualenvwrapper
If you tried removing the Python 2.7 location addition to your PATH and it didn't work, you have to remember that changes to your .bash_profile are not automatically reflected in your terminal. You have to create a new terminal session and that new session will read your new .bash_profile.
If you are trying to get the command virtualenv to resolve to two different Pythons, it's simply not possible. It will always resolve to one specific Python based on the ordering of the locations on your PATH. VirtualEnv does install alternate versions of the command with the version of the Python built into the name. Use the commands virtualenv-3.4 and virtualenv-2.7 to create virtual environments for each different Python.
The key to all of this is remembering that PATH is used to resolve to an absolute path location of a program this is run. Use the echo $PATH and which commands to help you understand how that final path is being resolved.
It may seem tedious, but typing out the full absolute path is always going to side-step the magic of PATH resolution:
$ /Library/Frameworks/Python.framework/Versions/2.7/bin/virtualenv
Try using smaller steps on your way to manipulating the path. Use just virutalenv-3.4 to make a Python 3.4 virtual environment and only bring in virtualenvwrapper as you need it and after your understanding of shell environments is more solid.
Finally, even pro's can get tripped up by playing games manipulating the path. You are ultimately trying to take a bunch of different application locations and flatten them all into one namespace. At some point conflicts can become unresolveable. A Python installation tool such as Buildout has it's own learning curve, but it hard-codes the absolute path to the python of each python script that it installs. Hard-coding absolute paths is the only way to deal with extreme corner cases such as having two builds of Python 3.4 side-by-side where you have two scripts which each need to run on two different builds of the same Python. Hard-coding absolute locations is also desirable in production environments, because then your application is immune to any changes to the bash shell. If you forget about a required ordering of your PATH, or another sysadmin tinkers with the PATH, you won't find your application breaking unexpectedly.