I want to activate a virtualenv instance from a Python script.
I know it's quite easy to do, but all the examples I've seen use it to run commands within the env and then close the subprocess.
I simply want to activate the virtualenv and return to the shell, the same way that bin/activate does.
Something like this:
$me: my-script.py -d env-name
$(env-name)me:
Is this possible?
Relevant:
virtualenv › Invoking an env from a script
If you want to run a Python subprocess under the virtualenv, you can do that by running the script using the Python interpreter that lives inside virtualenv's /bin/ directory:
import subprocess
# Path to a Python interpreter that runs any Python script
# under the virtualenv /path/to/virtualenv/
python_bin = "/path/to/virtualenv/bin/python"
# Path to the script that must run under the virtualenv
script_file = "must/run/under/virtualenv/script.py"
subprocess.Popen([python_bin, script_file])
However, if you want to activate the virtualenv under the current Python interpreter instead of a subprocess, you can use the activate_this.py script:
# Doing execfile() on this file will alter the current interpreter's
# environment so you can import libraries in the virtualenv
activate_this_file = "/path/to/virtualenv/bin/activate_this.py"
execfile(activate_this_file, dict(__file__=activate_this_file))
The simplest solution to run your script under virtualenv's interpreter is to replace the default shebang line with path to your virtualenv's interpreter like so at the beginning of the script:
#!/path/to/project/venv/bin/python
Make the script executable:
chmod u+x script.py
Run the script:
./script.py
Voila!
It turns out that, yes, the problem is not simple, but the solution is.
First I had to create a shell script to wrap the "source" command. That said I used the "." instead, because I've read that it's better to use it than source for Bash scripts.
#!/bin/bash
. /path/to/env/bin/activate
Then from my Python script I can simply do this:
import os
os.system('/bin/bash --rcfile /path/to/myscript.sh')
The whole trick lies within the --rcfile argument.
When the Python interpreter exits it leaves the current shell in the activated environment.
Win!
To run another Python environment according to the official Virtualenv documentation, in the command line you can specify the full path to the executable Python binary, just that (no need to active the virtualenv before):
/path/to/virtualenv/bin/python
The same applies if you want to invoke a script from the command line with your virtualenv. You don't need to activate it before:
me$ /path/to/virtualenv/bin/python myscript.py
The same for a Windows environment (whether it is from the command line or from a script):
> \path\to\env\Scripts\python.exe myscript.py
Just a simple solution that works for me. I don't know why you need the Bash script which basically does a useless step (am I wrong ?)
import os
os.system('/bin/bash --rcfile flask/bin/activate')
Which basically does what you need:
[hellsing#silence Foundation]$ python2.7 pythonvenv.py
(flask)[hellsing#silence Foundation]$
Then instead of deactivating the virtual environment, just Ctrl + D or exit. Is that a possible solution or isn't that what you wanted?
The top answer only works for Python 2.x
For Python 3.x, use this:
activate_this_file = "/path/to/virtualenv/bin/activate_this.py"
exec(compile(open(activate_this_file, "rb").read(), activate_this_file, 'exec'), dict(__file__=activate_this_file))
Reference: What is an alternative to execfile in Python 3?
The child process environment is lost in the moment it ceases to exist, and moving the environment content from there to the parent is somewhat tricky.
You probably need to spawn a shell script (you can generate one dynamically to /tmp) which will output the virtualenv environment variables to a file, which you then read in the parent Python process and put in os.environ.
Or you simply parse the activate script in using for the line in open("bin/activate"), manually extract stuff, and put in os.environ. It is tricky, but not impossible.
For python2/3, Using below code snippet we can activate virtual env.
activate_this = "/home/<--path-->/<--virtual env name -->/bin/activate_this.py" #for ubuntu
activate_this = "D:\<-- path -->\<--virtual env name -->\Scripts\\activate_this.py" #for windows
with open(activate_this) as f:
code = compile(f.read(), activate_this, 'exec')
exec(code, dict(__file__=activate_this))
I had the same issue and there was no activate_this.py in the Scripts directory of my environment.
activate_this.py
"""By using execfile(this_file, dict(__file__=this_file)) you will
activate this virtualenv environment.
This can be used when you must use an existing Python interpreter, not
the virtualenv bin/python
"""
try:
__file__
except NameError:
raise AssertionError(
"You must run this like execfile('path/to/active_this.py', dict(__file__='path/to/activate_this.py'))")
import sys
import os
base = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if(sys.platform=='win32'):
site_packages = os.path.join(base, 'Lib', 'site-packages')
else:
site_packages = os.path.join(base, 'lib', 'python%s' % sys.version[:3], 'site-packages')
prev_sys_path = list(sys.path)
import site
site.addsitedir(site_packages)
sys.real_prefix = sys.prefix
sys.prefix = base
# Move the added items to the front of the path:
new_sys_path = []
for item in list(sys.path):
if item not in prev_sys_path:
new_sys_path.append(item)
sys.path.remove(item)
sys.path[:0] = new_sys_path
Copy the file to the Scripts directory of your environment and use it like this:
def activate_virtual_environment(environment_root):
"""Configures the virtual environment starting at ``environment_root``."""
activate_script = os.path.join(
environment_root, 'Scripts', 'activate_this.py')
execfile(activate_script, {'__file__': activate_script})
activate_virtual_environment('path/to/your/venv')
Refrence: https://github.com/dcreager/virtualenv/blob/master/virtualenv_support/activate_this.py
You should create all your virtualenvs in one folder, such as virt.
Assuming your virtualenv folder name is virt, if not change it
cd
mkdir custom
Copy the below lines...
#!/usr/bin/env bash
ENV_PATH="$HOME/virt/$1/bin/activate"
bash --rcfile $ENV_PATH -i
Create a shell script file and paste the above lines...
touch custom/vhelper
nano custom/vhelper
Grant executable permission to your file:
sudo chmod +x custom/vhelper
Now export that custom folder path so that you can find it on the command-line by clicking tab...
export PATH=$PATH:"$HOME/custom"
Now you can use it from anywhere by just typing the below command...
vhelper YOUR_VIRTUAL_ENV_FOLDER_NAME
Suppose it is abc then...
vhelper abc
Related
I thought it will be as simple as adding these locations to Path or PYTHONPATH. I added them to PYTHONPATH and added PYTHONPATH to Path.
When running SET of window's terminal I can see my newly set paths;
E:\Tests> SET
Path=E:\Tests\PythonTests
PYTHONPATH=E:\Tests\PythonTests
(I simplified the list for readability)
I then create a very simple python file test.py inside E:\Tests\PythonTests with a single line:
print ("Hello world")
Now, if I cd \Tests\PythonTests I can run it successfully:
E:\Tests\PythonTests> python test.py
Hello world
If I cd \Tests I can:
E:\Tests> python pythonTests/test.py
Hello world
But if I try
E:\Tests> python test.py
python: can't open file 'test.py': [error 2] No such file or directory
Python version:
E:\Tests\PythonTests>python --version
Python 3.8.0
Am I'm missing something? What am I doing wrong?
The PYTHONPATH env var does not control where the python command searches for arbitrary Python programs. It controls where modules/packages are searched for. Google "pythonpath environment variable" for many explanations what the env var does. This is from python --help:
PYTHONPATH : ':'-separated list of directories prefixed to the
default module search path. The result is sys.path.
Specifying a file from which to read the initial Python script is not subject to any env var manipulation. In other words, running python my_prog.py only looks in the CWD for my_prog.py. It does not look at any other directory.
Returns error "OSError : no Such file or directory". We were trying to activate our newly created virtual env venvCI using the steps in builder with shellCommand.Seems like we cant activate the virtualenv venvCI. Were only new in this environment so please help us.Thanks.
from buildbot.steps.shell import ShellCommand
factory = util.BuildFactory()
# STEPS for example-slave:
factory.addStep(ShellCommand(command=['virtualenv', 'venvCI']))
factory.addStep(ShellCommand(command=['source', 'venvCI/bin/activate']))
factory.addStep(ShellCommand(command=['pip', 'install', '-r','development.pip']))
factory.addStep(ShellCommand(command=['pyflakes', 'calculator.py']))
factory.addStep(ShellCommand(command=['python', 'test.py']))
c['builders'] = []
c['builders'].append(
util.BuilderConfig(name="runtests",
slavenames=["example-slave"],
factory=factory))
Since the buildsystem creates a new Shell for every ShellCommand you can't source env/bin/activate since that only modifies the active shell's environment. When the Shell(Command) exits, the environment is gone.
Things you can do:
Give the environment manually for every ShellCommand (read what
activate does) env={...}
Create a bash script that runs all your commands in
a single shell (what I've done in other systems)
e.g.
myscript.sh:
#!/bin/bash
source env/bin/activate
pip install x
python y.py
Buildbot:
factory.addStep(ShellCommand(command=['bash', 'myscript.sh']))
blog post about the issue
Another option is to call the python executable inside your virtual environment directly, since many Python tools that provide command-line commands are often executable as modules:
from buildbot.steps.shell import ShellCommand
factory = util.BuildFactory()
# STEPS for example-slave:
factory.addStep(ShellCommand(command=['virtualenv', 'venvCI']))
factory.addStep(ShellCommand(
command=['./venvCI/bin/python', '-m', 'pip', 'install', '-r', 'development.pip']))
factory.addStep(ShellCommand(
command=['./venvCI/bin/python', '-m', 'pyflakes', 'calculator.py']))
factory.addStep(ShellCommand(command=['python', 'test.py']))
However, this does get tiresome after a while. You can use string.Template to make helpers:
import shlex
from string import Template
def addstep(cmdline, **kwargs):
tmpl = Template(cmdline)
factory.addStep(ShellCommand(
command=shlex.split(tmpl.safe_substitute(**kwargs))
))
Then you can do things like this:
addstep('$python -m pip install pytest', python='./venvCI/bin/python')
These are some ideas to get started. Note that the neat thing about shlex is that it will respect spaces inside quoted strings when doing the split.
How can I run a python script with my own command line name like myscript without having to do python myscript.py in the terminal?
Add a shebang line to the top of the script:
#!/usr/bin/env python
Mark the script as executable:
chmod +x myscript.py
Add the dir containing it to your PATH variable. (If you want it to stick, you'll have to do this in .bashrc or .bash_profile in your home dir.)
export PATH=/path/to/script:$PATH
The best way, which is cross-platform, is to create setup.py, define an entry point in it and install with pip.
Say you have the following contents of myscript.py:
def run():
print('Hello world')
Then you add setup.py with the following:
from setuptools import setup
setup(
name='myscript',
version='0.0.1',
entry_points={
'console_scripts': [
'myscript=myscript:run'
]
}
)
Entry point format is terminal_command_name=python_script_name:main_method_name
Finally install with the following command.
pip install -e /path/to/script/folder
-e stands for editable, meaning you'll be able to work on the script and invoke the latest version without need to reinstall
After that you can run myscript from any directory.
I usually do in the script:
#!/usr/bin/python
... code ...
And in terminal:
$: chmod 755 yourfile.py
$: ./yourfile.py
Another related solution which some people may be interested in. One can also directly embed the contents of myscript.py into your .bashrc file on Linux (should also work for MacOS I think)
For example, I have the following function defined in my .bashrc for dumping Python pickles to the terminal, note that the ${1} is the first argument following the function name:
depickle() {
python << EOPYTHON
import pickle
f = open('${1}', 'rb')
while True:
try:
print(pickle.load(f))
except EOFError:
break
EOPYTHON
}
With this in place (and after reloading .bashrc), I can now run depickle a.pickle from any terminal or directory on my computer.
The simplest way that comes to my mind is to use "pyinstaller".
create an environment that contains all the lib you have used in your code.
activate the environment and in the command window write pip install pyinstaller
Use the command window to open the main directory that codes maincode.py is located.
remember to keep the environment active and write pyinstaller maincode.py
Check the folder named "build" and you will find the executable file.
I hope that this solution helps you.
GL
I've struggled for a few days with the problem of not finding the command py -3 or any other related to pylauncher command if script was running by service created using Nssm tool.
But same commands worked when run directly from cmd.
What was the solution? Just to re-run Python installer and at the very end click the option to disable path length limit.
I'll just leave it here, so that anyone can use this answer and find it helpful.
I have a simple bash script that allows cron to execute a series of Python scripts in a virtualenv. The script keeps raising No such file or directory errors.
~/nightly.sh works fine:
#!/bin/bash
source virt_env/myproject/bin/activate
cd virt_env/myproject/main
python script1.py
python script2.py
I want to keep everything in ~/virt_env/myproject/main/ to simplify deployment. I thought I could call bash virt_env/myproject/main/nightly.sh on this:
#!/bin/bash
MAINDIR=`dirname $0`
cd $MAINDIR
source ../bin/activate
python script1.py
python script2.py
but I get the No such file or directory. If I manually cd to ~/virt_env/myproject/main/, then I can run the main commands no problem. Clearly I'm missing something about how dirname and cd work in this context.
How can I point bash at the right place?
Solution
As proposed in the accepted answer, it's best to avoid calling cd from within the script and use an explicit path variable instead. Here's the working version of virt_env/myproject/main/nightly.sh:
#!/bin/bash
MAINDIR=`dirname $0`
echo "The main directory is" $MAINDIR
# Activate virtual environment
source $MAINDIR/../bin/activate
# Run Python scripts
python $MAINDIR/python1.py
python $MAINDIR/python2.py
Because the Python scripts are now called from an arbitrary path, I needed to update the python scripts to be smarter about path awareness as well.
This code fails because os.path.basename omits path information:
# works when called with "python python1.py"
# fails when called with "python $MAINDIR/python1.py"
CONFIG_FILE = os.path.basename(__file__)[:-3] + ".config"
f = open(CONFIG_FILE,"r")
Updating it to use os.path.abspath fixes the problem:
# works regardless of how it is called
CONFIG_FILE = os.path.abspath(__file__)[:-3] + ".config"
f = open(CONFIG_FILE,"r")
Perhaps it would simply be better to eliminate the 'cd' command. Invoke everything from a full path specification. In your example add $MAINDIR/ to the executables.
Your bash script can then be in any directory where the executables are reachable. You are not exposed to the problems of what happens when cd fails.
Example:
cd yourdir
rm -f yourglob # oops things got removed from where you started if yourdir did not exist.
Two things:
Are you sure you know what the dirname command is doing? It's going to remove the top-level directory as well as the leading slash on whatever you call it on. I would make absolutely sure that the output of dirname is exactly what you think it is.
For example, /home/user/ will output /home.
You're using ~, which references the $HOME variable in your environment. You didn't mention where the cron is listed, but make sure it's not being run as a different user. Root's ~ and your ~ will be two completely different directories.
That's all I can think of. I hope that helps!
Add echo $MAINDIR after
MAINDIR=`dirname $0`
cd $MAINDIR
So you can see, if the contents of MAINDIR is correct.
Also you can run sh with -x or add set -x to the beginning of the script to see what happens.
I have a python script let's name it script1.py. I can run it in the terminal this way:
python /path/script1.py
...
but I want to run like a command-line program:
arbitraryname
...
how can i do it ?
You use a shebang line at the start of your script:
#!/usr/bin/env python
make the file executable:
chmod +x arbitraryname
and put it in a directory on your PATH (can be a symlink):
cd ~/bin/
ln -s ~/some/path/to/myscript/arbitraryname
There are three parts:
Add a 'shebang' at the top of your script which tells how to execute your script
Give the script 'run' permissions.
Make the script in your PATH so you can run it from anywhere.
Adding a shebang
You need to add a shebang at the top of your script so the shell knows which interpreter to use when parsing your script. It is generally:
#!path/to/interpretter
To find the path to your python interpretter on your machine you can run the command:
which python
This will search your PATH to find the location of your python executable. It should come back with a absolute path which you can then use to form your shebang. Make sure your shebang is at the top of your python script:
#!/usr/bin/python
Run Permissions
You have to mark your script with run permissions so that your shell knows you want to actually execute it when you try to use it as a command. To do this you can run this command:
chmod +x myscript.py
Add the script to your path
The PATH environment variable is an ordered list of directories that your shell will search when looking for a command you are trying to run. So if you want your python script to be a command you can run from anywhere then it needs to be in your PATH. You can see the contents of your path running the command:
echo $PATH
This will print out a long line of text, where each directory is seperated by a semicolon. Whenever you are wondering where the actual location of an executable that you are running from your PATH, you can find it by running the command:
which <commandname>
Now you have two options: Add your script to a directory already in your PATH, or add a new directory to your PATH. I usually create a directory in my user home directory and then add it the PATH. To add things to your path you can run the command:
export PATH=/my/directory/with/pythonscript:$PATH
Now you should be able to run your python script as a command anywhere. BUT! if you close the shell window and open a new one, the new one won't remember the change you just made to your PATH. So if you want this change to be saved then you need to add that command at the bottom of your .bashrc or .bash_profile
Add the following line to the beginning script1.py
#!/usr/bin/env python
and then make the script executable:
$ chmod +x script1.py
If the script resides in a directory that appears in your PATH variable, you can simply type
$ script1.py
Otherwise, you'll need to provide the full path (either absolute or relative). This includes the current working directory, which should not be in your PATH.
$ ./script1.py
You need to use a hashbang. Add it to the first line of your python script.
#! <full path of python interpreter>
Then change the file permissions, and add the executing permission.
chmod +x <filename>
And finally execute it using
./<filename>
If its in the current directory,