We Keep Coding

Activating Python Virtual env in Batch file

I'm trying to make a batch script (called run_windows) that check if the python virtual environment exists and if not, create it then activate it, install the requirements and finally run some python code.
set "VIRTUAL_ENV=mat_visualizer_env"
:read_mat
%VIRTUAL_ENV%\Scripts\activate
pip install -r app_files/requirements.txt
python -c "import sys; sys.path.insert(1,'app_files'); from main import visualize_mat_eeg; visualize_mat_eeg('%1')"
pause
EXIT /B 0
IF EXIST "%VIRTUAL_ENV%\Scripts\activate.bat" (
CALL :read_mat
) ELSE (
pip install virtualenv
python -m venv mat_visualizer_env
CALL :read_mat
)
However, when I run my script, the code exits at line 4: %VIRTUAL_ENV%\Scripts\activate with no errors:

Few things:
Running the .bat or bash version of activate in powershell is not predictable, Activate.ps1 is found in newer releases. It could probably be used with older versions if copied from a newer release.
Scripts in windows (.bat or .cmd in windows) processes from top down, :<ANCHOR> is not skipped like a Sub or Function would be.
I have only worked with the Conda version of activate.ps1 and it adds some nice commands. Had a quick look in standard Python 3.10.5 and it does not add commands but should work fine.
Edit: Added that while working, is not the best option to use the bash version of activate

Here is a quick example of your batch file rearranged, and using the Call command as previously instructed.
#Echo Off
Set "VIRTUAL_ENV=mat_visualizer_env"
If Not Exist "%VIRTUAL_ENV%\Scripts\activate.bat" (
pip.exe install virtualenv
python.exe -m venv %VIRTUAL_ENV%
)
If Not Exist "%VIRTUAL_ENV%\Scripts\activate.bat" Exit /B 1
Call "%VIRTUAL_ENV%\Scripts\activate.bat"
pip.exe install -r app_files/requirements.txt
python.exe -c "import sys; sys.path.insert(1,'app_files'); from main import visualize_mat_eeg; visualize_mat_eeg('%~1')"
Pause
Exit /B 0
Please note, that there is no working directory defined in this script, as was yours, and therefore no way to guarantee that when run, the relative paths point to the intended locations. Additionally, you have not made it clear what %1 was supposed to be, or where it was coming from, so I cannot guarantee whether that is correct.

Run python method in virtualenv from crontab

I am currently struggling with how to run a method from a python file in a virtual environment via crontab.
I have a directory that looks as follows: /home/ubuntu/project has the file file.py
and the folder venv in it. In file.py there is a method() that I want to execute regularly via crontab, using the python and dependencies of the virtual environment.
I have already figured out that I need to use the python from inside the virtual environment, so instead of
python3
I use
/home/ubuntu/project/venv/bin/python3.
Now, I have also found answers to the question how to run a method from the command line, namely via
python3 -c 'import foo; print foo.hello()'.
I have tried to combine the two, but unfortunately
/home/ubuntu/project/venv/bin/python3 -c 'import /home/ubuntu/project/file; print(file.method())'
is invalid syntax. Also
/home/ubuntu/project/venv/bin/python3 -c 'from /home/ubuntu/project/ import file; print(file.method())'
only results in errors. On the other hand,
/home/ubuntu/project/venv/bin/python3 -c 'import file; print(file.method())'
results in file not being found.
How do I do this properly?
Thank you very much for considering this question.

The argument to import is not a file name. The simplest workaround is probably to cd into the directory, then run the script with the virtual environment's Python interpreter.
42 17 * * * cd project && ./venv/bin/python3 -c 'import file; file.method()'
from the crontab of the user whose home directory is /home/ubuntu.
More generally, the directory you want to import from needs to be on your PYTHONPATH,so you could equivalently set that instead of cd into the directory. A third alternative is to make the code in file.py into an installable module, and install it in the virtual environment. For a one-off, this may be an unnecessary chore, but it is definitely the most robust and sustainable solution.

sudo privileges within python virtualenv

Environment
Windows Subsystem for Linux with Serial Communication to a GPS.
Adafruit GPS connected to a Arduino Nano which is connected to COM10. In Windows Subsystem for Linux this is equivalent to /dev/ttyS10
Requirements: pyserial
I have written a simple script to read information from the GPS module:
import serial
def select_sentence():
""" This function sends serial data to the GPS module to display only GPGGA and GPRMC"""
def read_gps():
ser = serial.Serial("/dev/ttyS10", 9600)
while True:
print(ser.readline().decode('utf-8'))
if __name__ == "__main__":
select_sentence()
read_gps()
In the virtualenv I chose Python3 and when I executed it I got Permission Error for the serial port /ttyS10 so I chose to sudo chmod 666 /dev/ttyS10 to use the script in the virtualenv.
However is there an alternative to the above mentioned chmod /dev/serial in order to avoid the PermissionErrors?
I am aware that even in the virtualenv when one uses sudo the packages installed in the virtualenv are no considered and instead sudo looks for your global pip packages.

When you activate a virtualenv (by source venv/bin/activate or similar), that basically just tells your shell: "hey, when you search for a command, look in venv/bin before you look anywhere else", by updating the $PATH environment variable. That way, when you run a command like python, your shell sees and runs the python in venv/bin instead of in /usr/bin or wherever. That copy of Python is configured to look in venv/lib for packages rather than /usr/lib, so you can use the packages in your virtualenv instead of the ones installed globally.
However, when you run a program with sudo, it ignores $PATH. Why does it do that? Because in the historical days of *nix, it was common to have sudo set up so that users could execute only specific commands with it, like (say) sudo iftop1, so that anyone could check what the network was being used for, but still nobody could run sudo rm -rf /*. If sudo respected the user's $PATH, you could just copy /bin/rm to ~/bin/iftop, add ~/bin to your $PATH, then run sudo iftop – but you would actually be running rm as root!
So, sudo ignores $PATH by default. But you can still execute specific programs by giving sudo the full path to the program, so you can execute the Python in your virtualenv as root by running something like sudo ./venv/bin/python (assuming your virtualenv is called venv). That will make you root while still having access to the packages in your virtualenv, like pyserial.
1: I don't actually know of any command that would be set up like this, this is a bad example, sorry.

Also make sure you have created virtualenv without sudo command, since that may cause permissions issues on using virtual env without sudo later. If that's the case run the command below:
sudo chown -R your_username:your_username path/to/virtuaelenv/
Than you can enable permissions for reading the /dev/ttyS10 for your user and run python script by that user.
NOTE: Also you want to add shebang line to to the top of your python script with the path to python interpreter which sits in your env. So you will be able to call it without interpreter.
#!/usr/bin/env python
See more on that SO Answer: Should I put #! (shebang) in Python scripts, and what form should it take?

Here is my workaround on bash. Put this in an executable file on your PATH (e.g. vesudo):
#!/bin/bash
if [ -z "$VIRTUAL_ENV" ]; then
echo "Error: Virtual environment not found" >&2
exit 1
fi
_args=''
for _a in "$#"; do
_a="${_a//\\/\\\\}"
_args="$_args \"${_a//\"/\\\"}\""
done
sudo bash <<_EOF
source "$VIRTUAL_ENV/bin/activate"
$_args
_EOF
The logic is simple: Escape input arguments, run a privileged subshell, source virtual environment and pass arguments to the subshell.
Example usage:
~/tmp$ source .venv/bin/activate
(.venv) ~/tmp$ which python
/home/omer/tmp/.venv/bin/python
(.venv) ~/tmp$ vesudo which python
/home/omer/tmp/.venv/bin/python
(.venv) ~/tmp$ which pip
/home/omer/tmp/.venv/bin/pip
(.venv) ~/tmp$ vesudo which pip
/home/omer/tmp/.venv/bin/pip
(.venv) ~/tmp$ vesudo python -c 'import sys; print(sys.argv)' it works 'with spaced arguments' as well
['-c', 'it', 'works', 'with spaced arguments', 'as', 'well']
(.venv) ~/tmp$ vesudo echo '$HOME'
/root
I put this script in a repo for convenience.

Add alias in your linux machine:
# ~/.bash_aliases
alias spython='sudo $(printenv VIRTUAL_ENV)/bin/python3'
NOTE: make sure you have virtual env activated.
Run python script with spython command :)
spython file.py

Virtualenv uses wrong python, even though it is first in $PATH

I had a problem where python was not finding modules installed by pip while in the virtualenv.
I have narrowed it down, and found that when I call python when my virtualenv in activated, it still reaches out to /usr/bin/python instead of /home/liam/dev/.virtualenvs/noots/bin/python.
When I use which python in the virtualenv I get:
/home/liam/dev/.virtualenvs/noots/bin/python
When I look up my $PATH variable in the virtualenv I get:
bash: /home/liam/dev/.virtualenvs/noots/bin:/home/liam/bin:/home/liam/.local/bin:/home/liam/bin:/home/liam/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin: No such file or directory
and yet when I actually run python it goes to /usr/bin/python
To make things more confusing to me, if I run python3.5 it grabs python3.5 from the correct directory (i.e. /home/liam/dev/.virtualenvs/noots/bin/python3.5)
I have not touched /home/liam/dev/.virtualenvs/noots/bin/ in anyway. python and python3.5 are still both linked to python3 in that directory. Traversing to /home/liam/dev/.virtualenvs/noots/bin/ and running ./python, ./python3 or ./python3.5 all work normally.
I am using virtualenvwrapper if that makes a difference, however the problem seemed to occur recently, long after install virtualenv and virtualenvwrapper

My problem was that i recently moved my project with virtualenv to another location, due to this activate script had wrong VIRTUAL_ENV path.
$ cat path_to_your_env/bin/activate
... # some declarations
VIRTUAL_ENV="/path_to_your_env/bin/python" # <-- THIS LINE
export VIRTUAL_ENV
... # some declarations
To fix this, just update VIRTUAL_ENV in activate script.
Also you maybe need to fix first line of your bin/pip to link to real python path.

As tdelaney suggested in the comments, I ran alias and found that I had previously aliased python to /usr/bin/python3.5 in my .bashrc.
I removed that alias from my .bashrc, ran unalias python, and source ~/.bashrc and the problem was solved.

If you don't get the program that which says you should get, you need to look higher up the chain than the platform executor. Shells typically have a way to alias commands and on most unixy shells you can just enter alias to see which commands have been remapped. Then its just a matter of going to the config files for your shell and removing the alias.
Sometimes people alias python to try to sort out which python they should be using. But there are usually other, better ways. On my linux machine, for example, python3 is in the path but is a symlink to the real python I am using.
td#mintyfresh ~ $ which python3
/usr/bin/python3
td#mintyfresh ~ $ ls -l /usr/bin/python3
lrwxrwxrwx 1 root root 9 Feb 17 2016 /usr/bin/python3 -> python3.4
td#mintyfresh ~ $
This is nice because non-shell programs running python get the same one I do and virtual environments work naturally.

On Cygwin, I still have a problem even after I created symlink to point /usr/bin/python to F:\Python27\python.exe. Here, after source env/Scripts/activate, which python is still /usr/bin/python.
After a long time, I figured out a solution. Instead of using virtualenv env, you have to use virtualenv -p F:\Python27\python.exe env even though you have created a symlink.

I'm currently having the same problem. Virtualenv was created in Windows, now I'm trying to run it from WSL.
In virtualenv I renamed python.exe to python3.exe(as I have only python3 command in WSL). In $PATH my virtualenv folder is first, there is no alias for python. I receive which python3
/usr/bin/python3. In /usr/bin/python3 there is symlink `python3 -> python3.6. I suppose it doesn't matter for order resolution.

Had the exact same problem.
I ran:
virtualenv -p /venv/bin/python3 env
and got a permission denied.
so i tried:
sudo chmod 777 -R /venv/bin

which python and print(sys.executable)
were not agreeing for me. This meant that with an active virtualenv pip install <package> would install to the virtualenv, but running python would be the base install.
I eventually got around this by running
virtualenv -p \path\to\python.exe --always-copy <venvName>
I'm not sure if specifying the path to the original python is really necessary, but can't hurt. According to the man page:
--copies, --always-copy try to use copies rather than symlinks, even when symlinks are the default for the platform (default: False)
I'm using windows powershell with msys64.

How can I run a python script using pythonbrew venv from the command line?

I recently came across this in a cron script at my place of work:
/bin/bash -c "[[ -s $HOME/.pythonbrew/etc/bashrc ]] && source $HOME/.pythonbrew/etc/bashrc && pythonbrew use 2.6.7 && pythonbrew venv use someapp && python /opt/someapp/bin/someapp.py"
This is for a system-wide (multi-user) installation of Pythonbrew.
It works. But please tell me there's a better way.
Addendum
To clarify what I'm looking for: I'd like a one-line command to run my script though a virtualenv tied to pythonbrew. With virtualenv alone, I could do something like this:
/opt/someapp/venv/bin/python /opt/someapp/bin/someapp.py
What I don't want is another script to run my script (like that cron command above).

I believe it can be done by using the python binary directly from you pythonbrew virtual environment.
By default its in ~/.pythonbrew/venvs/Python-<version>/<name of venv>/bin/python
But I think you can change the path with an environmental variable.
So just change the first half of the line you added to reference the pythonbrew virtual environment python binary and it should work.

On the first line on your python script add a shebang (#!) followed by a path to your target python. Then make the python script executable. It can then be executed directly from the command line (crontab, another bash script, whatever).
make a virtual env in your temp dir:
$ cd /tmp
$ virtualenv venv
the path to your python in that venv is /tmp/venv/bin/python
Using an editor create a simple script containing all of the following:
#!/tmp/venv/bin/python
print("hello world")
Save it in your home directory as "mypyscript.py"
make it executable:
$ chmod 755 mypyscript.py
Now you should be able to execute it using the filename directly on the command line:
$ ./mypyscript.py
hello world
Do this to your someapp.py substituting the relevant path to your python and that should work.

The trick turned out to be locating the pythonbrew virtualenv's python binary. Mark's answer pointed me in the right direction. But here's a complete rundown for future reference:
With pythonbrew installed, I did the following (as root on the server):
pythonbrew install 2.6.6
pythonbrew switch 2.6.6
pythonbrew venv create --no-site-packages myapp
I had a pip freeze file, so I set up my virtualenv using that:
/usr/local/pythonbrew/venvs/Python-2.6.6/myapp/bin/pip install -r /tmp/requirements.pip
Now my python binary can be found at /usr/local/pythonbrew/venvs/Python-2.6.6/myapp/bin/python. So to run my script:
/usr/local/pythonbrew/venvs/Python-2.6.6/myapp/bin/python /opt/myapp/bin/myapp.py