Why is %PYTHONPATH% being ignored by Python using cmd on Windows? - python

I developed a Python package on Linux, which works and my pytest succeeds.
I am making it Windows compatible.
I have say 'my_package' that sits in say 'C:\import_path\my_package'
I set the PYTHONPATH at the command line in my virtual environment like so:
set PYTHONPATH="C:\import_path"
echo %PYTHONPATH%
"C:\import_path"
# Python script identifies this
print('PYTHONPATH = {}'.format(os.environ.get('PYTHONPATH')))
"C:\import_path"
However any Python code referencing my_package when run from a Command Prompt where this PYTHONPATH has been explicitly set, will error:
ModuleNotFoundError: No module named 'my_package'
However the code will run fine with PYTHONPATH = C:\import_path being set in the System Environment variables.
It also runs fine in VSCode with:
"python.analysis.extraPaths": [
"C:\\import_path"
]
Why is python.exe not reading the %PYTHONPATH% variable that was set in the cmd, but python.exe will read it when running in debug from VSCode or from the os system env variable?

In all probability you are running from different shell session or calling different shell sessions and the variable is not parsed to all sessions. You could try setting it like this:
setx PYTHONPATH "C:\import_path" /M
When you set a variable you should set it system-wide if you intend to use multiple sessions.

Related

Auto activate virtual environment in Visual Studio Code

I want VS Code to turn venv on run, but I can't find how to do that.
I already tried to add to settings.json this line:
"terminal.integrated.shellArgs.windows": ["source${workspaceFolder}\env\Scripts\activate"]
But, it throws me an 127 error code. I found what 127 code means. It means, Not found. But how it can be not found, if I see my venv folder in my eyes right now?
I think it's terminal fault. I'm using Win 10 with Git Bash terminal, that comes when you install Git to your machine.
This is how I did it in 2021:
Enter Ctrl+Shift+P in your vs code.
Locate your Virtual Environment:
Python: select interpreter > Enter interpreter path > Find
Once you locate your virtual env select your python version:
your-virtual-env > bin > python3.
Now in your project you will see .vscode directory created open settings.json inside of it and add:
"python.terminal.activateEnvironment": true
don't forget to add comma before to separate it with already present key value pair.
Now restart the terminal.
You should see your virtual environment activated automatically.
Actually the earlier suggested solutions didn't work for me, instead I added the following in my settings:
"settings": {
"python.terminal.activateEnvInCurrentTerminal": true,
"python.defaultInterpreterPath": "~/venv/bin/python"
}
Of course replace the defaultInterpreterPath (used to be pythonPath) setting with your own path (so don't copy/paste the second line).
You don't need this line at all. Just remove it and switch your Python interpreter to point to the one within your venv. Here's a relevant documentation (italicized emphasis mine):
To select a specific environment, use the Python: Select Interpreter
command from the Command Palette (Ctrl+Shift+P).
... and
opening a terminal with the Terminal: Create New Integrated Terminal
command. In the latter case, VS Code automatically activated the
selected environment.
Once you switch the interpreter VS code should create a .vscode folder within your workspace with a settings.json indicating the python interpreter. This will give VS code the direction of where to locate the venv.
There is a new flag that one can use: "python.terminal.activateEnvironment": true
my scenario was pretty much the same. I am running VSCode on Windows, wanting to used git bash as my default Terminal but after the venv got created, it was doing some weird stuff when a Terminal would open where it couldn't find the correct python interpeter in the venv/Scripts folder even though I did ctrl-shift-p a bunch of times to reset it to the python.exe there. I also wanted to make sure the activate script was run on Terminal open. I just couldn't get the debugger to work right and it kept complaining that it could not find the python interpreter, so I basically couldn't debug at all.
So for anyone who is having weird stuff happen trying to use a Git Bash Terminal in VSCode in Windows related to a python project using a virtual env, here is what I found out;
I noticed that when I opened a new Git Bash Terminal, and looked at the $PATH variable to make sure it could find the interpreter in the venv, the path to the venv/Scripts folder would be prepended to the $PATH, but not with linux path separators like everything else in $PATH but with a Windows style path;
echo $PATH
C:\Users\blah\Documents\blah\Stock-down\Dev\this_api\venv/Scripts:/c/Users/blah/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/blah/bin:/c/Program Files/Go/bin:/c/Python39/Scripts:
So whenever I would try to debug, or just even run which python, It would get confused and thought the interpreter was in here;
which python
C:\Users\blah\Documents\blah\Stock-down\Dev\this_api\venv/Scripts/C/Users/blah/Documents/blah/Stock-down/Dev/this_api/venv/Scripts/python.exe
I ran the "printenv" command in the Terminal to see what env vars were getting set and why the venv interpreter path was getting messed up. And I found a env var I didn't know existed - VIRTUAL_ENV. But I didn't know how it was getting set or where it came from. After some pain, and hunting around I found it - when you run "python -m venv venvname" to create the virtual env in the project folder, as you know, it creates the activate (and activate.bat for windows) scripts in the ./venv/Scripts folder. Inside these files this VIRTUAL_ENV variable is not only exported, but but prepended to the $PATH variable on Terminal open with the "/Scripts" folder name added in linux path style. The problem with is that it sets the VIRTUAL_ENV value with windows type path - I know its painful and wrong to do this, but I just changed it to what Git Bash is expecting, see below;
#unset irrelevant variables
deactivate nondestructive
#VIRTUAL_ENV="C:\Users\blah\Documents\blah\Stock-down\Dev\this_api\venv"
VIRTUAL_ENV="/c/Users/blah/Documents/blah/Stock-down/Dev/st_api/venv"
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/Scripts:$PATH"
export PATH
Now when a new Git Bash Terminal is opened it prepends the venv/Scripts path correctly to $PATH;
echo $PATH
/c/Users/blah/Documents/blah/Stock-down/Dev/st_api/venv/Scripts:/c/Users/blah/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/blah/bin:/c/Program Files/Go/bin:/c/Python39/Scripts:
I also made a copy of the executable python.exe in the ./venv/Scripts dir and just called it "python" and now the command "which python" works and I can debug fine. I still set ctrl-shift-P when I choose the interpreter in VSCODE to "python.exe" when I just use "find" during the selection.
For more information on VIRTUAL_ENV var, see this doc -- python venv docs
It feels like jumping through a bunch of hoops I know, but this way I can open a new Git Bash Terminal, have the activate script run correctly on Terminal Open, debug, and operate normally without having to float between Git Bash AND WSL Ubuntu AND Powershell, etc.
My environment started to activate automatically after the advice from this article
When you create a new virtual environment, a prompt will be displayed
to allow you to select it for the workspace. This will add the path to
the Python interpreter from the new virtual environment to your
workspace settings. That environment will then be used when installing
packages and running code through the Python extension. For examples
of using virtual environment in projects, see the Python, Django, and
Flash tutorials.
My solution was to create /.vscode/settings.json manually
Here is the tree;
├───django
│ ├───.vscode
│ │ ├───settings.json
I created /.vscode/settings.json then added this code in settings.json (I am using windows so I used double backslash for path location otherwise it gives unicode error, and don't copy paste this, find your own .virtualenvs)
{
"python.defaultInterpreterPath": "C:\\Users\\Talha\\.virtualenvs\\django-okd21pq9\\Scripts\\python.exe"
}

VS Code integrated terminal not starting with virtual environment activated while the right interpreter is being selected

Platform and software versions:
Mac OS Mojave, VS Code 1.38.0, Python extension installed. Created
virtual environment in project directory using command
python3 -m env.
Modified setting in Python extension,
"python.venvPath": "bin", to handle the bin directory where the python for the virtual
environment is stored.
Situation:
When I launch VS Code using code ., and then open a python file in the folder, the interpreter selected is ./bin/python, however the integrated terminal is not set to the right python executable. If I launch a new terminal it sources the virtual environment (which may be due to the Python extension setting "python.terminal.activateEnvironment": true)
Question:
Is there a way to have the integrated terminal also have the virtual environment sourced?
Or is there a better way to have VS Code activate virtual environment created by python3 -m env .?
Thank you.
Edit:
Just reread the VS Code documentation here - https://code.visualstudio.com/docs/python/environments and this time noticed this below. Wondering if there is a way to kill the existing terminal and then launch one upon VS Code launch...
However, launching VS Code from a shell in which a certain Python environment is activated does not automatically activate that environment in the default integrated terminal. Use the Terminal: Create New Integrated Terminal command after VS Code is running.
Two things. One, "python.venvPath" is meant to point at a directory that contains other virtual environments, not the bin/ directory that has a Python interpreter from a virtual environment. (I also don't know what python3 -m env is supposed to do; did you mean python3 -m venv?)
Two, there isn't a way to make VS Code automatically launch and complete the loading of the Python extension before VS Code creates a terminal if you have the terminal frame open at start-up.
At the time of this writting vscode now has
"python.terminal.activateEnvInCurrentTerminal": true,
so I my global settings.json, F1 > preference: Open Settings (JSON)
"python.venvPath": "D:/miniconda3/envs",
"python.terminal.activateEnvInCurrentTerminal": true,
and in my workspace's settings.json, F1 > preference: Open Workspace Settings (JSON)
"python.defaultInterpreterPath": "D:/miniconda3/envs/my-workspace-venv/python.exe"
And it works for me.

PyCharm: activate venv when running a script

I am new in PyCharm, I use the 2019 CE version.
I have set an environment variable in venv/bin/activate:
# User defined variables:
export DATABASE="postgresql://postgres:postgres#localhost:5432/postgres"
And I can access it using bash:
$ source venv/bin/activate
(venv) $ echo $DATABASE
postgresql://postgres:postgres#localhost:5432/postgres
And when running my script in this environment, it works as expected:
(venv) $ python3 newproject/settings.py
{'database': 'postgresql://postgres:postgres#localhost:5432/postgres'}
It is also automatically done for PyCharm Terminal that natively activates the environment, but I could not succeed in setting it when I run a script.
Anyway, when I specify a Run Configuration to execute a script that needs to access this environment variable, the environment is not loaded automatically before running the script. I have searched almost everywhere, the logical place should be there:
I have found no option (or documentation) to enable virtualenv before running a script. I even tried to add an external tool to manually activate it before it runs the script. But it is useless since executed environments are disjoint: the script does not run in the same environment that venv.
Looking at the python interpreter it seems it is the correct environment.
I know that I can set Environment Variable in this peculiar setup form, but then variables are no longer available when I run the virtual environment, so it is not a solution to me.
My question is then: How can I setup a Run Configuration that activate my environment and run the script within this environment?

When I use Pycharm run a python file(in fact it is my first use), always report error, for example: [duplicate]

I need to debug an py-script in PyCharm that makes use of unix binaries like "which, grep" and so on.
If I start the py-script from Terminal (bash), unix binaries are found and all works like expected.
If I start the script from PyCharm "Debug" or "Run", there seems to be no "PATH" set => the unix binaries were not found.
Am I missing sth.?
Add the PATH environment variable to your Run Configuration (Run->Edit Configurations…) like this: /usr/local/bin:$PATH
Create a new environment variable in your run configuration named PATH, and set it equal to the output of running echo $PATH on your command line. This will let you get around the issue.
I have had to manually symlink binaries that are in your PATH, but not in /usr/bin/. For example, scripts running 'ffmpeg' in pycharm will not see /usr/local/bin/ffmpeg but after symlinking will be able to see /use/bin/ffmpeg. This is a python subprocess issue (they don't consult your .bashrc as bash does).

Environment $PATH different when using venv

I'm using PyCharm on a mac (OSX mavericks) to run a simple script shown below. All it does is print the PATH variable. I have a virtualenv in the project directory. I added a Run Configuration in PyCharm and tried it with different Pythons:
# file mytest.py
import os
print "PATH: ", os.environ['PATH']
When I run with the system default python (/usr/bin/python) it prints the correct value for PATH (i.e. the PATH as I have configured in my .bash_profile file,) which is kind of long and contains many directories.
But when I choose the venv's Python, the path is reduced to only: /usr/bin:/bin:/usr/sbin:/sbin:/Users/myname/projects/myproj/venv/bin
This doesn't happen if I run the script from a terminal window. In this case it shows the correct PATH for both the system's python and the venv python. It also doesn't happen if I deactivate the venv and run venv/bin/python mytest.py.
Anyone knows how to make the correct PATH value be set when running from PyCharm and using venv?
you should probably know that all environment variables are inherited.
When you define environment variable in your .bash_profile it becomes available in your terminal (bash),
and in all processes that will be started from terminal (These processes will be children for the bash process).
That's why you are getting expected values when running your script from within the terminal.
You start PyCharm not from a terminal, so it doesn't inherit PATH. And so do Python or venv (they launched from PyCharm).
To solve your issue you have 3 options here: just start PyCharm from terminal or move PATH variable definition from .bash_profile to session init scripts (PATH will be defined system-wide) or Duplicate your PATH in PyCharm's run configuration (it has such option over there)
Good luck!
I built something for SublimeLint that figures out your shell's PATH in Python.
https://github.com/lunixbochs/sublimelint/blob/st3/lint/util.py#L57
Basically, it runs $SHELL --login, echoes your path, and parses it.
The gist is:
import os
import subprocess
shell = os.path.basename(os.environ['SHELL'])
output = subprocess.Popen(
(shell, '--login', '-c', 'echo __SEP__$PATH'),
stdout=subprocess.PIPE).communicate()[0] or ''
print output.split('__SEP__', 1)[1].split(':')
The problem for me on windows was case related: it's the difference between Path and PATH.
If you choose:
Edit configurations
Environment Variables
Show (down in the bottom right corner)
This shows the environment variables. On my system there is no PATH, only Path.
Defining a new PATH environment variable wouldn't work, but Path worked fine.
There's a workaround in my case.
To modify the environment variable PATH, I usually just need to add extra paths to the PYTHONPATH variable in the run configuration's environment variables setting.
PYTHONPATH will then be added to PATH.

Categories