We Keep Coding

Prevent a Python-embedded to look in my default path C:\Python38 for modules

I'm using Cython in --embed mode to produce a .exe. I'm evaluating the Minimal set of files required to distribute an embed-Cython-compiled code and make it work on any machine. To do this, I only copy a minimal number of files from the Python Windows embeddable package.
In order to check this, I need to be sure that the current process I'm testing doesn't in fact use my system default Python install, i.e. C:\Python38.
To do this, I open a new cmd.exe and do set PATH= which temporarily removes everything from the PATH. Then I can test any self-compiled app.exe and make sure it doesn't reuse C:\Python38's files under the hood.
It works, except for the modules. Even after doing set PATH=, my code app.py
import json
print(json.dumps({"a":"b"}))
when Cython---embed-compiled into a .exe works, but it still uses C:\Python38\Lib\json\__init__.py! I know this for sure, because if I temporarily remove this file, my .exe now fails, because it cannot find the json module.
How to completely remove any link to C:\Python38 when debugging a Python program which shouldn't use these files?
Why isn't set PATH= enough? Which other environment variable does it use for modules? I checked all my system variables and I think I don't find any which seems related to Python.

Python has a quite complicated heuristic for finding its "installation" (see for example this SO-question or this description), so probably it doesn't find the installation you are providing but the "default" installation.
Probably the most simple way is to set the environment variable PYTHONPATH pointing to the desired installation prior to start of the embedded interpreter.
By examination of sys.path one can check whether the correct installation was found.

Thanks to #ead's answer and his link getpath.c finally redirecting to getpathp.c in the case of Windows, we can learn that the rule for building the path for module etc. is:
current directory first
PYTHONPATH env. variable
registry key HKEY_LOCAL_MACHINE\SOFTWARE\Python or the same in HKCU
PYTHONHOME env. variable
finally:
Iff - we can not locate the Python Home, have not had a PYTHONPATH
specified, and can't locate any Registry entries (ie, we have nothing
we can assume is a good path), a default path with relative entries is
used (eg. .\Lib;.\DLLs, etc)
Conclusion: in order to debug an embedded version of Python, without interfering with the default system install (C:\Python38 in my case), I finally solved it by temporarily renaming the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Python to HKEY_LOCAL_MACHINE\SOFTWARE\PythonOld.
Side note: I'm not sure I will ever revert this registry key back to normal: my normal Python install shouldn't need it anyway to find its path, since when I run python.exe from anywhere (it is in the PATH for everyday use), it will automatically look in .\Lib\ and .\DLL\ which is correct. I don't see a single use case in which my normal install python.exe wouldn't find its subdir .\Lib\ or .\DLL\ and requiring the registry for this. In which use case would the registry be necessary? if python.exe is started then its path has been found, and it can take its .\Lib subfolder, without help from registry. I think 99,99% of the time this registry feature is doing more harm than good, preventing a Python install to be really "portable" (i.e. that we can move from one folder to another).
Notes:
To be 100% sure, I also did this in command line, but I don't think it's necessary:
set PATH=
set PYTHONPATH=
set PYTHONHOME=
Might be helpful to do debugging of an embedded Python: import ctypes. If you haven't _ctypes.pyd and libffi-7.dll in your embedded install folder, it should fail. If it doesn't, this means it looks somewhere else (probably in your default system-wide Python install).

Running keras vscode from R but it says "Python was not found but can be installed from the Microsoft Store"

I'm running vscode using keras application on R with the following code (on R console):
library(foreign)
library(dplyr)
library(tidyverse)
library(tidytext)
library(keras)
library(data.table)
options(scipen=999)
dat <- read.csv("https://www.dropbox.com/s/31wmgva0n151dyq/consumers.csv?dl=1")
max_words <- 2000 # Maximum number of words to consider as features
maxlen <- 64 # Text cutoff after n words
# Prepare to tokenize the text
texts <- as.character(dat$consumer_complaint_narrative)
tokenizer <- text_tokenizer(num_words = max_words) %>%
fit_text_tokenizer(texts)
But it says:
Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640Error in python_config(python_version, required_module, python_versions) :
Error 9009 occurred running C:\Users\my_working_directory\AppData\Local\MICROS~1\WINDOW~1\python.exe
It seems to suggest that I have not installed python on my device, but I actually did because I ran similar keras Python code on my jupyter notebook without problem, and I just want to try doing this in R.
I have found others asked similar question before, but I could not figure on top of my mind what is the problem, at least for my case. It will be really appreciated if someone could help me on this.

Have you checked that Python is in the default PATH?
From the docs:
3.6. Configuring Python
To run Python conveniently from a command prompt, you might consider changing some default environment variables in Windows. While the installer provides an option to configure the PATH and PATHEXT variables for you, this is only reliable for a single, system-wide installation. If you regularly use multiple versions of Python, consider using the Python Launcher for Windows.
3.6.1. Excursus: Setting environment variables Windows allows environment variables to be configured permanently at both the User
level and the System level, or temporarily in a command prompt.
To temporarily set environment variables, open Command Prompt and use
the set command:
C:\>set PATH=C:\Program Files\Python 3.8;%PATH%
C:\>set PYTHONPATH=%PYTHONPATH%;
C:\My_python_lib
C:\>python
These changes will
apply to any further commands executed in that console, and will be
inherited by any applications started from the console.
Including the variable name within percent signs will expand to the
existing value, allowing you to add your new value at either the start
or the end. Modifying PATH by adding the directory containing
python.exe to the start is a common way to ensure the correct version
of Python is launched.
To permanently modify the default environment variables, click Start
and search for ‘edit environment variables’, or open System
properties, Advanced system settings and click the Environment
Variables button. In this dialog, you can add or modify User and
System variables. To change System variables, you need non-restricted
access to your machine (i.e. Administrator rights).
Note Windows will concatenate User variables after System variables,
which may cause unexpected results when modifying PATH. The PYTHONPATH
variable is used by all versions of Python 2 and Python 3, so you
should not permanently configure this variable unless it only includes
code that is compatible with all of your installed Python versions.
The path specified in the code snippet C:\Program Files\Python 3.8 must be adapted to reflect where your Python is actually located.

CMD opens Windows Store when I type 'python'

Today when I tried to run simple code on Sublime Text 3, the following message appeared:
Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640
And when I type Python in CMD, it opens the Windows Store for me to download Python 3.7. This problem started today for no good reason. I didn't change or download anything about Python and already tried reinstalling Python, and the Path environment variable is correct.

Use the Windows search bar to find "Manage app execution aliases". There should be two aliases for Python. Unselect them, and this will allow the usual Python aliases "python" and "python3". See the image below.
I think we have this problem when installing Python because in a new Windows installation the aliases are in the ON position as in image below. When turned on, Windows puts an empty or fake file named python.exe and python3.exe in the directory named %USERPROFILE%\AppData\Local\Microsoft\WindowsApps. This is the alias.
Then Microsoft put that directory at the top of the list in the "Path" environment variables.
When you enter "python" in cmd, it searches the directories listed in your "Path" environment variables page from top to bottom. So if you installed Python after a new Windows 10 install then get redirected to the Windows Store, it's because there are two python.exe's: The alias in the App Execution Alias page, and the real one wherever you installed Python. But cmd finds the App execution, alias python.exe, first because that directory is at the top of the Path.
I think the easiest solution is to just check the python.exe and python3.exe to OFF as I suggested before, which deletes the fake EXE file files. Based on this Microsoft Devblog, they stated they created this system partially for new Python users, specifically kids learning Python in school that had trouble installing it.
Creating this alias was to help kids just starting Python to install it and focus on learning to code. I think Windows probably deletes those aliases if you install Python from the Windows App Store. We are noticing that they do not get deleted if you manually install from another source.
(Also, the empty/fake python.exe is not really empty. It says 0 KB in the screenshot, but entering "start ms-windows-store:" in cmd opens the Windows App Store, so it probably just has a line with that and a way to direct it to the Python page.)
Finally, as Chipjust suggested, you can create a new alias for Python using something like DOSKEY as explained in this article for example:
How to set aliases for the command prompt in Windows

The main problem here is that the order in the path calls the windows from top to bottom, and that there is python.exe in %USERPROFILE%\AppData\Local\Microsoft\WindowsApps which is called first if there are no other python.exes in the PATH above that line.
To ensure that the correct python.exe is called, add the Python interpreter installation folder (containing python.exe) to the PATH, above %USERPROFILE%\AppData\Local\Microsoft\WindowsApps
Here is an example:
To get to this location, click "Start" → start typing "Env" → Select "Edit the system environment variables" → "Environment variables" button → Select the entry for "Path" in the upper list → Click "Edit".
Python components should be at the top, as in step 5. If not, move them up by pressing the button in step 6.

If the Python interpreter is already installed, then go to Apps & features from settings, select Python, and then select modify.
Again select modify and select Next:
Then this window will appear:
Select "add Python to environment variable" and click on the install button. Then again go to apps & features, click modify and click Repair.
Now go to CMD and type Python.
Problem solved.

This is a PowerShell script that does the magic.
Remove-Item $env:USERPROFILE\AppData\Local\Microsoft\WindowsApps\python*.exe

I had problems with this as well, where Windows didn't recognize Python or Anaconda in a double click or cmd (command) prompt.
Problem: unable to import libraries in "python" cmd in Windows. Instead the Windows "python" cmd took users somewhere they don't want to go.
Problem Cause: In Windows "Environmental Variables", Windows adds a python.exe and python3.exe (I don't know where these link to) in the "%USERPROFILE%\AppData\Local\Microsoft\WindowsApps" directory.
Solution: I tried deleting the python*.exe files in the WindowsApp directory, but Windows wouldn't allow it, so I opened a command prompt in the "%USERPROFILE%\AppData\Local\Microsoft\WindowsApps" directory and typed:
del python.exe
del python3.exe
Then I created an environment variable linking to the installed Python interpreter link. In my case, it was C:\Anaconda3; C:\Anaconda3\Scripts, and some others for good measure.

Because this is a common issue and this appears to be the canonical question, I want to try to give a complete overview of The Python 3.7 Windows Store Fiasco (TM).
Why is it possible for this to happen?
A convergence of two things: the previous introduction of the Python Launcher for Windows in 2011 (hereafter py), and a Windows 10 update in May 2019 that was apparently intended to make installing Python easier for Windows users.
Oops. Turns out that installation path isn't great; it bypasses the "lengthy" setup wizard... which contains some options that some users find very useful. It caused other issues, too. Not to mention that it just works in slightly non-standard ways, has limitations on file system access because it's a Store app, initially couldn't itself be launched by py....
Okay, but why do those factors result in the problem?
Since the introduction of py, by default, Windows Python installers do not add the new Python install to the PATH. Why? Because the entire point of py is that it uses its own logic to find a Python installation, based on some combination of command-line switches and possibly the source file's own shebang line. Now your source files can be associated with py instead of any particular python.exe, and you can get Linux-like behaviour when double-clicking a file. Meanwhile, by running py at the command line, you have easy access to whatever you need, and you don't have to think about which version of Python was installed most recently. So there's seemingly no good reason to put any of those Python installations on the PATH. It only risks confusing you when, for example, the most recently installed version isn't the most up-to-date one. Right?
In the update, Windows 10 put a "python.exe" into a Windows Apps-related folder, which is a wrapper app to open a Microsoft Store link. The idea is that it's on the PATH, but way near the end; so if you have an installed Python, it gets used, and otherwise the wrapper is invoked and helpfully prompts you to install Python - so that you can actually run that random, totally trusted .py file your friend sent you on Discord.
And it would work perfectly, if your installed Python were on the PATH.
Oops.
(But, you know, py had been introduced around 8 years prior. You'd think someone at Microsoft would have been aware of the potential issue. Maybe instead of a special shortcut link, they could have made an actual script that checks for the presence of C:\Windows\py.exe or something.)
So what are my options?
You can check the option to add new Python versions to the PATH when you install them, and deal with the fact that python at the command line means a specific one of them. If you need to change that, you can manually tweak your PATH variable.
You can just manually tweak the PATH variable after the fact. (or "Modify" a Python installation to fix it.) This is covered in several other answers.
Independently of that, you can disable the wrappers, as shown in the top answer. You should probably do this anyway; seeing python fail at the command line is less aggravating than dealing with a random GUI window popping up and offering to install something for you, especially when you know you have it already.
If you want to keep the PATH empty, consider using virtual environments for your projects. Whenever a virtual environment is active, the PATH is temporarily modified such that python means the Python installation of that environment. It's quite convenient, really.
You might be able to tell your IDEs to use py instead of a specific Python installation, and it might even be helpful to do so. I don't know. I don't use one.

As a person who does Python development in Sublime Text, I know you said the Python interpreter path was correct, but when you install the Python interpreter make sure to tick the option to add Python to PATH.
I had the same issue back in the day till I did this.

You can manually add Python to the Windows path by doing this:
Start the Run box and enter sysdm.cpl
Go to the Advanced tab and click the Environment Variables button
Now you’ll need to locate the relevant Python paths
Here is how a Python application path looks like:
And this is how a Python Scripts path looks like:
Select the Path variable, press edit and add both paths (Python application and Python Scripts)
If it doesn't show, press new instead of edit and fill the New User Variable box
This is how my Variable value looks like:
C:\Users\Ron\AppData\Local\Programs\Python\Python37-32;C:\Users\Ron\AppData\Local\Programs\Python\Python37-32\Scripts
That’s it! You just added Python to the Windows Path.
Source: Data To Fish

So, I had the same problem.
My answer was to add python to PATH not only for User variables, it was there, but also for System variables. And now everything works.
So, I've got the same problem in VENV.
I solved it by typing >> python**.exe**
But to add every time .exe a little bit nervous.
Also try to type python.exe in cmd.

This is an additional note for anyone using pyenv-win: after turning off the application execution aliases, run pyenv rehash. You may also need to close/reopen your CMD or PowerShell window.

I have put together a powershell snippet, which reorders WindowsApps and Python folder so that Python is first. Run this as Admin:
$appsFld="$env:USERPROFILE\AppData\Local\Microsoft\WindowsApps";
$pyPath=(Resolve-Path "$env:USERPROFILE\AppData\Local\Programs\Python\Python*\")
$Env:Path = (($Env:Path.Split(';') | Where-Object { $_ -ne "$appsFld" }) -join ';');
$Env:Path = (($Env:Path.Split(';') | Where-Object { $_ -ne "$pyPath" }) -join ';');
$Env:Path += ";$pyPath";
$Env:Path +=";$appsFld";
[Environment]::SetEnvironmentVariable("PATH", "$Env:Path", "Machine")
Then I can run python just fine:
> python
Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep 5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> quit

Adding a bit to the question. Even when I typed pip freeze, it wasn't showing anything.
Here's what I did:
There were multiple instances for the Python application in /AppData/Local/Microsoft/WindowApps.
I deleted those and then it worked.

PyCharm cannot find library

I am using PyCharm 5 to run a Python 2.7 (Anaconda) script in Ubuntu. My script imports a module with import tensorflow, but this causes the error ImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory. So, it seems that the library libcudart.so.7.0 is needed by this module, but it cannot be found.
Now, I have seen that this library is on my machine in /usr/local/cuda-7.0/targets/x86_64-linux/lib. So, in PyCharm, I went to Settings->Project Interpreters->Interpreter Paths. This had a list of paths, such as /home/karnivaurus/Libraries/Anaconda/python2.7. I then added to this list, the path mentioned above which contains the required library.
However, this did not fix the problem. I still get an error telling me that libcudart.so.7.0 cannot be found. If I run my script from the shell though (python myfile.py), then it runs fine.
How can I tell PyCharm where to find this library?
I have noticed that if I have print sys.path in my script, the paths it prints out are entirely different to those in Settings->Project Interpreters->Interpreter Paths... should they be the same?

I came across this problem just recently using a remote debugger, however I believe it's still the same solution. I just added the following to the Environment Variables section in the Run/Debug Configuration options found in Run > Edit Configurations... dialog: LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

The path to your cuda library seems strange to me. I would expect it to be /usr/local/cuda-7.0/lib64 or /usr/local/cuda-7.0/lib.
Did you follow all of the cuda installation procedure?
If you type env on the command line, do you see a path to cuda in your LD_LIBRARY_PATH?
Update from comments below:
The issue is that PyCharm was invoked from the desktop, and wasn't getting the right environment variables. Solution is to either:
invoke from the command line,
create a script to set environment and then invoke, and make a link to that script on the desktop,
or set environment variables on the desktop item

While some of these answers are correct, and could work, I haven't seen what the OP specifically asked for, and that is where to set environments for the python console. This can be accomplished inside pycharm at:
File > Settings > Build,Execution,Deployment > Console > Python Console
In the options there, you'll find a place to define Environment Variables. Set LD_LIBRARY_PATH there.

Edit your pycharm.desktop, specify the environment variable in exec, like below:
[Desktop Entry]
Version=1.0
Type=Application
Name=Pycharm
Exec=env LD_LIBRARY_PATH=:/usr/local/cuda/lib64:/usr/local/cuda/lib64 /home/cwh/software/pycharm-2016.1.4/bin/pycharm.sh
Icon=/home/cwh/software/pycharm-2016.1.4/bin/pycharm.png
Name[zh_CN]=Pycharm
so pycharm will find cuda

Have you selected the right python interpreter in your project's settings?
See here.
I had a similar issue and changing the interpreter solved it without having to create a new icon.

The following works for me on Community edition 2019.3
To set globally for a project:
Open File/Settings/Project/Project Interpreter
click on the cog icon next to the interpreter
choose show all
click on the little folder with tree icon bottom right
add the path to "Interpreter Paths"

importing matplotlib in eclipse

I'm running pydev in Ecplise (Python 2.7). Numpy and tkinter work fine, but I installed matplotlib and for some reason when I try
from matplotlib import *
I get the response it's not found. I added the folder with the module to PythonPath but keep getting same message. What am I doing wrong?

from pydev tutorial (in my experience, the two most common cases related with your problem):
1)
The most common error is having a problem in the environment variables
used from the shell that spawned Eclipse, in a way that for some
reason when getting the variables of one interpreter, it gathers the
info from another interpreter (thus mixing the interpreter and the
actual libraries).
2)
If you add something to your python installation, you need to either
add it manually as a 'new folder' in the System PYTHONPATH (if it's
still not under a folder in the PYTHONPATH) or (recommended) remove
your interpreter and add it again, then, press apply.