I need to debug an py-script in PyCharm that makes use of unix binaries like "which, grep" and so on.
If I start the py-script from Terminal (bash), unix binaries are found and all works like expected.
If I start the script from PyCharm "Debug" or "Run", there seems to be no "PATH" set => the unix binaries were not found.
Am I missing sth.?
Add the PATH environment variable to your Run Configuration (Run->Edit Configurations…) like this: /usr/local/bin:$PATH
Create a new environment variable in your run configuration named PATH, and set it equal to the output of running echo $PATH on your command line. This will let you get around the issue.
I have had to manually symlink binaries that are in your PATH, but not in /usr/bin/. For example, scripts running 'ffmpeg' in pycharm will not see /usr/local/bin/ffmpeg but after symlinking will be able to see /use/bin/ffmpeg. This is a python subprocess issue (they don't consult your .bashrc as bash does).
I am having an issue with implementing git hooks with the Github Desktop client for windows. When I created the hook, it runs a python script that checks the commit message. The weird part is, it runs on perfectly fine on Windows with command prompt, powershell, tortoisegit, webstorm IDE, git bash, also works on Linux terminal, EXCEPT for Github Windows client. reading the debug log, i get this error
GitHub.IO.ProcessException: /usr/bin/env: python: No such file or directory
My already attempted solutions:
adding 'python' as system variable
using ruby
removing '/usr/bin/env' (this will throw an error saying this file doesn't exists)
adding a .bashrc file solution i found online by putting the python path in my home directory
Does anyone have any idea how to fix this?
This error means that Git cannot find
To fix that, append C:\Python (or wherever you installed python) to your PATH environment variable in windows (control panel > System).
Once you set it up test the hook again.
Another option is to create a .bashrc file in your %USERPROFILE% directory and set the variable there:
export PATH=/c/Python:$PATH
Again - check to see that it fixed the problem.
Whenever I use sys.path.append, the new directory will be added. However, once I close python, the list will revert to the previous (default?) values. How do I permanently add a directory to PYTHONPATH?
If you're using bash (on a Mac or GNU/Linux distro), add this to your ~/.bashrc
export PYTHONPATH="${PYTHONPATH}:/my/other/path"
You need to add your new directory to the environment variable PYTHONPATH, separated by a colon from previous contents thereof. In any form of Unix, you can do that in a startup script appropriate to whatever shell you're using (.profile or whatever, depending on your favorite shell) with a command which, again, depends on the shell in question; in Windows, you can do it through the system GUI for the purpose.
superuser.com may be a better place to ask further, i.e. for more details if you need specifics about how to enrich an environment variable in your chosen platform and shell, since it's not really a programming question per se.
Instead of manipulating PYTHONPATH you can also create a path configuration file. First find out in which directory Python searches for this information:
python -m site --user-site
For some reason this doesn't seem to work in Python 2.7. There you can use:
python -c 'import site; site._script()' --user-site
Then create a .pth file in that directory containing the path you want to add (create the directory if it doesn't exist).
For example:
# find directory
SITEDIR=$(python -m site --user-site)
# create if it doesn't exist
mkdir -p "$SITEDIR"
# create new .pth file with our path
echo "$HOME/foo/bar" > "$SITEDIR/somelib.pth"
This works on Windows
On Windows, with Python 2.7 go to the Python setup folder.
Open Lib/site-packages.
Add an example.pth empty file to this folder.
Add the required path to the file, one per each line.
Then you'll be able to see all modules within those paths from your scripts.
In case anyone is still confused - if you are on a Mac, do the following:
Open up Terminal
Type open .bash_profile
In the text file that pops up, add this line at the end:
export PYTHONPATH=$PYTHONPATH:foo/bar
Save the file, restart the Terminal, and you're done
You could add the path via your pythonrc file, which defaults to ~/.pythonrc on linux. ie.
import sys
sys.path.append('/path/to/dir')
You could also set the PYTHONPATH environment variable, in a global rc file, such ~/.profile on mac or linux, or via Control Panel -> System -> Advanced tab -> Environment Variables on windows.
To give a bit more explanation, Python will automatically construct its search paths (as mentioned above and here) using the site.py script (typically located in sys.prefix + lib/python<version>/site-packages as well as lib/site-python). One can obtain the value of sys.prefix:
python -c 'import sys; print(sys.prefix)'
The site.py script then adds a number of directories, dependent upon the platform, such as /usr/{lib,share}/python<version>/dist-packages, /usr/local/lib/python<version>/dist-packages to the search path and also searches these paths for <package>.pth config files which contain specific additional search paths. For example easy-install maintains its collection of installed packages which are added to a system specific file e.g on Ubuntu it's /usr/local/lib/python2.7/dist-packages/easy-install.pth. On a typical system there are a bunch of these .pth files around which can explain some unexpected paths in sys.path:
python -c 'import sys; print(sys.path)'
So one can create a .pth file and put in any of these directories (including the sitedir as mentioned above). This seems to be the way most packages get added to the sys.path as opposed to using the PYTHONPATH.
Note: On OSX there's a special additional search path added by site.py for 'framework builds' (but seems to work for normal command line use of python): /Library/Python/<version>/site-packages (e.g. for Python2.7: /Library/Python/2.7/site-packages/) which is where 3rd party packages are supposed to be installed (see the README in that dir). So one can add a path configuration file in there containing additional search paths e.g. create a file called /Library/Python/2.7/site-packages/pip-usr-local.pth which contains /usr/local/lib/python2.7/site-packages/ and then the system python will add that search path.
On MacOS, Instead of giving path to a specific library. Giving full path to the root project folder in
~/.bash_profile
made my day, for example:
export PYTHONPATH="${PYTHONPATH}:/Users/<myuser>/project_root_folder_path"
after this do:
source ~/.bash_profile
On linux you can create a symbolic link from your package to a directory of the PYTHONPATH without having to deal with the environment variables. Something like:
ln -s /your/path /usr/lib/pymodules/python2.7/
For me it worked when I changed the .bash_profile file. Just changing .bashrc file worked only till I restarted the shell.
For python 2.7 it should look like:
export PYTHONPATH="$PYTHONPATH:/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python"
at the end of the .bash_profile file.
Adding export PYTHONPATH="${PYTHONPATH}:/my/other/path" to the ~/.bashrc might not work if PYTHONPATH does not currently exist (because of the :).
export PYTHONPATH="/my/other/path1"
export PYTHONPATH="${PYTHONPATH}:/my/other/path2"
Adding the above to my ~/.bashrc did the trick for me on Ubuntu 16.04
This is an update to this thread which has some old answers.
For those using MAC-OS Catalina or some newer (>= 10.15), it was introduced a new Terminal named zsh (a substitute to the old bash).
I had some problems with the answers above due to this change, and I somewhat did a workaround by creating the file ~/.zshrc and pasting the file directory to the $PATH and $PYTHONPATH
So, first I did:
nano ~/.zshrc
When the editor opened I pasted the following content:
export PATH="${PATH}:/Users/caio.hc.oliveira/Library/Python/3.7/bin"
export PYTHONPATH="${PYTHONPATH}:/Users/caio.hc.oliveira/Library/Python/3.7/bin"
saved it, and restarted the terminal.
IMPORTANT: The path above is set to my computer's path, you would have to adapt it to your python.
The script below works on all platforms as it's pure Python. It makes use of the pathlib Path, documented here https://docs.python.org/3/library/pathlib.html, to make it work cross-platform. You run it once, restart the kernel and that's it. Inspired by https://medium.com/#arnaud.bertrand/modifying-python-s-search-path-with-pth-files-2a41a4143574. In order to run it it requires administrator privileges since you modify some system files.
from pathlib import Path
to_add=Path(path_of_directory_to_add)
from sys import path
if str(to_add) not in path:
minLen=999999
for index,directory in enumerate(path):
if 'site-packages' in directory and len(directory)<=minLen:
minLen=len(directory)
stpi=index
pathSitePckgs=Path(path[stpi])
with open(str(pathSitePckgs/'current_machine_paths.pth'),'w') as pth_file:
pth_file.write(str(to_add))
Just to add on awesomo's answer, you can also add that line into your ~/.bash_profile or ~/.profile
The add a new path to PYTHONPATH is doing in manually by:
adding the path to your ~/.bashrc profile, in terminal by:
vim ~/.bashrc
paste the following to your profile
export PYTHONPATH="${PYTHONPATH}:/User/johndoe/pythonModule"
then, make sure to source your bashrc profile when ever you run your code in terminal:
source ~/.bashrc
Hope this helps.
I added permanently in Windows Vista, Python 3.5
System > Control Panel > Advanced system settings > Advanced (tap) Environment Variables > System variables > (if you don't see PYTHONPATH in Variable column) (click) New > Variable name: PYTHONPATH > Variable value:
Please, write the directory in the Variable value. It is details of Blue Peppers' answer.
Fix Python Path issues when you switch from bash to zsh
I ran into Python Path problems when I switched to zsh from bash.
The solution was simple, but I failed to notice.
Pip was showing me, that the scripts blah blah or package blah blah is installed in ~/.local/bin which is not in path.
After reading some solutions to this question, I opened my .zshrc to find that the solution already existed.
I had to simply uncomment a line:
Take a look
I found a solution to do this in a anaconda environment here: https://datacomy.com/python/anaconda/add_folder_to_path/
Just:
conda develop /your_path
In Python 3.6.4 you can persist sys.path across python sessions like this:
import sys
import os
print(str(sys.path))
dir_path = os.path.dirname(os.path.realpath(__file__))
print(f"current working dir: {dir_path}")
root_dir = dir_path.replace("/util", '', 1)
print(f"root dir: {root_dir}")
sys.path.insert(0, root_dir)
print(str(sys.path))
I strongly suggest you use virtualenv and virtualenvwrapper otherwise you will clutter your path
Inspired by andrei-deusteanu answer, here is my version. This allows you to create a number of additional paths in your site-packages directory.
import os
# Add paths here. Then Run this block of code once and restart kernel. Paths should now be set.
paths_of_directories_to_add = [r'C:\GIT\project1', r'C:\GIT\project2', r'C:\GIT\project3']
# Find your site-packages directory
pathSitePckgs = os.path.join(os.path.dirname(os.__file__), 'site-packages')
# Write a .pth file in your site-packages directory
pthFile = os.path.join(pathSitePckgs,'current_machine_paths.pth')
with open(pthFile,'w') as pth_file:
pth_file.write('\n'.join(paths_of_directories_to_add))
print(pthFile)
After multiple bashing into wall. Finally resolved, in my CentOS 8 the pip3 was old, which was showing error to install the recent packages.
Now, I had downloaded the Python source package, which is Python-3.10.4 and installed the usual way, however the post-installation check generated errors in bash.
And I could not remove the existing Python, because that would break the CentOS desktop features.
Solution:
For building
./configure //don't not add --prefix=/usr, which you need to set proper care
make -j8
sudo make install
Now, as you have multiple Python installed, you can set alias python=python3
And for setting PYTHONPATH
export PYTHONPATH="/usr/local/bin/python3.10:/usr/local/lib/python3.10/lib-dynload:/usr/local/lib/python3.10/site-packages"
Don't add PYTHONHOME
For those who (like me) don't want to get too deeply involved in Python file management (which seems hopelessly overcomplicated), creating a .pth file works perfectly on my Windows 11 laptop (I'm using Visual Studio Code in Windows). So just go to the folder for your virtual environment site packages - there's mine:
Create a text file with a .pth extension - I called mine wheal.pth:
Add paths to it:
The best thing about this in VS Code is that import statements recognise this path (I had to exit VS Code and go back in), so now more typing # type: ignore to suppress linting warning messages!
on Mac :
user#terminal$ env PYTHONPATH=module_path python3
>>> import sys
>>> sys.path
['module_path', 'plus_other_python3_paths',...]
Shortest path between A <-> B is a straight line;
import sys
if not 'NEW_PATH' in sys.path:
sys.path += ['NEW_PATH']
I'm using PyCharm on a mac (OSX mavericks) to run a simple script shown below. All it does is print the PATH variable. I have a virtualenv in the project directory. I added a Run Configuration in PyCharm and tried it with different Pythons:
# file mytest.py
import os
print "PATH: ", os.environ['PATH']
When I run with the system default python (/usr/bin/python) it prints the correct value for PATH (i.e. the PATH as I have configured in my .bash_profile file,) which is kind of long and contains many directories.
But when I choose the venv's Python, the path is reduced to only: /usr/bin:/bin:/usr/sbin:/sbin:/Users/myname/projects/myproj/venv/bin
This doesn't happen if I run the script from a terminal window. In this case it shows the correct PATH for both the system's python and the venv python. It also doesn't happen if I deactivate the venv and run venv/bin/python mytest.py.
Anyone knows how to make the correct PATH value be set when running from PyCharm and using venv?
you should probably know that all environment variables are inherited.
When you define environment variable in your .bash_profile it becomes available in your terminal (bash),
and in all processes that will be started from terminal (These processes will be children for the bash process).
That's why you are getting expected values when running your script from within the terminal.
You start PyCharm not from a terminal, so it doesn't inherit PATH. And so do Python or venv (they launched from PyCharm).
To solve your issue you have 3 options here: just start PyCharm from terminal or move PATH variable definition from .bash_profile to session init scripts (PATH will be defined system-wide) or Duplicate your PATH in PyCharm's run configuration (it has such option over there)
Good luck!
I built something for SublimeLint that figures out your shell's PATH in Python.
https://github.com/lunixbochs/sublimelint/blob/st3/lint/util.py#L57
Basically, it runs $SHELL --login, echoes your path, and parses it.
The gist is:
import os
import subprocess
shell = os.path.basename(os.environ['SHELL'])
output = subprocess.Popen(
(shell, '--login', '-c', 'echo __SEP__$PATH'),
stdout=subprocess.PIPE).communicate()[0] or ''
print output.split('__SEP__', 1)[1].split(':')
The problem for me on windows was case related: it's the difference between Path and PATH.
If you choose:
Edit configurations
Environment Variables
Show (down in the bottom right corner)
This shows the environment variables. On my system there is no PATH, only Path.
Defining a new PATH environment variable wouldn't work, but Path worked fine.
There's a workaround in my case.
To modify the environment variable PATH, I usually just need to add extra paths to the PYTHONPATH variable in the run configuration's environment variables setting.
PYTHONPATH will then be added to PATH.
I am trying to get virtualenv[wrapper] to work on my Windows machine through Cygwin. The install is successful, together with easy_install, based on these directions: http://www.doughellmann.com/docs/virtualenvwrapper/.
The problem comes in when I use the mkvirtualenv [name_of_vir_env]. I get the following output:
$ mkvirtualenv testenv
New python executable in testenv\Scripts\python.exe
Installing setuptools.................done.
bash: D:\.virtualenvs/testenv/bin/postactivate: No such file or directory
chmod: getting attributes of `D:\\.virtualenvs/testenv/bin/postactivate': No such file or directory
bash: D:\.virtualenvs/testenv/bin/predeactivate: No such file or directory
chmod: getting attributes of `D:\\.virtualenvs/testenv/bin/predeactivate': No such file or directory
bash: D:\.virtualenvs/testenv/bin/postdeactivate: No such file or directory
chmod: getting attributes of `D:\\.virtualenvs/testenv/bin/postdeactivate': No such file or directory
ERROR: Environment 'D:\.virtualenvs/testenv' does not contain an activate script.
Inside the testenv directory, there is no bin subdirectory, rather only Lib and Scripts. Scripts contains activate.bat which is supposed to be used to activate this particular environment, however if I try to run this out through bash I get an error:
$ ./activate.bat
./activate.bat: line 1: #echo: command not found
./activate.bat: line 4: syntax error near unexpected token `('
./activate.bat: line 4: `if not defined PROMPT ('
I can exit bash and call activate.bat and this changes to the desired environment. But then not being in bash I cannot use the workon command, or any other in virtualenvwrapper_bashrc.
How can I get the two to work together, that is, stay in bash so I can use the commands in virtualenvwrapper_bashrc?
I'm not experienced with virtualenvwrapper, but do use virtualenv regularly. I don't think the activate.bat is intended to be run under cygwin, it works when run in the regular windows shell. I think if you are using cygwin, you might want to use something more like bin/activate(the version for unix-like OS's).
The cygwin environment within bash could be drastically different from the standard environment activate.bat expects to run in, so finding an activate script that would work with bash(perhaps find a copy from a unix version) would probably get you to where you can run your virtualenv within bash.
This post looks promising
http://atbrox.com/2009/09/21/how-to-get-pipvirtualenvfabric-working-on-cygwin/
--
Quentin
This did it for me:
https://bitbucket.org/cliffxuan/virtualenvwrapper-for-cygwin-windows-python
The vanilla virtualenvwrapper does not seem to support the Cygwin environment.
A caveat however is that executable scripts under the Scripts directory of the virtualenv are only executable if you pass them explicitly to the python command and don't use ~ or anything else that Cygwin implicitly converts a path starting with /cygdriv/c/...—the native Windows Python cannot see those paths.
I don't know virtualenv, but I'm seeing what looks like a classic mixed path syntax issue from cygwin:
Your string is:
D:\.virtualenvs/testenv/bin/predeactivate
but Cygwin is interpreting the backslash as an escape for the "." character, producing:
D:.virtualenvs/testenv/bin/postactivate
which is in the error text you're quoting and is obviously a malformed path. Check your actual environment variable - it's probably in DOS/Windows path syntax, while the internal part is in cygwin/unix syntax.
If so, try using os.path.join to splice the two parts together and see if you get a consistent syntax out of that.