I'm trying to run the following bash script which runs a Python program after activating a conda environment.
send.bash
#!/bin/bash
source activate manage_oam_users
python ~/path/to/script/send.py
source deactivate
crontab
30 * * * * source /path/to/script/send.bash
I get the following error from cron, although running source send.bash works perfectly. I've also tried using bash send.bash which works fine when run manually, but results in the same error when run from cron.
/path/to/script/send.bash: line 2: activate: No such file or directory
activate and deactivate are probably scripts located somewhere an entry in your $PATH variable points to. Usually, software installed locally for one user adds statements to your .profile file or .bashrc that extend your $PATH variable so that you can use the software's scripts without using full paths.
While your bash loads .profile and .bashrc automatically, CRON won't do that. There are at least two solutions for this.
A) Full Paths everywhere
Either you use full paths in the script executed by your CRON job, like this:
#!/bin/bash
source /path/to/activate manage_oam_users
python $HOME/path/to/script/send.py
source /path/to/deactivate
Also use $HOME instead of ~. You can find out the full paths using which activate and which deactivate in your shell.
B) Source .profile or .bashrc
Alternatively you can source your .profile (or .bashrc; you will have to look which file extends your $PATH variable with the anaconda directories) in your CRON tab:
30 * * * * source $HOME/.profile; source /path/to/script/send.bash
Extra: What does source mean?
source is a Unix command that evaluates the file following the command, as a list of commands, executed in the current context.
– from Wikipedia, the something something great encyclopaedia
A commonly used alias for the source command is a single dot (. /path/to/script).
A related, but more generic question can be found on the UNIX and Linux Stack Exchange.
Since the cron does not run from the directory in which anaconda is installed, it is unable to find activate. Also your path seems to be missing the root anaconda directory.
Find the location of the activate command and add it to your PATH .
which activate
/Users/username/anaconda/bin/activate
In your bash_profile add
export PATH="/Users/username/anaconda/bin:$PATH"
additionally, the directory can be defined to be added automatically
script_dir=$(dirname $0)
#file_imports
source $script_dir"/functions.sh"
I put the following in my python_go.py
import os
os.system("cd some_dir") # This is the directory storing an existing virtual environment
os.system(". activate") #because I want to activate the virtual environment
os.system("cd another_dir") #this the directory I can start my work
I hope I can run the python_go.py, it can do the work mentioned above.
But when I run it, seems it can only do the first step, the rest of it, e.g. . activate seems not working.
Can someone tell me how to do it? Thank you!!
Most probably you don't have to change to some_dir to source activate so saving these lines
. some_dir/activate
cd another_dir
as, let's say go.sh and doing
. go.sh
has the same effect
If you're relying to os.system(". activate") to work if it's in the directory some_dir that won't work because the current directory won't persist across calls to os.system().
You're going to be better off calling a shell script that aggregates all three of the commands you want to do and execute that once from the python script.
Otherwise you want to set up the environment for the parent python process using os.chdir() before calling os.system on the activate call. Also, the os.system(". activate") call won't do what you want because the "dot space" notation will load information into a shell that's going to go away when the os.system call finishes.
Edited (to your followup comment):
Your shell script should look like this (do_activate.sh):
cd some_dir
. activate
cd another_dir
and the python code like this:
os.system("db_activate.sh").
Keep in mind that whatever environment variables were saved by ". activate" won't persist after the os.system call.
Your code does nothing. os.system() starts a new shell for each command i.e., all os.system() calls have no positive effect: cd and . activate may have effect only on the current shell (and possibly its children).
If all you want is to activate a virtualenv in the current shell then you should use a shell command:
$ . some_dir/activate && cd another_dir
Note: the command has effect only on the current (running) shell (and its descendants).
virtualenvwrapper provides several hooks that allows to execute commands before/after activating a virtualenv e.g., you could put cd another_dir into $VIRTUAL_ENV/bin/postactivate then it is enough to run:
$ workon <virtualenv-name>
to activate virtualenv-name virtualenv and run all the hooks (cd another_dir in this case).
You probably want to install virtualenvwrapper which does what you want:
workon envname will source the file and activate the virtualenv.
You can then do setvirtualenvproject in the desired directory and you'll automatically go to the directory where the project is located. You only need to do this command once as it'll then happen automatically from then on.
Whenever I use sys.path.append, the new directory will be added. However, once I close python, the list will revert to the previous (default?) values. How do I permanently add a directory to PYTHONPATH?
If you're using bash (on a Mac or GNU/Linux distro), add this to your ~/.bashrc
export PYTHONPATH="${PYTHONPATH}:/my/other/path"
You need to add your new directory to the environment variable PYTHONPATH, separated by a colon from previous contents thereof. In any form of Unix, you can do that in a startup script appropriate to whatever shell you're using (.profile or whatever, depending on your favorite shell) with a command which, again, depends on the shell in question; in Windows, you can do it through the system GUI for the purpose.
superuser.com may be a better place to ask further, i.e. for more details if you need specifics about how to enrich an environment variable in your chosen platform and shell, since it's not really a programming question per se.
Instead of manipulating PYTHONPATH you can also create a path configuration file. First find out in which directory Python searches for this information:
python -m site --user-site
For some reason this doesn't seem to work in Python 2.7. There you can use:
python -c 'import site; site._script()' --user-site
Then create a .pth file in that directory containing the path you want to add (create the directory if it doesn't exist).
For example:
# find directory
SITEDIR=$(python -m site --user-site)
# create if it doesn't exist
mkdir -p "$SITEDIR"
# create new .pth file with our path
echo "$HOME/foo/bar" > "$SITEDIR/somelib.pth"
This works on Windows
On Windows, with Python 2.7 go to the Python setup folder.
Open Lib/site-packages.
Add an example.pth empty file to this folder.
Add the required path to the file, one per each line.
Then you'll be able to see all modules within those paths from your scripts.
In case anyone is still confused - if you are on a Mac, do the following:
Open up Terminal
Type open .bash_profile
In the text file that pops up, add this line at the end:
export PYTHONPATH=$PYTHONPATH:foo/bar
Save the file, restart the Terminal, and you're done
You could add the path via your pythonrc file, which defaults to ~/.pythonrc on linux. ie.
import sys
sys.path.append('/path/to/dir')
You could also set the PYTHONPATH environment variable, in a global rc file, such ~/.profile on mac or linux, or via Control Panel -> System -> Advanced tab -> Environment Variables on windows.
To give a bit more explanation, Python will automatically construct its search paths (as mentioned above and here) using the site.py script (typically located in sys.prefix + lib/python<version>/site-packages as well as lib/site-python). One can obtain the value of sys.prefix:
python -c 'import sys; print(sys.prefix)'
The site.py script then adds a number of directories, dependent upon the platform, such as /usr/{lib,share}/python<version>/dist-packages, /usr/local/lib/python<version>/dist-packages to the search path and also searches these paths for <package>.pth config files which contain specific additional search paths. For example easy-install maintains its collection of installed packages which are added to a system specific file e.g on Ubuntu it's /usr/local/lib/python2.7/dist-packages/easy-install.pth. On a typical system there are a bunch of these .pth files around which can explain some unexpected paths in sys.path:
python -c 'import sys; print(sys.path)'
So one can create a .pth file and put in any of these directories (including the sitedir as mentioned above). This seems to be the way most packages get added to the sys.path as opposed to using the PYTHONPATH.
Note: On OSX there's a special additional search path added by site.py for 'framework builds' (but seems to work for normal command line use of python): /Library/Python/<version>/site-packages (e.g. for Python2.7: /Library/Python/2.7/site-packages/) which is where 3rd party packages are supposed to be installed (see the README in that dir). So one can add a path configuration file in there containing additional search paths e.g. create a file called /Library/Python/2.7/site-packages/pip-usr-local.pth which contains /usr/local/lib/python2.7/site-packages/ and then the system python will add that search path.
On MacOS, Instead of giving path to a specific library. Giving full path to the root project folder in
~/.bash_profile
made my day, for example:
export PYTHONPATH="${PYTHONPATH}:/Users/<myuser>/project_root_folder_path"
after this do:
source ~/.bash_profile
On linux you can create a symbolic link from your package to a directory of the PYTHONPATH without having to deal with the environment variables. Something like:
ln -s /your/path /usr/lib/pymodules/python2.7/
For me it worked when I changed the .bash_profile file. Just changing .bashrc file worked only till I restarted the shell.
For python 2.7 it should look like:
export PYTHONPATH="$PYTHONPATH:/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python"
at the end of the .bash_profile file.
Adding export PYTHONPATH="${PYTHONPATH}:/my/other/path" to the ~/.bashrc might not work if PYTHONPATH does not currently exist (because of the :).
export PYTHONPATH="/my/other/path1"
export PYTHONPATH="${PYTHONPATH}:/my/other/path2"
Adding the above to my ~/.bashrc did the trick for me on Ubuntu 16.04
This is an update to this thread which has some old answers.
For those using MAC-OS Catalina or some newer (>= 10.15), it was introduced a new Terminal named zsh (a substitute to the old bash).
I had some problems with the answers above due to this change, and I somewhat did a workaround by creating the file ~/.zshrc and pasting the file directory to the $PATH and $PYTHONPATH
So, first I did:
nano ~/.zshrc
When the editor opened I pasted the following content:
export PATH="${PATH}:/Users/caio.hc.oliveira/Library/Python/3.7/bin"
export PYTHONPATH="${PYTHONPATH}:/Users/caio.hc.oliveira/Library/Python/3.7/bin"
saved it, and restarted the terminal.
IMPORTANT: The path above is set to my computer's path, you would have to adapt it to your python.
The script below works on all platforms as it's pure Python. It makes use of the pathlib Path, documented here https://docs.python.org/3/library/pathlib.html, to make it work cross-platform. You run it once, restart the kernel and that's it. Inspired by https://medium.com/#arnaud.bertrand/modifying-python-s-search-path-with-pth-files-2a41a4143574. In order to run it it requires administrator privileges since you modify some system files.
from pathlib import Path
to_add=Path(path_of_directory_to_add)
from sys import path
if str(to_add) not in path:
minLen=999999
for index,directory in enumerate(path):
if 'site-packages' in directory and len(directory)<=minLen:
minLen=len(directory)
stpi=index
pathSitePckgs=Path(path[stpi])
with open(str(pathSitePckgs/'current_machine_paths.pth'),'w') as pth_file:
pth_file.write(str(to_add))
Just to add on awesomo's answer, you can also add that line into your ~/.bash_profile or ~/.profile
The add a new path to PYTHONPATH is doing in manually by:
adding the path to your ~/.bashrc profile, in terminal by:
vim ~/.bashrc
paste the following to your profile
export PYTHONPATH="${PYTHONPATH}:/User/johndoe/pythonModule"
then, make sure to source your bashrc profile when ever you run your code in terminal:
source ~/.bashrc
Hope this helps.
I added permanently in Windows Vista, Python 3.5
System > Control Panel > Advanced system settings > Advanced (tap) Environment Variables > System variables > (if you don't see PYTHONPATH in Variable column) (click) New > Variable name: PYTHONPATH > Variable value:
Please, write the directory in the Variable value. It is details of Blue Peppers' answer.
Fix Python Path issues when you switch from bash to zsh
I ran into Python Path problems when I switched to zsh from bash.
The solution was simple, but I failed to notice.
Pip was showing me, that the scripts blah blah or package blah blah is installed in ~/.local/bin which is not in path.
After reading some solutions to this question, I opened my .zshrc to find that the solution already existed.
I had to simply uncomment a line:
Take a look
I found a solution to do this in a anaconda environment here: https://datacomy.com/python/anaconda/add_folder_to_path/
Just:
conda develop /your_path
In Python 3.6.4 you can persist sys.path across python sessions like this:
import sys
import os
print(str(sys.path))
dir_path = os.path.dirname(os.path.realpath(__file__))
print(f"current working dir: {dir_path}")
root_dir = dir_path.replace("/util", '', 1)
print(f"root dir: {root_dir}")
sys.path.insert(0, root_dir)
print(str(sys.path))
I strongly suggest you use virtualenv and virtualenvwrapper otherwise you will clutter your path
Inspired by andrei-deusteanu answer, here is my version. This allows you to create a number of additional paths in your site-packages directory.
import os
# Add paths here. Then Run this block of code once and restart kernel. Paths should now be set.
paths_of_directories_to_add = [r'C:\GIT\project1', r'C:\GIT\project2', r'C:\GIT\project3']
# Find your site-packages directory
pathSitePckgs = os.path.join(os.path.dirname(os.__file__), 'site-packages')
# Write a .pth file in your site-packages directory
pthFile = os.path.join(pathSitePckgs,'current_machine_paths.pth')
with open(pthFile,'w') as pth_file:
pth_file.write('\n'.join(paths_of_directories_to_add))
print(pthFile)
After multiple bashing into wall. Finally resolved, in my CentOS 8 the pip3 was old, which was showing error to install the recent packages.
Now, I had downloaded the Python source package, which is Python-3.10.4 and installed the usual way, however the post-installation check generated errors in bash.
And I could not remove the existing Python, because that would break the CentOS desktop features.
Solution:
For building
./configure //don't not add --prefix=/usr, which you need to set proper care
make -j8
sudo make install
Now, as you have multiple Python installed, you can set alias python=python3
And for setting PYTHONPATH
export PYTHONPATH="/usr/local/bin/python3.10:/usr/local/lib/python3.10/lib-dynload:/usr/local/lib/python3.10/site-packages"
Don't add PYTHONHOME
For those who (like me) don't want to get too deeply involved in Python file management (which seems hopelessly overcomplicated), creating a .pth file works perfectly on my Windows 11 laptop (I'm using Visual Studio Code in Windows). So just go to the folder for your virtual environment site packages - there's mine:
Create a text file with a .pth extension - I called mine wheal.pth:
Add paths to it:
The best thing about this in VS Code is that import statements recognise this path (I had to exit VS Code and go back in), so now more typing # type: ignore to suppress linting warning messages!
on Mac :
user#terminal$ env PYTHONPATH=module_path python3
>>> import sys
>>> sys.path
['module_path', 'plus_other_python3_paths',...]
Shortest path between A <-> B is a straight line;
import sys
if not 'NEW_PATH' in sys.path:
sys.path += ['NEW_PATH']
I'm trying to install easy_install using setup tools for the purpose of installing scrapy on Windows XP. On the setup tools page it says:
Once installation is complete, you will find an easy_install.exe
program in your Python Scripts subdirectory. Be sure to add this
directory to your PATH environment variable, if you haven't already
done so.
But when i install I only find an easy_install.exe.manifest file, and I'm not able to run this file. Sorry if this is a stupid question, but I'm new to this. I'm not quite sure what this sentence means either:
Be sure to add this directory to your PATH environment variable, if
you haven't already done so.
It means that you should have the path to the directory containing the file when you do
echo %PATH% (windows)
echo $PATH (linux/cygwin/...)
in the command line. If it does not appear, you should add it.
To do so check out the following links
git - setting path variable (linux)
How to set ANT_HOME with Windows? (windows)
I'm having some trouble running the ./make command in my debian command line to install python 2.7.2.
I untarred my download from Python.org and ran ./configure which appeared to have worked fine. Unfortunately when I type in ./make I get the following error:
./make: No such file or directory
Not sure why this occurs, but I'd like to get an updated version of python to continue learning the language.
Thanks for your help,
Andy
When you type ./configure, it runs a executable script in the current directory (labeled with a .) called configure.
Make is an executable file, usually located somewhere like /usr/bin, which uses a file in the directory to run a bunch of commands depending on whether files are up to date.
When you just type make, your shell (the program that handles all your commands and sends their output to the terminal) will go looking through all the directories in the PATH environment variable to find an executable file called make, and run the first one it finds. But, when you type ./make, you're actually telling it to try and run an executable file in the current directory, called make. (It uses this approach, not searching the PATH variable, whenever you put a / in the command.)
You can use the . anywhere you could use a normal directory to specify the same directory, so for example: /usr/bin/././././ is the same as: /usr/bin. Similarly, you can use .. to specify the directory above, so /usr/bin/../bin/../bin/../lib is the same as /usr/lib.
So, after running the configure script located in ./, which generates a so-called makefile, you run the system wide version of make, located where ever, by just typing make, which uses the makefile to build the package.
Also, you can use the which command to find out where the command that'll run when you enter a command by itself - for example, which make.
(Apologies if any of this is condescending, I was going for completism. Also, I may have overused the code tags...)
its not ./make
try
"make"
as it is