I want to know how (if there is a way) to write a function and have it automatically defined when you start a python shell, or to keep certain defined functions after the shell closes so you don't have to keep re defining it if you use it a lot.
Yes, you can, you need to set your environment variable PYTHONSTARTUP to a python script you would like to be executed at every python startup.
By convention, such files are named by adding the rc suffix to the program name you're trying to tweak, and are usually located at the root of your home directory. In that case I would create the python file under $HOME/.pythonrc and then run export PYTHONSTARTUP=~/.pythonrc (for UNIX-like systems, it could be slightly different on Windows if you don't have a MinGW or equivalent).
Here's an example of .pythonrc file that you can play with: https://gist.github.com/twneale/5245670
Related
I am trying to access constants set in a python file from a bash script. The constants will be accessed by both python files and bash scripts. At first I thought to just set the variables in python like:
import os
...
constant_variable=1
constant_path='/path/to/file'
etc...
and then
source constants_file.py
in the bash script.
but I have been told to set these constants as environmental variables in python instead as the python file will likely include stuff that I don't want to load into bash using "source" in the future.
I have tried doing something like this in python...
os.environ['CONSTANT_VARIABLE'] = '1'
...
constant_variable = int(os.environ.get('CONSTANT_VARIABLE'))
and then accessing it in bash like so...
constant_variable=${CONSTANT_VARIABLE}
which does not work because I am no longer using source to read in the constants from the python file...
I don't know if this is even possible, but is there some way to just import env variables created by the python file without sourcing the whole file? Or am I way off base here?
Thanks!
If you set the environment variable in python, it will only be set for the process's child process. Unless you're running the bash script FROM the python process, you'll probably need to either generate a file to source with bash, or just define the variables in bash and then read them in python.
I have a Python script which imports the datetime module. It works well until someday I can run it in a directory which has a Python script named datetime.py. Of course, there are a few ways to resolve the issue. First, I run the script in a directory that does not contain the script datetime.py. Second, I can rename the Python script datetime.py. However, neither of the 2 approaches are perfect ways. Suppose one ship a Python script, he never knows where users will run the Python script. Another possible fix is to prevent Python from search the current working directory for modules. I tried to remove the empty path ('') from sys.path but it works in an interactive Python shell but not in a Python script. The invoked Python script still searches the current path for modules. I wonder whether there is any way to disable Python from searching the current path for modules?
if __name__ == '__main__':
if '' in sys.path:
sys.path.remove('')
...
Notice that it deosn't work even if I put the following code to the beginning of the script.
import sys
if '' in sys.path:
sys.path.remove('')
Below are some related questions on StackOverflow.
Removing path from Python search module path
pandas ImportError C extension when io.py in same directory
initialization of multiarray raised unreported exception python
Are you sure that Python is searching for that module in the current directory, and not on the script directory? I don't think Python adds the current directory to the sys.path, except in one case. Doing so could even be a security risk (akin to having . on the UNIX PATH).
According to the documentation:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first
So, '' as a representation of the current directory happens only if run from interpreter (that's why your interactive shell test worked) or if the script is read from the standard input (something like cat modquest/qwerty.py | python). Neither is a rather 'normal' way of running Python scripts, generally.
I'm guessing that your datetime.py stands side by side with your actual script (on the script directory), and that it just happens that you're running the script from that directory (that is script directory == current directory).
If that's the actual scenario, and your script is standalone (meaning just one file, no local imports), you could do this:
sys.path.remove(os.path.abspath(os.path.dirname(sys.argv[0])))
But keep in mind that this will bite you in the future, once the script gets bigger and you split it into multiple files, only to spend several hours trying to figure out why it is not importing the local files...
Another option is to use -I, but that may be overkill:
-I
Run Python in isolated mode. This also implies -E and -s. In isolated mode sys.path contains neither the script’s directory nor the user’s site-packages directory. All PYTHON* environment variables are ignored, too. Further restrictions may be imposed to prevent the user from injecting malicious code.
In the R programming language, there is a site.profile file that defines some code that R processes execute on start up. Is there similar functionality in Python?
Edit: to clarify, this script should be executed if the user calls python from the command line, but also if python is spawned from another process (e.g. if the user's script uses subprocess to spawn another python).
If you only want this for interactive sessions (as opposed to also happening every time you run a script with python myscript.py or ./myscript or a module with python -m mymodule), what you want is the environment variable PYTHONSTARTUP:
If this is the name of a readable file, the Python commands in that file are executed before the first prompt is displayed in interactive mode. The file is executed in the same namespace where interactive commands are executed so that objects defined or imported in it can be used without qualification in the interactive session…
If you want this to always happen forever, of course, you need to set this environment variable in some appropriate global place—e.g., your shell profile on most *nix platforms, or both your shell profile and your launchd profile on macOS, or the appropriate part of the Control Panel on Windows (the appropriate part changes with almost every new version of Windows, but it usually has "System" in the name).
If you want this to happen for all users, not just the current user… the details for how to set a system-wide environment variable are more platform-specific, but otherwise the idea is the same.
If you want this to happen for every Python session, even when some other program is running a Python script and you didn't even know it was doing that… what you want is either usercustomize or sitecustomize, as documented in the site documentation:
This module is automatically imported during initialization. The automatic import can be suppressed using the interpreter’s -S option.
…
After these path manipulations, an attempt is made to import a module named sitecustomize, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory.
After this, an attempt is made to import a module named usercustomize, which can perform arbitrary user-specific customizations, if ENABLE_USER_SITE is true. This file is intended to be created in the user site-packages directory (see below), which is part of sys.path unless disabled by -s…
So, you want to find an appropriate place to override this. First try this:
python3 -m site
Then, if this didn't give you sys.path (probably only on pretty old Python, but just in case…), also do this:
python3 -c 'import sys; print('\n'.join(sys.path))'
If you want this customization to happen only for the current user, you want to create a usercustomize.py file in the USER_SITE directory listed by python3 -m site. If the directory doesn't exist, create it.
If you want it to happen for all users, you want a sitecustomize.py file in one of the sys.path directories. The problem is that there may already be one. For example, most linux distros' builtin Python packages have their own sitecustomize modules. If there is, python3 -c 'import sitecustomize; print(sitecustomize.__file__) will tell you where it is. Then, you can edit, or you can copy it, edit that copy, and place that copy somewhere that comes earlier in sys.path than the original. As a general rule, /usr/local is probably better than /usr, and site-packages is probably better than dist-packages is probably better than anything else.
The Python mechanism is called... site. It is a module that is automatically imported and sets up your environment. If it finds modules sitecustomize.py and usercustomize.py, it will import them as well. So these two are where you would put site-wide and personal customizations that you want to be a permanent part of your Python environment. See the site documentation for more details.
File pointed by your environmental variable PYTHONSTARTUP would be run on starting an interactive python session
And USER_SITE/usercustomize.py will be run for non-interactive python session.
Run python -c "import site; print(site.USER_SITE)" to find the USER_SITE dir location
I have been looking into bash and shell recently and been trying to work out how git is able to make a terminal command that runs C scripts in the current directory e.g git init, git push/pull etc.
I have been trying to simulate it in python, by making an executable python script in my home directory,
#!/usr/bin/env python
import os
print("current_dir: ",os.getcwd()) #prints /Users/usr/folder/script.py
Then I create a .command file that calls the python script
cd
cd FOLDER/
python3 SCRIPT.py
and editing the bash profile to export a variable to run the .command file.
export mycommand=/Users/urn/folder/command.command
Although this is not even nearly close to the way git achieves its command line. For example, when I run my script is not actually a terminal command it is just an environment variable, hence the $.
$mycommand
Secondly, this goes to the directory of the python file and runs the script from with that directory, therefore the output will always be the same
/Users/usr/folder/script.py
Although in git it runs the file in the current directory. Therefore the print statement would change depending on the terminal directory.
How am I able to create my own 'terminal command' such as git init to run my python script in whatever directory I'm in. Ps, I'm on mac.
Any help would be greatly appreciated :)
It sounds like you are missing at least two basic concepts:
The search path: when you issue a command that is not a shell function or built-in and that has an unqualified name (one with no / characters), the shell searches a list of zero or more directories for a matching executable file. This list of directories is the "path", and it is stored in the PATH environment variable. You can change it if you wish. This is how the shell finds the git program when you do not specify a path to it.
Executable scripts: Python, shell, Perl, etc. programs that must be run via an interpreter can be made executable by name alone by including an appropriate shebang line as the very first line and assigning an executable mode. You include an appropriate shebang line in your example Python program, in fact, but you seem not to understand its significance, because you explicitly launch the script via the python3 command. The shebang line is just another comment to Python, but it is meaningful to the system.
It seems like you probably also are missing some other concepts, like the fact that your script doesn't need to be in the current working directory for you to run it via the python3 launcher, path notwithstanding. Just specify its full pathname. Alternatively, Python has its own variation on a path, PYTHONPATH, by which it can locate modules and packages. These are alternatives, however -- the first two points are enough to achieve your apparent objective. Specifically,
Keep the shebang line in your script, though your default python is probably v2.7, so if you really want to run it specifically via python3 then modify the shebang line to say so:
#!/usr/bin/env python3
Make sure the file has executable mode. For example,
chmod 0755 /Users/urn/bin/SCRIPT.py
Then you should be able to execute the script from anywhere via its full pathname.
To access it from anywhere via its simple name, ensure that the directory containing it is in your path. For this purpose, it would be wise to choose an appropriate directory, such as /usr/local/bin or /Users/urn/bin (you may need to create the directory first). Whichever you choose, ensure that that directory is in your PATH. For example, edit /Users/urn/.bash_profile, creating it if necessary, and ensure that it contains (say) the commands
PATH=$PATH:/Users/urn/bin
export PATH
That will take effect in new Terminal windows you open afterward, but not automatically in any that are already open. In those windows, you will be able to run the script, from anywhere, via its simple name.
I've just started learning Python using Learning Python by Mark Luts. In his book he offers an example of a simple script that is called through the Windows shell. In the example, he calls is as follows:
C:\code> python script1.py
I've gone and modified the Environment Variables on my machine so that I can call
C:\User\Example> python
to open up the interpreter and I can also call something like
C:\User\Example> script1
to run a script that I've written and placed in my desired directory. My issue is that I can not call
C:\User\Example> python script1.py
in my command line the same way he does in the book. He's mentioned something about a PYTHONPATH Environment Variable, however, this variable isn't present on my machine. I only have 'path', 'TEMP', and 'TMP'. Particulary, when I try to make such a call I get the error
python: can't open file 'script1.py': [Errno 2] No such file or directory
What do I have to do in order to get this sort of command to work properly on the command line?
From the book (p. 44, 4th Ed):
Finally, remember to give the full path to your script if it lives in a different directory from the one in which you are working.
For your situation, this means using
C:\User\Example> python C:\User\Example\my_scripts\script1.py
You could write a batch file that looks for the script in a predefined directory:
#echo off
setlocal
PATH=C:\User\Example\Python36;%PATH%
SCRIPT_DIR=C:\User\Example\my_scripts
python %SCRIPT_DIR\%*
You are calling python from within the context of C:\User\Example, and passing it a name of a file you want to run through the intepreter (script1.py). It is clear that the PATH variable is setup correctly such that you can call python from anywhere on you computer, since we can see that it is running but not actually able to find your script.
However, you stated in the comment that your scripts are actually located in C:\User\Example\my_scripts. In other words, you are passing python a name of a file that doesn't exist!! (at least from the contect of C:\User\Example).
You need to be in the directory of the script in order for the python executable to be able to find it.
Alternatively, you can run the python command and give it more information as to where the script is. For instance, you could run python .\my_scripts\script1.py if you are running from within the contect of C:\User\Example and your scripts are in C:\User\Example\my_scripts.