Context:
I'm working with a few developers, some on OSX and some on Linux. We use a collection of bash scripts to load environment variables, build docker images etc. The OSX users are using zsh instead of bash, so some tricks (specifically how we're getting the running script's directory) aren't compatible. What works on Linux (but not OSX) is:
$ cat ./scripts/env_script.sh
THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
# ... more variables ...
$ source ./scripts/env_script.sh
$ echo $THIS_DIR
# the absolute path to the scripts directory
Some cursory testing shows this doesn't work on OSX, presumably because BASH_SOURCE doesn't work the same with zsh. Fine.
Goal:
We're all working with a Python, we would like to replace our bash scripts with Python.
The same pattern is very easy in Python.
from pathlib import Path
THIS_DIR = str(Path(__file__).parent)
Not the cleanest, but it works anywhere. Ideally we could run this and share THIS_DIR with the shell that called the script. Something like
$ <answer to my question> ./scripts/env_script.py
$ echo $THIS_DIR
# absolute path to the scripts directory
Question:
Suppose we rewrote our ./scripts/env_script.sh into ./scripts/env_script.py and collected a list of variables we would like to share with the calling environment.
That is, we wrote the Python script above.
Is it possible to do something akin to source ./scripts/env_script.py so that we can export the variables from the Python script into the calling shell?
What I've tried:
I found a potential duplicate here though the answer is basically 'Python runs in a sub-process, source would only work if your shell is Python` which makes sense but doesn't answer the question.
If there isn't a nice way to exchange these variables, then another option would basically be to have the Python script spit the variables to stdout and then eval them within my shell. I.e. eval $(./script/env_script.py). I guess this could work, but it feels wrong at least because I was always taught eval is evil.
I spent a little more time on the second solution since I could imagine it working (although I'm not happy with it). Basically, I rewrite ./scripts/env_script.py to be
FOO='whatever'
THIS_DIR=str(Path(__file__).parent)
# and so on...
# prints all "normal" variables to stdout
def_vars = list(locals().items())
for name, val in def_vars:
if name.startswith("_"):
continue
print(f'{name}="{val}"')
So running it normally would output
$ ./scripts/env_script.py
FOO="whatever"
THIS_DIR="..."
...
$ echo $FOO
# nothing
Then, to jam those variables into the calling shell, we wrap that with eval and it does the job
$ eval $(./scripts/env_script.py)
$ echo $FOO
whatever
I guess with some further tweaking, the footer at the end of env_script.py could be made a little more robust, and then moved outside of the script. Some sort of alias could be defined to make it syntactically nicer, so then calling it could like pysource ./scripts/env_script.py. Still, I'm not satisfied with this answer.
Edit: I did the tweaking and wrote a downloadable script here.
Related
I would like to run a Python script setting in the shell where the interpreter must look for the modules.
Suppose that myscript.py contains only:
import mymodule ; mymodule.myfunction()
But mymodule is in /home/user/hello, whereas myscript.py is, say, in /home/user/Desktop. I want to run on a terminal something like:
$ python /home/user/Desktop/myscript.py LOCATION_OF_THE_MODULES=/home/user/hello.
Would it be possible? I think that an alternative solution is to define the location in the import statement from the code, but this is not what I am looking for. I want to set the location through a variable in the shell.
So, I've been exploring a little your question, turns out this isn't a Python question but a "prompt" question, because indeed there is a way to do that but, since Python can't hop into interactive from script, we can't make it using Python only, but the Python interactive command have some extra options we can use
see:
python3 -h
for more info.
Specifically there are 2 options that are interesting, -i and -c which stands for interactive mode and command string respectively, that way we can load the modules with -c and hop into interactive with -i, like so:
python3 -i -c "import os"
Obviously, we need to make it more advanced so it can load multiple modules without Python scripting, then we will be needing to make the actual command to run Python and load the scripts you want, there is a problem tho, since we need to issue a command to be able to load all the modules you want in a folder it might create incompatibilities with prompts since not all prompts have the same syntax. There might be another low-level answer to this problem but I couldn't get to it, however, I will leave a Bash Script for reference so you can use it and/or edit it so it works best with your prompt.
FINAL_VAR=""
cd $1
for f in *.py; do
FINAL_VAR+="import ${f%.py}"$'\n'
done
python3 -i -c "$FINAL_VAR"
Usage steps:
Copy and save the script
Give it run permissions (chmod +x file_name.sh)
Run it this way: ./file_name.sh "/full/path/to/your/modules"
It will load all the .py files and will hop into an interactive Python shell for your use
Note: You might want to change the last line so it works accordingly to your Python installation
(Asking a question in order to answer it, having been initially puzzled by this. Other answers obviously welcome.)
Using #!/usr/bin/env python is a common trick to allow the python interpreter to be found using a PATH lookup, rather than hard-coding the path to python. It could be convenient to be adapt this to add a PATH=... argument to env, in order to hard-code a list of candidate directories while not hard-coding a single exact path. (This would make use of the fact that env uses the specified PATH variable when locating python, aside from passing it to the python process.)
For example (in test.py):
#!/usr/bin/env PATH=/opt/python/bin:/usr/bin:/bin python
import sys
print(sys.executable) # show which executable it found
But if you try this, although the command works if executed explicitly from the shell command line:
$ echo $PATH
/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
$ /usr/bin/env PATH=/opt/python/bin:/usr/bin:/bin python ./test.py
/opt/python/bin/python
it fails when trying to run it via the shebang:
$ ./test.py
[hangs - infinite loop]
Why is this, and what can be done instead?
Regarding why it fails, it turns out that this is an example of the problem described here.
As regards what can be done about it, it is possible to adapt various this workaround to a similar problem, namely how to pass an argument to python in #!/usr/bin/env python (which although it is not exactly the same problem, also relates to wanting to have more than two items in the shebang line).
This gives:
#!/bin/sh
''''export PATH=/opt/python/bin:/usr/bin:/bin; exec python "$0" # '''
import sys
print(sys.executable)
Invoking it via the shebang, we get:
$ ./test.py
/opt/python/bin/python
Solutions involving #!/usr/bin/env -S PATH=... python (GNU coreutils >= 8.30) still appear not to be sufficiently portable (e.g. Ubuntu 18.04.4, which was the latest LTS release until April 2020, has coreutils 8.28).
I'm writing a simple script which ideally will help me conveniently change directories around my system.
The details of the implementation don't matter, but let's say ideally I will place this script in /usr/bin and call it with an argument denoting where I want to go to on the system: goto project1
I would expect that when the script exits, my terminal's current working would have changed to that of Project 1.
In order to accomplish this, I tried:
os.chdir('/')
subprocess.call('cd /', shell=True)
Neither of which work. The first changes the working directory in Python and the second spawns a shell at /.
Then I realized how naive I was being. When a program is run, the terminal is just forking a process, while reading from stdout, which my program is writing to. Whatever it does, it wouldn't affect the state of terminal.
But then I thought "I've been using cd for years, surely someone wrote code for that", thinking there might be something to go off of (system call or something?).
But cd is not even coreutils. Instead, the source of cd is this:
builtin `echo ${0##*/} | tr \[:upper:] \[:lower:]` ${1+"$#"}
So, a couple of questions come to mind:
What's actually going on behind the scenes when a user calls cd? (Meaning, how is the terminal and the system actually interacting?)
Is it possible to have something like a Python script alter the terminal location?
Thanks so much for your help!
You could do it as a pair of scripts:
directory.py
#!/usr/bin/python
import sys
directory = sys.argv[1]
# do something interesting to manipulate directory...
print directory + "tmp"
directory.csh
#!/bin/csh -f
cd `python directory.py $1`
Result:
> pwd
/Users/cdl
> source directory.csh "/"
> pwd
/tmp
>
Substitute your favorite shell and variant of Python as desired. Turn on execute for the Python script to simplify further.
Clearly the shell is changing the directory but Python can do all the clever logic you want to figure out where to send the shell.
I'm python developer and most frequently I use buildout for managing my projects. In this case I dont ever need to run any command to activate my dependencies environment.
However, sometime I use virtualenv when buildout is to complicated for this particular case.
Recently I started playing with ruby. And noticed very useful feature. Enviourement is changing automatically when I cd in to the project folder. It is somehow related to rvm nad .rvmrc file.
I'm just wondering if there are ways to hook some script on different bash commands. So than I can workon environment_name automatically when cd into to project folder.
So the logic as simple as:
When you cd in the project with folder_name, than script should run workon folder_name
One feature of Unix shells is that they let you create shell functions, which are much like functions in other languages; they are essentially named groups of commands. For example, you can write a function named mycd that first runs cd, and then runs other commands:
function mycd () {
cd "$#"
if ... ; then
workon environment
fi
}
(The "$#" expands to the arguments that you passed to mycd; so mycd /path/to/dir will call cd /path/to/dir.)
As a special case, a shell function actually supersedes a like-named builtin command; so if you name your function cd, it will be run instead of the cd builtin whenever you run cd. In that case, in order for the function to call the builtin cd to perform the actual directory-change (instead of calling itself, causing infinite recursion), it can use Bash's builtin builtin to call a specified builtin command. So:
function cd () {
builtin cd "$#" # perform the actual cd
if ... ; then
workon environment
fi
}
(Note: I don't know what your logic is for recognizing a project directory, so I left that as ... for you to fill in. If you describe your logic in a comment, I'll edit accordingly.)
I think you're looking for one of two things.
autoenv is a relatively simple tool that creates the relevant bash functions for you. It's essentially doing what ruakh suggested, but you can use it without having to know how the shell works.
virtualenvwrapper is full of tools that make it easier to build smarter versions of the bash functions—e.g., switch to the venv even if you cd into one of its subdirectories instead of the base, or track venvs stored in git or hg, or … See the Tips and Tricks page.
The Cookbook for autoenv, shows some nifty ways ways to use the two together.
Just found in the description of virtualenvwraper this topic
It describes exactly what I need.
What i'd like to have is a mechanism that all commands i enter on a Bash-Terminal are wrapped by a Python-script. The Python-script executes the entered command, but it adds some additional magic (for example setting "dynamic" environment variables).
Is that possible somehow?
I'm running Ubuntu and Debian Squeezy.
Additional explanation:
I have a property-file which changes dynamically (some scripts do alter it at any time). I need the properties from that file as environment variables in all my shell scripts. Of course i could parse the property-file somehow from shell, but i prefer using an object-oriented style for that (especially for writing), as it can be done with Python (and ConfigObject).
Therefore i want to wrap all my scripts with that Python script (without having to modify the scripts themselves) which handles these properties down to all Shell-scripts.
This is my current use case, but i can imagine that i'll find additional cases to which i can extend my wrapper later on.
The perfect way to wrap every command that is typed into a Bash Shell is to change the variable PROMPT_COMMAND inside the .bashrc. For example, if I want to do some Python stuff before every command, liked asked in my question:
.bashrc:
# ...
PROMPT_COMMAND="python mycoolscript.py; $PROMPT_COMMAND;"
export $PROMPT_COMMAND
# ...
now before every command the script mycoolscript.py is run.
Use Bash's DEBUG trap. Let me know if you need me to elaborate.
Edit:
Here's a simple example of the kinds of things you might be able to do:
$ cat prefix.py
#!/usr/bin/env python
print "export prop1=foobar"
print "export prop2=bazinga"
$ cat propscript
#!/bin/bash
echo $prop1
echo $prop2
$ trap 'eval "$(prefix.py)"' DEBUG
$ ./propscript
foobar
bazinga
You should be aware of the security risks of using eval.
I don't know of anything but two things that might help you follow
http://sourceforge.net/projects/pyshint/
The iPython shell has some functionality to execute shell commands in the iterpreter.
There is no direct way you can do it .
But you can make a python script to emulate a bash terminal and you can use the beautiful "Subprocess" module in python to execute commnands the way you like