Is it possible to run a python script with parameters in command line like this:
./hello(var=True)
or is it mandatory to do like this:
python -c "from hello import *;hello(var=True)"
The first way is shorter and simpler.
Most shells use parentheses for grouping or sub-shells. So you can't call any commands like command(arg) from a normal shell ...but you can write a python script (./hello.py) that takes an argument.
import optparse
parser = optparse.OptionParser()
parser.add_option('-f', dest="f", action="store_true", default=False)
options, remainder = parser.parse_args()
print ("Flag={}".format(options.f))
And the call it with python hello.py -f
./hello(var=True) would be impossible from REPL shell. In some case it could be useful to have python function available in your current shell session. Here a workaround to make your python functions available in your shell environment.
# python-tools.sh
#!/usr/bin/env bash
set -a # make all available export all variable)
function hello(){
cd "/app/python/commands"
python "test.py" $#
}
Content of the python script
#! /usr/bin/env python
# /app/python/commands/test.py script
import sys
def test(*args):
print(args)
if __name__ == '__main__':
if sys.argv[1] in globals().keys():
print(sys.argv[1])
globals()[sys.argv[1]](sys.argv[2:])
else:
print("%s Not known function" % sys.argv[1])
Then source python-tools.sh
source python-tools.sh
After the hello function is available
$ hello test arg2 arg2
test
(['arg2', 'arg2'],)
Related
I know that I can run a python script from my bash script using the following:
python python_script.py
But what about if I wanted to pass a variable / argument to my python script from my bash script. How can I do that?
Basically bash will work out a filename and then python will upload it, but I need to send the filename from bash to python when I call it.
To execute a python script in a bash script you need to call the same command that you would within a terminal. For instance
> python python_script.py var1 var2
To access these variables within python you will need
import sys
print(sys.argv[0]) # prints python_script.py
print(sys.argv[1]) # prints var1
print(sys.argv[2]) # prints var2
Beside sys.argv, also take a look at the argparse module, which helps define options and arguments for scripts.
The argparse module makes it easy to write user-friendly command-line interfaces.
Use
python python_script.py filename
and in your Python script
import sys
print sys.argv[1]
Embedded option:
Wrap python code in a bash function.
#!/bin/bash
function current_datetime {
python - <<END
import datetime
print datetime.datetime.now()
END
}
# Call it
current_datetime
# Call it and capture the output
DT=$(current_datetime)
echo Current date and time: $DT
Use environment variables, to pass data into to your embedded python script.
#!/bin/bash
function line {
PYTHON_ARG="$1" python - <<END
import os
line_len = int(os.environ['PYTHON_ARG'])
print '-' * line_len
END
}
# Do it one way
line 80
# Do it another way
echo $(line 80)
http://bhfsteve.blogspot.se/2014/07/embedding-python-in-bash-scripts.html
use in the script:
echo $(python python_script.py arg1 arg2) > /dev/null
or
python python_script.py "string arg" > /dev/null
The script will be executed without output.
I have a bash script that calls a small python routine to display a message window. As I need to use killall to stop the python script I can't use the above method as it would then mean running killall python which could take out other python programmes so I use
pythonprog.py "$argument" & # The & returns control straight to the bash script so must be outside the backticks. The preview of this message is showing it without "`" either side of the command for some reason.
As long as the python script will run from the cli by name rather than python pythonprog.py this works within the script. If you need more than one argument just use a space between each one within the quotes.
and take a look at the getopt module.
It works quite good for me!
Print all args without the filename:
for i in range(1, len(sys.argv)):
print(sys.argv[i])
I have a csh script, which is a wrapper to send a python script into a grid engine. This works find as long as I don't use argument which contain quotes.
The csh 'my.csh script looks like:
#!/bin/csh -f
<grid command> my_script.py $*
my_script.py looks like:
#!<path>/python3.6
from argparse import ArgumentParser
def setup_arg_parser():
parser = ArgumentParser()
parser.add_argument('-argument',help='Just an example.')
args = parser.parse_args()
return args
if __name__ == '__main__':
args = setup_arg_parser()
print (args)
When call the my.csh as follows it works fine
%> my.csh -argument \"-anything -is -fine\"
but when I do the folling the quotes will be removed and the python argument parser reports an error.
%> my.csh -argument "-this -does_not -work"
Is there a way to prevent this without escaping the quotes ? What I need is a way provide the quotes to the next script, here it is a python script.
So to be more precise, what I am trying to do is :
read a full shell command as argument of my python script like : python myPythonScript.py ls -Fl
Call that command within my python script when I'd like to (Make some loops on some folders and apply the command etc ...)
I tried this :
import subprocess
from optparse import OptionParser
from subprocess import call
def execCommand(cmd):
call(cmd)
if __name__ == '__main__':
parser = OptionParser()
(options,args) = parser.parse_args()
print args
execCommand(args)
The result is that now I can do python myPythonScript.py ls , but I don't know how to add options. I know I can use parser.add_option , but don't know how to make it work for all options as I don't want to make only specific options available, but all possible options depending on the command I am running.
Can I use something like parser.add_option('-*') ? How can I parse the options then and call the command with its options ?
EDIT
I need my program to parse all type of commands passed as argument : python myScript.py ls -Fl , python myScript.py git pull, python myScript rm -rf * etc ...
OptionParser is useful when your own program wants to process the arguments: it helps you turn string arguments into booleans or integers or list items or whatever. In your case, you just want to pass the arguments on to the program you're invoking, so don't bother with OptionParser. Just pass the arguments as given in sys.argv.
subprocess.call(sys.argv[1:])
Depending on how much your program depends on command line arguments, you can go with simple route.
Simple way of reading command line arguments
Use sys to obtain all the arguments to python command line.
import sys
print sys.argv[1:]
Then you can use subprocess to execute it.
from subprocess import call
# e.g. call(["ls", "-l"])
call(sys.argv[1:])
This sample below works fine for me.
import sys
from subprocess import call
print(sys.argv[1:])
call(sys.argv[1:])
How to get shell tab completion cooperating with argparse in a Python script?
#!/usr/bin/env python
import argparse
def main(**args):
pass
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('positional', choices=['spam', 'eggs'])
parser.add_argument('--optional', choices=['foo1', 'foo2', 'bar'])
args = parser.parse_args()
main(**vars(args))
With an executable flag set on the .py file, the expected results should be something like:
$ ./example.py sp<tab>
-> completes to "./example.py spam"
$ ./example.py --op<tab>
-> completes to "./example.py --optional"
$ ./example.py --optional b<tab>
-> completes to "./example.py --optional bar"
$ ./example.py --optional f<tab>
-> completes to "./example.py --optional foo"
and, additionally, prints "foo1 foo2" choices on stdout on a new line
Have a look at argcomplete by Andrey Kislyuk.
Install it with:
pip install argcomplete
Import the module and add one line in your source before calling parser.parse_args():
#!/usr/bin/env python
import argparse as ap
import argcomplete
def main(**args):
pass
if __name__ == '__main__':
parser = ap.ArgumentParser()
parser.add_argument('positional', choices=['spam', 'eggs'])
parser.add_argument('--optional', choices=['foo1', 'foo2', 'bar'])
argcomplete.autocomplete(parser)
args = parser.parse_args()
main(**vars(args))
and to make sure that bash knows about this script, you use
eval "$(register-python-argcomplete your_script)"
you should put that line in your ~/.bashrc or follow argcomplete's docs and activate 'global' completion.
After that you completion works as requested.
The way this works is that the eval line creates a function _python_argcomlete which is registered using complete. (Run register-python-argcomplete your_script to just have a look at what gets eval-ed into bash). The autocomplete function looks for environment variables set by the bash completion mechanism to see if it needs to act. If it acts, it exits the program. If it doesn't act, this is a normal call to the program that function does nothing and the normal flow of the program continues.
For auto-complete to work you need a bash function to generate the possible options, and then you need to run complete -F <function_name> <program_name>
The best way of doing this is to have the program generate the completion function based on it's own parsing algorithm to avoid duplication. However, at a quick glance on argparse, I could not find a way to access it's internal structure, but I suggest you look for it.
Here is a bash function that will do for the above program:
function _example_auto() {
local cur=${COMP_WORDS[COMP_CWORD]}
local prev=${COMP_WORDS[COMP_CWORD-1]}
case "$prev" in
--optional )
COMPREPLY=( $(compgen -W "foo1 foo2 bar" -- $cur) )
return 0
;;
*)
COMPREPLY=( $(compgen -W "--optional spam eggs" -- $cur) )
return 0
;;
esac
}
I have tested optcomplete working with the optparse module. Its example is a simple file so I could get that working. I also tested it using the argparse module as the prior one is deprecated. But I really do not understand how and by whom the python program gets called on tab presses. I suspect bash together with the shebang line and the argparse (or optparse) module are involved in some way. I have been trying to figure this out (now gonna read the source code).
I have a little more complex program structure, which includes a wrapper around the piece of code which handles the arguments. Its argparse.ArgumentParser() instantiation and calls to add_argument() - which are superclassed into another intermediate module to avoid duplicating code, and wrapper around that is being called - are inside a function.
I want to understand the way this tab completion works between bash and python (or for that matter any other interpretor like perl).
NOTE: I have a fair understanding of bash completion (which I learned just now), and I think I understand the bash(only) custom completion.
NOTE: I have read other similar SO questions, and none really answer this Q.
Edit: Here is the bash function.
I already understood how the python module gets to know about words typed in the command line, by reading os.environ values of variables
$COMP_WORDS
$COMP_CWORD
$COMP_LINE
$COMP_POINT
$COMPREPLY
These variables have values only on tab press.
My question is how does the python module gets triggered?
To understand what's happening here, let's check what that bash function actually does:
COMPREPLY=( $( \
COMP_LINE=$COMP_LINE COMP_POINT=$COMP_POINT \
COMP_WORDS="${COMP_WORDS[*]}" COMP_CWORD=$COMP_CWORD \
OPTPARSE_AUTO_COMPLETE=1 $1 ) )
See the $1 at the end? That means that it actually calls the Python file we want to execute with special environment variables set! To trace what's happening, let's prepare a little script to intercept what optcomplete.autocomplete does:
#!/usr/bin/env python2
import os, sys
import optparse, optcomplete
from cStringIO import StringIO
if __name__ == '__main__':
parser = optparse.OptionParser()
parser.add_option('-s', '--simple', action='store_true',
help="Simple really simple option without argument.")
parser.add_option('-o', '--output', action='store',
help="Option that requires an argument.")
opt = parser.add_option('-p', '--script', action='store',
help="Option that takes python scripts args only.")
opt.completer = optcomplete.RegexCompleter('.*\.py')
# debug env variables
sys.stderr.write("\ncalled with args: %s\n" % repr(sys.argv))
for k, v in sorted(os.environ.iteritems()):
sys.stderr.write(" %s: %s\n" % (k, v))
# setup capturing the actions of `optcomplete.autocomplete`
def fake_exit(i):
sys.stderr.write("autocomplete tried to exit with status %d\n" % i)
sys.stdout = StringIO()
sys.exit = fake_exit
# Support completion for the command-line of this script.
optcomplete.autocomplete(parser, ['.*\.tar.*'])
sys.stderr.write("autocomplete tried to write to STDOUT:\n")
sys.stderr.write(sys.stdout.getvalue())
sys.stderr.write("\n")
opts, args = parser.parse_args()
This gives us the following when we try to autocomplete it:
$ ./test.py [tab]
called with args: ['./test.py']
...
COMP_CWORD: 1
COMP_LINE: ./test.py
COMP_POINT: 10
COMP_WORDS: ./test.py
...
OPTPARSE_AUTO_COMPLETE: 1
...
autocomplete tried to exit with status 1
autocomplete tried to write to STDOUT:
-o -h -s -p --script --simple --help --output
So optcomplete.autocomplete just reads the environment, prepares the matches, writes them to STDOUT and exits. The result -o -h -s -p --script --simple --help --output is then put into a bash array (COMPREPLY=( ... )) and returned to bash to present the choices to the user. No magic involved :)