We Keep Coding

How to hide args to argparse when running script with exec?

I have 2 files, runner.py that runs target.py with subprocess or exec.
They both have command line options.
If runner runs target with subprocess it's ok:
$ python runner.py
run target.py with subprocess...
target.py: running with dummy = False
If runner runs target code with exec (with the -e option):
$ python runner.py -e
run target.py with exec...
usage: runner.py [-h] [-d]
runner.py: error: unrecognized arguments: -e
the command line argument -e is "seen" by target.py code (which accepts only one --dummy option) and raises an error.
How can I hide args to argparse when running script with exec?
Here's the code:
runner.py
import subprocess
import argparse
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("-e", "--exec", help="run with exec", action="store_true")
args = parser.parse_args()
target_filename = "target.py"
if args.exec:
print("run target.py with exec...")
source_code = open(target_filename).read()
compiled = compile(source_code, filename=target_filename, mode="exec")
exec(compiled) # OPTION 1 - error on argparse
# exec(compiled, {}) # OPTION 2 - target does not go inside "if main"
# exec(compiled, dict(__name__="__main__")) # OPTION 3 - same error as OPTION 1
else:
print("run target.py with subprocess...")
subprocess.run(["python3", target_filename])
I tried to hide the globals with the commented options above, but without luck.
Seems related to how argparse works.
target.py
import argparse
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("-d", "--dummy", help="a dummy option", action="store_true")
args = parser.parse_args()
print(f"target.py: running with dummy = {args.dummy}")

There is the argparse conflict_handler option so one can write
argparse.ArgumentParser(conflict_handler='resolve') in target script.
resolve removes the conflicting options, but that doesn't handle well similar cases where the options have the same name both in runner and target or the case where you can't or don't want to change the target file.
Here's the solution I have found.
Internally argparse uses sys.argv to retrieve options set with command line.
You can directly set sys.argv = [target_filename] which removes the options, but changing sys can give a lot of other problems.
Using unittest.mock.patch (python3.4+) the sys.argv can be securely altered like this:
from unittest.mock import patch
# [...]
source_code = open(target_filename).read()
compiled = compile(source_code, filename=target_filename, mode="exec")
# remove command args
with patch('sys.argv', [target_filename]):
exec(compiled)
So one can also run target script code with options:
# run target
with patch('sys.argv', [target_filename]):
exec(compiled)
# run target with -d
with patch('sys.argv', [target_filename, "-d"]):
exec(compiled)

Python : extract parameters created with argparse

I have a file called simple_example.py, which consists of 2 functions:
# import the necessary packages
import argparse
class simple:
#staticmethod
def func1():
# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-n", "--name", help="name of the user", default='host')
ap.add_argument('-num', '--number', required=True, help='choose a number')
args = vars(ap.parse_args())
# display a friendly message to the user
print("Hi there {}, it's nice to meet you! you chose {}".format(args['name'], args['age']))
#staticmethod
def func2():
# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-n", "--name", help="name of the user", default='host')
ap.add_argument('-num', '--number', required=True, help='choose a number')
ap.add_argument("-g", "--greet", help="say greetings", default='hello')
args = vars(ap.parse_args())
# display a friendly message to the user
print("{} there {}, it's nice to meet you! you chose {}".format(args['greet'], args['name'], args['age']))
I'd like to be able to call either func1() or func2() from the command line, so, I created another file called pyrun.py from this link
# !/usr/bin/env python
# make executable in bash chmod +x PyRun
import sys
import inspect
import importlib
import os
if __name__ == "__main__":
cmd_folder = os.path.realpath(os.path.abspath(os.path.split(inspect.getfile(inspect.currentframe()))[0]))
if cmd_folder not in sys.path:
sys.path.insert(0, cmd_folder)
# get the second argument from the command line
methodname = sys.argv[1]
# split this into module, class and function name
modulename, classname, funcname = methodname.split(".")
# get pointers to the objects based on the string names
themodule = importlib.import_module(modulename)
theclass = getattr(themodule, classname)
thefunc = getattr(theclass, funcname)
# pass all the parameters from the third until the end of what the function needs & ignore the rest
args = inspect.getargspec(thefunc)
print(args)
However, args in ArgSpec(args=[], varargs=None, keywords=None, defaults=None) shows an empty list.
How can I extract the parameters from either func1 or func2?
Is there a better way to run either func1 or func2 from the command line?

You probably want to use sub-commands. Here is an implementation of your example using sub-commands.
import argparse
def func1(args):
print("Hi there {}, it is nice to meet you! You chose {}.".format(args.name, args.number))
def func2(args):
print("{} there {}, it is nice to meet you! You chose {}.".format(args.greet, args.name, args.number))
#
# The top-level parser
#
parser = argparse.ArgumentParser('top.py', description='An example sub-command implementation')
#
# General sub-command parser object
#
subparsers = parser.add_subparsers(help='sub-command help')
#
# Specific sub-command parsers
#
cmd1_parser = subparsers.add_parser('cmd1', help='The first sub-command')
cmd2_parser = subparsers.add_parser('cmd2', help='The second sub-command')
#
# Assign the execution functions
#
cmd1_parser.set_defaults(func=func1)
cmd2_parser.set_defaults(func=func2)
#
# Add the common options
#
for cmd_parser in [cmd1_parser, cmd2_parser]:
cmd_parser.add_argument('-n', '--name', default='host', help='Name of the user')
cmd_parser.add_argument('-num', '--number', required=True, help='Number to report')
#
# Add command-specific options
#
cmd2_parser.add_argument('-g', '--greet', default='hello', help='Greeting to use')
#
# Parse the arguments
#
args = parser.parse_args()
#
# Invoke the function
#
args.func(args)
Example output:
$ python ./top.py cmd1 -n Mark -num 3
Hi there Mark, it is nice to meet you! You chose 3.
$ python ./top.py cmd2 -n Bob -num 7 -g Hello
Hello there Bob, it is nice to meet you! You chose 7.
And, of course, the help functions work for each of the sub-commands.
$ python ./top.py cmd2 -h
usage: top.py cmd2 [-h] [-n NAME] -num NUMBER [-g GREET]
optional arguments:
-h, --help show this help message and exit
-n NAME, --name NAME Name of the user
-num NUMBER, --number NUMBER
Number to report
-g GREET, --greet GREET
Greeting to use

If I put your first block of code in a file, I can import it into a ipython session and run your 2 functions:
In [2]: import stack49311085 as app
In [3]: app.simple
Out[3]: stack49311085.simple
ipython tab expansion (which uses some form of inspect) shows me that the module has a simple class, and the class itself has two static functions.
I can call func1, and get an argparse error message:
In [4]: app.simple.func1()
usage: ipython3 [-h] [-n NAME] -num NUMBER
ipython3: error: the following arguments are required: -num/--number
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
Similarly for func2:
In [7]: app.simple.func2()
usage: ipython3 [-h] [-n NAME] -num NUMBER [-g GREET]
ipython3: error: the following arguments are required: -num/--number
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
parse_args as a default parses the sys.argv[1:] list, which obviouslly is not tailored to its requirements.
def foo(argv=None):
parser = ....
....
args = parse.parse_args(argv=argv)
return args
is a more useful wrapper. With this I can pass a test argv list, and get back the parsed Namespace. If I don't give it such a list, it will used the sys.argv default. When testing a parser I like to return and/or display the whole Namespace.
I haven't used inspect enough to try to figure out what you are trying to do with it, or how to correct it. You don't need inspect to run code in an imported module like this.
I can test your imported parser by modifying the sys.argv
In [8]: import sys
In [9]: sys.argv
Out[9]:
['/usr/local/bin/ipython3',
'--pylab',
'--nosep',
'--term-title',
'--InteractiveShellApp.pylab_import_all=False']
In [10]: sys.argv[1:] = ['-h']
In [11]: app.simple.func2()
usage: ipython3 [-h] [-n NAME] -num NUMBER [-g GREET]
optional arguments:
-h, --help show this help message and exit
-n NAME, --name NAME name of the user
-num NUMBER, --number NUMBER
choose a number
-g GREET, --greet GREET
say greetings
An exception has occurred, use %tb to see the full traceback.
SystemExit: 0
Or following the help:
In [12]: sys.argv[1:] = ['-num=42', '-nPaul', '-gHI']
In [13]: app.simple.func2()
...
---> 30 print("{} there {}, it's nice to meet you! you chose {}".format(args['greet'], args['name'], args['age']))
KeyError: 'age'
Oops, there's an error in your code. You ask for args['age'], but didn't define a parser argument with that name. That's part of why I like to print the args Namespace` - to make sure it is setting all the attributes that I expect.
Normally we don't use different parsers for different inputs. It's possible to do that based on your own test of sys.avgv[1], but keep in mind that that string will still be on sys.argv[1:] list that your parser(s) read. Instead write one parser that can handle the various styles of input. The subparser mentioned in the other answer is one option. Another is to base your action on the value of the args.greet attribute. If not used it will be the default value.

python script envoke -h or --help if no options are chosen

Trying to make my script more generic so I added some flags. My problem is the help only works if you type -h , obviously. I want to envoke -h when no flags are selected.
For example:
python 0_log_cleaner.py
Traceback (most recent call last):
File "0_log_cleaner.py", line 51, in <module>
getFiles(options.path,options.org_phrase,options.new_phrase,options.org_AN,options.new_AN,options.dst_path)
File "0_log_cleaner.py", line 37, in getFiles
for filename in os.listdir(path):
TypeError: coercing to Unicode: need string or buffer, NoneType found
but if I add -h I get:
python 0_log_cleaner.py -h
Usage: Example:
python 0_log_cleaner.py --sp original_logs/ --dp clean_logs/ --od CNAME --nd New_CNAME --oan 10208 --nan NewAN
Options:
-h, --help show this help message and exit
--sp=PATH Path to the source logs ie original_logs/
--dp=DST_PATH Path to where sanitized logs will be written to ie
clean_logs
--od=ORG_PHRASE original domain name ie www.clientName.com, use the command
-od clientName
--nd=NEW_PHRASE domain name to replace -od. ie -od clientName -nd domain
makes all log that use to be www.clientName.com into
www.domain.com
--oan=ORG_AN original AN number
--nan=NEW_AN AN number to replace original. ie -oan 12345 -nan AAAA1
replaces all instances of the AN number 12345 with AAAA1
EDIT 3 ANSWER
sample of my code to produce ^
import argparse
import sys
usage = "Description of function"
parser = argparse.ArgumentParser(description=usage)
parser.add_argument("--sp", dest="path", help='Path to the source logs ie logs/')
...
...(additional add arugments)
args = parser.parse_args()
def getFiles(path,org_phrase,new_phrase,org_AN,new_AN,dst_path):
if not len(sys.argv) > 1:
parser.print_help()
else:
run your logic

borrowed from here : Argparse: Check if any arguments have been passed
Here's how the final code looks like:
import argparse
import sys
usage = "Description of function"
parser = argparse.ArgumentParser(description=usage)
parser.add_argument("--sp", dest="path", help='Path to the source logs ie logs/')
...
...(additional add arugments)
args = parser.parse_args()
def getFiles(path,org_phrase,new_phrase,org_AN,new_AN,dst_path):
if not len(sys.argv) > 1:
parser.print_help()
else:
run your logic

If someone is still interested in a (very simple) solution:
parser = argparse.ArgumentParser()
parser.add_argument("jfile", type=str, help="Give the JSON file name.")
parser.add_argument("--output", type=str, help="Type in the final excel files name.")
try:
args = parser.parse_args()
return args
except:
parser.print_help()
My professor wanted the script to force the -h / --help page even when there are too few arguments. Instead of going like "python SCRIPT.py -h".
So what I did here was like: "Try to parse the arguments. And if it works, give them back to the main methode. Otherwise, if you fail (except), print the help(). Okay? Nice". ;)

Without knowing the method you are parsing with, I will assume the following (comment me if I am wrong or edit your question with some code on how you handle your parsing):
You are parsing everything and putting it in a variable. let parsed be that variable.
You are checking parsed for the existence of any of your option flags.
You probably not checking for the non-existence of arguments:
parsed = '' <- empty string
# or if you are using a list:
# parsed = []
if parsed: <- if parsed is not empty ("" or []) returns true
Do your stuff here, because you have options now
else: <- Differently options were not provided
Invoke the same method that you invoke when the option is -h
Also as #dhke suggests, consider using argparse if you are not using it already!
EDIT #1:
Translated for your specific case:
args = parser.parse_args() <-- ending line of your provided code
if not args:
parser.print_help()
else:
Do your stuff

Set a default choice for optionparser when the option is given

I have a python option parsers that parses an optional --list-something option.
I also want the --list-something option to have an optional argument (an option)
Using the argument default="simple" does not work here, otherwise simple will always
be the default, not only when --list-something was given.
from optparse import OptionParser, OptionGroup
parser = OptionParser()
options = OptionGroup(parser, "options")
options.add_option("--list-something",
type="choice",
choices=["simple", "detailed"],
help="show list of available things"
)
parser.add_option_group(options)
opts, args = parser.parse_args()
print opts, args
The above code is producing this:
[jens#ca60c173 ~]$ python main.py --list-something simple
{'list_something': 'simple'} []
[jens#ca60c173 ~]$ python main.py --list-something
Usage: main.py [options]
main.py: error: --list-something option requires an argument
[jens#ca60c173 ~]$ python main.py
{'list_something': None} []
But I want this to hapen:
[jens#ca60c173 ~]$ python main.py --list-something simple
{'list_something': 'simple'} []
[jens#ca60c173 ~]$ python main.py --list-something
{'list_something': 'simple'} []
[jens#ca60c173 ~]$ python main.py
{'list_something': None} []
I would like something that works out of the box in python 2.4 up till 3.0 (3.0 not included)
Since argparse is only introduced in python 2.7 this is not something I could use.

Optparse does not have any options for doing this easily. Instead you'll have to create a custom callback for your option. The callback is triggered when your option is parsed, at which point, you can check the remaining args to see if the user put an argument for the option.
Check out the custom callback section of the docs, in particular Callback example 6: variable arguments.

There is no default for the Optparser in python.
However, you can use the follwing -
# show help as default
if len(sys.argv) == 1:
os.system(sys.argv[0] + " -h")
exit()
This will run the same script with the -h option, and exit.
please notice - you will need to import the sys module.

Need help with python script with bash commands

I copied this script from internet but idon't know how to use it. i am newbiw to python so please help. When i execute it using
./test.py then i can only see
usage: py4sa [option]
A unix toolbox
options:
--version show program's version number and exit
-h, --help show this help message and exit
-i, --ip gets current IP Address
-u, --usage gets disk usage of homedir
-v, --verbose prints verbosely
when i type py4sa then it says bash command not found
The full script is
#!/usr/bin/env python
import subprocess
import optparse
import re
#Create variables out of shell commands
#Note triple quotes can embed Bash
#You could add another bash command here
#HOLDING_SPOT="""fake_command"""
#Determines Home Directory Usage in Gigs
HOMEDIR_USAGE = """
du -sh $HOME | cut -f1
"""
#Determines IP Address
IPADDR = """
/sbin/ifconfig -a | awk '/(cast)/ { print $2 }' | cut -d':' -f2 | head -1
"""
#This function takes Bash commands and returns them
def runBash(cmd):
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
out = p.stdout.read().strip()
return out #This is the stdout from the shell command
VERBOSE=False
def report(output,cmdtype="UNIX COMMAND:"):
#Notice the global statement allows input from outside of function
if VERBOSE:
print "%s: %s" % (cmdtype, output)
else:
print output
#Function to control option parsing in Python
def controller():
global VERBOSE
#Create instance of OptionParser Module, included in Standard Library
p = optparse.OptionParser(description='A unix toolbox',
prog='py4sa',
version='py4sa 0.1',
usage= '%prog [option]')
p.add_option('--ip','-i', action="store_true", help='gets current IP Address')
p.add_option('--usage', '-u', action="store_true", help='gets disk usage of homedir')
p.add_option('--verbose', '-v',
action = 'store_true',
help='prints verbosely',
default=False)
#Option Handling passes correct parameter to runBash
options, arguments = p.parse_args()
if options.verbose:
VERBOSE=True
if options.ip:
value = runBash(IPADDR)
report(value,"IPADDR")
elif options.usage:
value = runBash(HOMEDIR_USAGE)
report(value, "HOMEDIR_USAGE")
else:
p.print_help()
#Runs all the functions
def main():
controller()
#This idiom means the below code only runs when executed from command line
if __name__ == '__main__':
main()

It seems to me you have stored the script under another name: test.py rather than py4sa. So typing ./test.py, like you did, is correct for you. The program requires arguments, however, so you have to enter one of the options listed under 'usage'.
Normally 'py4sa [OPTIONS]' would mean that OPTIONS is optional, but looking at the code we can see that it isn't:
if options.verbose:
# ...
if options.ip:
# ...
elif options.usage:
# ...
else:
# Here's a "catch all" in case no options are supplied.
# It will show the help text you get:
p.print_help()
Note that the program probably would not be recognized by bash even if you renamed it to py4sa, as the current directory is often not in bash's PATH. It says 'usage: py4sa (..)' because that's hard-coded into the program.

The script is called "test.py". Either invoke it as such, or rename it to "py4sa".

you run a Python script using the interpreter, so
$ python py4sa