We Keep Coding

How to parse arguments in the style of git or conda in python

At least a few current command line tools follow the general pattern of:
$> program-name action [zero or more action specific options & arguments
For example
$> git checkout -b new-branch
$> git diff HEAD HEAD~2
... and so on ...
The idea is that we have different actions/commands (checkcout, diff, commit etc.) and the arguments/options for each command can be different.
How do you implement this pattern in python (ideally using argparse)?

(FTFM!) This is directly supported by argparse
>>> # create the top-level parser
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo', action='store_true', help='foo help')
>>> subparsers = parser.add_subparsers(help='sub-command help')
>>>
>>> # create the parser for the "a" command
>>> parser_a = subparsers.add_parser('a', help='a help')
>>> parser_a.add_argument('bar', type=int, help='bar help')
>>>
>>> # create the parser for the "b" command
>>> parser_b = subparsers.add_parser('b', help='b help')
>>> parser_b.add_argument('--baz', choices='XYZ', help='baz help')
>>>
>>> # parse some argument lists
>>> parser.parse_args(['a', '12'])
Namespace(bar=12, foo=False)
>>> parser.parse_args(['--foo', 'b', '--baz', 'Z'])
Namespace(baz='Z', foo=True)

python get changed (non-default) cli arguments?

Given an argument parser with n arguments, where I change the default value of only a small subset every run from the command line, is there a clean way of extracting a dict/namespace of all the non-default k,v arguments?
parser = argparse.ArgumentParser()
parser.add_argument("--a",type=str,default='a')
parser.add_argument("--b",type=str,default='b')
parser.add_argument("--c",type=str,default='c')
parser.add_argument("--d",type=str,default='d')
And
python run.py --a "e"
I would like to have
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--a",type=str,default='a')
parser.add_argument("--b",type=str,default='b')
parser.add_argument("--c",type=str,default='c')
parser.add_argument("--d",type=str,default='d')
non_default = parse_non_default(parser) # non_default = {'a':'e'}

You could lookup the parser and compare which values differenciate:
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--a", type=str, default='a')
parser.add_argument("--b", type=str, default='b')
parser.add_argument("--c", type=str, default='c')
parser.add_argument("--d", type=str, default='d')
parser.add_argument("--n", type=int, default=999)
args = parser.parse_args(['--a', 'e']) # Test CLI arguments!
non_default = {
opt.dest: getattr(args, opt.dest)
for opt in parser._option_string_actions.values()
if hasattr(args, opt.dest) and opt.default != getattr(args, opt.dest)
}
print(non_default)
main()
Out:
{'a': 'e'}

argparse - how to pass argument from args into function?

import argparse
from queries import most_common_cities
parser = argparse.ArgumentParser(description='A script that does operations with database data and returns values')
parser.add_argument('-c', '--most_common_cities',
nargs=1,
type=positive_int,
help='Specify how many common cities.')
args = parser.parse_args()
if args.most_common_cities:
result = most_common_cities(n) # "n" should be an arg passed by user
print(result)
How could I pass arguments from CLI to my function arg?
When someone use command:
python argp.py --most_common_cities 5
It should return 5 most common cities.

Remove nargs=1, then args.most_common_cities will be the actual value passed in.
nargs=1 wraps it in a list.
parser.add_argument('-c', '--most_common_cities',
type=int,
help='Specify how many common cities.')
args = parser.parse_args(['-c', '5'])
n = args.most_common_cities
print(n)
print(type(n))
# 5
# <class 'int'>

I started your script with following command:
python3 test.py --most_common_cities 5
You can access the arguments with:
import argparse
parser = argparse.ArgumentParser(description='A script that does operations with database data and returns values')
parser.add_argument('-c', '--most_common_cities',
nargs=1,
type=int,
help='Specify how many common cities.')
args = parser.parse_args()
arguments = vars(parser.parse_args())
print(arguments) #{'most_common_cities': [5]}
#then you can access the value with:
arguments['most_common_cities']

Python argparse, run one or more sub-commands

I'm trying to write a program that is able to execute multiple sub-commands. The argparse module is very helpful, but I think it is lacking the ability to specify more than one sub-command. For example, if I have the following code:
parser = argparse.ArgumentParser(prog='My Prog')
sub_parsers = parser.add_subparsers()
subcommand_a = sub_parsers.add_parser('subcommand_a', help='a help')
subcommand_a.add_argument('req1', help='required argument 1 help')
subcommand_a.add_argument('--opt1', help='option 1 help')
subcommand_a.add_argument('--opt2', nargs='+', help='option 2 help')
subcommand_b = sub_parsers.add_parser('subcommand_b', help='b help')
subcommand_b.add_argument('req1', help='required argument 1 help')
subcommand_b.add_argument('--opt1', help='option 1 help')
subcommand_b.add_argument('--opt2', help='option 2 help')
subcommand_b.add_argument('--opt3', nargs='+', help='option 3 help')
parser.parse_args()
I cannot specify both subcommand_a and subcommand_b on the command line. I can only do one of them at a time. I'd imagine that this would require a custom action or possibly even subclassing argparse, but I'm not sure where to start. I'd like to be able to call this program like the following:
./prog.py subcommand_a FOO --opt1=bar --opt2 1 2 3 -- subcommand_b BAR --opt1='foo' --opt3 a b c --
Any ideas?

Your problem is essentially the same as the one asked a few months back
Multiple invocation of the same subcommand in a single command line
That one wanted to call the same subcommand several times, but the issue the same - how to handle more than one subparsers argument.
The solutions there are either to split the command line before passing it in pieces that are parsed separately, or collect the 'unused' pieces from one parsing for using in a second or third.
Here's tweak to your code that returns 2 commands:
import argparse
parser = argparse.ArgumentParser(prog='My Prog')
sub_parsers = parser.add_subparsers(dest='cmd')
subcommand_a = sub_parsers.add_parser('subcommand_a', help='a help')
subcommand_a.add_argument('req1', help='required argument 1 help')
subcommand_a.add_argument('--opt1', help='option 1 help')
subcommand_a.add_argument('--opt2', nargs=3, help='option 2 help')
subcommand_b = sub_parsers.add_parser('subcommand_b', help='b help')
subcommand_b.add_argument('req1', help='required argument 1 help')
subcommand_b.add_argument('--opt1', help='option 1 help')
subcommand_b.add_argument('--opt2', help='option 2 help')
subcommand_b.add_argument('--opt3', nargs=3, help='option 3 help')
argv = "subcommand_a FOO --opt1=bar --opt2 1 2 3 subcommand_b BAR --opt1=foo --opt3 a b c"
rest = argv.split()
while rest:
[args, rest] = parser.parse_known_args(rest)
print args
print rest
which prints:
Namespace(cmd='subcommand_a', opt1='foo', opt2=['1', '2', '3'], req1='FOO')
['subcommand_b', 'BAR', '--opt3', 'a', 'b', 'c']
Namespace(cmd='subcommand_b', opt1=None, opt2=None, opt3=['a', 'b', 'c'], req1='BAR')
[]
I removed the '--' (see the end of my other answer)
I changed opt2 and opt3 to take 3 arguments, not the variable +. With + it can't tell where the list for opt2 ends and the next command begins.
It is also important that the different commands take different optionals (flags). Note that the first opt1 ends up with the second value, 'foo', leaving none for the 2nd command. See the other thread for a discussion of the issues and ways around that.

I've done some testing and changing the subcommand to other string worked.
if you use:
parser = argparse.ArgumentParser(prog='My Prog')
sub_parsers = parser.add_subparsers()
subcommand_a = sub_parsers.add_parser('subcommand_a', help='a help')
subcommand_a.add_argument('req1', help='required argument 1 help')
subcommand_a.add_argument('--opt1', help='option 1 help')
subcommand_a.add_argument('--opt2', nargs='+' help='option 2 help')
subcommand_b = sub_parsers.add_parser('subcommand_b', help='b help')
subcommand_b.add_argument('req1', help='required argument 1 help')
subcommand_b.add_argument('--opt3', help='option 1 help')
subcommand_b.add_argument('--opt4', help='option 2 help')
subcommand_b.add_argument('--opt5', nargs='+', help='option 3 help')
parser.parse_args()
it works.
I've done some research and not found a subcommand with a equal string. A overview of the documentation don't say nothing either. I presume it's a limitation ( by choice ) or a bug.
Maybe you can check it in the Source Code Hosting

Option accepted with and without value

I have a small script and I need it to be able to accept parameter with value and withou value.
./cha.py --pretty-xml
./cha.py --pretty-xml=5
I have this.
parser.add_argument('--pretty-xml', nargs='?', dest='xml_space', default=4)
But when I use --pretty-xml in xml_space will be 'none'. If I dont write this parameter in xml_space is stored the default value. I would need the exact opposite.

Use the const keyword:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--pretty-xml", nargs="?", type=int, dest="xml_space", const=4)
print(parser.parse_args([]))
print(parser.parse_args(['--pretty-xml']))
print(parser.parse_args(['--pretty-xml=5']))
results in
Namespace(xml_space=None)
Namespace(xml_space=4)
Namespace(xml_space=5)

Leave out the default parameter and use a custom Action instead:
class PrettyXMLAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if not values:
values = 4
setattr(namespace, self.dest, values)
parser.add_argument('--pretty-xml', nargs='?', type=int, dest='xml_space', action=PrettyXMLAction)
Demo:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--pretty-xml', nargs='?', type=int, dest='xml_space', action=PrettyXMLAction)
PrettyXMLAction(option_strings=['--pretty-xml'], dest='xml_space', nargs='?', const=None, default=None, type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args('--pretty-xml'.split())
Namespace(xml_space=4)
>>> parser.parse_args('--pretty-xml=5'.split())
Namespace(xml_space=5)
>>> parser.parse_args(''.split())
Namespace(xml_space=None)