We Keep Coding

Ordered and repeated flags using argparse

Trying to implement a CLI using argparse without writing my own parser if possible. Basically I would like to use the command line in order to invoke different methods sequentially in a program, i.e.:
my_program.py --doA --doB --doA --doA
I've seen threads with good ideas on how to obtain ordered arguments, but I suppose I would end up with a list like this ['doA', 'doB', 'doA', 'doB'] where I would still need to parse the arguments myself.
Is there any way to exploit tools in argparse in order to help execute these commands in order? Thanks!

Here's a parser that will collect these arguments repeatedly and in order:
In [1]: import argparse
In [2]: parser = argparse.ArgumentParser()
In [3]: parser.add_argument('--doA', dest='cmd', const='doA', action='append_con
...: st');
In [4]: parser.add_argument('--doB', dest='cmd', const='doB', action='append_con
...: st');
test:
In [5]: args = parser.parse_args('--doA --doB --doA --doA'.split())
In [6]: print(args)
Namespace(cmd=['doA', 'doB', 'doA', 'doA'])
do something:
In [7]: for action in args.cmd:
...: print('action:', action)
...:
action: doA
action: doB
action: doA
action: doA
If I had defined:
def doA(*args):
# do something
pass
and used const=doA, then the args.cmd list would be
Namespace(cmd=[doA, doB, ...])
and we could write
for action in args.cmd:
action(arguments)
argparse docs has something like this for subparses and the set_default command.
If these are the only arguments you accept this would be over kill. But if there are other options that need argparse this would fit in. But accepting a
parser.add_argument('--cmd', nargs='*', ...)
would be fine. It could use choices to restrict the input strings, and even a type function to translate the strings into function objects.

Command line arguments can separated into 3 groups:
Options - prefixed by - or -- and usually specify how the command should behave (--verbose) and may or may not take values, and typically the order does not matter
Sub commands - are one of a few program defined keywords telling to program to perform a certain subset of the program's full suite of functionality (pull in git pull), usually only one can be specified at a time
Arguments - Are the values usually passed at the end of the command, and specify the values to use during execution, and typically these values are simply consumed without any modification
Most argument parsers treat options as unordered and tend to simply set values to variables or properties in a namespace for the programmer to use. This makes ordered options generally not possible without writing a custom parser.
For your tool you just want to specify an ordered list of actions to take, so it might make more sense to pass them in as either values to an option, or simple arguments.
my_program.py --actions doA doB doC doD
my_program.py doA doB doC doD
Argparsers (like python's argparse) do not mutate the values given to options or values passed as arguments, meaning the order of the actions would be preserved.

Python argparse with choices and nargs=2

I am trying to create an option that takes two arguments. The first argument should be validated by choices, but the second argument is an arbitrary user-supplied value. For example.
> app -h
usage: app [--option {count,label} arg]
Correct usage examples:
> app --option count 1
> app --option count 912
> app --option label userfoo
I tried setting it up like this:
parser.add_argument('--option', choices=['count','label'], nargs=2)
This does not work, as it tries to validate BOTH arguments using the choices. The help string shows this:
usage: app [--option {count,label} {count,label}]
There are several ways I could do it manually:
remove the choices field and manually validate the first argument in code.
separate it into --option count --value 3, but that is awkward as --value is required by option but invalid without it. It is really a single option with two values
make --option have a compound value, for example --option count=3 and then parse the value
Part of what I want is to have the auto-generated help string show the choices for the first argument. I would also like for argparse to detect and report errors whenever possible with a minimum of custom code. The reason is that we have a very complex CLI, and this helps maintain consistency.
Is there a way to do this with argparse?

parser._get_values does, when nargs is number:
value = [self._get_value(action, v) for v in arg_strings]
for v in value:
self._check_value(action, v)
_get_value applies the type function, while _check_value test the choices. After this the values list is passed to the store Action.
So the normal processing applies the same type and choices test to each string.
I can imagine writing a type function that accepts both numbers and strings from a list. But it couldn't distinguish between the first and second arguments. A custom Action will see both, and could do further testing.
But often it's simpler to just do your own value testing after parsing. The parser doesn't have to do everything. It's primary function is to figure out what the user wants. Checking values and raising a standardized error is nice part of parsing, but isn't its primary purpose.
Also think about how you'd specify the usage/help for the expected input.

How can I most efficiently parse these arguments in python?

So I'm trying to come up with a strategy using the argparse library.
Here's how I want to interface with my program:
$ program list [<number>]
$ program check <name>
$ program watch <name> [<quality>]
Right now I have an argument parser like the following:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('list')
group.add_argument('check')
group.add_argument('watch')
But how can I add an optional argument, say an integer, to an existing argument?
Meaning the user could invoke the list command in the following ways:
program list
Where the list action would be called with a default value, or:
program list 10
Where the list action would be called with an argument of 10.
I saw the subcommands option in the documentation, but I ran into the problem where I would have a sub parser for list arguments, but then I would have to add a flag, such as -n and then provide the number. Perhaps this is a better way of doing it? But I like the idea of just providing the number if you want to, or omitting it if you don't.
Is what I'm trying to achieve good practice? Is it possible with argparse?

This sample set me off in the wrong direction. I've sketched a subparser implementation at the end that I think does the trick.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('list')
group.add_argument('check')
group.add_argument('watch')
It expects 3 strings, and will assign them to the 3 attributes.
However, you cannot put 3 'positional` arguments in a mutually exclusive group. One optional positional (yes the terminology is confusing) can be in such a group, but the rest must 'optionals' (flagged).
Going back to your initial list. Are these different patterns that you'd like to accept
program list [integer]
program check name
program watch name [quality]
where 'list','check','watch' are literal strings, while 'integer','name','quality' are variable names.
If that is the case, then subparsers is probably the best choice. nargs='?' can be used to make positional arguments 'optional'.
parser = argparse.ArgumentParser()
sp = parser.add_subparsers(dest='cmd') # put the command string in `cmd` attribute
p1 = sp.add_parser('list')
p2 = sp.add_parser('check')
p3 = sp.add_parser('watch')
p1.add_argument('value',dtype=int, nargs='?') # ok with 0 or 1 values
p2.add_argument('name')
p3.add_argument('name')
p3.add_argument('quality',nargs='?')
value and quality will get the default value if none is explicitly given. The default default is None. But you can define a meaningful alternative, e.g. default=0 for the integer value.
You could also define a parser like:
parser = ...
parser.add_argument('command', choices=['list','check','watch'])
parser.add_argument('rest', nargs='*')
This will expect one of the 3 'command' strings, and put anything else in the 'rest' attribute (as a list of strings). You could then interpret those strings however you want.

python argparse: how to use other parsed argument as parameter at calling function in type keyword?

I am trying to create an user interface using argparse module.
One of the argument need to be converted, so I use the type keyword:
add_argument('positional', ..., type=myfunction)
and there is another optional argument:
add_argument('-s', dest='switch', ...)
in addition, I have
parsed_argument=parse_args()
However, in myfunction, I hope I can use an additional parameter to control the behavior, which is the optional argument above, i.e.
def myfunction(positional, switch=parsed_argument.switch):
...
How can I achieve that?

Simple answer: You can’t. The arguments are parsed separately, and there is no real guarantee that some order is maintained. Instead of putting your logic into the argument type, just store it as a string and do your stuff after parsing the command line:
parser.add_argument('positional')
parser.add_argument('-s', '--switch')
args = parser.parse_args()
myfunction(args.positional, switch=args.switch)

I'm not sure I did understand correctly what you want to achieve, but if what you want to do is something that looks like:
myprog.py cmd1 --switcha
myprog.py cmd2 --switchb
yes you can, you need to use subparsers. I wrote a good example of it for a little PoC I wrote to access stackoverflow's API from CLI. The whole logic is a bit long to put thoroughly here, but mainly the idea is:
create your parser using parser = argparse.ArgumentParser(...)
create the subparsers using subparsers = parser.add_subparsers(...)
add the commands with things like `subparser.add_parser('mycommand', help='Its only a command').set_defaults(func=mycmd_fn) where
mycmd_fn takes args as parameters where you have all the switches you issued to the command!
the difference from what you ask, is that you'll need one function per command, and not one function with the positional argument as first argument. But you can leverage that easily by having mycmd_fn being like: mycmd_fn = lambda *args: myfunction('mycmd', *args)
HTH

From the documentation:
type= can take any callable that takes a single string argument and returns the converted value:
Python functions like int and float are good examples of a type function should be like. int takes a string and returns a number. If it can't convert the string it raises a ValueError. Your function could do the same. argparse.ArgumentTypeError is another option. argparse isn't going to pass any optional arguments to it. Look at the code for argparse.FileType to see a more elaborate example of a custom type.
action is another place where you can customize behavior. The documentation has an example of a custom Action. Its arguments include the namespace, the object where the parser is collecting the values it will return to you. This object contains any arguments have already been set. In theory your switch value will be available there - if it occurs first.
There are many SO answers that give custom Actions.
Subparsers are another good way of customizing the handling of arguments.
Often it is better to check for the interaction of arguments after parse_args. In your case 'switch' could occur after the positional and still have effect. And argparse.Error lets you use the argparse error mechanism (e.g. displaying the usage)

Use another variable when it's not specified in Python's argparse module

Is it feasible to use a variable which is also another argument variable when the first argument is not specified in argparse?
So in the following code:
import argparse
parser = argparse.ArgumentParser()
parser.add_option('-f', '--file', dest='outputfile')
parser.add_option('-d', '--db', dest='outpufDB')
And when I run the above script via script.py -f file_name, and don't specify the argument on outputDB, then I want to set the same value on outputDB as on outputfile. I know using default argument enables to set default value, but is it also feasible to set default value derived from another argument?
Thanks.

The traditional way to do this (the way people did it going back to old Unix getopt in C) is to give outputDB a useless "sentinel" default value. Then, after you do the parse, if outputDB matches the sentinel, use outputfile's value instead.
See default in the docs for full details on all of the options available. But the simplest—as long as it doesn't break any of your other params—seems to be to leave it off, and pass argument_default=SUPPRESS. If you do that, args.outputDB just won't exist if nothing was passed, so you can just check for that with hasattr or in or try.
Alternatively, you can pass an empty string as the default, but then of course a user can always trigger the same thing with --outputDB='', which you may not want to allow.
To get around that, you can not give it a type, which means you can give any default value that isn't a string, and there's no way the user can pass you the same thing. The pythonic way to get a sentinel when None or the falsey value of the appropriate type isn't usable is:
sentinel = object()
x = do_stuff(default_value=sentinel)
# ...
if x is sentinel:
# x got the default value
But here I don't think that's necessary. The default None should be perfectly fine (there's no way a user can specify that, since they can only specify strings).
(It's quite possible that argparse has an even cooler way to do this that I haven't discovered, so you might want to wait a while for other answers.)

The simple approach - do your own checking after parse_args.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--file', dest='outputfile')
parser.add_argument('-d', '--db', dest='outpufDB')
args = parser.parse_args()
if args.outpufDB is None:
args.outpufDB = args.outputfile
The default default value is None, which is easy test for.
Doing something entirely within argparse is useful if it is reflected in the help or you want the standardize error message. But you are just filling in a default after parsing.
I can imagine doing the same with a custom '-f' Action, but the logic would be more complex.

I'm not sure if this is a good practice, but I tried this on a similar problem. I have adapted the following snippet for your code, hope it helps.
import argparse
parser = argparse.ArgumentParser()
parser.add_option('-f', '--file', dest='outputfile', default='dummy_file')
parser.add_option('-d', '--db', dest='outputDB', default=parser.parse_args().outputfile)
If you don't pass any arguments the name of both, the 'outputfile' and 'outputDB' will be 'dummy_file'.