We Keep Coding

Command line with Python

I'm developing a simple project with the purpose or learning Python, actually I have version 3.6 and I wanted to build a command line tool to generate password with specific criteria. So fare here is what I got:
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-a", "-auto", help="auto mode", action="store_true")
group.add_argument("-m", "-manual", help="manual mode", action="store_true")
parser.parse_args()
the dead end is I have no idea how to limit a command, example -l -lenght to the reqisite of having -m another stop is, how do I define -l so that it can accept a value, for example -l:8 to specify a password of 8 characters, I also want to put a range limit on -l values, example -l:8-256 and finally, I dont understand the difference between - and -- when defining the parameters.
I have already done all the part of generating passwords, just need a way to control its flow from outside so implementing parameters looked like a good way of doing this.

What you are looking for is the choices option.
add_argument('--length', type=int, choices=range(8,257)
This will only accept integer values between 8 and 256 (inclusive).
As for your second question, - indicates the shorthand for an option, which is common in most CLI tools, well -- indicates the long hand. It is common practice is CLI tools to provide both a long and a short hand for options.

You can define a custom type to check for valid values:
from argparse import ArgumentTypeError
def passwd_len(s):
try:
s = int(s)
except ValueError:
raise ArgumentTypeError("'%s' is not an integer" % (s,))
if not (8 <= s <= 256):
raise ArgumentTypeError("%d is not between 8 and 256, inclusive" % (s,))
return s
parser.add_argument("--length", "-l", type=passwd_len)
The difference between -- and - is one of conventions. Historically, options were single-letter arguments prefixed with a -, and groups of options could be combined: -ab was the same as -a -b. In order to
support that convention and allow longer option names, a new convention
of using -- to prefix multi-character names was devised. --length is a single option, not a group of six options -l, -e, -n, -g, -t, and -h. Long options cannot be grouped.
I would define the -a and -m options like this:
group = parser.add_mutually_exclusive_group()
group.add_argument("-a", "--auto", help="auto mode", action="store_const", const="auto", dest="mode")
group.add_argument("-m", "--manual", help="manual mode", action="store_const", const="manual", dest="mode")
group.add_argument("--mode", choices=["auto", "manual"])
Now instead of having two Boolean attributes that can never have the same value, you have just one attribute whose value you can check directly. Think of --mode as being the canonical way of choosing a mode, with -a and -m as shortcuts for selecting a specific mode.

Using argparse, how can I process a "chdir" argument before fromfile expansion?

I want to support a sub-command CLI model, like used by git The particular bit I'm having trouble with is the "change directory" option. Like git, I want a -C DIR option which will have the program change to the specified directory before doing the sub-command. Not really a problem, using sub-parsers, BUT I also want to use the argparse.ArgumentParser(fromfile_prefix_chars='#') mechanism after the -C DIR argument is applied during parsing.
Here's the rub: fromfile argument expansion is performed by argparse before all other argument processing. Thus, any such fromfile arguments must either use absolute paths, or paths relative to the CWD at time the parser is invoked. I don't want absolute paths; I "need" to use fromfile paths that are relative to the -C DIR option. I wrote my own class ChdirAction(argparse.Action) to do the obvious. It worked fine, but since fromfile arguments were already expanded, it didn't give me what I want. (After discovering this not-what-I-want behavior, I looked at python3.5/argparse.py and found the same frustration embedded in cold, hard, unforgiving code.)
Here's a directory diagram that might help explain what I want:
/ foo / aaa / iii / arg.txt
| |
| + jjj / arg.txt
| |
| + arg.txt
|
+ bbb / iii / arg.txt
|
+ jjj / arg.txt
Consider when the CWD is either aaa or bbb at the time command line arguments are parsed. If I run with something like prog -C ./iii #arg.txt
I want the parser to expand #arg.txt with arguments from /foo/aaa/iii/arg.txt. What actually happens is that fromfile expands from the contents of /foo/aaa/arg.txt. When CWD is /foo/aaa this is the "wrong" file; when /foo/bbb it raises "error: [Errno 2] No such file or directory: 'arg.txt'"
More generally, prog -C ./DIR #arg.txt should expand from /foo/aaa/DIR/arg.txt which should work even the fromfile has "up-directory" parts, e.g. prog -C ./iii #../arg.txt should expand from /foo/aaa/arg.txt.
If this behavior can be made to happen, then I could -C DIR to any of {aaa,bbb}/{iii,jjj} and obtain consitent behaviour from a common command line construction.
As described, my problem isn't much of a problem. If can provide the -C DIR, to be realized by an os.chdir(DIR) after argument parsing, then I can also construct appropriate fromfile arguments. They could be either absolute or relative to the CWD at parsing (prior to any -C DIR taking effect). This might look like:
cd /foo/aaa; prog -C ./DIR #arg.txt #./DIR/arg.txt
I don't like it, but it would be okay. The REAL problem is that the actual change-directory argument I'm using is more like -C PATTERN. In my real problem case, PATTERN could be a simple path (absolute or relative). Or, it might be a glob pattern, or a partial name that has "non-trivial" resolution logic to find the actual directory for os.chdir(DIR). In this case (which I am struggling with), I can't have the invoker of the program resolve the actual location of the fromfile path.
Actually, I could, but that would put an inappropriate burden on the invoker. AND, when that invoker is an Eclipse launcher, I don't really have the control-flow power necessary to do it. So, it's back to having the program take care of it's own needs; a nicer abstraction, but how do I implement it?

Even as I was fleshing out the question, I came up with an idea. So I tried it out and it's kinda, sorta, okay(ish). I can get a constrained version of what I really want, but it's good enough for me (for now), so I thought I might as well share. It might be good enough for you, too. Even better, it might elicit a true solution from somewhere, maybe S.Bethard?
My hack is to do parsing in two phases: the first, is just enough to get the -C PATTERN argument by way of ArgumentParser.parse_known_args(...) without enabling the fromfile mechanism. If the result of that first (minimal) parsing yields a directory change argument, then I process it. The program aborts if more than a single -C PATTERN was specified, or the PATTERN can't be unambiguously resolved.
Then, I use a completely separate ArgumentParser object, configured with the full set of argument specifications that I actually want and parse it with the fromfile mechanism enabled.
There is some monkey business to get the --help argument to work (setting the proper conflict resolution policy, then merely accepting the arg in the first parser just to pass along to the second, which actually has all the "real" argument specs). Also, the first parser should support the same verbose/quiet options that the second one does, honoring their setting and also passing along from first to second parser.
Here's a simplified version of my application-level arg parser method. It doens't support verbose/quiet options at the first parser stage. I've elided the complexity of how a -C PATTERN is resolved to an actual directory. Also, I cut out the majority of the second parser's argument specification, leaving just the second parser's -C PATTERN argument (needed for --help output).
NOTE: Both parsers have a -C PATTERN argument. In the chdirParser it is meaningful; in the argParser it's present only so it will show up in the help output. Something similar should be done for verbose/quiet options - probably not that trixy, but it's not (yet) important to me so I don't mind always reporting a change of directory, even in quiet mode.
def cli_args_from_argv():
import argparse
import glob
import os
import sys
chdirParser = argparse.ArgumentParser(conflict_handler='resolve')
chdirParser.add_argument("-C", dest="chdir_pattern", action="append" , default=None)
chdirParser.add_argument("--help", "-h", dest="help", action="store_true", default=False)
(partial, remainder) = chdirParser.parse_known_args()
if partial.help:
remainder = ['--help']
elif partial.chdir_pattern:
if len(partial.chdir_pattern) > 1:
print(r'Too many -C options - at most one may be given, but received: {!r}'.format(partial.chdir_pattern), file=sys.stderr)
sys.exit(1)
pattern = partial.chdir_pattern[0]
resolved_dir = pattern
if os.path.exists(resolved_dir):
resolved_dir = pattern
else:
### ELIDED: resolution of pattern into an unambiguous and existing directory
if not resolved_dir:
print("Failed to resolve -C {!r}".format(pattern), file=sys.stderr)
sys.exit(1)
print("Changing to directory: {!r}".format(resolved_dir))
print("");
os.chdir(target_dir)
argParser = argparse.ArgumentParser(usage="usage: PROG [common-args] SUBCMD [subcmd-args]", fromfile_prefix_chars=':')
### ELIDED: a bunches of add_argument(...)
argParser.add_argument("-C", dest="chdir_spec", action="store", default=None, help="Before anything else, chdir to SPEC", metavar="SPEC")
return argParser.parse_args(args=remainder)
I have a feeling that there's probably a better way... Do you know?

I think the resolve bit can be replaced with
chdirParser = argparse.ArgumentParser(add_help=False)
and omit the -h definition and save. Let the second parser handle sys.argv unchanged (since you are including (but ignoring) the -C argument).
That append and test for len(partial.chdir_pattern) > 1 should work if you expect the user to use several -C dir1 ... -C dir2... commands. The alternative to use the default store action, which ends up saving the last of those repetitions. Why might the user repeat the -C, and why should you care? Usually we just ignore repetitions.
You might replace
print("Failed to resolve -C {!r}".format(pattern), file=sys.stderr)
sys.exit(1)
with
parser.error("Failed to resolve -C {!r}".format(pattern)')
It prints the usage (with only -C) and does ansys.exit(2)`. Not quite the same, but may be close enough.
For the second parser, the -C might be simplified (using defaults):
argParser.add_argument("-C", "--chdir-spec", help="Before anything else, chdir to SPEC", metavar="SPEC")
And use the full sys.argv.
return argParser.parse_args()
Otherwise, using 2 parsers makes sense, since the fromfile is present in the changed directory (and you want to ignore any such file in the initial directory).
I thought maybe a :arg.txt string the commandline would give problems in the first parser. But with parse_known_args it will just treat it as an unknown positional. But the proof's in the testing.

Argparse using optional argument with value in conjunction with positional argument

In my script I have, for simplicity, three arguments:
parser.add_argument("-c", "--compile")
parser.add_argument("--verbose",
help = "stores compilation results in specified log file as they come (default name: %(const)s)",
nargs = '?',
const = DEFAULT_LOG_FILE_NAME,
metavar = "LOGFILE_NAME")
parser.add_argument("path", nargs = "*")
-c and --verbose are both optional, as well as path, which is a positional argument. In addition, the argument to --verbose is also optional. If none is provided,
Say I want to combine these three in a single command.
I would run it as follows:
myscript.py -c --verbose path1 path2 path3
The problem here is that in this case, the script will interpret path1 as an argument to --verbose, unless I use --verbose=<log_name>. As far as I have been able to find, there is no way of restricting argparse to only allowing the = syntax instead of a space. I cannot count on my users understanding that either = must be used, or put --verbose as one of the last arguments.
How would I fix this? Any help is appreciated.

In this case, you are overloading --verbose to do 2 things: as an on/off flag, and as a log file option. Consider separating it into two different options: --verbose and --log=LOGFILE_NAME

Parsing limited switches with python argparse

Is there a way to parse only limited number of switches in a function using argparse? Say, my command is:
python sample.py -t abc -r dfg -h klm -n -p qui
And I want argparse to parse from -t to -h and leave the remaining, also show help for these only.
Next I want to parse any switch after -h into another function and see corresponding help there.
Is this behavior possible in argparse? Also is there a way I can modify sys.arg it is using internally?
Thanks.

python sample.py -t abc -r dfg -h klm -n -p qui
And I want argparse to parse from -t to -h and leave the remaining, also show help for these only. Next I want to parse any switch after -h into another function and see corresponding help there.
There are some issues with your specification:
Is -h the regular help? If so it has priority, producing the help without parsing the other arguments. The string after -h suggests you are treating it like a normal user define argument, which would then require initiating the parser with help turned off. But then how would you ask for help?
What sets the break between the two parsings/help? The number of arguments, the -h flag (regardless of order), or the id of the flags. Remember argparse accepts flagged arguments in any order.
You could define one parser that knows about -t and -r, and another that handles -n and -p. Calling each with parse_known_args lets it operate without raising a unknown argument error.
You can also modify the sys.argv. parse_args (and the known variant), takes an optional argv argument. If that is none, then it uses sys.argv[1:]. So you could either modify sys.argv itself (deleting items), or you could pass a subset of sys.argv to the parser.
parser1.parse_known_args(sys.argv[1:5])
parser2.parse_known_args(['-n','one','-o','two'])
parser3.parse_args(sys.argv[3:])
Play with those ideas, and come back to us if there are further questions.

You can always modify sys.args and put anything you wish there.
As for your main question, you can have two parsers. One of them will have arguments -t to -h, the second -n and -p. Then you can use argparse's parse_known_args() method on each parser, which will parse only the arguments defined for each of them.

How to make a custom command line interface using OptionParser?

I am using the OptionParser from optparse module to parse my command that I get using the raw_input().
I have these questions.
1.) I use OptionParser to parse this input, say for eg. (getting multiple args)
my prompt> -a foo -b bar -c spam eggs
I did this with setting the action='store_true' in add_option() for '-c',now if there is another option with multiple argument say -d x y z then how to know which arguments come from which option? also if one of the arguments has to be parsed again like
my prompt> -a foo -b bar -c spam '-f anotheroption'
2.) if i wanted to do something like this..
my prompt> -a foo -b bar
my prompt> -c spam eggs
my prompt> -d x y z
now each entry must not affect the other options set by the previous command. how to accomplish these?

For part 2: you want a new OptionParser instance for each line you process. And look at the cmd module for writing a command loop like this.

You can also solve #1 using the nargs option attribute as follows:
parser = OptionParser()
parser.add_option("-c", "", nargs=2)
parser.add_option("-d", "", nargs=3)

optparse solves #1 by requiring that an argument always have the same number of parameters (even if that number is 0), variable-parameter arguments are not allowed:
Typically, a given option either takes
an argument or it doesn’t. Lots of
people want an “optional option
arguments” feature, meaning that some
options will take an argument if they
see it, and won’t if they don’t. This
is somewhat controversial, because it
makes parsing ambiguous: if "-a" takes
an optional argument and "-b" is
another option entirely, how do we
interpret "-ab"? Because of this
ambiguity, optparse does not support
this feature.
You would solve #2 by not reusing the previous values to parse_args, so it would create a new values object rather than update.