We Keep Coding

python module called with cat and command line arguments parser

cat hello.txt | python main.py foobar
what is the reason of using cat here?
I understand foobar is command line argument so I need to handle it with arguments parser. how to handle it if i do not specify parameter e.g. --parameters ?
what about hello.txt used after cat?
is it the name of the file that i am passing into python main.py call as other argument or am I dumping python main.py execution results into hello.txt? how and what exactly? am I catching what python main.py is printing with cat and writing it into hello.txt?
EDIT:
Am i capturing it correct?
import argparse
import sys
def parse_arguments():
parser = argparse.ArgumentParser()
parser.add_argument('a', nargs='*')
args, unknown = parser.parse_known_args()
return args
file_content = data = sys.stdin.read()
params = parse_arguments()
print(file_content)
print(params)

Parse variable number of commands with python argparse

I'm developing a command line tool with Python whose functionality is broken down into a number of sub-commands, and basically each one takes as arguments input and output files. The tricky part is that each command requires different number of parameters (some require no output file, some require several input files, etc).
Ideally, the interface would be called as:
./test.py ncinfo inputfile
Then, the parser would realise that the ncinfo command requires a single argument (if this does not fit the input command, it complains), and then it calls the function:
ncinfo(inputfile)
that does the actual job.
When the command requires more options, for instance
./test.py timmean inputfile outputfile
the parser would realise it, check that indeed the two arguments are given, and then then it calls:
timmean(inputfile, outputfile)
This scheme is ideally generalised for an arbitrary list of 1-argument commands, 2-argument commands, and so on.
However I'm struggling to get this behaviour with Python argparse. This is what I have so far:
#! /home/navarro/SOFTWARE/anadonda3/bin/python
import argparse
# create the top-level parser
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# create the parser for the "ncinfo" command
parser_1 = subparsers.add_parser('ncinfo', help='prints out basic netCDF strcuture')
parser_1.add_argument('filein', help='the input file')
# create the parser for the "timmean" command
parser_2 = subparsers.add_parser('timmean', help='calculates temporal mean and stores it in output file')
parser_2.add_argument('filein', help='the input file')
parser_2.add_argument('fileout', help='the output file')
# parse the argument lists
parser.parse_args()
print(parser.filein)
print(parser.fileout)
But this doesn't work as expected. First, when I call the script without arguments, I get no error message telling me which options I have. Second, when I try to run the program to use ncinfo, I get an error
./test.py ncinfo testfile
Traceback (most recent call last):
File "./test.py", line 21, in <module>
print(parser.filein)
AttributeError: 'ArgumentParser' object has no attribute 'filein'
What am I doing wrong that precludes me achieving the desired behaviour? Is the use of subparsers sensible in this context?
Bonus point: is there a way to generalise the definition of the commands, so that I do not need to add manually every single command? For instance, grouping all 1-argument commands into a list, and then define the parser within a loop. This sounds reasonable, but I don't know if it is possible. Otherwise, as the number of tools grows, the parser itself is going to become hard to maintain.

import argparse
import sys
SUB_COMMANDS = [
"ncinfo",
"timmean"
]
def ncinfo(args):
print("executing: ncinfo")
print(" inputfile: %s" % args.inputfile)
def timmean(args):
print("executing: timmean")
print(" inputfile: %s" % args.inputfile)
print(" outputfile: %s" % args.outputfile)
def add_parser(subcmd, subparsers):
if subcmd == "ncinfo":
parser = subparsers.add_parser("ncinfo")
parser.add_argument("inputfile", metavar="INPUT")
parser.set_defaults(func=ncinfo)
elif subcmd == "timmean":
parser = subparsers.add_parser("timmean")
parser.add_argument("inputfile", metavar="INPUT")
parser.add_argument("outputfile", metavar="OUTPUT")
parser.set_defaults(func=timmean)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('-o', '--common-option', action='store_true')
subparsers = parser.add_subparsers(help="sub-commands")
for cmd in SUB_COMMANDS:
add_parser(cmd, subparsers)
args = parser.parse_args(sys.argv[1:])
if args.common_option:
print("common option is active")
try:
args.func(args)
except AttributeError:
parser.error("too few arguments")
Some usage examples:
$ python test.py --help
usage: test.py [-h] [-o] {ncinfo,timmean} ...
positional arguments:
{ncinfo,timmean} sub-commands
optional arguments:
-h, --help show this help message and exit
-o, --common-option
$ python test.py ncinfo --help
usage: test.py ncinfo [-h] INPUT
positional arguments:
INPUT
optional arguments:
-h, --help show this help message and exit
$ python test.py timmean --help
usage: test.py timmean [-h] INPUT OUTPUT
positional arguments:
INPUT
OUTPUT
optional arguments:
-h, --help show this help message and exit
$ python test.py -o ncinfo foo
common option is active
executing: ncinfo
inputfile: foo
$ python test.py -o timmean foo bar
common option is active
executing: timmean
inputfile: foo
outputfile: bar

Get command line parameters with argparse

I´m trying to use argparse of Python but I cannot get a command line parameter.
Here my code:
DEFAULT_START_CONFIG='/tmp/config.json'
parser = argparse.ArgumentParser(description='Start the Cos service and broker for development purposes.')
parser.add_argument('-c', '--config', default=DEFAULT_START_CONFIG, action=FileAction, type=str, nargs='?',
help='start configuration json file (default:' + DEFAULT_START_CONFIG + ')')
args = parser.parse_args()
But then when I run my python script like:
./start.py -c /usr/local/config.json
Instead of getting this path it is getting the default value defined (/tmp/config.json).
print args.config ---> "/tmp/config.json"
What I´m doing wrong here?

The standard documentation doesn't mention FileAction. Instead there's a class FileType intended for type argument, not for action.
So I would write something like this:
DEFAULT_START_CONFIG='/tmp/config.json'
parser = argparse.ArgumentParser(description='Start the Cos service and broker for development purposes.')
parser.add_argument('-c', '--config', default=DEFAULT_START_CONFIG,
type=argparse.FileType('r'), help='start configuration json file')
args = parser.parse_args()
print(args)
This gives me the following:
$ python test3.py
Namespace(config=<open file '/tmp/config.json', mode 'r' at 0x7fd758148540>)
$ python test3.py -c
usage: test3.py [-h] [-c CONFIG]
test3.py: error: argument -c/--config: expected one argument
$ python test3.py -c some.json
usage: test3.py [-h] [-c CONFIG]
test3.py: error: argument -c/--config: can't open 'some.json': [Errno 2] No such file or directory: 'some.json'
$ touch existing.json
$ python test3.py -c existing.json
Namespace(config=<open file 'existing.json', mode 'r' at 0x7f93e27a0540>)
You may subclass argparse.FileType to something like JsonROFileType which would check if the supplied file is actually a JSON of expected format etc, but this seems to be out of the scope of the question.

Unable to get the value of a command argument using argparse

I'm trying to parse the command line arguments in a very simple way:
$ python main.py --path /home/me/123
or
$ python main.py --path=/home/me/123
And then:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--path')
args = parser.parse_args()
And args returns nothings:
(Pdb) args
(Pdb) args.path
How can I access the value of --path?

You can print args.path and it will show your line argument. For more details you can check the below link for more details about argparse
Argparse Tutorial
You can also use sys to parse your command line arguments, such as
>>> import sys
>>> path = sys.argv[1] # sys.argv always start at 1
>>> print path
Check the below link for more details.
Python Command Line Arguments
Hope it helps.

It works fine for me...
>>> args
Namespace(path='/home/me/123')
So you can access it via args.path

Flexible UNIX command line interface with Python

I was wondering how to create a flexible CLI interface with Python. So far I have come up with the following:
$ cat cat.py
#!/usr/bin/env python
from sys import stdin
from fileinput import input
from argparse import ArgumentParser, FileType
def main(args):
for line in input():
print line.strip()
if __name__ == "__main__":
parser = ArgumentParser()
parser.add_argument('FILE', nargs='?', type=FileType('r'), default=stdin)
main(parser.parse_args())
This handles both stdin and file input:
$ echo 'stdin test' | ./cat.py
stdin test
$ ./cat.py file
file test
The problem is it doesn't handle multiple input or no input the way I would like:
$ ./cat.py file file
usage: cat.py [-h] [FILE]
cat.py: error: unrecognized arguments: file
$ ./cat.py
For multiple inputs it should cat the file multiple times and for no input input should ideally have same the behaviour as -h:
$ ./cat.py -h
usage: cat.py [-h] [FILE]
positional arguments:
FILE
optional arguments:
-h, --help show this help message and exit
Any ideas on creating a flexible CLI interface with Python?

Use nargs='*' to allow for 0 or more arguments:
if __name__ == "__main__":
parser = ArgumentParser()
parser.add_argument('FILE', nargs='*', type=FileType('r'), default=stdin)
main(parser.parse_args())
The help output now is:
$ bin/python cat.py -h
usage: cat.py [-h] [FILE [FILE ...]]
positional arguments:
FILE
optional arguments:
-h, --help show this help message and exit
and when no arguments are given, stdout is used.
If you want to require at least one FILE argument, use nargs='+' instead, but then the default is ignored, so you may as well drop that:
if __name__ == "__main__":
parser = ArgumentParser()
parser.add_argument('FILE', nargs='+', type=FileType('r'))
main(parser.parse_args())
Now not specifying a command-line argument gives:
$ bin/python cat.py
usage: cat.py [-h] FILE [FILE ...]
cat.py: error: too few arguments
You can always specify stdin still by passing in - as an argument:
$ echo 'hello world!' | bin/python cat.py -
hello world!

A pretty good CLI interface the handles file input, standard input, no input, file output and inplace editing:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
def main(args, help):
'''
Simple line numbering program to demonstrate CLI interface
'''
if not (select.select([sys.stdin,],[],[],0.0)[0] or args.files):
help()
return
if args.output and args.output != '-':
sys.stdout = open(args.output, 'w')
try:
for i, line in enumerate(fileinput.input(args.files, inplace=args.inplace)):
print i + 1, line.strip()
except IOError:
sys.stderr.write("%s: No such file %s\n" %
(os.path.basename(__file__), fileinput.filename()))
if __name__ == "__main__":
import os, sys, select, argparse, fileinput
parser = argparse.ArgumentParser()
parser.add_argument('files', nargs='*', help='input files')
group = parser.add_mutually_exclusive_group()
group.add_argument('-i', '--inplace', action='store_true',
help='modify files inplace')
group.add_argument('-o', '--output',
help='output file. The default is stdout')
main(parser.parse_args(), parser.print_help)
The code simply emulates nl and numbers the lines but should serve as a good skeleton for many applications.