Conflict using both memory_profiler and docopt - python

I have a Python program that uses docopt and appears to parse command-line arguments just fine. However, when I attempt to include memory_profiler in the invocation (with -m memory_profiler), docopt fails to parse the command and prints the usage statement. Here is a sample program:
"""
Usage:
ids_transform [options]
ids_transform --block BLOCK_MSG
ids_transform --unblock
ids_transform --version
Examples:
ids_transform.py --block '2013-03-15 IM#123456 database down'
ids_transform.py -c ../shared/etc/ids_transform.yaml
Options:
-h --help show this help message and exit
-c CONFIG --config=CONFIG config file
-d --debug begin debugging
--force override block file; force run
--profile use cProfile to gather statistics on process
"""
from docopt import docopt
if __name__ == '__main__':
arguments = docopt(__doc__, version='1.0.0rc2')
print(arguments)
Here is a successful invocation:
$ python ids.py -d --force -c foo.yml
{'--block': False,
'--config': 'foo.yml',
'--debug': True,
'--force': True,
'--help': False,
'--profile': False,
'--unblock': False,
'--version': False,
'BLOCK_MSG': None}
And here is the error when utilizing the memory_profiler:
$ python -m memory_profiler ids.py -d --force -c foo.yml
Usage:
ids_transform [options]
ids_transform --block BLOCK_MSG
ids_transform --unblock
ids_transform --version
What am I missing?

It seems memory_profiler doesn't strip itself from sys.argv, so a hack would (I guess) be to do it yourself:
if sys.argv[0].endswith('memory_profiler.py'):
del sys.argv[0]

Related

Python command line argument option -d

I am trying to figure out the python command-line option -d. From the documentation it says -d Turn on parser debugging output.
But when I test, it shows me nothing but an argument:
main.py
import sys
def main(argv):
print(argv)
if __name__ == "__main__":
main(sys.argv[1:])
Execute in cmd:
$ python main.py -d /path/to/file
O/P:
['-d', 'C:/Program Files/Git/path/to/file']
Here -d print as an argument. Can anybody tell me the purpose of the -d option and how to use it?
-d is python option, not yours. So proper invocation is
$ python -d main.py /path/to/file

Output Test names in unittest discover

I'm running Python unittest using the discover mode:
% python -m unittest discover
The system prints a dot for each test, but I'd rather see a test name.
Is there an option that makes this happen?
The verbose flag (-v) is what you're looking for:
$ python -m unittest discover -v
test_a (tests.test_a.TestA) ... ok
test_b (tests.test_b.TestB) ... ok
...
For more options, check:
$ python -m unittest --help
usage: python -m unittest [-h] [-v] [-q] [--locals] [-f] [-c] [-b]
[tests [tests ...]]
positional arguments:
tests a list of any number of test modules, classes and test
methods.
optional arguments:
-h, --help show this help message and exit
-v, --verbose Verbose output
-q, --quiet Quiet output
--locals Show local variables in tracebacks
-f, --failfast Stop on first fail or error
-c, --catch Catch Ctrl-C and display results so far
-b, --buffer Buffer stdout and stderr during tests
...

Multiple bash command in Nomad

I have an application that runs multiple Python scripts in order. I can run them in docker-compose as follow:
command: >
bash -c "python -m module_a &&
python -m module_b &&
python -m module_c"
Now I'm, scheduling the job in Nomad, and added the below command under configuration for Docker driver:
command = "/bin/bash"
args = ["-c", "python -m module_a", "&&","
"python -m module_b", "&&",
"python -m module_c"]
But Nomad seems to escape &&, and just runs the first module, and issue exit code 0. Is there any way to run the multiline command similar to docker-compose?
The following is guaranteed to work with the exec driver:
command = "/bin/bash"
args = [
"-c", ## next argument is a shell script
"for module; do python -m \"$module\" || exit; done", ## this is that script.
"_", ## passed as $0 to the script
"module_a", "module_b", "module_c" ## passed as $1, $2, and $3
]
Note that only a single argument is passed as a script -- the one immediately following -c. Subsequent arguments are arguments to that script, not additional scripts or script fragments.
Even simpler, you could run:
command = "/bin/bash"
args = ["-c", "python -m module_a && python -m module_b && python -m module_c" ]

how to load python script in interactive shell

I am trying sudo python get_gps.py -c and expecting it to load the script and then present the interactive shell to debug the script live as opposed to typing it in manually.
From the docs:
$ python --help
usage: /usr/bin/python2.7 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-B : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : debug output from parser; also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
use -i option

Python argparse: Mutually exclusive required group with a required option

I am trying to have a required mutually exclusive group with one required parameter. Below is the code which I have put
#!/usr/bin/python
import argparse
import sys
# Check for the option provided as part of arguments
def parseArgv():
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-v", "--verbose", choices=[1,2,3,4],
help = "Increase verbosity")
group.add_argument("-q", "--quiet", action="store_true", help = "Run quietly")
name = parser.add_mutually_exclusive_group(required=True)
name.add_argument("-n", "--name", help = "Name of the virtual machine")
name.add_argument("-t", "--template", help = "Name of the template to use \
for creating vm. If path is not provided then it will be looked \
under template directory.")
parser.add_argument("-s", "--save", help = "Save the machine template. If \
path is not provided then it will be saved under template directory.");
#parser.add_argument("-k", "--kick_start", required = True, help = "Name of the \
# kick start file. If path is not provided then it will be look into http \
# directory.")
if len(sys.argv) == 1:
parser.print_help()
args = parser.parse_args()
if __name__ == '__main__':
parseArgv()
Now the output of this program as follow
$ python test.py
usage: test.py [-h] [-v {1,2,3,4} | -q] (-n NAME | -t TEMPLATE) [-s SAVE]
optional arguments:
-h, --help show this help message and exit
-v {1,2,3,4}, --verbose {1,2,3,4}
Increase verbosity
-q, --quiet Run quietly
-n NAME, --name NAME Name of the virtual machine
-t TEMPLATE, --template TEMPLATE
Name of the template to use for creating vm. If path
is not provided then it will be looked under template
directory.
-s SAVE, --save SAVE Save the machine template. If path is not provided
then it will be saved under template directory.
usage: test.py [-h] [-v {1,2,3,4} | -q] (-n NAME | -t TEMPLATE) [-s SAVE]
test.py: error: one of the arguments -n/--name -t/--template is required
But if I un-comment the from line 20 - 22 then the output change as below
$ python test.py
usage: test.py [-h] [-v {1,2,3,4} | -q] (-n NAME | -t TEMPLATE) [-s SAVE] -k
KICK_START
optional arguments:
-h, --help show this help message and exit
-v {1,2,3,4}, --verbose {1,2,3,4}
Increase verbosity
-q, --quiet Run quietly
-n NAME, --name NAME Name of the virtual machine
-t TEMPLATE, --template TEMPLATE
Name of the template to use for creating vm. If path
is not provided then it will be looked under template
directory.
-s SAVE, --save SAVE Save the machine template. If path is not provided
then it will be saved under template directory.
-k KICK_START, --kick_start KICK_START
Name of the kick start file. If path is not provided
then it will be look into http directory.
usage: test.py [-h] [-v {1,2,3,4} | -q] (-n NAME | -t TEMPLATE) [-s SAVE] -k
KICK_START
test.py: error: argument -k/--kick_start is required
But I want that either -n / -t along with -k become mandatory. How to achieve the same.
You have already achieved it! Argparse only prints the first error it finds, so while it may look like it's only checking -k, it actually recuires -n/-t too. You can see this by actually giving it the -k argument.
If you provide the -k argument, the error message will change from test.py: error: argument -k/--kick_start is required to test.py: error: one of the arguments -n/--name -t/--template is required.

Categories