I need to pass a dictionary to a python module via a command line. I have not found a way to get Python to cooperate. The following is a very simplified example of what I'd like to accomplish. This produces the following error TypeError: _get_positional_kwargs() missing 1 required positional argument: 'dest'
. Adding a dest argument to the add_argument line results in TypeError: __init__() got an unexpected keyword argument 'kwargs', indicating the sad possibility that argparse simply may not take a dictionary as an argument. Is a pickle file truly the only option for passing dictionaries back and forth from command line calls?
def run(**kwargs):
for i in kwargs.items():
print(f'{i}: {kwargs[i]}')
# ...some code to manipulate the dictionary
return kwargs
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Test script")
parser.add_argument(kwargs='**', description='A dictionary to be manipulated and returned')
args = parser.parse_args()
x = run(args)
pass
Additional info - I know that I can pass a list to a *nargs argument with a command line that includes space-delimited arguments, but I need to do something similar with a dictionary.
UPDATE - WHAT WORKED:
Based on the helpful input from #chepner, I was able to get it to work with the json solution he mentioned in his answer. On my Windows platform, I did have to remove all spaces from the dictionary string and escape characters are required in the final command line string.
{\"this\":1,\"is\":2,\"a\":5,\"dictionary\":7}
Arguments are strings, period. What you want to pass is a string that contains an encoded dict, for example using JSON.
import argparse
import json
parser = argparse.ArgumentParser()
parser.add_argument('data', type=json.loads)
args = parser.parse_args()
If you run your script with something like
myscript '{"foo": 3, "bar": "hello"}'
then args.data["foo"] == 3 and args.data["bar"] == "hello".
Related
I am working on a project with many command line arguments and I'd like to be able to specify all of the arguments in the form of a file (i.e. JSON), and load this file into an argparse object, instead of pasting them into the terminal every time. I also need to enforce the presence of required arguments. I've found a couple options but none of them do exactly what I need.
If I have something like this
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--image_path", type=str, required=True)
parser.add_argument("--count", type=int, required=True)
args = parser.parse_args()
One approach might be to add something like this to update the dictionary with the JSON contents
args_dict = vars(args)
with open('path/to/args.json', 'rb') as f:
args_dict.update(json.load(f))
But an error occurs at the parse_args() line, since it didn't see the required arguments passed in.
The other approach I considered is to use
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
and just pass the file name, prefixed with #, to the command. The problem here is that I have to convert the dictionary-like format (JSON) to a line-separated list of arguments, as noted in the documentation. This is not trivial in my case.
I also considered just saving the arguments as a line-separated list to begin with, but the way that I know how to do that involves iterating over sys.argv and printing to a text file, which is problematic because I have some args that are assigned default values, which are not represented in sys.argv. I need to record the default values that were used for later reference.
It seems like there should be a simple way to do this, but I'm stumped at the moment.
I'm trying to pass two arguments to python, one read-only JSON file and one char together. In fact I want the arguments both positional, but I couldn't do it either, so let the char one optional. At the end I will read the dictionary and the char will be my key.
When I try like below, the second argument is passed as None. I can't understand why it's empty. Do you have any idea?
Thanks a lot!
python3 myProg.py dict.json -a
import json
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(description = 'program')
parser.add_argument('json', nargs = 1, type = argparse.FileType('r'))
parser.add_argument('-args', nargs ="?")
arguments = parser.parse_args()
dictionary = json.load(arguments.json[0])
arg_start = arguments.args
Recieving this error:
Namespace(args=None, json=[<_io.TextIOWrapper name='dict.json' mode='r' encoding='UTF-8'>])
It looks like you got the json file ok. If you'd omitted the nargs=1 you could have used:
dictionary = json.load(arguments.json)
-args is an nargs='?'. That means it it is ok to omit an argument. If not used at all its value will be the Actions default (currently None).
If provided as just -a, the value is the Action's const (also None).
If provided as -a foo, it will set that string.
If you'd define that argument as positional, plain 'args' (without the -) then it would have accepted the next string after the json file name.
I'm trying to use the argparse library in Python. I want to have the user do something like:
python my_script.py csv_name.csv [--dryrun]
where --dryrun is an optional parameter.
I then have the user enter an API key and secret key. When I run my code, I get past entering the API and secret keys and then I get:
usage: my_script.py [-h] csv dryrun
salesforceImporter.py: error: too few arguments
Here's my code:
def main():
api_key = getpass.getpass(prompt='Enter API Key: ')
secret_key = getpass.getpass(prompt='Enter Secret Key: ')
parser = argparse.ArgumentParser()
parser.add_argument("csv")
parser.add_argument("dryrun")
args = parser.parse_args()
validate_csv_name(args.csv)
is_dry_run = args.dryrun == '--dryrun'
Any idea where I'm going wrong?
Thanks!
When you use the following syntax:
parser.add_argument("csv")
parser.add_argument("dryrun")
You're adding these as positional -- required -- arguments. Only arguments with a leading dash or two are optional.
See the docs here:
The add_argument() method must know whether an optional argument, like -f or --foo, or a positional argument, like a list of filenames, is expected. The first arguments passed to add_argument() must therefore be either a series of flags, or a simple argument name. For example, an optional argument could be created like:
>>> parser.add_argument('-f', '--foo')
To add an optional --dry-run argument, you may use the following snippet:
parser.add_argument('--dry-run', action='store_true')
Calling your script using python my_script.py csv_name.csv --dry-run will result to args.dry_run being True. Not putting the option will result to it being False
This question already has answers here:
type=dict in argparse.add_argument()
(13 answers)
Closed 9 years ago.
I'm trying to accept an argument of type=dict with argparse but no matter the input it gives an error of invalid dict value.
#!/usr/bin/env python
import argparse
MYDICT = {'key': 'value'}
parser = argparse.ArgumentParser()
parser.add_argument("-m", "--mydict", action="store",
required=False, type=dict,
default=MYDICT)
args = parser.parse_args()
print args.mydict
This is what happens when I try and pass a dictionary to the script
./argp.py -m "{'key1': 'value1'}"
usage: argp.py [-h] [-m MYDICT]
argp.py: error: argument -m/--mydict: invalid dict value: "{'key1': 'value1'}"
Looking at the documents I would think that this would be possible.
http://docs.python.org/dev/library/argparse.html
“Any object that supports the in operator can be passed as the choices value, so dict objects, set objects, custom containers, etc. are all supported.”
I do not think it is possible to pass a dictionary as an argument in the command line because there doesn't exist a conversion function from string to dict (EDIT: A hack is possible which gives similar behaviour, see below). What you are essentially telling python to do is:
dict("{'key1': 'value1'}")
Which if you try it out in the python console, does not work.
What the phrase:
"Any object that supports the in operator can be passed as the choices value, so dict objects, set objects, custom containers, etc. are all supported."
refers to is the choices argument that can be passed with the add_argument function - not to the type argument.
Your best bet is to probably accept your argument as a string and then convert it using the json capabilities of python:
parser.add_argument('-m', '--my-dict', type=str)
args = parser.parse_args()
import json
my_dictionary = json.loads(args.my_dict)
You can then pass a dictionary in the form of a string. You can try the json encoder/decoder out for yourself in the python console to see how it works:
>>>json.loads('{"value1":"key1"}')
{u'value1': u'key1'}
EDIT: hpaulj has pointed out to me that you can "hack" the type parameter by passing it json.loads which allows you to pass JSON that is similar looking to a dictionary.
import json
parser.add_argument('-d', '--my-dict', type=json.loads)
args = parse.parse_args()
mydict = args.my_dict # Will return a dictionary
NOTE: The input format you pass is not the same as python dictionary but is probably similar enough for your use case.
The reason this works is actually quite interesting because internally argparse will just use the parameter value as a function to convert the argument. i.e. if type=int then it will use int(arg) or if type=json.loads then json.loads(arg)
This also means that you can pass any function which takes a single parameter in as the argument to type and perform custom conversions if you need to :)
This first link has the same question in the first section, but it is unanswered
(python argparse: parameter=value). And this second question is similar, but I can't seem to get it working for my particular case
( Using argparse to parse arguments of form "arg= val").
So my situation is this -- I am re-writing a Python wrapper which is used by many other scripts (I would prefer not to modify these other scripts). Currently, the Python wrapper is called with command line arguments of the form --key=value for a number of different arguments, but was parsed manually. I would like to parse them with argparse.
N.B. The argument names are unwieldy, so I am renaming using the dest option in add_argument.
parser = argparse.ArgumentParser(description='Wrappin Ronnie Reagan')
parser.add_argument("--veryLongArgName1", nargs=1, dest="arg1", required=True)
parser.add_argument("--veryLongArgName2", nargs=1, dest="arg2")
parser.add_argument("--veryLongArgName3", nargs=1, dest="arg3")
userOpts = vars(parser.parse_args())
Which, while apparently parsing the passed command lines correctly, displays this as the help:
usage: testing_argsparse.py [-h] --veryLongArgName1 ARG1
[--veryLongArgName2 ARG2]
[--veryLongArgName3 ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
But what I want is that all parameters are specified with the --key=value format, not --key value. i.e.
usage: testing_argsparse.py [-h] --veryLongArgName1=ARG1
[--veryLongArgName2=ARG2]
[--veryLongArgName3=ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
testing_argsparse.py --veryLongArgName1=foo
works. argparse module accepts both --veryLongArgName1=foo and --veryLongArgName1 foo formats.
What exact command line arguments are you trying to pass to argparse that's causing it to not work?
A little late but for anyone with a similar request as the OP you could use a custom HelpFormatter.
class ArgFormatter(argparse.HelpFormatter):
def _format_args(self, *args):
result = super(ArgFormatter, self)._format_args(*args)
return result and '%%%' + result
def _format_actions_usage(self, *args):
result = super(ArgFormatter, self)._format_actions_usage(*args)
return result and result.replace(' %%%', '=')
This can then be passed to ArgumentParser to give the wanted behavior.
parser = argparse.ArgumentParser(
description='Wrappin Ronnie Reagan',
formatter_class=ArgFormatter)
This intercepts the args (ARG1, ARG2, ...) and adds a custom prefix which is later replaced (along with the unwanted space) for an = symbol. The and in the return statements makes sure to only modify the result if it's non-empty.