We Keep Coding

How do I pass arguments from command line to the python script to read and update values of certain keys present in a json file

I have certain data in a json file (say, example.json),
example.json
data = {
'name' : 'Williams',
'working': False,
'college': ['NYU','SU','OU'],
'NYU' : {
'student' : True,
'professor': False,
'years' : {
'fresher' : '1',
'sophomore': '2',
'final' : '3'
}
}
}
I wish to write a code wherein I can give the arguments on Command line, i.e. suppose if a script is saved in a file 'script.py', then,
In the terminal: If I enter *$ python3* script.py --get name --get NYU.student Then it outputs name=Williams
NYU.student=True
If I enter *$ python3* script.py' --set name=Tom --set NYU.student=False
Then, it updates name and NYU.student keys in the dictionay to Tom and False and outputs NYU.student=Tom and NYU.student=False on the command line.
I have tried the following code for the python script (i.e. script.py)
script.py
import json
import pprint
import argparse
if __name__== "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--get", help="first command")
parser.add_argument("--set", help="second command")
args=parser.parse_args()
with open('example.json','r') as read_file:
data=json.load(read_file)
if args.set == None:
key = ' '.join(args.get[:])
path = key.split('.')
now = data
for k in path:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
print(now)
elif args.get == Null:
key, value = ' '.join(args.set[:]).split('=')
path = key.split('.')
now = data
for k in path[:-1]:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
now[path[-1]] = value
with open('example.json','w') as write_file: #To write the updated data back to the same file
json.dump(data,write_file,indent=2)
However, my script is not working as I expect it to? Kindly, help me with the script

Your code has the following issues:
When joining the argument values in line number 23 and 35, you use a space. This leads to the "Error key" value. Removing the space will solve the issue.
key = ''.join(arg[:])
You defined the arguments to only pass one value. Not multiple. Therefore even if you pass multiple --get or --set values, the script only gets one value. Adding action="append" to line number 9 and 10 will solve the issue.
parser.add_argument("--get", help="first command", action="append")
parser.add_argument("--set", help="second command", action="append")
Full code:
import json
import pprint
import argparse
if __name__== "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--get", help="first command", action="append")
parser.add_argument("--set", help="second command", action="append")
args=parser.parse_args()
try:
with open('example.json','r') as read_file:
data=json.load(read_file)
except IOError:
print("ERROR: File not found")
exit()
if args.set == None:
for arg in args.get:
key = ''.join(arg[:])
path = key.split('.')
now = data
for k in path:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
print(f"{arg} = {now}")
elif args.get == None:
for arg in args.set:
key, value = ''.join(arg[:]).split('=')
path = key.split('.')
now = data
for k in path[:-1]:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
print(f"{arg}")
now[path[-1]] = value
with open('example.json','w') as write_file: #To write the updated data back to the same file
json.dump(data,write_file,indent=2)

here is the get part of the question, I hope that you can continue the set part of your assignment. good luck
python test.py --get name NYU.student
import json
import pprint
import argparse
def match(data: dict, filter: str):
current = data
for f in filter.split("."):
if f not in current:
return False
current = current[f]
return current == True
if __name__== "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--get", nargs="*", help="first command")
args = parser.parse_args()
with open('example.json','r') as f:
data = json.loads(f.read())
if args.get is not None and len(args.get) == 2:
attr_name = args.get[0]
if match(data, args.get[1]):
print("{}={}".format(attr_name, data[attr_name]))

In order to pass arguments using command line make use of sys module in python3. The sys module reads the command line arguments as a list of strings. The first element in the list is always the name of the file and subsequent elements are arg1, arg2 ..... so on.
Hope the following example helps to understand the usage of sys module.
Example Command :
python filename.py 1 thisisargument2 4
The corresponding code
import sys
# Note that all the command line args will be treated as strings
# Thus type casting will be needed for proper usage
print(sys.argv[0])
print(sys.argv[1])
print(sys.argv[2])
print(sys.argv[3])
Corresponding Output
filename.py
1
thisisargument2
4
Also please make a thorough google search before posting a question on stackoverflow.

command line argument followed by argparse option

I'm new to python and currently playing around with argpase. I'm trying to call a function using a directory path given as a command line argument followed by an argparse option(-name) and a regex that goes through all the files in the directory and spits out all the matches to the regex as so:
./find.py ../seek -name '[a-z]*\.txt'
However, I'm getting a error that looks like
usage: find.py [-h] [--path PATH] [-name] [--regex REGEX]
find.py: error: unrecognized arguments: . . / s e e k / p r o g r a m . c
And without the -name its just printing all the files inside the path.
Here is what I have so far:
#!/usr/bin/python2.7
import os, sys, argparse,re
from stat import *
def parse(argv=None):
parser = argparse.ArgumentParser()
parser.add_argument('--path', help='path of directory', action='store')
parser.add_argument('-name', '--name', action='store_true')
parser.add_argument('--regex', default=r"[a-z0-9A-Z]")
args = parser.parse_args(argv)
print(args)
return args
def main(argv=None):
direc = sys.argv[1]
files = []
for f in os.listdir(direc):
pathname = os.path.join(direc, f)
mode = os.stat(pathname).st_mode
if S_ISREG(mode):
args = parse(pathname)
if args.name:
dirls = [re.match(args.regex, pathname)]
print(dirls)
else:
print pathname
if __name__ == '__main__':main()
Any thoughts?

Argument Parser PATH Example : Different type of arguments with custom handlers added. For path here you can pass '-path' followed by path value as argument
import os
import argparse
from datetime import datetime
def parse_arguments():
parser = argparse.ArgumentParser(description='Process command line arguments.')
parser.add_argument('-path', type=dir_path)
parser.add_argument('-e', '--yearly', nargs = '*', help='yearly date', type=date_year)
parser.add_argument('-a', '--monthly', nargs = '*',help='monthly date', type=date_month)
return parser.parse_args()
def dir_path(path):
if os.path.isdir(path):
return path
else:
raise argparse.ArgumentTypeError(f"readable_dir:{path} is not a valid path")
def date_year(date):
if not date:
return
try:
return datetime.strptime(date, '%Y')
except ValueError:
raise argparse.ArgumentTypeError(f"Given Date({date}) not valid")
def date_month(date):
if not date:
return
try:
return datetime.strptime(date, '%Y/%m')
except ValueError:
raise argparse.ArgumentTypeError(f"Given Date({date}) not valid")
def main():
parsed_args = parse_arguments()
if __name__ == "__main__":
main()

In order for your program to operate, you need a path. So, the --path option must take an argument. Modify your parse() function to change the line
parser.add_argument('--path', help='path of directory', action='store')
to
parser.add_argument('--path', help='path of directory', action='store', required=True)
You need to call parse_args() only once. Remove the parse() invocation to the top of the loop.
And you needn't do
direc = sys.argv[1]
if you are using argparse.
re.match() returns a match object, which is probably not what you want to print.
You might want to take a look at match() versus search().
The match() function only checks if the RE matches at the beginning of the string while search() will scan forward through the string for a match.
If you wanted to print the file names matching the regex, you could do
if S_ISREG(mode):
#args = parse(pathname)
if args.name:
#dirls = re.match(args.regex, pathname)
dirls = re.search(args.regex, pathname)
if( dirls ):
print(pathname)
else:
print pathname
So main() should be something like
def main(argv=None):
args = parse(sys.argv[1:])
print(args)
#direc = sys.argv[1]
direc = args.path
files = []
for f in os.listdir(direc):
pathname = os.path.join(direc, f)
mode = os.stat(pathname).st_mode
if S_ISREG(mode):
#args = parse(pathname)
if args.name:
#dirls = re.match(args.regex, pathname)
dirls = re.search(args.regex, pathname)
if( dirls ):
print(pathname)
else:
print pathname
In order to specify the regex matching the file names, you must specify the regex using the --regex option. By default, you've made it to match names having only numbers and (English) letters.
./find.py --path ../seek -name --regex [a-z]\*.txt
or
./find.py --path ../seek -name --regex '[a-z]*.txt'

Append to file succeeding but calling error from another def

When I specify the -a switch to add to config file for some reason this calls the read_config file also. For example I use ./script.py -a newfile.txt and the file is added correctly but then returns "Config File not found".
parser = argparse.ArgumentParser(description='Copy multiple Files from a specified data file')
parser.add_argument('-c', '--configfile', default="config.dat", help='file to read the config from')
parser.add_argument('-l', '--location', default="/home/admin/Documents/backup/",help='Choose location to store files')
parser.add_argument('-a', '--add', help='add new line to config data')
def read_config(data):
try:
dest = '/home/admin/Documents/backup/'
# Read in date from config.dat
data = open(data)
# Interate through list of files '\n'
filelist = data.read().split('\n')
# Copy through interated list and strip white spaces and empty lines
for file in filelist:
if file:
shutil.copy(file.strip(), dest)
except FileNotFoundError:
logger.error("Config file not found")
print ("Config File not found")
Not quite sure why this section below is working but read_config is called and finding the except error. Where am I going wrong?
def addtofile(add):
f = open('config.dat', "a")
f.write(add + '\n')
f.close()
args = vars(parser.parse_args())
read = read_config(args['configfile'])
add = addtofile(args['add'])

parser = argparse.ArgumentParser()
parser.add_argument('-c', action='store_true')
a = parser.parse_args()
if a.c:
{
print("get custom config")
}
else:
{
print("using default config file")
}

I think your solution will be:
parser = argparse.ArgumentParser()
parser.add_argument('-c', action='store_true')
a = parser.parse_args()
if a.c:
{
print("get custom config")
}
else:
{
print("using default config file")
}

Adding an additional argument to an argparse argument

I'm new to using the arparse module in python and am hoping someone can help me with the following problem.
I am specifying a variable number of files as inputs using:
parser = argparse.ArgumentParser(description='Get Files')
parser.add_argument('-i','--input', help='Input file(s)',required=True, nargs='+')
args = parser.parse_args()
I would like to specify a variable number of file inputs, each with an associated value of 1 or 2 and am not sure how to do this.
I would like the program to work so that my command line entry should be:
MyProgram.py -i myfile.txt 2 secondfile.txt 1 ...
Once I have this working how do I call each file in the program?

Your code is functional. You could use the grouper recipe to loop through args.input two items at a time:
import argparse
parser = argparse.ArgumentParser(description='Get Files')
parser.add_argument('-i','--input', help='Input file(s)',required=True, nargs='+')
args = parser.parse_args()
for filename, num in zip(*[iter(args.input)]*2):
print(filename, num)
# with open(filename) as f:
# ....
yields
('myfile.txt', '2')
('secondfile.txt', '1')

It would be clearer to have -i once for each pair of inputs, like this:
parser.add_argument("-i", "--input", nargs=2, action='append')
Now, args.input would be a list of lists, like so
[ ['myfile.txt', 2], ['secondfile.txt', 1] ]
It does require slightly more typing for the user, since -i needs to be explicitly typed once per file.
Another option is to specify each argument as a single word, then parse the word using the type argument. I would get rid of the -i argument as well and use positional arguments for required "options".
parser.add_argument('input', nargs='+', type=lambda x: x.rsplit(":", 2))
Usage would be
myscript.py myfile.txt:1 secondfile.txt:2 ...

You might be best off using sys.argv as,
import sys, os
variables = list()
filenames = list()
if( len( sys.argv ) > 1 ):
if( sys.argv[1] == '-i' )or( sys.argv[1] == '--input' ):
N = len( sys.argv[1:] )
for i in range( 2, N, 2 ):
variables.append( int( sys.argv[i] ) )
filenames.append( sys.argv[i+1] )
for file in filenames:
# do something here
os.system( 'less '+file )
I haven't tested this, but that should work.

Try this:
parser = argparse.ArgumentParser(description='Get Files')
parser.add_argument('-i','--input', help='Input file(s)',
required=True, nargs='+', action='append')
args = parser.parse_args()

append dictionary, save to a file and increment dictionary regularly

I create a dictionary and save it to a file using json. The code takes input and updates the dictionary regularly, but unfortunately I can't get it to write the dictionary properly.
Following is the code that I have written. Have a look in here :
import os, sys, pickle, re, json
from optparse import OptionParser
parser = OptionParser("Store Daily Intakes \n python [Options] <-h help>")
parser.add_option("-n", "--ndays", dest="ndays", action="store", type="int", help="Input the day")
parser.add_option("-m", "--morning", dest="morning", action="store", type="string", help="Input the morning intake format <Banana-1pc,Bread-1pc,CottageChees-2tbs>")
parser.add_option("-l", "--lunch", dest="lunch", action="store", type="string", help="Input the Lunch intake format <Rice-2tbs,Roti-1pc,ChickenCurry-2tbs,Dal-2tbs>")
parser.add_option("-a", "--afternoon", dest="afternoon", action="store", type="string", help="Input the afternoon intake format <Cornflakes-2tbs,Banana-1pc>")
parser.add_option("-d", "--dinner", dest="dinner", action="store", type="string", help="Input the dinner intake format <Pasta-20gms, Cheese-2slice>")
(options, args) = parser.parse_args()
if options.ndays is None or options.morning is None or options.lunch is None or options.afternoon is None or options.dinner is None :
print parser.print_help()
exit(-1)
if os.path.isfile("./DailyInTakeFile.json") is True :
jout = file('./DailyInTakeFile.json','r') # read mode
CurDct = json.load(jout)
print CurDct
DailyInTake = dict()
DailyInTake["%d" % options.ndays] = {}
din = DailyInTake["%s" % options.ndays]
din['Morning'] = options.morning
din['Lunch'] = options.lunch
din['Afternoon'] = options.afternoon
din['Dinner'] = options.dinner
saved = sys.stdout
ofile = file('DailyInTakeFile.json', 'a') # append mode
for idx in CurDct.keys() :
if int(idx) == options.ndays :
print idx, options.ndays
print "The Intake for day # %d exists" %options.ndays
print "Are you sure you want to overwrite: Type [yes/no]"
lett=sys.stdin.read()
if "yes" in lett :
CurDct[idx]['Morning'] = options.morning
CurDct[idx]['Lunch'] = options.lunch
CurDct[idx]['Afternoon'] = options.afternoon
CurDct[idx]['Dinner'] = options.dinner
ofile.close()
sys.exit("Exiting after updating day # %d" % options.ndays)
else :
ofile.close()
sys.exit("Exiting without update")
else :
sys.stdout = ofile
print json.dumps(DailyInTake)
print ","
sys.stdout = saved
ofile.close()
else :
DailyInTake = dict()
DailyInTake["%d" % options.ndays] = {}
din = DailyInTake["%s" % options.ndays]
din['Morning'] = options.morning
din['Lunch'] = options.lunch
din['Afternoon'] = options.afternoon
din['Dinner'] = options.dinner
#print DailyInTake
saved = sys.stdout
ofile = file('DailyInTakeFile.json', 'a') # append mode
sys.stdout = ofile
print json.dumps(DailyInTake)
print ","
sys.stdout = saved
ofile.close()
from datetime import date, timedelta
from subprocess import call
call("cp DailyInTakeFile.json DailyInTakeFile.json.%s" % str(date.today()), shell=True)
The output json file from this code is the following for example :
{"1": {"Lunch": "l3", "Dinner": "d3", "Afternoon": "a3", "Morning": "m3"}}
{"2": {"Lunch": "l3", "Dinner": "d3", "Afternoon": "a3", "Morning": "m3"}}
As you can see it is just adding a single dictionary each time rather than appending to the first one created. I just can't think it out anymore. Any help will be appreciated.
UPDATE WITH CODE THAT MAINLY CHANGED
saved = sys.stdout
for idx in CurDct.keys() :
if int(idx) == options.ndays :
print idx, options.ndays
print "The Intake for day # %d exists" %options.ndays
print "Are you sure you want to overwrite: Type [yes/no]"
lett=sys.stdin.read()
if "yes" in lett :
ofile = file('DailyInTakeFile.json', 'w') # write mode
sys.stdout = ofile
CurDct.update(DailyInTake)
print json.dumps(CurDct)
sys.stdout = saved
ofile.close()
sys.exit("Exiting after updating day # %d" % options.ndays)
else :
sys.exit("Exiting without update")
else :
ofile = file('DailyInTakeFile.json', 'w') # write mode
sys.stdout = ofile
CurDct.update(DailyInTake)
print json.dumps(CurDct)
sys.stdout = saved
ofile.close()

According to the code, you create a new dictionary every time. And don't append to the old one in the file. DailyInTake = dict() So output to the file, just appends a new dictionary.
My suggestion would be.
To add the new dictionary index to CurDct as CurDct[index] = DailyInTake[index], then dump the whole dictionary back to the file. You can open the file for writing other than appending.

Using JSON serialisation as a mutable data storage backend seems like a rather odd solution. Without looking at your code in detail, I suggest using one of the solutions that are meant to be used this way. The most suitable one for this case seems to be the shelve module.