We Keep Coding

How to call input parameters into another python file using argparse?

I have the following files in a python pipeline
#in_para.py
nx = 31
c = 1
#solver.py
import numpy
import os
def simple(output):
ny = nx + 5
c_2 = c + 2
with open(os.path.join(output, 'Output.txt'), 'w') as f:
print("Hello stackoverflow!", file=f)
print("I have a question.", file=f)
if __name__=='__main__':
simple()
#main.py
import os
import numpy
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-C','--Chk',type=str, help='Choose arg')
parser.add_argument('-O','--output',type=str, default="./Output", help=' Provide the output Directory')
args = vars(parser.parse_args())
output = args['output']
if not os.path.exists(output):
os.makedirs(output)
if args['Chk'] == 'compo1':
simple(output)
if __name__=='__main__':
main()
I would like to call the input file in_para.py through command line argument such that
python3 main.py -I in_para -C compo1 -O Ouput_dir
gives me the desired output:
this is a simple test 36
this is a simple test2 3
I know if I do from in_para import * it will solve the problem (in a different manner), but I would like to call the input parameters from the command line as a positional argument and pass it on to solver.py. I haven't been able to find an example in the documentation similar to above task. Besides, the above is just an example, in_para.py and solver.py has several input parameters and several lines of code, so I dont want the user of main.py to go into either file and modify it.

Change the implementation in solver.py
def simple(nx, c, output): # provide parameters as function arguments
ny = nx + 5
c_2 = c + 2
print("this is a simple test", ny)
print("this is a simple test2", c_2)
with open(os.path.join(output, 'Output.txt'), 'w') as f:
print("Hello stackoverflow!", file=f)
print("I have a question.", file=f)
In the main parse the parameters file
... # other imports
import json
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-C','--Chk', type=str, help='Choose arg')
parser.add_argument('-I','--input', type=argparse.FileType("r"), help='Input parameters')
parser.add_argument('-O','--output', type=str, help='output dir')
args = parser.parse_args()
output = args.output
# implement any way of parsing the input file, e.g. a json file
params = json.load(args.input)
... # do something else with the arguments
if args.Chk == 'compo1':
simple(output=output, **params)
... # further code lines
Your inputs file would look like
{"nx": 31, "c": 1}
Comment on edit:
Either you use args = vars(parser.parse_args()) which results in a dictionary or you just use the args = parser.parse_args() to select arguments by attribute. In your original post you used the first approach which I adopted in the first answer. However, I would prefer not using vars.
Note, that the code snippet to create the directory does not add value to the actual question of the post, so I removed it to avoid any distraction and added some placeholder instead.

How do I pass arguments from command line to the python script to read and update values of certain keys present in a json file

I have certain data in a json file (say, example.json),
example.json
data = {
'name' : 'Williams',
'working': False,
'college': ['NYU','SU','OU'],
'NYU' : {
'student' : True,
'professor': False,
'years' : {
'fresher' : '1',
'sophomore': '2',
'final' : '3'
}
}
}
I wish to write a code wherein I can give the arguments on Command line, i.e. suppose if a script is saved in a file 'script.py', then,
In the terminal: If I enter *$ python3* script.py --get name --get NYU.student Then it outputs name=Williams
NYU.student=True
If I enter *$ python3* script.py' --set name=Tom --set NYU.student=False
Then, it updates name and NYU.student keys in the dictionay to Tom and False and outputs NYU.student=Tom and NYU.student=False on the command line.
I have tried the following code for the python script (i.e. script.py)
script.py
import json
import pprint
import argparse
if __name__== "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--get", help="first command")
parser.add_argument("--set", help="second command")
args=parser.parse_args()
with open('example.json','r') as read_file:
data=json.load(read_file)
if args.set == None:
key = ' '.join(args.get[:])
path = key.split('.')
now = data
for k in path:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
print(now)
elif args.get == Null:
key, value = ' '.join(args.set[:]).split('=')
path = key.split('.')
now = data
for k in path[:-1]:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
now[path[-1]] = value
with open('example.json','w') as write_file: #To write the updated data back to the same file
json.dump(data,write_file,indent=2)
However, my script is not working as I expect it to? Kindly, help me with the script

Your code has the following issues:
When joining the argument values in line number 23 and 35, you use a space. This leads to the "Error key" value. Removing the space will solve the issue.
key = ''.join(arg[:])
You defined the arguments to only pass one value. Not multiple. Therefore even if you pass multiple --get or --set values, the script only gets one value. Adding action="append" to line number 9 and 10 will solve the issue.
parser.add_argument("--get", help="first command", action="append")
parser.add_argument("--set", help="second command", action="append")
Full code:
import json
import pprint
import argparse
if __name__== "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--get", help="first command", action="append")
parser.add_argument("--set", help="second command", action="append")
args=parser.parse_args()
try:
with open('example.json','r') as read_file:
data=json.load(read_file)
except IOError:
print("ERROR: File not found")
exit()
if args.set == None:
for arg in args.get:
key = ''.join(arg[:])
path = key.split('.')
now = data
for k in path:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
print(f"{arg} = {now}")
elif args.get == None:
for arg in args.set:
key, value = ''.join(arg[:]).split('=')
path = key.split('.')
now = data
for k in path[:-1]:
if k in now:
now = now[k]
else:
print('Error: Invalid Key')
print(f"{arg}")
now[path[-1]] = value
with open('example.json','w') as write_file: #To write the updated data back to the same file
json.dump(data,write_file,indent=2)

here is the get part of the question, I hope that you can continue the set part of your assignment. good luck
python test.py --get name NYU.student
import json
import pprint
import argparse
def match(data: dict, filter: str):
current = data
for f in filter.split("."):
if f not in current:
return False
current = current[f]
return current == True
if __name__== "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--get", nargs="*", help="first command")
args = parser.parse_args()
with open('example.json','r') as f:
data = json.loads(f.read())
if args.get is not None and len(args.get) == 2:
attr_name = args.get[0]
if match(data, args.get[1]):
print("{}={}".format(attr_name, data[attr_name]))

In order to pass arguments using command line make use of sys module in python3. The sys module reads the command line arguments as a list of strings. The first element in the list is always the name of the file and subsequent elements are arg1, arg2 ..... so on.
Hope the following example helps to understand the usage of sys module.
Example Command :
python filename.py 1 thisisargument2 4
The corresponding code
import sys
# Note that all the command line args will be treated as strings
# Thus type casting will be needed for proper usage
print(sys.argv[0])
print(sys.argv[1])
print(sys.argv[2])
print(sys.argv[3])
Corresponding Output
filename.py
1
thisisargument2
4
Also please make a thorough google search before posting a question on stackoverflow.

using python gooey how to open another GUI after clicking one out of multiple button

Hi I have following code which is working properly for python GUI based on wxpython-gooey. However my task is to create a main GUI with multiple buttons and after clicking any button respective GUI should open and process the respective task and close. The code i have is not for main GUI but after clicking button on main GUI the below code's GUI must open, the sub-GUI code is as follows
from __future__ import print_function
import pandas as pd
import numpy as np
import glob
import os
import json
from argparse import ArgumentParser
from gooey import Gooey, GooeyParser
#Gooey(program_name="Jio Project")
def parse_args():
""" Use GooeyParser to build up the arguments we will use in our script
Save the arguments in a default json file so that we can retrieve them
every time we run the script.
"""
stored_args = {}
# get the script name without the extension & use it to build up
# the json filename
script_name = os.path.splitext(os.path.basename(__file__))[0]
args_file = "{}-args.json".format(script_name)
# Read in the prior arguments as a dictionary
if os.path.isfile(args_file):
with open(args_file) as data_file:
stored_args = json.load(data_file)
parser = GooeyParser(description='Qualcomm-Samsung a Jio project')
parser.add_argument('data_directory',
action='store',
default=stored_args.get('data_directory'),
widget='DirChooser',
help="Source directory that contains Excel files")
parser.add_argument('output_directory',
action='store',
widget='DirChooser',
default=stored_args.get('output_directory'),
help="Output directory to save summary report")
parser.add_argument('cust_file',
action='store',
default=stored_args.get('cust_file'),
widget='FileChooser',
help='Customer Account Status File')
parser.add_argument('-d', help='Start date to include',
default=stored_args.get('d'),
widget='DateChooser')
args = parser.parse_args()
# Store the values of the arguments so we have them next time we run
with open(args_file, 'w') as data_file:
# Using vars(args) returns the data as a dictionary
json.dump(vars(args), data_file)
return args
def combine_files(src_directory):
""" Read in all of the sales xlsx files and combine into 1
combined DataFrame
"""
all_data = pd.DataFrame()
for f in glob.glob(os.path.join(src_directory, "sales-*.xlsx")):
df = pd.read_excel(f)
all_data = all_data.append(df, ignore_index=True)
all_data['date'] = pd.to_datetime(all_data['date'])
return all_data
def add_customer_status(sales_data, customer_file):
""" Read in the customer file and combine with the sales data
Return the customer with their status as an ordered category
"""
df = pd.read_excel(customer_file)
all_data = pd.merge(sales_data, df, how='left')
# Default everyone to bronze if no data included
all_data['status'].fillna('bronze', inplace=True)
# Convert the status to a category and order it
all_data["status"] = all_data["status"].astype("category")
all_data["status"].cat.set_categories(["gold", "silver", "bronze"], inplace=True)
return all_data
def save_results(sales_data, output):
""" Perform a summary of the data and save the data as an excel file
"""
summarized_sales = sales_data.groupby(["status"])["unit price"].agg([np.mean])
output_file = os.path.join(output, "sales-report.xlsx")
writer = pd.ExcelWriter(output_file, engine='xlsxwriter')
summarized_sales = summarized_sales.reset_index()
summarized_sales.to_excel(writer)
if __name__ == '__main__':
# maingui=main_args()
conf = parse_args()
print("Reading sales files")
sales_df = combine_files(conf.data_directory)
print("Reading customer data and combining with sales")
customer_status_sales = add_customer_status(sales_df, conf.cust_file)
print("Saving sales and customer summary data")
save_results(customer_status_sales, conf.output_directory)
print("Done")

In order to create a real custom interactive gui, I would advice you to get familier with a framwork such as mentioned here: https://wiki.python.org/moin/GuiProgramming
But in order to answer the specific question, Gooey has some little disadvantages, which make it difficult to span a new Gooey window from within the original window (I will not get into detail here), but what you can do is, spawn a new window as a subprocess. Minimal example below:
import sys
import os
from argparse import ArgumentParser
from subprocess import Popen, PIPE
from gooey import Gooey
from gooey import GooeyParser
#Gooey()
def main():
parser = GooeyParser(
description='''Test''')
required = parser.add_argument_group('Optional arguments')
parser.add_argument(
'-i', '--input',
required=False,
dest = 'input',
help='Test_file',
widget='FileChooser',
)
args = parser.parse_args()
########################################
# call the next gooey as subprocess from here
# should work on any system
########################################
PYTHON_PATH = sys.executable
process = Popen([PYTHON_PATH, 'spawn_next.py'], stdout=PIPE, stderr=PIPE)
output, error = process.communicate()
print(output)
print(error)
if __name__ == "__main__":
main()
Then you can call the file spawn_next.py, in which you can open another Goopy window like:
from gooey import Gooey
from gooey import GooeyParser
#Gooey()
def main():
parser = GooeyParser(
description='''Test''')
required = parser.add_argument_group('Optional arguments')
parser.add_argument(
'-i', '--input',
required=False,
dest = 'input',
help='Test_file',
widget='FileChooser',
)
args = parser.parse_args()
main()

pass json file as command line argument through parser , is this possible?

I need to overwrite json file parameters to a python dictionary through command line argument parser. Since, json file is located in the current working directory but its name can be dynamic , so i want something like below :-
python python_script --infile json_file
python_script:
if __name__ == "__main__":
profileInfo = dict()
profileInfo['profile'] = "enterprisemixed"
profileInfo['nodesPerLan'] = 50
json_file:
{
"profile":"adhoc",
"nodesPerLan" : 4
}
I tried to add the following lines, but don't know how to load this json data to the python dictionary :-
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--infile', nargs = 1, help="JSON file to be processed",type=argparse.FileType('r'))
arguments = parser.parse_args()

Read the JSON file with the name given to --infile and update your profileInfo:
import json
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--infile', nargs=1,
help="JSON file to be processed",
type=argparse.FileType('r'))
arguments = parser.parse_args()
# Loading a JSON object returns a dict.
d = json.load(arguments.infile[0])
profileInfo = {}
profileInfo['profile'] = "enterprisemixed"
profileInfo['nodesPerLan'] = 50
print(profileInfo)
# Overwrite the profileInfo dict
profileInfo.update(d)
print(profileInfo)

how to create dynamic configuration file using python

I have a python script which is controlled by a config file called system.config .the structure of config file is like bellow with some default values.
[company]
companyname: XYZ
[profile]
name: ABC
joining: 1/1/2014
the code for config file is : config_parser_details.py
import ConfigParser
import sys
Config = ConfigParser.ConfigParser()
Config.read("system.config")
filename = "system.config"
def ConfigSectionMap(section):
dict1 = {}
options = Config.options(section)
for option in options:
try:
dict1[option] = Config.get(section, option)
if dict1[option] == -1:
DebugPrint("skip: %s" % option)
except:
print("exception on %s!" % option)
dict1[option] = None
return dict1
company = ConfigSectionMap("company")['companyname']
name = ConfigSectionMap("profile")['name']
joindate = ConfigSectionMap("profile")['joining']
now the code for my script is : test.py
import config_parser_details as p
import sys
import warnings
import os
company = p.company
name = p.name
date = p.joindate
print("%s\n" %company)
print("%s\n" %name)
output is
XYZ
ABC
now I want to give input in the config file through command line.
like
python test.py --compname ="testing"
if any argument is missing in the command line than default value will be the input.

You could use argparse library to parse command line arguments.
So your test.py file looks like below :
import config_parser_details as p
import sys
import warnings
import os
import argparse
commandLineArgumentParser = argparse.ArgumentParser()
commandLineArgumentParser.add_argument("-c", "--compname", help="Company name", default=p.company)
commandLineArguments = commandLineArgumentParser.parse_args()
company = commandLineArguments.compname
name = p.name
date = p.joindate
print("%s\n" %company)
print("%s\n" %name)

I'd advise looking into a tool like docopt.
For a quick fix though, you can try doing this
def ConfigSectionMap(section):
options = Config.options(section)
arg_dict = {}
for command_line_argument in sys.argv[1:]:
arg = command_line_argument.split("=")
arg_dict[arg[0][2:]] = arg[1]
for key in arg_dict:
options[key] = arg_dict[key]
return options
This will load up all the default option. Any options put on the command line will override or add to the options dict.

First of all, I'd move code into a main section so that you can import config_parser_details without executing code:
if __name__ == '__main__':
main()
def main():
Config = ConfigParser.ConfigParser()
Config.read("system.config")
filename = "system.config"
company = ConfigSectionMap("company")['companyname']
name = ConfigSectionMap("profile")['name']
joindate = ConfigSectionMap("profile")['joining']
Secondly, I'd use STB land's suggestion of parsing the command line with argparse, something like:
def main():
# do the parsing thing first, then:
filename = args.filename
do_stuff(filename)
This way you can neatly use python's own unit test framework or nosetests to write test file that don't require you to manually specify parameters:
def test_basic():
# create a temporary file with tempfile.NamedTemporaryFile
tmpfile = tempfile.NamedTemporaryFile()
# add test data to tmpfile
do_stuff(tmpfile)
# check the output
assert ....
This comes with the added benefit of not having global variables, which will complicate your life later.