We Keep Coding

if else statement for input arguments of a function in python

I run my bash script my_file.sh in a python file as follows:
import subprocess
def rest_api():
params = {
'query': 'indepedence day',
'formats': '["NEWSPAPER"]',
}
subprocess.call(['bash',
'my_file.sh',
f'QUERY={params.get("query")}',
f'DOC_TYPE={params.get("formats")}',
f'LANGUAGE={params.get("lang")}', # returns None!
])
if __name__ == '__main__':
rest_api()
Several of my input arguments in subprocess.call do not normally exist in a dictionary params={} (here I provided f'LANGUAGE={params.get("lang")}' as one example). I handle such unavailability in my_file.sh to initialize with something, for instance:
if [ -z "$LANGUAGE" ]; then LANGUAGE="${LANGUAGE:-[]}"; fi
What I want is to apply some sort of if else statement in subprocess.call function with this logic:
if params.get("lang") is None, do not even send it as an input to bash file, e.g., treat it as I never provided such input for my_file.sh.
Therefore, I tried to rewrote my code like this:
subprocess.call(['bash',
'my_file.sh',
f'QUERY={params.get("query")}',
f'DOC_TYPE={params.get("formats")}',
if params.get("lang"): f'LANGUAGE={params.get("lang")}', # syntax Error
])
which is wrong I get the following invalid syntax error:
Traceback (most recent call last):
File "nationalbiblioteket_logs.py", line 13, in <module>
from url_scraping import *
File "/home/xenial/WS_Farid/DARIAH-FI/url_scraping.py", line 17, in <module>
from utils import *
File "/home/xenial/WS_Farid/DARIAH-FI/utils.py", line 53
if params.get("lang"): f'LANGUAGE={params.get("lang")}',
^
SyntaxError: invalid syntax
Do I have a wrong understanding of applying if else statement for the input arguments of a python function or is there an easier or cleaner way doing it?
Cheers,

You can specify the default when calling .get(), so use an empty string.
f'LANGUAGE={params.get("lang", "")}'

If you don't want the LANGUAGE= argument at all when no value is provided, you need to build the list dynamically.
cmd = ['bash',
'my_file.sh',
f'QUERY={params.get("query")}',
f'DOC_TYPE={params.get("formats")}']
if (lang := params.get("lang")) is not None:
cmd += [f'LANGUAGE={lang}']
subprocess.call(cmd)

How to detect method signatures of imported python modules

I have a python app which can load and run extensions. It looks like this:
import importlib, sys
sys.path.append('.')
module = importlib.import_module( 'extension' )
Extension = getattr(module, 'Extension')
instance = Extension()
instance.FunctionCall(arg1='foo')
The users can create their own loadable extension by implementing a class I defined. In this case, it is in extension.py:
class Extension:
def FunctionCall(self, arg1):
print(arg1)
This works great for now, but I would like to extend my interface to pass another argument into Extension.FunctionCall
instance.FunctionCall(arg1='foo', arg2='bar')
How can I do this while maintaining reverse compatibility with the existing interface?
Here's what happens to existing users if I make the change:
$ python3 loader.py
Traceback (most recent call last):
File "loader.py", line 9, in <module>
instance.FunctionCall(arg1='foo', arg2='bar')
TypeError: FunctionCall() got an unexpected keyword argument 'arg2'
One option is to catch the TypeError, but is this the best solution? It looks like it could get unmaintainable with several iterations of interfaces.
try:
instance.FunctionCall(arg1='foo', arg2='bar')
except TypeError:
instance.FunctionCall(arg1='foo')
Another option is to define the "interface version" as a property of the extension and then use that in a case statement, but it would be nicer to simply detect what is supported instead of pushing overhead to the users.
if hasattr(Extension, 'Standard') and callable(getattr(Extension, 'Standard')):
standard=instance.Standard()
else:
standard=0
if standard == 0:
instance.FunctionCall(arg1='foo')
elif standard == 1:
instance.FunctionCall(arg1='foo', arg2='bar')

How to convert type `file` in arg parser from Python 2 to Python 3?

I need to convert an xsd to a PostgreSQL schema, for which I found this Python script. Unfortunately it's a Python 2 script, so I want to convert it to Python 3. It fails on this part however:
parser.add_argument(
'xsd',
metavar='FILE',
type=file,
nargs='+',
help='XSD file to base the Postgres Schema on'
)
The error says:
Traceback (most recent call last):
File "xsd2pgsql.py", line 183, in <module>
type=file,
NameError: name 'file' is not defined
I know there's no type file any more in Python 3. How can I fix this?

The type argument will accept any callable which will take the string used on the command line and return some object of the desired type.
So in this case, just doing type=open would be equivalent to the action of type=file in Python 2, which is to return the handle of a file opened in read-only mode.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('infile', type=file)
args = parser.parse_args()
print(args.infile.readline())
gives:
$ python2 test.py /etc/passwd
root:x:0:0:root:/root:/bin/bash
$ python3 test.py /etc/passwd
Traceback (most recent call last):
File "test.py", line 5, in <module>
parser.add_argument('filename', type=file)
NameError: name 'file' is not defined
After changing to type=open
$ python3 test.py /etc/passwd
root:x:0:0:root:/root:/bin/bash
(This will also work with python 2.)
However, a greater range of file opening options is available using argparse.FileType (see documentation).
This needs to be instantiated, so the basic usage is argparse.FileType(), but for example you could use argparse.FileType(mode="w") (or just argparse.FileType("w")) to get a file opened for writing. So this code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('outfile', type=argparse.FileType(mode="w"))
args = parser.parse_args()
args.outfile.write("hello\n")
args.outfile.close()
would give this:
$ python3 test.py myoutput
$ cat myoutput
hello
although this would not be necessary for conversion of the Python 2 example in the question.

argparse comes with a file wrapper that will open a file for you.
parser.add_argument(
'xsd',
metavar='FILE',
type=argparse.FileType('r'),
help='XSD file to base the Postgres Schema on'
)
This would have been preferred in Python 2 as well.

Custom Ansible module is giving param extra params error

I am trying to implement hostname like module and my target machine in an amazon-ec2. But When I am running the script its giving me below error:
[ansible-user#ansible-master ~]$ ansible node1 -m edit_hostname.py -a node2
ERROR! this task 'edit_hostname.py' has extra params, which is only allowed in the following modules: meta, group_by, add_host, include_tasks, import_role, raw, set_fact, command, win_shell, import_tasks, script, shell, include_vars, include_role, include, win_command
My module is like this:
#!/usr/bin/python
from ansible.module_utils.basic import *
try:
import json
except ImportError:
import simplejson as json
def write_to_file(module, hostname, hostname_file):
try:
with open(hostname_file, 'w+') as f:
try:
f.write("%s\n" %hostname)
finally:
f.close()
except Exception:
err = get_exception()
module.fail_json(msg="failed to write to the /etc/hostname file")
def main():
hostname_file = '/etc/hostname'
module = AnsibleModule(argument_spec=dict(name=dict(required=True, type=str)))
name = module.params['name']
write_to _file(module, name, hostname_file)
module.exit_json(changed=True, meta=name)
if __name__ == "__main__":
main()
I don't know where I am making the mistake. Any help will be greatly appreciated. Thank you.

When developing a new module, I would recommend to use the boilerplate described in the documentation. This also shows that you'll need to use AnsibleModule to define your arguments.
In your main, you should add something like the following:
def main():
# define available arguments/parameters a user can pass to the module
module_args = dict(
name=dict(type='str', required=True)
)
# seed the result dict in the object
# we primarily care about changed and state
# change is if this module effectively modified the target
# state will include any data that you want your module to pass back
# for consumption, for example, in a subsequent task
result = dict(
changed=False,
original_hostname='',
hostname=''
)
module = AnsibleModule(
argument_spec=module_args
supports_check_mode=False
)
# manipulate or modify the state as needed (this is going to be the
# part where your module will do what it needs to do)
result['original_hostname'] = module.params['name']
result['hostname'] = 'goodbye'
# use whatever logic you need to determine whether or not this module
# made any modifications to your target
result['changed'] = True
# in the event of a successful module execution, you will want to
# simple AnsibleModule.exit_json(), passing the key/value results
module.exit_json(**result)
Then, you can call the module like so:
ansible node1 -m mymodule.py -a "name=myname"

ERROR! this task 'edit_hostname.py' has extra params, which is only allowed in the following modules: meta, group_by, add_host, include_tasks, import_role, raw, set_fact, command, win_shell, import_tasks, script, shell, include_vars, include_role, include, win_command
As explained by your error message, an anonymous default parameter is only supported by a limited number of modules. In your custom module, the paramter you created is called name. Moreover, you should not include the .py extension in the module name. You have to call your module like so as an ad-hoc command:
$ ansible node1 -m edit_hostname -a name=node2
I did not test your module code so you may have further errors to fix.
Meanwhile, I still strongly suggest you use the default boilerplate from the ansible documentation as proposed in #Simon's answer.

How to execute multiple class instance in Python with Argparse

My project has several categories with multiple scripts inside, each script allows you to perform a unique task.
To run my project, I give my main.py arguments that I retrieve with argparse.
Example: ./main.py --category web --script S1_HTTP_HEADER --url https://qwerty.com --port 1234
However, I need to create an argument to run all my scripts at once.
Example : ./main.py --category web --all --url https://qwerty.com --port 1234
Actually, my main.py looks like this :
if args.category == "web":
if args.script == 'S1_HTTP_HEADER':
from scripts.WEB_S1_HTTPHEADER import RequestHeader
make_request = RequestHeader(args.url, args.port)
make_request.insert_value()
Do you have solutions to run all scripts at once with only one argument?
For information, each script has a class that I have to instantiate with a URL and a PORT. Of course I have to import my class in my main.py before the operation.
Thank's !

I would define a custom parsing action to load all callables respective to the category.
That class defines a mapping binding both categories and associated callables.
Each callable is a tuple composed of the script name and the associated callable object.
An additional argument "scripts" is created by the custom action as a list object. Empty by default.
Import mechanism needs to be changed a little too.
from scripts import WEB_S1_HTTPHEADER, WEB_S2_HTTPHEADER, WEB_S3_HTTPHEADER
class GetScripts(argparse.Action):
"""
"""
SCRIPTS = {"web": [("S1_HTTPHEADER", WEB_S1_HTTPHEADER.RequestHeader), ("S2_HTTPHEADER", WEB_S2_HTTPHEADER.RequestHeader), ("S3_HTTPHEADER", WEB_S3_HTTPHEADER.RequestHeader)]}
def __init__(self, option_strings, dest, **kwargs):
super(GetScripts, self).__init__(option_strings, dest, **kwargs)
def __call__(self, parsobj, namespace, values, option_string=None):
setattr(namespace, self.dest, values)
if values:
category = getattr(namespace, "category")
setattr(namespace, "scripts", self.SCRIPTS.get(category, []))
Define the keyword argument "--all" associated to the custom action:
your_parser.add_argument("--all", nargs="?", default=False, const=True, action=GetScripts)
You can then iterate over the "scripts" argument to run each configured callable.
main.py now looks like:
if args.all:
for _, func in args.scripts:
make_request = func(args.url, args.port)
make_request.insert_value()
A new category or a new script means however changing the class member SCRIPTS.
Does it answer to your need?