I'm writing a simulation software which should support reading parameters from a config file or from the command line. It is very important to be able to track what was the configuration of a simulation, I'm committing the config file to a local git repository at the start of simulation.
Now if I have parameters on the command line they have higher priority than the ones in the config file. But I also want to commit them. I guess I could save the python objects of a configured simulation, just before it is started. But it would be more elegant, if I could just update the config file with the command line parameters before committing it.
The reason I write the config file in python is that I have to define some python objects in it. I have something like
import SomeSimulationClass
SIMULATOR = SomeSimulationClass
in my config file and the SIMULATOR can then be swapped easily.
If I want to use something like configparser I can't have objects I believe.
Is there any easy way to update a python config file? All variable names in it are already defined, I just want to change the values. The only thing I can think of is parsing the file, comparing strings between the file and the command line parameters ...
You can write whatever you want into a file, and then later, Configparser can read
from it using values from your variables. Here is an example on how I used Configparser to read environment from config file.
import os
from ConfigParser import SafeConfigParser
conf_filename = os.getenv("CONFIG_FILE")
src_dir = os.getenv("CONFIG_DIR")
conf_file = os.path.join(src_dir,conf_filename)
parser = SafeConfigParser()
parser.read(conf_file)
section = env
server = parser.get(section, 'host')
db_port = parser.get(section, 'db_port')
ws_port = parser.get(section, 'ws_port')
and the config file itself:
[PROD]
host=xxx-yyy-15
db_port=1521
ws_port=8280
ora_server=xxx-xxx-xxx.com
sid=XXXXX
userid=xxxx
passwd=xxxx
[STAGE]
host=xxx-yyy-04
db_port=1521
ws_port=8280
ora_server=yyy-yyy-yyy.com
sid=YYYYYY
userid=yyyy
passwd=yyyy
I found a way to do what I want. Some small modifications were necessary to my python config module to allow it to be rewritten with the following script, but it works for my purposes:
with open('merged_config.py', 'w') as merged_config, \
open(base_config_module.__file__, 'r') as base_config:
for line in base_config:
if 'import' in line:
# copy imports from bas config
merged_config.write(line)
for item in dir(base_config_module):
if item.startswith("__"):
# ignore __variables like '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__' ...
continue
if item == 'SimulationSteps':
# ingoring my imports
continue
item_val = getattr(base_config_module, item)
# I had to overwrite the __repr__() method of Enums which I used. Everyting else worked fine.
merged_config.write('%s = %s\n' % (item, repr(item_val)))
Related
I am trying to implement hostname like module and my target machine in an amazon-ec2. But When I am running the script its giving me below error:
[ansible-user#ansible-master ~]$ ansible node1 -m edit_hostname.py -a node2
ERROR! this task 'edit_hostname.py' has extra params, which is only allowed in the following modules: meta, group_by, add_host, include_tasks, import_role, raw, set_fact, command, win_shell, import_tasks, script, shell, include_vars, include_role, include, win_command
My module is like this:
#!/usr/bin/python
from ansible.module_utils.basic import *
try:
import json
except ImportError:
import simplejson as json
def write_to_file(module, hostname, hostname_file):
try:
with open(hostname_file, 'w+') as f:
try:
f.write("%s\n" %hostname)
finally:
f.close()
except Exception:
err = get_exception()
module.fail_json(msg="failed to write to the /etc/hostname file")
def main():
hostname_file = '/etc/hostname'
module = AnsibleModule(argument_spec=dict(name=dict(required=True, type=str)))
name = module.params['name']
write_to _file(module, name, hostname_file)
module.exit_json(changed=True, meta=name)
if __name__ == "__main__":
main()
I don't know where I am making the mistake. Any help will be greatly appreciated. Thank you.
When developing a new module, I would recommend to use the boilerplate described in the documentation. This also shows that you'll need to use AnsibleModule to define your arguments.
In your main, you should add something like the following:
def main():
# define available arguments/parameters a user can pass to the module
module_args = dict(
name=dict(type='str', required=True)
)
# seed the result dict in the object
# we primarily care about changed and state
# change is if this module effectively modified the target
# state will include any data that you want your module to pass back
# for consumption, for example, in a subsequent task
result = dict(
changed=False,
original_hostname='',
hostname=''
)
module = AnsibleModule(
argument_spec=module_args
supports_check_mode=False
)
# manipulate or modify the state as needed (this is going to be the
# part where your module will do what it needs to do)
result['original_hostname'] = module.params['name']
result['hostname'] = 'goodbye'
# use whatever logic you need to determine whether or not this module
# made any modifications to your target
result['changed'] = True
# in the event of a successful module execution, you will want to
# simple AnsibleModule.exit_json(), passing the key/value results
module.exit_json(**result)
Then, you can call the module like so:
ansible node1 -m mymodule.py -a "name=myname"
ERROR! this task 'edit_hostname.py' has extra params, which is only allowed in the following modules: meta, group_by, add_host, include_tasks, import_role, raw, set_fact, command, win_shell, import_tasks, script, shell, include_vars, include_role, include, win_command
As explained by your error message, an anonymous default parameter is only supported by a limited number of modules. In your custom module, the paramter you created is called name. Moreover, you should not include the .py extension in the module name. You have to call your module like so as an ad-hoc command:
$ ansible node1 -m edit_hostname -a name=node2
I did not test your module code so you may have further errors to fix.
Meanwhile, I still strongly suggest you use the default boilerplate from the ansible documentation as proposed in #Simon's answer.
I'm new to Python. This is my first Ansible module in order to delete the SimpleDB domain from ChaosMonkey deletion.
When tested in my local venv with my Mac OS X, it keeps saying
Module unable to decode valid JSON on stdin. Unable to figure out
what parameters were passed.
Here is the code:
#!/usr/bin/python
# Delete SimpleDB Domain
from ansible.module_utils.basic import *
import boto3
def delete_sdb_domain():
fields = dict(
sdb_domain_name=dict(required=True, type='str')
)
module = AnsibleModule(argument_spec=fields)
client = boto3.client('sdb')
response = client.delete_domain(DomainName='module.params['sdb_domain_name']')
module.exit_json(changed = False, meta = response)
def main():
delete_sdb_domain()
if __name__ == '__main__':
main()
And I'm trying to pass in parameters from this file: /tmp/args.json.
and run the following command to make the local test:
$ python ./delete_sdb_domain.py /tmp/args.json
please note I'm using venv test environment on my Mac.
If you find any syntax error in my module, please also point it out.
This is not how you should test your modules.
AnsibleModule expects to have specific JSON as stdin data.
So the closest thing you can try is:
python ./delete_sdb_domain.py < /tmp/args.json
But I bet you have your json file in wrong format (no ANSIBLE_MODULE_ARGS, etc.).
To debug your modules you can use test-module script from Ansible hacking pack:
./hacking/test-module -m delete_sdb_domain.py -a "sdb_domain_name=zzz"
How to know which file is calling which file in filesystem, like file1.exe is calling file2.exe
so file2.exe is modified,
and file1.exe is entered in log file.
winos
I have searched INTERNET but not able to find any samples.
In order know which file is calling which file you can use the Trace module
exp: if you have 2 files
***file1.py***
import file2
def call1():
file2.call2()
***file2.py***
def call2():
print "---------"
u can use it using console:
$ python -m trace --trackcalls path/to/file1.py
or within a program using a Trace object
****tracefile.py***
import trace,sys
from file1 import call1
#specify what to trace here
tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix], trace=0, count=1)
tracer.runfunc(call1) #call the function call1 in fille1
results = tracer.results()
results.write_results(summary=True, coverdir='.')
I've been trying to find the answer to this but everything i look at is reading a file from another directory or running a file from another directory. I'm sure what i want to do isn't that hard but i am new at this and don't know how to describe what it's called.
I have a python script run.py that is in the /src directory. I run that script form the /src directory. run.py calls two files (configuration.py and server.py). These two files are in a folder called lib (src/lib).All folders have an empty __init__.py file.
When i take these files out of lib and put them just in src i can run the script when the script looks like it does below.
import os
import inspect
import sys
import configuration
import server
# Initialize Parameters
f_path = os.path.abspath(inspect.getfile(inspect.currentframe()))
absolute_path = os.path.dirname(f_path)
if __name__ == "__main__":
from optparse import OptionParser, OptionGroup
parser = OptionParser()
parser.usage = "usage: %prog [options] "
parser.description = "Primary application entry point."
parser.add_option("-v", "--verbose", dest="verbose", action="store_true",
default=False, help="Run verbose.")
group = OptionGroup(parser, "Node operations")
group.add_option("--do-queue-job", dest="do_queue_job", action="store_true",
help="Run the next job in the quasar-server queue.")
parser.add_option_group(group)
(options, args) = parser.parse_args()
# Clear argv to prevent issues with web.py automatically using arguments to
# bind ip addresses.
sys.argv = []
configuration = configuration.Configuration("/home/mscarpa/PhpstormProjects/quasar-node/quasar-node/quasar-node/src/config.yml")
if (options.do_queue_job):
# Get server instance
server_connection = server.QuasarConnection(configuration)
#return server_connection
# Get next job from server
next_job = server_connection.get_next_job()
#return next_job
The two parts of the code i know i have to change if i move the two files to /src/lib are the following:
configuration = configuration.Configuration("/home/mscarpa/PhpstormProjects/quasar-node/quasar-node/quasar-node/src/config.yml")
server_connection = server.QuasarConnection(configuration)
i am thinking that i would just have to put.lib before them like so, but every time i try it it says lib is not defined.
configuration = lib.configuration.Configuration("/home/mscarpa/PhpstormProjects/quasar-node/quasar-node/quasar-node/src/config.yml")
server_connection = lib.server.QuasarConnection(configuration)
This is probably a noob question, but does anyone know how to target these files if the are in the src/lib directory as opposed to just the src directory
You just need to change your import statement to reflect the module's new location:
import lib.configuration as configuration
import lib.server as server
And the rest of your script doesn't really need to change.
I got it. I think your answer may have worked in certain cases but i think my problem being new at this is figuring out what to search.
it was a sys.arg thing so i had to include that path to that lib folder before i imported the files.
sys.path.insert(0, '/home/mscarpa/PhpstormProjects/quasar-node/quasar-node/quasar-node/src/lib')
import configuration
import server
I'm learning Python and I'm have some issues.
I got this error message:
Traceback (most recent call last):
File "main_console.py", line 8, in <module>
from util import Util
File "../utils/util.py", line 13, in <module>
class Util:
File "../utils/util.py", line 73, in Util
config.write(configfile)
NameError: name 'config' is not defined
Following is my code (this is inside a function):
config = ConfigParser.ConfigParser()
for index, list in enumerate(my_list):
config.add_section(str(index))
config.set(str(index), 'id', list.name)
config.set(str(index), 'host', list.host)
# Creating the folder
myFolder = "/etc/element/"
if not os.path.exists(myFolder):
os.makedirs(myFolder)
# Creating the file
filePath = "/etc/element/settings.cfg"
with open(filePath, 'wb') as configfile:
config.write(configfile)
Note: I'm using Sublime Text 3 as IDE. "myFolder" have the same problem a little time ago - if I type filePath = myFolder + '/settings.cfg' I got the same error of "not defined". Finally, I imported ConfigParser as following: import ConfigParser.
Is Python forgetting my variable name or I'm doing something wrong?
It is possible ConfigParser or some other import also defines a config variable that is being used by your config.write().
Sometimes it is safer to just import the functions you need, that way you know exactly what is defined in your file. It means you have to explicitly import everything that you use in other files, but it prevents any unknown duplication in your namespace.
You would do this like:
from ConfigParser import ConfigParser # instead of import ConfigParser
# Then
config = ConfigParser()
Secondly, config is a common variable - try renaming it to something like myConfig and see if it still happens.
configfile is the object that you want to use for write(). You are using a parser to write.
Thanks for replies, but I found the solution.
I go to Nano and edit the file with this error, so I see the indentation is absolutaly wrong. It was just a Sublime text 3 issue, now's solved.