I'm trying to write my first module in Ansible, which is essentially a wrapper around another module. Here is my module:
#!/usr/bin/python
import ansible.runner
import sys
def main():
module.exit_json(changed=False)
from ansible.module_utils.basic import *
main()
and here is the error it gives me (stripped from 'msg'):
ImportError: No module named ansible.runner
I am on ubuntu and installed ansible with aptitude, version is 1.9.1
Any ideas?
Modules have to essentially be standalone. The boilerplate gets injected at runtime (the text of the boilerplate replaces the import at the bottom), and the combined text of the module + boilerplate is squirted to the remote machine and run there. As such, you can't import things from ansible core like the runner (unless you install ansible on the remote machine- don't be that guy). "module" is one of the items that you have to create from stuff defined in the boilerplate. Here's a sample module skeleton I wrote:
#! /usr/bin/python
import json
def main():
module = AnsibleModule(
argument_spec = dict(
state = dict(default='present', choices=['present', 'absent'])
),
supports_check_mode = True
)
p = module.params
changed = False
state = p['state']
if not module.check_mode:
# do stuff
pass
#module.fail_json(msg='it broke')
module.exit_json(changed=changed)
from ansible.module_utils.basic import *
main()
I just checked a module I wrote a while back and I don't have such an import line. The only import I have is from ansible.module_utils.basic import *. The module object I create myself in main:
module = AnsibleModule(
argument_spec=dict(
paramA=dict(required=True),
paramB=dict(required=False),
paramC=dict(required=False),
),
add_file_common_args=True,
supports_check_mode=True
)
Related
I have a module that works fine in python 3.5+ but not in 3.4. The only possible change that may be effecting it is how circular imports are handled in 3.5+. I cannot find any circular imports though so there may be something else going on.
module/
module/
__init__.py
file_a.py
from module import settings
from module.file_b import SomeBClass
def stuff():
settings.init()
stuff = SomeBClass()
def run():
stuff()
def main():
run()
file_b.py
from module.settings import config, properties
class SomeBClass():
....
file_c.py
class SomeClass():
connect to db...
settings.py
from module.file_c import SomeClass
def init():
global config
global properties
config = SomeClass()
properties = config.get_it()
when running I get the following error:
File "/home/somewhere/module/module/file_b.py", line 11, in <module>
from module.settings import config, properties
ImportError: cannot import name 'config'
I have tried running the module with python -mv to see if something gets imported more than once but I cannot see anything alarming.
Anyone have experience dealing with the differences between 3.4 and 3.5+? Does trying to access attributes from globals in the settings.init cause issues?
Hello I did got into circular dependency what is not refactori-zable other than doubling code.
I have something like this (only much more complex):
myParser.py:
import sys
import main #comment this to make it runnable
def parseEvnt():
sys.stdout.write("evnt:")
main.parseCmd(1) #comment this to make it runnable
tbl.py:
import myParser
tblx = {
1:("cmd",),
2:("evnt",myParser.parseEvnt),
}
main.py:
import tbl
def parseCmd(d):
print(tbl.tblx[d][0])
data=[1,2]
for d in data:
if(d<2):
parseCmd(d)
else:
fce = tbl.tblx[d][1]
fce()
Obvious error I'm getting is:
File "D:\Data\vbe\workspace\marsPython\testCircular2\main.py", line 1, in <module>
import tbl
File "D:\Data\vbe\workspace\marsPython\testCircular2\tbl.py", line 1, in <module>
import myParser
File "D:\Data\vbe\workspace\marsPython\testCircular2\myParser.py", line 2, in <module>
import main
File "D:\Data\vbe\workspace\marsPython\testCircular2\main.py", line 7, in <module>
parseCmd(d)
File "D:\Data\vbe\workspace\marsPython\testCircular2\main.py", line 3, in parseCmd
print(tbl.tblx[d][0])
AttributeError: module 'tbl' has no attribute 'tblx'
In C I think I would just tell by declaration in tbl.py hey there is function parseEvnt(). I would not need to include myParser and there would be no circular include.
In python I do not know how to do it.
I read few threads and there is always some wise guy recommending refactorizing. But in this case parseCmd() needs to see tblx which needs to see parseEvnt() (unless function declaration) and parseEvnt() need to call parseCmd() (cos evnt contains triggering cmd and I do not want to double the decoding cmd code).
Is there way how to make it working in python?
You can frequently get away with circular dependencies as long as the modules don't try to use each other's data until all importing is complete - as a practical matter, that means referencing with namespace (from module import something is forbidden) and only using the other modules inside functions and methods (no mystuff = module.mystuff in the global space). That's because when importing starts, python puts the module name in sys.modules and won't try to import that module again.
You ran into trouble because when you run main.py, python adds __main__ to sys.modules. When the code finally came around to import main, there was no "main" in the module list and so main.py was imported again... and its top level code tried to run.
Lets rearrange your test case and throw in a few print statements to tell when import happens.
myParser.py
print(' + importing myParser')
import sys
print('import parsecmd')
import parsecmd
def parseEvnt():
sys.stdout.write("evnt:")
parsecmd.parseCmd(1)
tbl.py
print(' + importing tbl')
print('import myParser')
import myParser
tblx = {
1:("cmd",),
2:("evnt",myParser.parseEvnt),
}
Parsecmd.py (new)
print(' + importing parsecmd')
print('import tbl')
import tbl
def parseCmd(d):
print(tbl.tblx[d][0])
main.py
print('running main.py')
print('import parsecmd')
import parsecmd
if __name__ == "__main__":
data=[1,2]
for d in data:
if(d<2):
parsecmd.parseCmd(d)
else:
fce = parsecmd.tbl.tblx[d][1]
fce()
When I run it I get
running main.py
import parsecmd
+ importing parsecmd
import tbl
+ importing tbl
import myParser
+ importing myParser
import parsecmd <-- didn't reimport parsecmd
cmd
evnt:cmd
If you're insistent on not refactoring (which is the real solution to this - not being a wise guy), you could move your problematic import into your function in myParser.py
import sys
def parseEvnt():
import main ## import moved into function
sys.stdout.write("evnt:")
main.parseCmd(1)
Again, see if you can redesign your code so such interdependencies are avoided.
The above solution is sort of a hack and won't solve future problems you might run into due to this dependency.
Circular imports should be avoided. Refactoring is required, any workaround that still requires a circular import is not a good solution.
That being said, the refactoring doesn't have to be extensive. There are at least a couple of fairly simple solutions.
Solution 1: move shared functions to a shared module
Since you want to use parseCmd from more than one place, move it to a separate file. That way both main.py and myParser.py can import the function.
Solution 2: have main pass parseCmd to parseEvnt.
First, make parseEvnt accept an argument to tell it which function to run:
# myParser.py
import sys
def parseEvnt(parseCmd):
sys.stdout.write("evnt:")
parseCmd(1)
Next, when you call myParser.parseEvnt, pass in a reference to main.parseCmd:
# main.py:
...
else:
fce = tbl.tblx[d][1]
fce(parseCmd)
There are other ways to accomplish the same thing. For example, you could add a "configure" method in myParser, and then have main.py call the configure method and pass in a reference to its parseCmd. The configure method can then store this reference in a global variable.
Another choice is to import main in the function that uses it:
main.py
import sys
def parseEvnt():
import main
sys.stdout.write("evnt:")
main.parseCmd(1)
I want to create a global logging object in a Python module that is imported by many small scripts. The Python module is designed to provide a consistent setup of things like logging, logos, timing etc. for all scripts. I'm trying to set up the logging in this module in order to make it easy to change the logging characteristics of all of these scripts at once.
The relevant part of the Python module is as follows:
if engageLog:
global log
log = logging.getLogger(__name__)
logging.root.addHandler(technicolor.ColorisingStreamHandler())
How could I write this such that the logging object is available in the scripts without requiring any setup beyond importing the module?
In the following example script, the module is called propyte:
#!/usr/bin/env python
"""
################################################################################
# #
# script-1 #
# #
################################################################################
Usage:
script-1 [options]
Options:
-h, --help display help message
--version display version and exit
-v, --verbose verbose logging
-u, --username=USERNAME username
--data=FILENAME input data file [default: data.txt]
"""
name = "script-1"
version = "2015-10-21T1331Z"
import os
import sys
import time
import docopt
import propyte
def main(options):
global program
program = propyte.Program(options = options)
# access options and arguments
input_data_filename = options["--data"]
log.info("")
log.info("input data file: {filename}".format(
filename = input_data_filename
))
print("")
program.terminate()
if __name__ == "__main__":
options = docopt.docopt(__doc__)
if options["--version"]:
print(version)
exit()
main(options)
Just use the log attribute on the module:
import propyte
propyte.log # the global log object in the propyte module
Note however that the logging module configuration is already global. You can use logging.getLogger('somename') and it'll return a singleton logger object associated with that name. Configuration applied to that object will still be present when you retrieve the same logger in another module.
I know that if I import a module by name import(moduleName), then I can reload it with reload(moduleName)
But, I am importing a bunch of modules with a Kleene star:
from proj import *
How can I reload them in this case?
I think there's a way to reload all python modules. The code for Python 2.7 is listed below: Instead of importing the math module with an asterisk, you can import whatever you need.
from math import *
from sys import *
Alfa = modules.keys()
modules.clear()
for elem in Alfa:
str = 'from '+elem+' import *'
try:
exec(str)
except:
pass
This is a complicated and confusing issue. The method I give will reload the module and refresh the variables in the given context. However, this method will fall over if you have multiple modules using a starred import on the given module as they will retain their original values instead of updating. In generally, even having to reload a module is something you shouldn't be doing, with the exception of when working with a REPL. Modules aren't something that should be dynamic. You should consider other ways of providing the updates you need.
import sys
def reload_starred(module_name, context):
if context in sys.modules:
context = vars(sys.modules[context])
module = sys.modules[module_name]
for name in get_public_module_variables(module):
try:
del context[name]
except KeyError:
pass
module = reload(module)
context.update(get_public_module_variables(module))
def get_public_module_variables(module):
if hasattr(module, "__all__"):
return dict((k,v) for (k,v) in vars(module).items()
if k in module.__all__)
else:
return dict((k,v) for (k,v) in vars(module).items()
if not k.startswith("_"))
Usage:
reload_starred("my_module", __name__)
reload_starred("my_module", globals())
reload_starred("my_module", "another_module")
def function():
from my_module import *
...
reload_starred("my_module", locals())
Either it's lack of sleep but I feel silly that I can't get this. I have a plugin, I see it get loaded but I can't instantiate it in my main file:
from transformers.FOMIBaseClass import find_plugins, register
find_plugins()
Here's my FOMIBaseClass:
from PluginBase import MountPoint
import sys
import os
class FOMIBaseClass(object):
__metaclass__ = MountPoint
def __init__(self):
pass
def init_plugins(self):
pass
def find_plugins():
plugin_dir = os.path.dirname(os.path.realpath(__file__))
plugin_files = [x[:-3] for x in os.listdir(plugin_dir) if x.endswith("Transformer.py")]
sys.path.insert(0, plugin_dir)
for plugin in plugin_files:
mod = __import__(plugin)
Here's my MountPoint:
class MountPoint(type):
def __init__(cls,name,bases,attrs):
if not hasattr(cls,'plugins'):
cls.plugins = []
else:
cls.plugins.append(cls)
I see it being loaded:
# /Users/carlos/Desktop/ws_working_folder/python/transformers/SctyDistTransformer.pyc matches /Users/carlos/Desktop/ws_working_folder/python/transformers/SctyDistTransformer.py
import SctyDistTransformer # precompiled from /Users/carlos/Desktop/ws_working_folder/python/transformers/SctyDistTransformer.pyc
But, for the life of me, I can't instantiate the 'SctyDistTransformer' module from the main file. I know I'm missing something trivial. Basically, I want to employ a class loading plugin.
To dymically load Python modules from arbitrary folders use imp module:
http://docs.python.org/library/imp.html
Specifically the code should look like:
mod = imp.load_source("MyModule", "MyModule.py")
clz = getattr(mod, "MyClassName")
Also if you are building serious plug-in architecture I recommend using Python eggs and entry points:
http://wiki.pylonshq.com/display/pylonscookbook/Using+Entry+Points+to+Write+Plugins
https://github.com/miohtama/vvv/blob/master/vvv/main.py#L104