I am implementing a new module for specific needs in my environment. I would like to print certain outputs (such as some variables) by this module, similar as debug module prints with msg parameter, but in a more customized way.
AnsibleModule class has fail_json() method which accepts msg argument to print on a failure, but I cannot find a way to print a message on success with exit_json()
I also don't know how builtin debug module works, found almost nothing except DOCUMENTATION and EXAMPLES in the module script.
Everything you want to be done on Ansible controller is done by action plugins (they are module's companions).
Take a look at some very simple plugin/module here.
You want to execute module, inspect it's result for your custom message, use display.v or display.warning or anything else to display this message and then return module's result back to Ansible core.
For this very reason debug is an action plugin, and it's module only contains documentation, because all the work is done by plugin itself.
Related
Ok, so I know this has been asked before in many different threads but I find myself still trying to reduce my doubt.
I have an application that allows the user to pass a dictionary of strings and 'choose' a backend function from a library to process it. The functions are part of a 'workflow' library and loaded by the system admin on the backend. Available functions are stored in the backend in a manifest file.
The mechanics are such that the users send the dictionary as json to the web app and nominate which function from the library should process it. The function is is then loaded and executed via the python exec() or eval() functions.
Before the execution, the requested function is checked against a list of available functions (whitelist) from the manifest file.
My basic question is, can whitelisting make exec() and eval safe? Could it be made 'safer'?
If I understand it, the function is trusted by the admin and that makes it as safe as any python module you install. Just make sure that the exec part is only done on the trusted code. Here is an example where functions with the same name as their file are loaded and executed.
import json
from pathlib import Path
# files named the same as the function, no .py
FUNCTION_DIR = Path("/my/functions/are/here")
def run_func(name, data):
try:
func_namespace = {}
exec(open(FUNCTION_DIR/name).read(), func_namespace)
return func_namespace[name](json.dumps(data))
except Exception as e:
return "Hey, what kind of game are you playing here? " + str(e)
The the function is naturally whitelisted just because its in the known-safe directory.
I am trying to code up a module which has two classes. First class is called as TextProcessing
class TextProcessing(object):
""" To carry out text processing
"""
def __init__(self,):
pass
It has various methods in there for pre-processing text.
Similary other class is for other data wrangling on pre-processed data.
I am saving these two classes in a python file to make it a module.
Now lets say a user downloads this python module and would now want to run the various methods of each class.
I wanted to provide some sort of documentation about the module, methods of each class to a user when she imports the module so that she is aware of which function to call and what parameters to pass.
Think of how a scikit learn documentation is on their documentation page.
http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html
Even the documentation we get to see when we do
help(some_python_module)
is fine too.
Issue is I don't have a documentation page like sklearn to show documentation. And I wanted a user to know documentation of various methods she can use once she imports the module in python console.
Is there a way I can print that documentation info to the console when a user imports the module?
It can show the doc string of each Class and Method.
This is a very weird thing to do, but it's definitely possible.
The easiest thing to do is just to call help. While it's intended to be called from the interactive prompt, there's nothing stopping you from calling it from your own code.
Of course you could instead extract the docstrings (they're stored as __doc__ on every module, class, and function), textwrap them yourself, and print them out, but if you're trying to reproduce the same thing help does, that's a lot of work for no real benefit.
The only tricky bit is that the thing you want to invoke the help system on is "this current module". How do you refer to that? It's a bit clunky, but you have this current module's name as __name__, so you can look it up in sys.modules.
So:
"""Helpful module"""
import sys
class Spam:
"""Classy class"""
def eggs(self):
"Functional function"
return 2
help(sys.modules[__name__])
Now, when you import helpful for the first time in a session, it will print out the help.
Of course that will be pretty odd if someone's trying to run a script that does an import helpful, rather than doing it from an interactive session. So you may want to only do this in interactive sessions, by checking sys.flags:
if sys.flags.interactive:
help(sys.modules[__name__])
What if someone does an import otherthing, and that otherthing does an import helpful? You'll get the same help, which may be confusing.
If that's a problem, the only real option I can think of is to check whether the calling frame comes from the top-level script (and that the flags are interactive). That's pretty hacky, and something you shouldn't even consider unless you really need to, so I'll just direct you to the inspect module and hope you don't need it.
Is there any case where doing:
import __main__
might lead to an ImportError? All cases I've tried seem to indicate that this always works. The docs on __main__ don't seems to state anything on the matter.
To give some context: I am trying to inject some names in __main__.__dict__ using the usersitecustomize hook in order to (mainly) have them available when the REPL fires up.
Granted that no redefinitions of __import__ occur (as a comment stated), this essentially boils down to if I need to wrap it in a try-except or not.
It probably is. Python initializes __main__ in this file:
https://github.com/python/cpython/blob/master/Python/pylifecycle.c#L1327
However please note that modules like runpy and IPython replace the __main__ module with their own dynamically created ones to prevent collisions with their own launch scripts and to provide expected behaviour in case of runpy.
runpy itself is part of the Python Standard Library and provides the implementation of the -m flag which allows arbitrary modules to be executed as script.
An alternative is IPython which offers the feature to execute code at the launch of a new REPL.
For more details, see here: http://ipython.readthedocs.io/en/stable/config/intro.html?highlight=exec_lines
I'm using Sphinx to document a project that depends on wxPython, using the autodocs extension so that it will automatically generate pages from our docstrings. The autodocs extension automatically operates on every module you import, which is fine for our packages but is a problem when we import a large external library like wxPython. Thus, instead of letting it generate everything from wxPython I'm using the unittest.mock library module (previously the external package Mock). The most basic setup works fine for most parts of wxPython, but I've run into a situation I can't see an easy way around (likely because of my relative unfamiliarity with mock until this week).
Currently, the end of my conf.py file has the following:
MOCK_MODULES = ['wx.lib.newevent'] # I've skipped irrelevant entries...
for module_name in MOCK_MODULES:
sys.modules[module_name] = mock.Mock()
For all the wxPython modules but wx.lib.newevent, this works perfectly. However, here I'm using the newevent.NewCommandEvent() function[1] to create an event for a particular scenario. In this case, I get a warning on the NewCommandEvent() call with the note TypeError: 'Mock' object is not iterable.
While I can see how one would use patching to handle this for building out unit tests (which I will be doing in the next month!), I'm having a hard time seeing how to integrate that at a simple level in my Sphinx configuration.
Edit: I've just tried using MagicMock() as well; this still produces an error at the same point, though it now produces ValueError: need more than 0 values to unpack. That seems like a step in the right direction, but I'm still not sure how to handle this short of explicitly setting it up for this one module. Maybe that's the best solution, though?
Footnotes
Yes, that's a function, naming convention making it look like a class notwithstanding; wxPython follows the C++ naming conventions which are used throughout the wxWidgets toolkit.
From the error, it looks like it is actually executing newevent.NewCommandEvent(), so I assume that somewhere in your code you have a top-level line something like this:
import wx.lib.newevent
...
event, binder = wx.lib.newevent.NewCommandEvent()
When autodoc imports the module, it tries to run this line of code, but since NewCommandEvent is actually a Mock object, Python can't bind its output to the (event, binder) tuple. There are two possible solutions. The first is to change your code to that this is not executed on import, maybe by wrapping it inside if __name__ == '__main__'. I would recommend this solution because creating objects like this on import can often have preblematic side effects.
The second solution is to tell the Mock object to return appropriate values thus:
wx.lib.newevent.NewCommandEvent = mock.Mock(return_value=(Mock(), Mock()))
However, if you are doing anything in your code with the returned values you might run into the same problem further down the line.
How does one get (finds the location of) the dynamically imported modules from a python script ?
so, python from my understanding can dynamically (at run time) load modules.
Be it using _import_(module_name), or using the exec "from x import y", either using imp.find_module("module_name") and then imp.load_module(param1, param2, param3, param4) .
Knowing that I want to get all the dependencies for a python file. This would include getting (or at least I tried to) the dynamically loaded modules, those loaded either by using hard coded string objects or those returned by a function/method.
For normal import module_name and from x import y you can do either a manual scanning of the code or use module_finder.
So if I want to copy one python script and all its dependencies (including the custom dynamically loaded modules) how should I do that ?
You can't; the very nature of programming (in any language) means that you cannot predict what code will be executed without actually executing it. So you have no way of telling which modules could be included.
This is further confused by user-input, consider: __import__(sys.argv[1]).
There's a lot of theoretical information about the first problem, which is normally described as the Halting problem, the second just obviously can't be done.
From a theoretical perspective, you can never know exactly what/where modules are being imported. From a practical perspective, if you simply want to know where the modules are, check the module.__file__ attribute or run the script under python -v to find files when modules are loaded. This won't give you every module that could possibly be loaded, but will get most modules with mostly sane code.
See also: How do I find the location of Python module sources?
This is not possible to do 100% accurately. I answered a similar question here: Dependency Testing with Python
Just an idea and I'm not sure that it will work:
You could write a module that contains a wrapper for __builtin__.__import__. This wrapper would save a reference to the old __import__and then assign a function to __builtin__.__import__ that does the following:
whenever called, get the current stacktrace and work out the calling function. Maybe the information in the globals parameter to __import__ is enough.
get the module of that calling functions and store the name of this module and what will get imported
redirect the call the real __import__
After you have done this you can call your application with python -m magic_module yourapp.py. The magic module must store the information somewhere where you can retrieve it later.
That's quite of a question.
Static analysis is about predicting all possible run-time execution paths and making sure the program halts for specific input at all.
Which is equivalent to Halting Problem and unfortunately there is no generic solution.
The only way to resolve dynamic dependencies is to run the code.