I have a function name stored in a variable like this:
myvar = 'mypackage.mymodule.myfunction'
and I now want to call myfunction like this
myvar(parameter1, parameter2)
What's the easiest way to achieve this?
funcdict = {
'mypackage.mymodule.myfunction': mypackage.mymodule.myfunction,
....
}
funcdict[myvar](parameter1, parameter2)
It's much nicer to be able to just store the function itself, since they're first-class objects in python.
import mypackage
myfunc = mypackage.mymodule.myfunction
myfunc(parameter1, parameter2)
But, if you have to import the package dynamically, then you can achieve this through:
mypackage = __import__('mypackage')
mymodule = getattr(mypackage, 'mymodule')
myfunction = getattr(mymodule, 'myfunction')
myfunction(parameter1, parameter2)
Bear in mind however, that all of that work applies to whatever scope you're currently in. If you don't persist them somehow, you can't count on them staying around if you leave the local scope.
def f(a,b):
return a+b
xx = 'f'
print eval('%s(%s,%s)'%(xx,2,3))
OUTPUT
5
Easiest
eval(myvar)(parameter1, parameter2)
You don't have a function "pointer". You have a function "name".
While this works well, you will have a large number of folks telling you it's "insecure" or a "security risk".
Why not store the function itself? myvar = mypackage.mymodule.myfunction is much cleaner.
modname, funcname = myvar.rsplit('.', 1)
getattr(sys.modules[modname], funcname)(parameter1, parameter2)
eval(compile(myvar,'<str>','eval'))(myargs)
compile(...,'eval') allows only a single statement, so that there can't be arbitrary commands after a call, or there will be a SyntaxError. Then a tiny bit of validation can at least constrain the expression to something in your power, like testing for 'mypackage' to start.
I ran into a similar problem while creating a library to handle authentication. I want the app owner using my library to be able to register a callback with the library for checking authorization against LDAP groups the authenticated person is in. The configuration is getting passed in as a config.py file that gets imported and contains a dict with all the config parameters.
I got this to work:
>>> class MyClass(object):
... def target_func(self):
... print "made it!"
...
... def __init__(self,config):
... self.config = config
... self.config['funcname'] = getattr(self,self.config['funcname'])
... self.config['funcname']()
...
>>> instance = MyClass({'funcname':'target_func'})
made it!
Is there a pythonic-er way to do this?
Related
I'm looking for a safe mechanism in Python to execute potentially unsafe script code coming from a user.
Code example:
def public(v):
print('Allowed to trigger this with '+v)
def secret():
print('Not allowed to trigger this')
unsafe_user_code = '''
def user_function(a):
if 'o' in a:
public('parameter')
a = 'hello'
user_function(a)
'''
run_code(unsafe_user_code, allowed=['public'])
This could be easily achieved with exec() but as I understand there is no way of using exec() in a safe way in Python.
Here are my requirements:
The syntax of the script should ideally be similar to Python (but could also be something like JavaScript)
Standard mechanisms like string operations, if/else/while and definition of own variables and functions need to be available in the script
The script should only be able to execute certain functions (in the example, only public() would be allowed)
I would like to rely on Python-based implementations/libraries only as I don't want to have dependencies on Python-external software
It should not introduce a security risk
The only way I found so far is to use a parsing library, where I would have to define everything by myself (e.g. this one: https://github.com/lark-parser/lark ).
Is there a better way to achieve something like this?
Thanks!
Answer
Here's the full python file if you want to copy:
from copy import copy
def public(v):
print('Allowed to trigger this with '+v)
def secret():
print('Not allowed to trigger this')
unsafe_user_code = '''
def user_function(a):
if 'o' in a:
public('parameter')
# secret()
# If you uncomment it, it'll throw an error saying 'secret' does not exist
a = 'hello'
user_function(a)
'''
def run_code(code_to_run,allowed:list[str]):
g = globals()
allowed_dict = {f"{name}": g[name] for name in allowed}
code = compile(code_to_run,'','exec')
def useless():
pass
useless.__code__ = code
g = copy(useless.__globals__)
for x in g:
del useless.__globals__[x]
for x in allowed_dict:
useless.__globals__[x] = allowed_dict[x]
useless()
run_code(unsafe_user_code, allowed=['public'])
Explaination
I don't think you need a parsing library.
You can use one of python's Built-in Functions called compile().
You can compile code like this:
text_to_compile = "print('hello world')"
code = compile(
text_to_compile, # text to compile
'file_name', # file name
'exec' # compile mode
)
more about compile modes
and then you can run it simply by:
exec(code) # prints 'hello world' in the terminal
or, you can put that code inside of a function and run the function:
def useless():
pass
useless.__code__ = code
useless() # prints 'hello world' in the terminal
now for changing the scope of the function, we can access its __global__ dictionary and remove everything from it.
then we add the functions/variables that we want to it.
# from copy import copy
def run_code(code_to_run,allowed:list[str]):
g = globals()
allowed_dict = {f"{name}": g[name] for name in allowed}
code = compile(code_to_run,'','exec')
def useless():
pass
useless.__code__ = code
g = copy(useless.__globals__)
for x in g:
del useless.__globals__[x]
for x in allowed_dict:
useless.__globals__[x] = allowed_dict[x]
useless()
alright let's talk about what's happening:
def run_code(code_to_run,allowed:list[str]):
the :list[str] is just defining the type of the variable allowed.
g = globals()
using the globals() method we are able to get all of the variables that the function run_code has access to. this includes the public and the secret functions.
allowed_dict = {f"{name}": g[name] for name in allowed}
and use the variable names that you passed down into the run_code function and get the variables from the globals of this function.
the it's pretty much the same as saying:
allowed_dict = {}
for name in allowed:
allowed_dict[name] = g[name]
alright then we compile the code.
we also make a useless function and put the compiled code into the function:
code = compile(code_to_run,'','exec')
def useless():
pass
useless.__code__ = code
after that, we copy the globals of the useless function:
g = copy(useless.__globals__)
we copy the globals in the function so that later in the code we can loop over it and delete it.
we do this because we cannot delete something from a dictionary while a for loop is iterating it.
for x in g:
del useless.__globals__[x]
we delete everything in the useless function's global.
the useless function is an empty function but it's global is filled with all sort of stuff, like the public and the secret functions.
and then we just put all the allowed variables into the globals of the function so that the code inside of the function can use the variables/functions.
for x in allowed_dict:
useless.__globals__[x] = allowed_dict[x]
then we can just run the useless function:
useless()
and that's pretty much it all.
I have a module testing system in Python where individual modules call something like:
class Hello(object):
_DOC_ATTR = { 'greeting': '''
a greeting message.
>>> h = Hello()
>>> h.greeting = 'hi there'
>>> h.greeting
'hi there'
''' }
def __init__(self):
self.greeting = "hello"
class Test(unittest.TestCase):
# tests here
if __name__ == '__main__':
import tester
tester.test(Test)
inside tester, I run the tests in Test along with a doctest on "__main__". This works great and has worked fine for a long time. Our specialized _DOC_ATTR dictionary documents individual attributes on the function when we build into Sphinx. However, doctests within this dictionary are not called. What I would like to do is within tester.test() to run doctests on the values in each class's _DOC_ATTR as well.
The problem that I'm having is trying to find a way within tester.test() to figure out all the variables (specifically classes) defined in __main__. I've tried looking at relevant places in traceback to no avail. I thought that because I was passing in a class from __main__, namely __main__.Test that I'd be able to use the .__module__ from Test to get access to the local variables there, but I can't figure out how to do it.
I would rather not need to alter the call to tester.test(Test) since it's used in hundreds of modules and I've trained all the programmers working on the project to follow this paradigm. Thanks for any help!
I think that I may have found the answer:
import inspect
stacks = inspect.stack()
if len(stacks) > 1:
outerFrame = stacks[1][0]
else:
outerFrame = stacks[0][0]
localVariables = outerFrame.f_locals
for lv in list(localVariables.keys()):
lvk = localVariables[lv]
if (inspect.isclass(lvk)):
docattr = getattr(lvk, '_DOC_ATTR', None)
if docattr is not None:
# ... do something with docattr ...
Another solution: since we are passing the "Test" class in, and in order to run there needs to be a "runTest" function defined, one could also use the func_globals on that function. Note that it cannot be a function inherited from a superclass, such as __init__, so it may have limited functionality for wider use cases.
import inspect
localVariables = Test.runTest.func_globals
for lv in list(localVariables.keys()):
lvk = localVariables[lv]
if (inspect.isclass(lvk)):
#### etc.
I'm new to Python and I'm trying to mock a function only when a specific argument is passed. If other than the desired argument is passed, I'd like to call the original function instead.
In Python 2.7 I tried something like this:
from foo import config
def test_something(self):
original_config = config # config is a Module.
def side_effect(key):
if key == 'expected_argument':
return mocked_result
else:
return original_config.get(key)
config.get = Mock(side_effect=side_effect)
# actualy_test_something...
It won't work 'cause original_config is not a copy of config. It references the same module ending up in an infinite loop. I could try cloning the original config module instead but that seems to be overkill.
Is there something similar to RSpec's mocks I could use? e.g:
obj.stub(:message).with('an_expected_argument').and_return('a_mocked_result')
Any help would be appreciated. Thanks.
You'd need to store a reference to the unpatched function first:
def test_something(self):
original_config_get = config.get
def side_effect(key):
if key == 'expected_argument':
return mocked_result
else:
return original_config_get(key)
config.get = Mock(side_effect=side_effect)
Here original_config_get references the original function before you replaced it with a Mock() object.
I am trying to make generic config, and thus config parser. There are two config files say A and B. I want to parse sections and make global values from them according to hardcoded list.
Here is an example:
in config file:
[section]
var1 = value1
var2 = value2
In global scope:
some_global_list = [ ["var1","var2"], ["var3","var4"] ]
in function to unpack this values, by ConfigParser:
configparser = ConfigParser.RawConfigParser()
configparser.read(some_config_filename)
for variables in some_global_list:
globals()[section]=dict()
for element in configparser.items(section):
globals()[section].update({element[0]:element[1]})
This works nicely...however. Scope of globals() seem to be limited to function which is obviously not what I intended. I can access variable only while in that function.
Could someone share better yet simple idea?
I know that i might move code to main and not to worry, but I don't think it is a good idea.
I thought also about making some generator (sorry for pseudocode here):
in global scope:
for x in some_global_list:
globals()[x] = x
also tried adding this to function:
for x in some_global_list[0]:
global x
but got nowhere.
Edit :
After discussion below, here it is:
Problem solved like this:
removed whole configuration from main script to module
imported (from module import somefunction) config from module
removed globals() in fact didnt need them, since function was changed a little like so:
in function:
def somefunction:
#(...)
configparser = ConfigParser.RawConfigParser()
configparser.read(some_config_filename)
temp_list=[]
for variables in some_global_list:
tmp=dict()
for element in configparser.items(section):
tmp.update({element[0]:element[1]})
temp_list.append (tmp)
return temp_list #this is pack for one file.
now in main script
tmp=[]
for i,conf_file in enumerate([args.conf1,args.conf2,args.conf3]):
if conf_file:
try:
tmp.append([function(params...)])
except:
#handling here
#but since i needed those config names as global variables
for j,variable_set in enumerate(known_variable_names[i]):
globals()[variable_set] = tmp[i][j]
so unfortunate hack presists. But seems to work. Thx for Your help guys.
I'm accepting (if thats possible) below answer since it gave me good idea :)
A simple way to solve this issue is in your application package within the __init__.py you can do something similar to the following:
app_config = read_config()
def read_config():
configparser = ConfigParser.RawConfigParser()
configparser.read(some_config_filename)
return configparser.as_dict() #An imaginery method which returns the vals as dict.
The "app_config" variable can be imported into any other module within the application.
In django apps like celery you can write a function, put it in a file (func.py) in a specific folder (myFunc). Now you can create a Object in django with a reference to this function to be run by a scheduler.
I don't want to write a new celery, I'd like to know what technique is used to accomplish a behavior like this: Running a function referenced by a string/or a CharField.
Short example
Folder structure:
myApp
---myFunc
-----func.py
-models.py
func.py
def test():
print "foo"
models.py
class RunAFunction(models.Model):
function = models.CharField(max_length=100)
python manage.py shell
> from myApp.models import RunAFunction
> func = RunAFunction(function="test()")
> func.save()
Now I want to run myFunc.func.test() with my RunAFunction() class.
Please don't tell me that I need to use eval() ;)
Split the string on the last . to get a module and function name, then use importlib and getattr() to get the object:
import importlib
modulename, funcname = string.rsplit('.', 1)
module = importlib.import_module(modulename)
function = getattr(module, funcname)
result = function()
So if string = 'myApp.myFunc.test', then the above code splits that to 'modulename = 'myApp.myFunc' and funcname = 'test', then calls importlib.import_module('myApp.myFunc'), and getattr(module, 'test') on the result, giving you a reference to the function, that then can be called.
The first step is to identify your function by name only, so "test" instead of "test()". Then you should be able to look it up within a global namespace or other module by using something like getattr(). Once you have the function in a variable, you can put the parens on and call it.