I want to proxy an API over a network. I have the API in a dictionary. I'd like to create a class with the API methods from the dictionary so I can use the API as if I was local. The trouble is finding the name of my dynamically created method. (My approach is based on Adding a Method to an Existing Object and Python dynamic class methods.)
class MainClass(object):
def build_API(self):
methods = dict(meth1='arg1', meth2='arg2')
for key in methods.iterkeys():
setattr(self, key, MethodType(self.default_API, self))
def default_API(self, *args, **kwargs)
called_as_name = ????
self.send_message(called_as_name, args, kwargs)
def send_message(self, called_as_name, *args, **kwargs)
...
# Send API command over network
....
To use this:
api = MainClass()
api.build_API()
api.meth1()
However, everything I try for "called_as_name" always returns "default_API" and never "meth1". How can I get "called_as_name = meth1" when I type "api.meth1()" and "called_as_name = meth2" when I type "api.meth2()"?
I have tried:
curframe = inspect.currentframe()
calframe = inspect.getouterframes(curframe, 2)
called_as_name = calframe[1][3]
from Python: How to get the caller's method name in the called method?
called_as_name = inspect.stack()[1][5]
from Getting the caller function name inside another function in Python?
called_as_name = sys._getframe(1).f_code.co_name
from Getting the caller function name inside another function in Python?
Trying to do this with actual methods and grabbing the names from the stack frame with that sort of introspection trickery is a recipe for disaster. Instead, make the "methods" be custom callable objects that know their names. Here's a sketch:
class FakeMethod(object):
def __init__(self, name, parent):
self.name = name
self.parent = parent
def __call__(self, *args, **kwargs):
self.parent.send_message(self.name, args, kwargs)
class MainClass(object):
def build_API(self):
methods = dict(meth1='arg1', meth2='arg2')
for key in methods.iterkeys():
setattr(self, key, FakeMethod(key, self))
def send_message(self, called_as_name, *args, **kwargs):
print("Sending message:", called_as_name, args, kwargs)
Then:
>>> api = MainClass()
>>> api.build_API()
>>> api.meth1()
Sending message: meth1 ((), {}) {}
>>> api.meth2()
Sending message: meth2 ((), {}) {}
In theory you could even use __getattr__ on the MainClass to dynamically generate a FakeMethod every time an attribute name is accessed that is not defined but is listed in some list of API method names.
Related
I have the following class in python:
class MatrixOfCarriers(DataLoader):
def load_data(self):
brands = self.brands
try:
data = self._load_data()
print("'Carrier Matrix' data loaded successfully.\n")
except Exception as e:
print(
"\nCouldn't load 'Carier Matrix' data due to the following "
+ "error: '{}'".format(str(e))
)
raise e
return data
I want to decorate method MatrixOfCarriers.load() with the following decorator:
def cast_pct_buffer_columns(brands):
def inner(func):
def wrapper(*args, **kwargs):
data = func(*args, **kwargs)
for brand in func.brands.values():
if brand["pct_buffer_column"] in data.columns:
data[brand["pct_buffer_column"]] = (
pd.to_numeric(
data[brand["pct_buffer_column"]].str.replace(
"[,%]", "", regex=True
)
)
/ 100
)
return data
return wrapper
return inner
Thr problem is that such method requires a parameter brand that is available as an instance variable, but I can't send it using:
#cast_pct_buffer_columns(self.brands)
def load_data(self):
but self.brands is not in scope out of the body of an instance method.
I also tried to set brands = self.brands in the body of method load_data() and then called brands = func.brands from the decorator, but it didn't work neither.
How can I do this?
If you are always decorating a method, and will always need to get the .brands attribute, there are two things that can be changed: first, Python will pass the instance itself as the first argument to your wrapper - just as it pass the self parameter - so you can just use the .brands attribute in that first parameter.
Since the wrapper method can have access to the needed data, there is no need for an intermediate level of the decorator, to pass it the parameters, so it could be simply:
def cast_pct_buffer_columns(func):
def wrapper(instance, *args, **kwargs):
data = func(instance, *args, **kwargs)
for brand in instance.brands.values():
if brand["pct_buffer_column"] in data.columns:
data[brand["pct_buffer_column"]] = (
pd.to_numeric(
data[brand["pct_buffer_column"]].str.replace(
"[,%]", "", regex=True
)
)
/ 100
)
return data
return wrapper
(here, I wrote "instance" as the parameter that will take the value that is usually called self inside a method. You have to add it explicitly to the inner call of func)
Now, if the decorator will be used in other situations and classes, and the desired values won't always be in the .brands attribute - let's suppose in another class, the same decorator were applied, but it'd need to pick the vendor attribute - then, that can be passed as a string to the decorator - and you can use the getattr built-in function to get to the values themselves:
def cast_pct_buffer_columns(attrname):
def wrapper(instance, *args, **kwargs):
data = func(*args, **kwargs)
for item in getattr(instance, attrname).values():
# do stuff
....
return data
return wrapper
return inner
class MatrixOfVendorCarriers(DataLoader):
#cast_pct_buffer_columns("vendors")
def load_data(self):
vendors self.vendors
...
return data
I posted this a while ago. I have been making some progress, and this appears to be trickier than I thought. Python: Getting the name of a callable function argument
I have made progress with #wraps, and now I am able to get a bit further. This is a separate question that merits its own thread.
Shortly, how can I access somehow the class instance, whose member a function call is? Here is a complete and working code snippet illustrating the issue (python 3 only).
The reason why I need to do this is explained in my other post if anyone cares. When I call b.set_something(), the parameter is a method call to a. But as there is no method "set_something_else" in A but it is grabbed by the __getattr__() hack, I am struggling a bit to identify what I am working with, as I need to "explain" over a messaging protocol to the remote server, which class instance executed which functions, with possible callable parameters to other classes.
So I thought to add an id to every class instance and use this as a reference. When my b object receives the call to b.get_something() with a.get_something_else as an argument, I can now because of #wraps detect in my b that
The function is of class A
The function name is "get_something_else"
Now the only missing link is to grab the "id" from the instance of A, whose member get_something_else happens to be. This would allow me to link everything together on the remote side. But how do I reference it from B? The only information I have is the callable parameter a.get_something_else. I am able to freely modify classes Foo and Bar but not the final part how variables a and b are constructed.
from functools import wraps
import uuid
class Bar:
def __init__(self):
self.id = str(uuid.uuid4())
def __getattr__(self, name):
#wraps(name)
def foo(*args, **kwargs):
_kwa = {}
for k, v in kwargs.items():
if callable(v):
cn = "{}.{}".format(v.__qualname__.split(".")[0], v.__wrapped__)
# How to get "id" from the object whose member v is??
_kwa[k] = cn
continue
else:
_kwa[k] = v
x = {"args": args, "kwargs": _kwa}
print(x)
return foo
class Foo:
def __init__(self):
self.id = str(uuid.uuid4())
def __getattr__(self, name):
#wraps(name)
def foo(*args, **kwargs):
_kwa = {}
for k, v in kwargs.items():
if callable(v):
cn = "{}.{}".format(v.__qualname__.split(".")[0], v.__wrapped__)
_kwa[k] = cn
continue
else:
_kwa[k] = v
x = {"args": args, "kwargs": _kwa}
print(x)
return foo
a = Foo()
b = Bar()
b.set_something(command=a.set_something_else)
Not sure if this is exactly what you’d want but have you checked out the super function?
Class Parent():
Def init(self):
super(Parent, self).init()
I have a simple method which accepts a function to call this back later:
def SimpleFunc(parm1):
print(parm1)
class CallMe:
def __init__(self, func):
self.func = func
def Call(self, parm):
self.func(parm)
caller = CallMe(SimpleFunc)
caller.Call("Hallo")
That works fine!
But I want to use a class method and want to call the method on a defined object as callback:
class WithClassMethod:
def __init__( self, val ):
self.val = val
def Func(self, parm):
print( "WithClass: ", self.val, parm )
obj = WithClassMethod(1)
caller = CallMe( ??? )
caller.Call("Next")
How can I bind an object/method pair to a callable object?
Attention: The code from CallMe is not under my control. It comes from a webserver which needs a handler function.
You could simply pass the method object to the class:
called = CallMe(obj.Func)
To expand a bit, instance methods are really just the original class function:
>>> obj.Func.__func__
<function __main__.WithClassMethod.Func>
which, during access on an instance (obj.Func) are transformed via a descriptor (__get__) that attaches self (the instance) to them:
>>> obj.Func.__self__
<__main__.WithClassMethod at 0x7fbe740ce588>
so you can pretty much do anything you want with methods as with functions.
I'm attempting to implement a decorator on certain methods in a class so that if the value has NOT been calculated yet, the method will calculate the value, otherwise it will just return the precomputed value, which is stored in an instance defaultdict. I can't seem to figure out how to access the instance defaultdict from inside of a decorator declared outside of the class. Any ideas on how to implement this?
Here are the imports (for a working example):
from collections import defaultdict
from math import sqrt
Here is my decorator:
class CalcOrPass:
def __init__(self, func):
self.f = func
#if the value is already in the instance dict from SimpleData,
#don't recalculate the values, instead return the value from the dict
def __call__(self, *args, **kwargs):
# can't figure out how to access/pass dict_from_SimpleData to here :(
res = dict_from_SimpleData[self.f.__name__]
if not res:
res = self.f(*args, **kwargs)
dict_from_SimpleData[self.f__name__] = res
return res
And here's the SimpleData class with decorated methods:
class SimpleData:
def __init__(self, data):
self.data = data
self.stats = defaultdict() #here's the dict I'm trying to access
#CalcOrPass
def mean(self):
return sum(self.data)/float(len(self.data))
#CalcOrPass
def se(self):
return [i - self.mean() for i in self.data]
#CalcOrPass
def variance(self):
return sum(i**2 for i in self.se()) / float(len(self.data) - 1)
#CalcOrPass
def stdev(self):
return sqrt(self.variance())
So far, I've tried declaring the decorator inside of SimpleData, trying to pass multiple arguments with the decorator(apparently you can't do this), and spinning around in my swivel chair while trying to toss paper airplanes into my scorpion tank. Any help would be appreciated!
The way you define your decorator the target object information is lost. Use a function wrapper instead:
def CalcOrPass(func):
#wraps(func)
def result(self, *args, **kwargs):
res = self.stats[func.__name__]
if not res:
res = func(self, *args, **kwargs)
self.stats[func.__name__] = res
return res
return result
wraps is from functools and not strictly necessary here, but very convenient.
Side note: defaultdict takes a factory function argument:
defaultdict(lambda: None)
But since you're testing for the existence of the key anyway, you should prefer a simple dict.
You can't do what you want when your function is defined, because it is unbound. Here's a way to achieve it in a generic fashion at runtime:
class CalcOrPass(object):
def __init__(self, func):
self.f = func
def __get__(self, obj, type=None): # Cheat.
return self.__class__(self.f.__get__(obj, type))
#if the value is already in the instance dict from SimpleData,
#don't recalculate the values, instead return the value from the dict
def __call__(self, *args, **kwargs):
# I'll concede that this doesn't look very pretty.
# TODO handle KeyError here
res = self.f.__self__.stats[self.f.__name__]
if not res:
res = self.f(*args, **kwargs)
self.f.__self__.stats[self.f__name__] = res
return res
A short explanation:
Our decorator defines __get__ (and is hence said to be a descriptor). Whereas the default behaviour for an attribute access is to get it from the object's dictionary, if the descriptor method is defined, Python will call that instead.
The case with objects is that object.__getattribute__ transforms an access like b.x into type(b).__dict__['x'].__get__(b, type(b))
This way we can access the bound class and its type from the descriptor's parameters.
Then we create a new CalcOrPass object which now decorates (wraps) a bound method instead of the old unbound function.
Note the new style class definition. I'm not sure if this will work with old-style classes, as I haven't tried it; just don't use those. :) This will work for both functions and methods, however.
What happens to the "old" decorated functions is left as an exercise.
I'm trying to build a nice base around py.test
Some of our tests needs certain test data to work.
Today we just specify a mock object as a function argument and do the setup in the generator, this is clearly not desirable.
Here is an example of how it could look today:
def test_something(self, some_data):
# some_data is unused in the test
I'd like to do something like this:
#uses_some_data
def test_something(self):
# The data is loaded when the test is run
Though I have not figured out how to do this properly.
I cannot use class setup because I want the data to be persistant over the entire session, not setup/torn down on every test class.
My first idea was to still use funcargs but instead of letting the test have the funcarg we let the decorator request the funcarg for the function, basically hiding the ugliness.
The problem with this is that I need a py.test object to request a funcarg.
Is there any way I can get such an object or is this the wrong approach all together?
It would be an awesome bonus if the data did not have to be loaded if none of the collected tests requires the data, this is the downside of using decorators seeing as they are always run no matter if the test will be run or not.
Here is something that may work as-is, and if not will hopefully point you in the right direction.
class TestData(object):
def __getattr__(self, name):
if name not in ('data1', 'data2', 'data3'):
raise AttributeError("TestData has no %s" % name)
if name == 'data1':
result = self._generate_data('data1')
setattr(self.__class__, name, result)
elif name == 'data2':
result = self._generate_data('data2')
setattr(self.__class__, name, result)
elif name == 'data3':
result = self._generate_data('data3')
setattr(self.__class__, name, result)
return result
def _generate_data(self, data_name):
return data_name * int(data_name[-1])
The TestData class uses the __getattr__ method to generate the data as it is needed, and by saving the generated date back to the class (not the instance!), the data is kept around for future use as well.
class uses_some_data(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
global test_data
test_data = TestData()
return self.func(*args, **kwargs)
A simple decorator to set the global name binding for test_data. In fact, this version of the decorator is so simple it can be easily replaced with a module level name binding of test_data = TestData().
#uses_some_data
def testing_test():
print(test_data.data2)
And a test function.
If you don't like the global level of test_data you could get fancier with the decorator and assign test_data to the function itself:
class uses_some_data(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
self.func.test_data = TestData()
return self.func(*args, **kwargs)
In this case, make sure your testing functions reference themselves
#uses_some_data
def testing_test():
print(testing_test.test_data.data2)
After playing around some I found that this works:
def pytest_funcarg__some_data(request):
def create():
# Load the test data here
print 'Test data loaded'
return request.cached_setup(
setup=create,
scope='session',
extrakey='some_data'
)
def uses_some_data(func):
# The funcarg is actually requested here
def wrapper(self, some_data):
return func
return wrapper
class TestSomething(object):
#uses_some_data
def test_something(self):
# "Some data" is now available
pass