I have a series of subclasses which each have their own functions for accessing data. Sometimes, it can be more complex, but by default it calls that method (see example below). The problem arises when I try to simply call the defined function and it passes self as an argument. The data access function signatures aren't defined for that and are used other ways, so it doesn't make sense to add self as an argument. How can I accomplish this design with the right implementation?
# data.py
def get_class_a_records(connection, date):
pass
def get_class_b_records(connection, date):
pass
class Parent:
def get_records(self, connection, **kwargs):
self.data_method(connection=connection, **kwargs)
class A(Parent):
data_method = get_class_a_records
class B(Parent):
data_method = get_class_b_records
class C(Parent):
def get_records(self, connection, **kwargs):
# logic for custom code/post-processing
pass
Now if we instantiate one of these classes, we run into an issue:
a = A()
a.get_records(connection=None, date='test')
TypeError: get_class_a_records() got multiple values for argument 'connection'
This is because the call self.data_method actually passes self as an argument and we see get_class_a_records clearly doesn't have self as an argument.
You can call the first parameter of an instance method anything you want, but python will always insert a reference to the instance object in that position when the method is called. By assigning the get_class_a_records function to a class variable, congratulations, you've made it an instance method. More technically, python will make it an instance method when you call it.
The rules for class and static methods are different. For a class, the instance's class object is passed. And for a static method, nothing is added. That's the one you want. Just convert the function to a staticmethod and it will work.
def get_class_a_records(connection, date):
print("class a records", connection)
def get_class_b_records(connection, date):
pass
class Parent:
def get_records(self, connection, **kwargs):
self.data_method(connection=connection, **kwargs)
class A(Parent):
data_method = staticmethod(get_class_a_records)
class B(Parent):
data_method = staticmethod(get_class_b_records)
class C(Parent):
def get_records(self, connection, **kwargs):
# logic for custom code/post-processing
pass
A().data_method("connection", "date")
Related
I was writing a simple implementation of classmethod decorator for a better understanding of both decorator and classmethod. Here is the problem I faced. When I called the class method with an instance of my class, everything was fine, but calling the method with the class object fails with error:
>>**TypeError: wrapper() missing 1 required positional argument: 'cls'**
and when I called the method with the explicit class object as parameter it succeed, but a call to classmethod from class object should pass the class object itself as the first parameter, right?
import functools
import inspect
def myclassmethod(meth):
#functools.wraps(meth)
def wrapper(cls, *args, **kwargs):
#print(f'obj:{cls}, cls:{cls.__class__}, isclass:{inspect.isclass(cls)}')
return meth(cls if inspect.isclass(cls) else cls.__class__, *args, **kwargs)
return wrapper
class MyDecoratedMethods(object):
_name = 'ClassName'
def __init__(self):
self._name = 'InstanceName'
def __repr__(self):
return f'{self.__class__.__name__}({self._name!r})'
#myclassmethod
def classname(cls):
return cls._name
MyDecoratedMethods().classname()
#MyDecoratedMethods.classname()
MyDecoratedMethods.classname(MyDecoratedMethods) # This works
To see what's going on, I removed the #functools.wraps(meth) line and then ran:
print(MyDecoratedMethods.classname)
# <function __main__.myclassmethod.<locals>.wrapper(cls, *args, **kwargs)>
This shows us, that MyDecoratedMethods.classname is simply the function you created inside your decorator. And this function knows nothing about the class it is called from.
However, we can override this behavior with Descriptors. A descriptor "knows" when it is accessed from a class or an instance and, most importantly, can distinguish between those cases (which is how regular methods are created).
Here is a first try:
class ClassMethod:
def __init__(self, function):
self.function = function
def __get__(self, instance, cls):
print(cls, instance)
class MyDecoratedMethods(object):
...
#ClassMethod
def classname(cls):
...
MyDecoratedMethods.classname
# prints <class '__main__.MyDecoratedMethods'> None
MyDecoratedMethods().classname
# prints <class '__main__.MyDecoratedMethods'> <__main__.MyDecoratedMethods object ...>
So we see that accessing the class method from the class sets instance to None and accessing it from an instance sets instance to that very object.
But actually we don't need the instance at all. We can implement the logic without it.
from functools import partial
class ClassMethod:
def __init__(self, function):
self.function = function
def __get__(self, instance, cls):
# create a new function and set cls to the first argument
return partial(self.function, cls)
...
MyDecoratedMethods().classname()
# "ClassName"
MyDecoratedMethods.classname()
# "ClassName"
And we are done. Our custom descriptor accomplished two things:
It prevented the function from binding the instance to the first argument when calling it from an instance of the class (like a function would normally do to become a method)
It always binds the class to the first argument when it is accessed from either the class or an instance.
Side note: Your approach to check whether an instance or the class called the function is also flawed (inspect.isclass(cls)). It will work for "normal" classes but not for meta classes, because then inspect.isclass returns True for the class and its instances.
I decorated some methods with #bot_thinking, which stores some information about the decorated method in the functions attribute.
One piece of information is 'class_name', but my program needs the class type as a variable, e.g. RandomBot. I would like to get this class.
Here is some sample code:
class DepthPrunedMinimaxAgent(Agent):
#bot_thinking(associated_name="minimax profondeur")
def select_move(self, game_state: GameState):
Above is the decorated part of the code.
The decorator:
functions = {}
def bot_thinking(associated_name, active=True):
def _(func):
if active:
class_name = func.__qualname__.rsplit('.')[-2]
import sys
# class_name_2=getattr(sys.modules[__name__], class_name)
# module=importlib.import_module('sources.agent')
functions[associated_name] = (associated_name, class_name,
globals()[class_name], func)
else:
functions.pop(associated_name)
return _
bot_thinking isn't a real decorator, it's a decorator factory.
From the func function, I get the class_name, but I can't use the accepted answer by #m.kocikowski, to find the correct class because this class is decorated, so it already imports the annotation module, so importing from the module of the annotation the annotated module would result in a cyclic import, which python does not seem to permit.
Do you see a method to get the class from its name?
ps:
ps:
to be clearer : the annotation part of the code need an import to the annotated classes(to retrieve the class from its name), which also need an importation of the annotation (for the annotation to work).
You can do what you want if you use a descriptor class, rather than a function, as the decorator, at least if you're using Python 3.6 or newer. That's because there's a new method added to the descriptor protocol, __set_name__. It gets called when the descriptor object is saved as a class variable. While most descriptors will use it to record the name they're being saved as, you can use it to get the class you're in.
You do need to make your decorator object wrap the real function (implementing calling and descriptor lookup methods), rather than being able to return the unmodified function you were decorating. Here's my attempt at a quick and dirty implementation. I don't really understand what you're doing with functions, so I may not have put the right data in it, but it should be close enough to get the idea across (owner is the class the method stored in).
functions = {}
def bot_thinking(associated_name, active=True):
class decorator:
def __init__(self, func):
self.func = func
def __set_name__(self, owner, name):
if active:
functions[associated_name] = (associated_name, owner.__name__,
owner, self.func)
else:
functions.pop(associated_name)
def __get__(self, obj, owner):
return self.func.__get__(obj, owner)
def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)
return decorator
The problem is the class hasn't been defined yet when the bot_thinking() decorator factory (and decorator itself) are executing. The only workaround I can think of would be to patch things up after the class is defined, as illustrated below:
from pprint import pprint, pformat
functions = {}
def bot_thinking(associated_name, active=True):
def _(func):
if active:
class_name = func.__qualname__.split(".")[-2]
functions[associated_name] = (associated_name, class_name, class_name, func)
else:
functions.pop(associated_name, None)
return func # Decorators must return a callable.
return _
class Agent: pass
class GameState: pass
class DepthPrunedMinimaxAgent(Agent):
#bot_thinking(associated_name="minimax profondeur")
def select_move(self, game_state: GameState):
pass
# After class is defined, update data put into functions dictionary.
for associated_name, info in functions.items():
functions[associated_name] = (info[0], info[1], globals()[info[2]], info[3])
pprint(functions)
Output:
{'minimax profondeur': ('minimax profondeur',
'DepthPrunedMinimaxAgent',
<class '__main__.DepthPrunedMinimaxAgent'>,
<function DepthPrunedMinimaxAgent.select_move at 0x00F158A0>)}
Say I have a class which contains several functions.
class Person:
def __init__(self): pass
def say(self, speech): pass
def walk(self, destination): pass
def jump(self): pass
When the user instantiates a Person, I'd like them to be able to call any method of the class. If the requested method does not exist (e.g. Person.dance()), a default function should be called instead.
I imagine that this could be done via a theoretical magic method -
class Person:
def __init__(self): pass
def say(self, speech): pass
def walk(self, destination): pass
def jump(self): pass
def sleep(self): print("Zzz")
def __method__(self, func):
if func.__name__ not in ['say','walk','jump']:
return self.sleep
else
return func
billy = Person()
billy.dance()
>> "Zzz"
However, I know of no such magic method.
Is there a way to make non-existent methods within a class redirect to another class?
It's important that the end-user doesn't have to do anything - from their perspective, it should just work.
The standard way to catch an undefined attribute is to use __getattr__:
# But see the end of the answer for an afterthought
def __getattr__(self, attr):
return self.sleep
Python does not differentiate between "regular" attributes and methods; a method call starts with an ordinary attribute lookup, whose result just happens to be callable. That is,
billy.run()
is the same as
f = billy.run
f()
This means that __getattr__ will be invoked for any undefined attribute; there is no way to tell at lookup time whether the result is going to be called or not.
However, if all you want is to define "aliases" for a common method, you can do that with a loop after the class statement.
class Person:
def __init__(self): pass
def say(self, speech): pass
def walk(self, destination): pass
def jump(self): pass
def do_all(self): pass
for alias in ["something", "something_else", "other"]:
setattr(Person, alias, Person.do_all)
You can also make hard-coded assignments in the class statement, but that would be unwieldy if there are, as you mention, hundreds of such cases:
class Person:
def do_all(self): pass
something = do_all
something_else = do_all
(I did not experiment with using exec to automate such assignments in a loop; it might be possible, though not recommended.)
You can also embed the list of aliases in the definition of __getattr__, come to think of it:
def __getattr__(self, attr):
if attr in ["something", "something_else", "other"]:
return self.sleep
else:
raise AttributeError(f"type object 'Person' has no attribute '{attr}'")
Your users might find the API behavior confusing. However, if you're sure you need this pattern, you can try something like
# getattr will get the function attribute by a string name version
# None is returned if no function is found
my_func = getattr(self, 'my_func', None)
# callable ensures `my_func` is actually a function and not a generic attribute
# Add your if-else logic here
if callable(my_func):
my_func(args)
else:
...
You could nest your "default" function inside __getattr__ in order to gain access to the called non-existent method's name and arguments.
class Test:
def __getattr__(self, attr):
def default(*args, **kwargs):
return attr, args, kwargs
return default
test = Test()
print(test.test('test'))
# ('test', ('test',), {})
I have a class who's job is to wrap another class (code I don't control), intercept all calls to the wrapped class, perform some logic, and pass along the call to the underlying class. Here's an example:
class GithubRepository(object):
def get_commit(self, sha):
return 'Commit {}'.format(sha)
def get_contributors(self):
return ['bobbytables']
class LoggingGithubRepositoryWrapper(object):
def __init__(self, github_repository):
self._github_repository = github_repository
def __getattr__(self, name):
base_func = getattr(self._github_repository, name)
def log_wrap(*args, **kwargs):
print "Calling {}".format(name)
return base_func(*args, **kwargs)
return log_wrap
if __name__ == '__main__':
git_client = LoggingGithubRepositoryWrapper(GithubRepository())
print git_client.get_commit('abcdef1245')
print git_client.get_contributors()
As you can see, the way that I do this is by implementing __getattr__ on the wrapping class and delegating to the underlying class. The downside to this approach is that users of LoggingGithubRepositoryWrapper don't know which attributes/methods the underlying GithubRepository actually has.
This leads me to my question: is there a way to define or document the calls handled by __getattr__? Ideally, I'd like to be able to autocomplete on git_client. and be provided a list of supported methods. Thanks for your help in advance!
You can do this a few different ways, but they wont involve the use of __getattr__.
What you really need to do is dynamically create your class, or at least dynamically create the wrapped functions on your class. There are a few ways to do this in python.
You could build the class definition using type() or metaclasses, or build it on class instantiation using the __new__ method.
Every time you call LoggingGithubRepositoryWrapper(), the __new__ method will be called. Here, it looks at all the attributes on the github_repository argument and finds all the non-private methods. It then creates a function on the instantiated LoggingGithubRepositoryWrapper class instance that wraps the repo call in a logging statement.
At the end, it passes back the modified class instance. Then __init__ is called.
from types import MethodType
class LoggingGithubRepositoryWrapper(object):
def __new__(cls, github_repository):
self = super(LoggingGithubRepositoryWrapper, cls).__new__(cls)
for name in dir(github_repository):
if name.startswith('__'):
continue
func = getattr(github_repository, name)
if isinstance(func, MethodType):
setattr(self, name, cls.log_wrap(func))
return self
#staticmethod
def log_wrap(func):
def wrap(*args, **kwargs):
print 'Calling {0}'.format(func.__name__)
return func(*args, **kwargs)
return wrap
def __init__(self, github_repository):
... # this is all the same
I have a method to save data in DB, and a decorator to manage the connection, but I can't figure out how to make it work.
method to save:
class DA_Row(DABase):
#DABase.connectAndDisconnect
def save(self):
"""
Guarda el spin en la base de datos
"""
self.__cursor.callproc('sp_insert_row', (
"value 1",
"value 2"
)
)
and I have here the inherited class with a function decorator that does not work.
class DABase():
def __init__(self):
self.__cursor = None
#staticmethod
def connectAndDisconnect(func):
def deco(*args):
returnValue = None
self.DBconnect()
try:
self.__cursor = self.db.cursor()
returnValue = func(*args)
finally:
self.desconectarDB()
return returnValue
return deco
....
Shown this...
How can I redefine DABase.__cursor from a decorator?
If is not possible, how to solve this problem in a different way?
Thank you for your time!
self is just a name like everything else, it does not magically appear like Java's this. You need to add it to your decorator. Try this:
#staticmethod
def connectAndDisconnect(func):
# deco will be a method, so it needs self (ie a DA_Row instance)
def deco(self, *args):
returnValue = None
self.DBconnect()
try:
self.__cursor = self.db.cursor()
# func was supposed to be a method to, so it needs self
returnValue = func(self, *args)
finally:
self.desconectarDB()
return returnValue
return deco
It would help if you showed the error that you're getting. However, I can take a guess...
Decorating a method of a class is hard. How is connectAndDisconnect supposed to know what self should be? connectAndDisconnect is a static method of the base class, which gets called when the derived class gets created, long before any instances of the derived class get created.
There's a trick which lets the decorator figure out what self should be, but it's a complicated hack and fragile in a way that I'll explain at the end. The trick is, use a class as the decorator, and make that class an descriptor (i.e. define __get__) to give you a chance to determine what self should be. In your case it would look something like:
class DABase(object):
def __init__(self):
self.__cursor = None
class connectAndDisconnect(object):
def __init__(self, method):
self._method = method # method is the thing being decorated
# note that it's an *unbound* method
self._instance = None # no way to know what the instance is yet
def __get__(self, instance, owner):
'This will be called when the decorated method is accessed'
self._instance = instance
return self
def __call__(self, *args):
'This is where the actual decoration takes place'
returnValue = None
# 'self' is the connectAndDisconnect object. 'self._instance' is the decorated object.
self._instance.DBConnect()
try:
self._instance.__cursor = self._instance.db.cursor()
# Since self._method is unbound, we have to pass the instance explicitly
returnValue = self._method(self._instance, *args)
finally:
self._instance.desconectarDB()
return returnValue
The derived class is unchanged:
class DA_Row(DABase):
#DABase.connectAndDisconnect
def save(self):
# ...
Now DA_Row.save is actually an instance of the connectAndDisconnect class. If d is a DA_Row object and someone calls d.save(), the first thing that happens is that connectAndDisconnect.__get__ gets called because someone tried to access d.save. This sets the _instance variable to equal d. Then connectAndDisconnect.__call__ gets called and the actual decoration takes place.
This works, most of the time. But it's fragile. It only works if you call save in the "normal" way, i.e. through an instance. If you try to do funny stuff like calling DA_Row.save(d) instead, it won't work because connectAndDisconnect.__get__ won't be able to figure out what the instance should be.