I have a base class like so:
class Token:
def __init__(self, value):
self.value = value.strip()
self.tokens = None
def get_value(self):
return self.value
def tokenize(self):
pass # abstract stub
def __str__(self):
return type(self).__name__ + ': '+ re.sub(r'\s+', ' ', self.value)
And a ton of it's child classes:
class T_DefineDirective(Token):
def __init__(self, value):
super().__init__(value)
class T_IncludeDirective(Token):
def __init__(self, value):
super().__init__(value)
class T_IfdefDirective(Token):
def __init__(self, value):
super().__init__(value)
class T_Identifier(Token):
def __init__(self, value):
super().__init__(value)
class T_Rvalue(Token):
def __init__(self, value):
super().__init__(value)
def tokenize(self):
pass # stuff here
Now I'm a DRY programmer. I hate repetition. If you look at the code, the __init__ piece is copy-pasted in all the child classes.
My question, is there some way to avoid the repetition, or is this really the right way?
(note that the example is a bit shortened, so it may not make too much sense. But you can see the issue I mean).
If you do not have any additional setup work to do in the Token subclasses, then it is safe not to override __init__.
If you do have to perform some subclass-specific initialisation, then the patten that you're using is fine and 'pythonic'.
To clarify:
if __init__ is not defined on a class, then Python will use the __init__ method defined on (one of) its parent class(es), if possible
this is because there aren't any special rules for overriding 'magic' methods like __init__
even if the initialiser on a parent class is used, an instance of the subclass will be created
this is because the actual creation happens in __new__; the newly created object is then passed to __init__ for initialisation
If you really want to eliminate as much boilerplate as possible:
First, you don't need __init__ if all it does is call super(); special methods are inherited just like any other methods, as sapi's answer explains.
Second, you can dynamically create a bunch of classes:
token_classes = {
'T_{}'.format(name): type('T_{}'.format(name), (Token,), {})
for name in 'DefineDirective IncludeDirective IfdefDirective Identifier'.split()
}
And you can use them straight out of that dict, but if you really want to make them into globals you can:
globals().update(token_classes)
However, the whole goal of avoiding repetition is to make your code more readable and maintainable, and in this case, I think we're achieving the opposite. :)
Related
So I have some god forsaken legacy code that uses the reserved word property, um wrong. In a base class that gets inherited they have basically implemented.
class TestClass(object):
def __init__(self, property):
self._property = property
#property
def property(self):
return self._property
test = TestClass('test property')
print(test.property)
Which runs without error. If you add another method below that you get,
class TestClass2(object):
def __init__(self, property):
self._property = property
#property
def property(self):
return self._property
#property
def other_property(self):
return 'test other property'
test = TestClass2('test property')
print(test.property)
print(test.other_property)
Which throws:
---> 10 #property
11 def other_property(self):
12 print('test other property')
TypeError: 'property' object is not callable
Because you know you have overwritten property in the local namespace.
class TestClass3(object):
def __init__(self, property):
self._property = property
#property
def other_property(self):
return 'test other property'
#property
def property(self):
return self._property
test = TestClass3('test property')
print(test.property)
print(test.other_property)
You can work around this if you always define your property overwrite at the bottom of your class. If the property method is only defined on the base class you inherit from things also work out, because namespaces.
class TestClass4(TestClass):
def __init__(self, property):
super(TestClass4, self).__init__(property)
#property
def other_property(self):
return 'test other property'
test = TestClass4('test property')
print(test.property)
print(test.other_property)
My righteous indignation says that we MUST update this variable name in the huge amount of legacy code, because GAAAAH, but other than having to remember to add new methods above the definition of property definition in the rarely modified base class, this doesn't actually break anything right?
dont shadow builtins... with almost no refactoring at all you can avoid shadowing the builtin entirely
use __getattr__ instead of #property to return your _property member ...
class TestClass(object):
def __init__(self):
self._property = 12
def __getattr__(self,item):
if item == "property":
#do your original getter code for `property` here ...
# now you have not overwritten the property keyword at all
return getattr(self,"_property") # just return the variable
class TestClass2(TestClass):
def __init__(self):
self._property = 67
print TestClass2().property
class MySubClass(TestClass):
#property
def a_property(self):
return 5
print MySubClass().property
print MySubClass().a_property
really, as an aside, theres not any good reason imho to use #property in python. all it does is end up confusing other programmers later and obscuring the fact that you are actually calling a function. I used to do it regularly ... I now avoid it unless I have a very very compelling reason not to
Yes, if you always add new methods above the definition of the property method nothing will break. So put a nice big comment to that effect in the code. Hopefully, anyone wanting to override property in a derived class will look at the base class first and see your comment...
BTW, the property arg to the __init__ method also shadows property, but I guess that's not an issue.
Ideally, someone should clean this mess up, but I understand that it may not be cost-effective to do that.
Also, I'm somewhat baffled why the original coder made property an #property in the first place. That construct should only be used when the value of the attribute has to be calculated dynamically, not for simply returning a static attribute. Perhaps they were new to Python and they were told that's the way to do getters in Python...
You could always remap property to another name. So long as you choose a name that doesn't match your other class attributes, and it won't be exposed in the external interface for the class, so it doesn't really matter what you name it.
tproperty = property
class Test(...)
#tproperty
def property(self):
....
In Python, how do I get a reference to the current class object within a class statement? Example:
def setup_class_members(cls, prefix):
setattr(cls, prefix+"_var1", "hello")
setattr(cls, prefix+"_var2", "goodbye")
class myclass(object):
setup_class_members(cls, "coffee") # How to get "cls"?
def mytest(self):
print(self.coffee_var1)
print(self.coffee_var2)
x = myclass()
x.mytest()
>>> hello
>>> goodbye
Alternatives that I've written off are:
Use locals(): This gives a dict in a class statement that can be written to. This seems to work for classes, however the documentation tells you not to do this. (I might be tempted to go with this alternative if someone can assure me that this will continue to work for some time.)
Add members to the class object after the class statement: My actual application is to derive a PyQt4 QWidget class with dynamically created pyqtProperty class attributes. QWidget is unusual in that it has a custom metaclass. Very roughly, the metaclass compiles a list of pyqtProperties and stores it as additional member. For this reason, properties that are added to the class after creation have no effect. An example to clear this up:
from PyQt4 import QtCore, QtGui
# works
class MyWidget1(QtGui.QWidget):
myproperty = QtCore.pyqtProperty(int)
# doesn't work because QWidget's metaclass doesn't get to "compile" myproperty
class MyWidget2(QtGui.QWidget):
pass
MyWidget2.myproperty = QtCore.pyqtProperty(int)
Please note that the above will work for most programming cases; my case just happens to be one of those unusual corner cases.
For Python 3, the class must be declared as
class myclass(object, metaclass = Meta):
prefix = "coffee"
...
A few other points:
The metaclass may be a callable, not just a class (Python 2&3)
If the base class of your class already has a non-standard metaclass, you have to make sure you call it's __init__() and __new__() methods instead of type's.
The class statement accepts keyword parameters that are passed on to the metaclass (Python 3 only)
A rewrite of mouad's solution in Python 3 using all of the above is...
def MetaFun(name, bases, attr, prefix=None):
if prefix:
attr[prefix+"_var1"] = "hello"
attr[prefix+"_var2"] = "goodbye"
return object.__class__(name, bases, attr)
class myclass(object, metaclass = MetaFun, prefix="coffee"):
def mytest(self):
print(self.coffee_var1)
print(self.coffee_var2)
AFAIK there is two way to do what you want:
Using metaclass, this will create your two variables in class creation time (which i think is what you want):
class Meta(type):
def __new__(mcs, name, bases, attr):
prefix = attr.get("prefix")
if prefix:
attr[prefix+"_var1"] = "hello"
attr[prefix+"_var2"] = "goodbye"
return type.__new__(mcs, name, bases, attr)
class myclass(object):
__metaclass__ = Meta
prefix = "coffee"
def mytest(self):
print(self.coffee_var1)
print(self.coffee_var2)
Create your two class variable in instantiation time:
class myclass(object):
prefix = "coffee"
def __init__(self):
setattr(self.__class__, self.prefix+"_var1", "hello")
setattr(self.__class__, self.prefix+"_var2", "goodbye")
def mytest(self):
print(self.coffee_var1)
print(self.coffee_var2)
N.B: I'm not sure what you want to achieve because if you want to create dynamic variables depending on the prefix variable why are you accessing like you do in your mytest method ?! i hope it was just an example.
Two more approaches you might use:
A class decorator.
def setup_class_members(prefix):
def decorator(cls):
setattr(cls, prefix+"_var1", "hello")
setattr(cls, prefix+"_var2", "goodbye")
return cls
return decorator
#setup_class_members("coffee")
class myclass(object):
# ... etc
Especially if you need to add attributes in various combinations, the decorator approach is nice because it does not have any effect on inheritance.
If you are dealing with a small set of of attributes that you wish to combine in various ways, you can use mixin classes. A mixin class is a regular class, it's just intended to "mix in" various attributes to some other class.
class coffee_mixin(object):
coffee_var1 = "hello"
coffee_var2 = "goodbye"
class tea_mixin(object):
tea_var1 = "good morning old bean"
tea_var2 = "pip pip cheerio"
class myclass(coffee_mixin, tea_mixin):
# ... etc
See zope.interface.declarations._implements for an example of doing this kind of magic. Just be warned that it's a serious maintainability and portability risk.
This question already has answers here:
Making functions non override-able
(6 answers)
Closed 3 years ago.
Is there any way I can prevent a child class from overriding a method in the base class?
My guess is that there is not, but I'm coming from the .NET world, and I'm trying to make my API as robust as possible, so any input is greatly appreciated.
class Parent:
def do_something(self):
'''This is where some seriously important stuff goes on'''
pass
class Child(Parent):
def do_something(self):
'''This should not be allowed.'''
pass
Is it possible to enforce this? I know the compiler won't help, so maybe by means of some runtime check? Or is it just not a pythonic way of going about things?
You are right: what you are attempting is contrary to Python's structure and its culture.
Document your API, and educate your users how to use it. It's their program, so if they still want to override your function, who are you to prevent them?
If a API lets you provide subclasses of a certain class and calls your (legally) overridden methods, but also other API methods of that class with simple names like "add", accidentally overriding those methods could lead to hard-to-track-down bugs. It's better to at least warn the user.
The cases where a user wants/needs to override a method that will completely break the API is practically zero. The cases where a user accidentally overrides something that he shouldn't and needs hours to find the culprit are far more frequent. Debugging faulty behaviour caused by this can be cumbersome.
This is how I use to warn or protect attributes from being accidentally overridden:
def protect(*protected):
"""Returns a metaclass that protects all attributes given as strings"""
class Protect(type):
has_base = False
def __new__(meta, name, bases, attrs):
if meta.has_base:
for attribute in attrs:
if attribute in protected:
raise AttributeError('Overriding of attribute "%s" not allowed.'%attribute)
meta.has_base = True
klass = super().__new__(meta, name, bases, attrs)
return klass
return Protect
You can use it like this:
class Parent(metaclass=protect("do_something", "do_something_else")):
def do_something(self):
'''This is where some seriously important stuff goes on'''
pass
class Child(Parent):
def do_something(self):
'''This will raise an error during class creation.'''
pass
uzumaki already provided one metaclass as a possible solution to the question asked above, but here is another with example usage. Following an attempt to create a Child class, another way of making it difficult to override methods is shown. Putting two underscores before but not after an attribute name will automatically cause name mangling to be invoked. See this answer to another question for an easy-to-use way of accessing this ability manually.
#! /usr/bin/env python3
class Access(type):
__SENTINEL = object()
def __new__(mcs, name, bases, class_dict):
private = {key
for base in bases
for key, value in vars(base).items()
if callable(value) and mcs.__is_final(value)}
if any(key in private for key in class_dict):
raise RuntimeError('certain methods may not be overridden')
return super().__new__(mcs, name, bases, class_dict)
#classmethod
def __is_final(mcs, method):
try:
return method.__final is mcs.__SENTINEL
except AttributeError:
return False
#classmethod
def final(mcs, method):
method.__final = mcs.__SENTINEL
return method
class Parent(metaclass=Access):
#Access.final
def do_something(self):
"""This is where some seriously important stuff goes on."""
pass
try:
class Child(Parent):
def do_something(self):
"""This should not be allowed."""
pass
except RuntimeError:
print('Child cannot be created.')
class AnotherParent:
def __do_something(self):
print('Some seriously important stuff is going on.')
def do_parent_thing(self):
self.__do_something()
class AnotherChild(AnotherParent):
def __do_something(self):
print('This is allowed.')
def do_child_thing(self):
self.__do_something()
example = AnotherChild()
example.do_parent_thing()
example.do_child_thing()
Goal: Make a decorator which can modify the scope that it is used in.
If it worked:
class Blah(): # or perhaps class Blah(ParentClassWhichMakesThisPossible)
def one(self):
pass
#decorated
def two(self):
pass
>>> Blah.decorated
["two"]
Why? I essentially want to write classes which can maintain specific dictionaries of methods, so that I can retrieve lists of available methods of different types on a per class basis. errr.....
I want to do this:
class RuleClass(ParentClass):
#rule
def blah(self):
pass
#rule
def kapow(self):
pass
def shazam(self):
class OtherRuleClass(ParentClass):
#rule
def foo(self):
pass
def bar(self):
pass
>>> RuleClass.rules.keys()
["blah", "kapow"]
>>> OtherRuleClass.rules.keys()
["foo"]
You can do what you want with a class decorator (in Python 2.6) or a metaclass. The class decorator version:
def rule(f):
f.rule = True
return f
def getRules(cls):
cls.rules = {}
for attr, value in cls.__dict__.iteritems():
if getattr(value, 'rule', False):
cls.rules[attr] = value
return cls
#getRules
class RuleClass:
#rule
def foo(self):
pass
The metaclass version would be:
def rule(f):
f.rule = True
return f
class RuleType(type):
def __init__(self, name, bases, attrs):
self.rules = {}
for attr, value in attrs.iteritems():
if getattr(value, 'rule', False):
self.rules[attr] = value
super(RuleType, self).__init__(name, bases, attrs)
class RuleBase(object):
__metaclass__ = RuleType
class RuleClass(RuleBase):
#rule
def foo(self):
pass
Notice that neither of these do what you ask for (modify the calling namespace) because it's fragile, hard and often impossible. Instead they both post-process the class -- through the class decorator or the metaclass's __init__ method -- by inspecting all the attributes and filling the rules attribute. The difference between the two is that the metaclass solution works in Python 2.5 and earlier (down to 2.2), and that the metaclass is inherited. With the decorator, subclasses have to each apply the decorator individually (if they want to set the rules attribute.)
Both solutions do not take inheritance into account -- they don't look at the parent class when looking for methods marked as rules, nor do they look at the parent class rules attribute. It's not hard to extend either to do that, if that's what you want.
Problem is, at the time the decorated decorator is called, there is no object Blah yet: the class object is built after the class body finishes executing. Simplest is to have decorated stash the info "somewhere else", e.g. a function attribute, then a final pass (a class decorator or metaclass) reaps that info into the dictionary you desire.
Class decorators are simpler, but they don't get inherited (so they wouldn't come from a parent class), while metaclasses are inherited -- so if you insist on inheritance, a metaclass it will have to be. Simplest-first, with a class decorator and the "list" variant you have at the start of your Q rather than the "dict" variant you have later:
import inspect
def classdecorator(aclass):
decorated = []
for name, value in inspect.getmembers(aclass, inspect.ismethod):
if hasattr(value, '_decorated'):
decorated.append(name)
del value._decorated
aclass.decorated = decorated
return aclass
def decorated(afun):
afun._decorated = True
return afun
now,
#classdecorator
class Blah(object):
def one(self):
pass
#decorated
def two(self):
pass
gives you the Blah.decorated list you request in the first part of your Q. Building a dict instead, as you request in the second part of your Q, just means changing decorated.append(name) to decorated[name] = value in the code above, and of course initializing decorated in the class decorator to an empty dict rather than an empty list.
The metaclass variant would use the metaclass's __init__ to perform essentially the same post-processing after the class body is built -- a metaclass's __init__ gets a dict corresponding to the class body as its last argument (but you'll have to support inheritance yourself by appropriately dealing with any base class's analogous dict or list). So the metaclass approach is only "somewhat" more complex in practice than a class decorator, but conceptually it's felt to be much more difficult by most people. I'll give all the details for the metaclass if you need them, but I'd recommend sticking with the simpler class decorator if feasible.
The problem: I have a class which contains a template method execute which calls another method _execute. Subclasses are supposed to overwrite _execute to implement some specific functionality. This functionality should be documented in the docstring of _execute.
Advanced users can create their own subclasses to extend the library. However, another user dealing with such a subclass should only use execute, so he won't see the correct docstring if he uses help(execute).
Therefore it would be nice to modify the base class in such a way that in a subclass the docstring of execute is automatically replaced with that of _execute. Any ideas how this might be done?
I was thinking of metaclasses to do this, to make this completely transparent to the user.
Well, if you don't mind copying the original method in the subclass, you can use the following technique.
import new
def copyfunc(func):
return new.function(func.func_code, func.func_globals, func.func_name,
func.func_defaults, func.func_closure)
class Metaclass(type):
def __new__(meta, name, bases, attrs):
for key in attrs.keys():
if key[0] == '_':
skey = key[1:]
for base in bases:
original = getattr(base, skey, None)
if original is not None:
copy = copyfunc(original)
copy.__doc__ = attrs[key].__doc__
attrs[skey] = copy
break
return type.__new__(meta, name, bases, attrs)
class Class(object):
__metaclass__ = Metaclass
def execute(self):
'''original doc-string'''
return self._execute()
class Subclass(Class):
def _execute(self):
'''sub-class doc-string'''
pass
Is there a reason you can't override the base class's execute function directly?
class Base(object):
def execute(self):
...
class Derived(Base):
def execute(self):
"""Docstring for derived class"""
Base.execute(self)
...stuff specific to Derived...
If you don't want to do the above:
Method objects don't support writing to the __doc__ attribute, so you have to change __doc__ in the actual function object. Since you don't want to override the one in the base class, you'd have to give each subclass its own copy of execute:
class Derived(Base):
def execute(self):
return Base.execute(self)
class _execute(self):
"""Docstring for subclass"""
...
execute.__doc__= _execute.__doc__
but this is similar to a roundabout way of redefining execute...
Look at the functools.wraps() decorator; it does all of this, but I don't know offhand if you can get it to run in the right context
Well the doc-string is stored in __doc__ so it wouldn't be too hard to re-assign it based on the doc-string of _execute after the fact.
Basically:
class MyClass(object):
def execute(self):
'''original doc-string'''
self._execute()
class SubClass(MyClass):
def _execute(self):
'''sub-class doc-string'''
pass
# re-assign doc-string of execute
def execute(self,*args,**kw):
return MyClass.execute(*args,**kw)
execute.__doc__=_execute.__doc__
Execute has to be re-declared to that the doc string gets attached to the version of execute for the SubClass and not for MyClass (which would otherwise interfere with other sub-classes).
That's not a very tidy way of doing it, but from the POV of the user of a library it should give the desired result. You could then wrap this up in a meta-class to make it easier for people who are sub-classing.
I agree that the simplest, most Pythonic way of approaching this is to simply redefine execute in your subclasses and have it call the execute method of the base class:
class Sub(Base):
def execute(self):
"""New docstring goes here"""
return Base.execute(self)
This is very little code to accomplish what you want; the only downside is that you must repeat this code in every subclass that extends Base. However, this is a small price to pay for the behavior you want.
If you want a sloppy and verbose way of making sure that the docstring for execute is dynamically generated, you can use the descriptor protocol, which would be significantly less code than the other proposals here. This is annoying because you can't just set a descriptor on an existing function, which means that execute must be written as a separate class with a __call__ method.
Here's the code to do this, but keep in mind that my above example is much simpler and more Pythonic:
class Executor(object):
__doc__ = property(lambda self: self.inst._execute.__doc__)
def __call__(self):
return self.inst._execute()
class Base(object):
execute = Executor()
class Sub(Base):
def __init__(self):
self.execute.inst = self
def _execute(self):
"""Actually does something!"""
return "Hello World!"
spam = Sub()
print spam.execute.__doc__ # prints "Actually does something!"
help(spam) # the execute method says "Actually does something!"