I have a chain of inheritance in Python, and I want each child class to be able to add on new custom parameters. Right now I'm doing this:
class A(object):
PARAM_NAMES = ['blah1']
...
class B(A):
PARAM_NAMES = A.PARAM_NAMES + ['blah2']
...
I'm wondering if there's a slicker method, though, without referencing A twice? Can't use super() because it's not within a method definition, afaik. I suppose I could use a class method, but that'd be annoying (since I really would want a property).
What's the right way to do this?
of coarse there is always black magic you can do ... but the question is just because you can ... should you?
class MyMeta(type):
items = []
def __new__(meta, name, bases, dct):
return super(MyMeta, meta).__new__(meta, name, bases, dct)
def __init__(cls, name, bases, dct):
MyMeta.items.extend(cls.items)
cls.items = MyMeta.items[:]
super(MyMeta, cls).__init__(name, bases, dct)
class MyKlass(object):
__metaclass__ = MyMeta
class A(MyKlass):
items=["a","b","c"]
class B(A):
items=["1","2","3"]
print A.items
print B.items
since this creates a copy it will not suffer from the same problem as the other solution
(please note that I dont really recommend doing this ... its just to show you can)
This may or may not be smart, but it's technically possible to use a metaclass for this. Unlike Joran's method, I use a property, so that it retains full dynamic nature (that is, if you modify any class's private _PARAM_NAMES list after defining the class, the corresponding PARAM_NAME property of every other derived class reflects that change). For this reason I put an add_param method on the base class.
Python 3 is assumed here, and the PARAM_NAMES property returns a set to avoid duplicate items.
class ParamNameBuilderMeta(type):
def __new__(mcl, name, bases, dct):
names = dct.get("PARAM_NAMES", [])
names = {names} if isinstance(names, str) else set(names)
dct["_PARAM_NAMES"] = names
dct["PARAM_NAMES"] = property(lambda s: type(s).PARAM_NAMES)
return super().__new__(mcl, name, bases, dct)
#property
def PARAM_NAMES(cls):
# collect unique list items ONLY from our classes in the MRO
return set().union(*(c._PARAM_NAMES for c in reversed(cls.__mro__)
if isinstance(c, ParamNameBuilderMeta)))
Usage:
class ParamNameBuilderBase(metaclass=ParamNameBuilderMeta):
#classmethod
def add_param(self, param_name):
self._PARAM_NAMES.add(param_name)
class A(ParamNameBuilderBase):
PARAM_NAMES = 'blah1'
class B(A):
PARAM_NAMES = 'blah1', 'blah2'
class C(B):
pass
Check to make sure it works on both classes and instances thereof:
assert C.PARAM_NAMES == {'blah1', 'blah2'}
assert C().PARAM_NAMES == {'blah1', 'blah2'}
Check to make sure it's still dynamic:
C.add_param('blah3')
assert C.PARAM_NAMES == {'blah1', 'blah2', 'blah3'}
The behavior you've described is actually quite specific. You've said that you
want each child class to be able to add on new custom paramters
But the way you've implemented it, this will result in unpredictable behaviour. Consider:
class A(object):
PARAM_NAMES = ['blah1']
class B(A):
PARAM_NAMES = A.PARAM_NAMES + ['blah2']
class C(A):pass
print(A.PARAM_NAMES)
print(B.PARAM_NAMES)
print(C.PARAM_NAMES)
A.PARAM_NAMES.append('oops')
print(C.PARAM_NAMES)
What we notice is that the classes that choose to add new parameters have a new reference to the parameter list, while ones that do not add new parameters have the same reference as their parent. Unless carefully controlled, this is unsafe behaviour.
It is more reliable to only use constants as class properties, or to redefine the list entirely each time (make it a tuple), which is not "slicker". Otherwise, I'd reccomend class methods, as you suggest, and making the property an instance variable
Related
I have a class parent and two subclasses child1(parent) and child2(parent) sort of like the following near code.
(edited to more properly show that the parent class is doing something)
class parent(object):
name = None
def __init__(self,e):
# process the common attributes
name = e.attrib['name']
def __new__(cls,e):
if e.attrib['type'] == 'c1':
return child1(e)
elif e.attrib['type'] == 'c2':
return child2(e)
else:
raise
class child1(parent):
extra1 = None
def __init__(self,e):
super(e)
# set attributes from e that are specific to type c1
class child2(parent):
extra2 = None
def __init__(self,e):
super(e)
# set attributes from e that are specific to type c2
The goal is to be able to get the "right" class based on the value of the parameter. So if I can say obj = parent(element) and obj will be either child1 or child2 depending on what the value of element.attrib['type'] is.
The problem is that inside parent.__new__, you're calling child1(e), while calls child1.__new__, which finds the implementation in parent.__new__ and calls it with the same e, which calls child1(e), which… so you get infinite recursion.
There are better ways to design this, but if you just want to fix your design, there are three options:
If you define __new__ in all of your subclasses, it won't fall through the parent.__new__. You can do this in a single step by interposing an intermediate class between parent and childN, so you only need intermediate.__new__. Or use a mixin that they all inherit, or …
Get rid of the inheritance. Is there really any reason child1 is-a parent here?
You seem to be looking for what in Smalltalk/ObjC terms is called a "class cluster", and you don't need the "visible face" of the cluster to be the base class in Python any more than you do in those languages.
For example:
class base(object):
pass
class parent(base):
def __new__(cls, e):
# same as before
class child1(base):
# etc.
In Python, you can even make parent an ABC, and register each childN with it so you can use isinstance and friends with it.
Finally, you can just catch the recursion by only handling __new__ on parent, not its subclasses:
def __new__(cls, e):
if cls is not parent:
return super(parent, cls).__new__(cls)
This is a lot easier if you don't have parent be a class at all, but just a normal function.
Using the base class is a pattern from languages where that's the only real way to do this. It's not necessary or helpful in Python.
Is there a way to make a Python #property act as a setter and getter all at once?
I feel like I've seen this somewhere before but can't remember and can't recreate the solution myself.
For example, instead of:
class A(object):
def __init__(self, b): self.b = b
def get_c(self): return self.b.c
def set_c(self, value): self.b.c = value
c = property(get_c, set_c)
we could somehow signal that for A objects, the c attribute is really equivalent to b.c for getter, setter (and deleter if we like).
Motivation:
This would be particularly useful when we need A to be a proxy wrapper around B objects (of which b is an instance) but share only the data attributes and no methods. Properties such as these would allow the A and B objects' data to stay completely in sync while both are used by the same code.
I think you are looking for this forwardTo class as posted on ActiveState.
This recipe lets you transparently forward attribute access to another
object in your class. This way, you can expose functionality from some
member of your class instance directly, e.g. foo.baz() instead of
foo.bar.baz().
class forwardTo(object):
"""
A descriptor based recipe that makes it possible to write shorthands
that forward attribute access from one object onto another.
>>> class C(object):
... def __init__(self):
... class CC(object):
... def xx(self, extra):
... return 100 + extra
... foo = 42
... self.cc = CC()
...
... localcc = forwardTo('cc', 'xx')
... localfoo = forwardTo('cc', 'foo')
...
>>> print C().localcc(10)
110
>>> print C().localfoo
42
Arguments: objectName - name of the attribute containing the second object.
attrName - name of the attribute in the second object.
Returns: An object that will forward any calls as described above.
"""
def __init__(self, objectName, attrName):
self.objectName = objectName
self.attrName = attrName
def __get__(self, instance, owner=None):
return getattr(getattr(instance, self.objectName), self.attrName)
def __set__(self, instance, value):
setattr(getattr(instance, self.objectName), self.attrName, value)
def __delete__(self, instance):
delattr(getattr(instance, self.objectName), self.attrName)
For a more robust code, you may want to consider replacing getattr(instance, self.objectName) with operator.attrgetter(self.objectName)(instance). This would allow objectName to be a dotted name (e.g., so you could have A.c be a proxy for A.x.y.z.d).
If you're trying to delegate a whole slew of properties from any A object to its b member, it's probably easier to do that inside __getattr__, __setattr__, and __delattr__, e.g.:
class A(object):
delegated = ['c', 'd', 'e', 'f']
def __getattr__(self, attr):
if attr in A.delegated:
return getattr(self.b, attr)
raise AttributeError()
I haven't shown the __setattr__ and __delattr__ definitions here, for brevity, and to avoid having to explain the difference between __getattr__ and __getattribute__. See the docs if you need more information.
This is readily extensible to classes that want to proxy different attributes to different members:
class A(object):
b_delegated = ['c', 'd', 'e', 'f']
x_delegated = ['y', 'z']
def __getattr__(self, attr):
if attr in A.b_delegated:
return getattr(self.b, attr)
elif attr in A.x_delegated:
return getattr(self.x, attr)
else:
raise AttributeError()
If you need to delegate all attributes, dynamically, that's almost as easy. You just get a list of self.b's attributes (or self.b.__class__'s) at init time or at call time (which of the four possibilities depends on exactly what you want to do), and use that in place of the static list b_delegated.
You can of course filter this by name (e.g., to remove _private methods), or by type, or any arbitrary predicate (e.g., to remove any callable attributes).
Or combine any of the above.
At any rate, this is the idiomatic way to do (especially dynamic) proxying in Python. It's not perfect, but trying to invent a different mechanism is probably not a good idea.
And in fact, it's not really meant to be perfect. This is something you shouldn't be doing too often, and shouldn't be trying to disguise when you do it. It's obvious that a ctypes.cdll or a pyobjc module is actually delegating to something else, because it's actually useful for the user to know that. If you really need to delegate most of the public interface of one class to another, and don't want the user to know about the delegation… maybe you don't need it. Maybe it's better to just expose the private object directly, or reorganize your object model so the user is interacting with the right things in the first place.
There's the decorator syntax for creating properties, then there are full blown custom-defined descriptors, but since the setter/getter pseudo-private pattern is actively discouraged in Python and the Python community, there isn't really a widely distributed or commonly used way to do what you are looking for.
For proxy objects, you can use __getattr__, __setattr__, and __getattribute__, or try to manipulate things earlier in the process by fooling around with __new__ or a metaclass.
def make_property(parent, attr):
def get(self):
return getattr(getattr(self, parent), attr)
def set(self, value):
setattr(getattr(self, parent), attr, value)
return property(get, set)
class A(object):
def __init__(self, b): self.b = b
c = make_property('b', 'c')
Here's another way of doing it, statically forwarding properties from one object to another, but with economy.
It allows to forward get/set property in two lines, and aread-only property in one line, making use of dynamic property definition at the class level and lambdas.
class A:
"""Classic definition of property, with decorator"""
_id = ""
_answer = 42
#property
def id(self):
return self._id
#id.setter
def id(self, value):
self._id = value
#property
def what(self):
return self._answer
class B:
obj = A()
# Forward "id" from self.obj
id = property(lambda self: self.obj.id,
lambda self, value: setattr(self.obj, "id", value))
# Forward read-only property from self.obj
what = property(lambda self: self.obj.what)
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Can I get a reference to the 'owner' class during the init method of a descriptor?
Code is worth a thousand words:
>>> class ShortRib(object):
>>> def __init__(self, owner):
>>> self.owner = owner
>>>
>>> ... some more methods and stuff ...
>>>
>>>
>>> class Cow(object):
>>> shortRib = ShortRib(self)
>>>
>>>
>>> class BrownCow(Cow):
>>> pass
>>>
>>> BrownCow.shortRib.owner
<class '__main__.BrownCow'>
This doesn't work, though i wish it would. Basically, I want each class to have some static/class variables (i'm not sure which it is in this case?) but need each of those guys to know who (which class) it belongs to. Unfortunately, I can't "get" at the class in the body of the class declaration. Of course, I could always do this using a decorator:
>>> def vars(**kwargs):
>>> def wrap(cls):
>>> for k, w in kwargs.items():
>>> setattr(cls, k, w(cls))
>>> return cls
>>> return wrap
>>>
>>> #vars(shortRib=lambda cls: ShortRib(cls)
>>> class BrownCow(Cow):
>>> ...
>>>
>>> BrownCow.shortRib.owner
which would work. Another way would to have a class decorator that goes through all the shortRibs and similar static variables and sets their owner after the class declaration is complete. However, this seems like an incredibly roundabout and unintuitive way of doing what should be a pretty simple operation: having the static/class members of a class know who they belong to.
Is there a "proper" way of doing this?
Clarification:
I want these members to belong to the class, not to the instances. I'm trying to go for a almost-purely-functional style, using classes only for inheritance of shared behavior, and not creating instances of them at all. Instances would tend to give my functions access to arbitrary instance data shared across all functions, which would break the pure-functioness I am trying for. I could just use empty instances which I don't touch, but I think using pure classes would be cleaner.
You can easily do this in __new__:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class Cow(object):
shortRib = None
def __new__(cls, *args, **kwargs):
if cls.shortRib == None:
cls.shortRib = ShortRib(cls)
return super(Cow, cls).__new__(cls, *args, **kwargs)
Cow()
Cow.shortRib.owner
Or even __init__, if you don't mind referencing self.__class___.
You can also do it with a metaclass:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class MetaCow(type):
def __new__(cls, name, base, attrs):
attrs['shortRib'] = ShortRib(cls)
return super(MetaCow, cls).__new__(cls, name, base, attrs)
class Cow(object):
__metaclass__ = MetaCow
Cow.shortRib.owner
Why not let the instances of the Cow class have shortRibs, instead of the class itself?:
class ShortRib(object):
def __init__(self,owner):
self.owner=owner
class Cow(object):
def __init__(self):
self.shortRib=ShortRib(self)
class BrownCow(Cow):
pass
print(BrownCow().shortRib.owner)
# <__main__.BrownCow object at 0xb76a8d6c>
(Otherwise, you'll need a class decorator or metaclass -- as you've already mentioned. But simple is better than complex, so why not choose simple?)
By the way, if you really do want to use classes instead of instances:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class MetaCow(type):
def __init__(cls, name, base, attrs):
super(MetaCow, cls).__init__(name, base, attrs)
cls.shortRib = ShortRib(cls)
class Cow(object):
__metaclass__ = MetaCow
class BrownCow(Cow):
pass
print(Cow.shortRib.owner)
# <class '__main__.Cow'>
print(BrownCow.shortRib.owner)
# <class '__main__.BrownCow'>
Using
class MetaCow(type):
def __new__(cls, name, base, attrs):
is incorrect. The signature for type.__new__ is
class MetaCow(type):
def __new__(meta, name, base, attrs):
Since you want to modify the attributes of cls, not meta, use the MetaCow.__init__ not MetaCow__new__.
Two methods to to do what you want:
You can override the __getattr__ method in any class to return anything you desire when you ask for the value of an attribute.
You can use a property, which has a getter that returns the object you want it to return.
Both __getattr__ methods and properties are inherited.
I'm trying to create a metaclass for the class I created here: ctypes variable length structures
I want to simplify the Points class so it looks like this (Python 3.2):
class Points(c.Structure, metaclass=VariableMeta):
_fields_ = [
('num_points', c.c_uint32),
('points', 'Point*self.num_points')
]
def __init__(self):
self.num_points = 0
self.points = [0,]*MAX_SIZE
This is the metaclass I have so far:
class VariableMeta(type):
def __new__(cls, name, bases, dct):
dct['_inner_fields'] = dct['_fields_']
dct['_fields_'] = [('_buffer', c.c_byte*MAX_PACKET_SIZE)]
return type.__new__(cls, name, bases, dct)
def parse(self):
fields = []
for name, ctype in self._inner_fields:
if type(ctype) == str:
ctype = eval(ctype)
fields.append((name, ctype))
class Inner(c.Structure, PrettyPrinter):
_fields_ = fields
inner = Inner.from_address(c.addressof(self._buffer))
setattr(self, name, getattr(inner, name))
self = inner
return self
def pack(self):
fields = []
for name, ctype in self._inner_fields:
if type(ctype) == str:
ctype = eval(ctype)
fields.append((name, ctype))
class Inner(c.Structure, PrettyPrinter):
_fields_ = fields
inner = Inner()
for name, ctype in self._inner_fields:
value = getattr(self, name)
if type(value) == list:
l = getattr(inner, name)
for i in range(len(l)):
l[i] = getattr(self, name)[i]
else:
setattr(inner, name, value)
return inner
It looks like it should work, but when I run it I get the error: TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases.
I searched for hints to the solution of this problem, but ctypes Structure looks to be implemented in a c library. I am not sure how to fix this, any help or the specific solution is appreciated!
The problem is that ctypes.Structure uses its own custom metaclass: _ctypes.StructType. Since you inherit the metaclass from Structure, Python does not know which metaclass to use when constructing your class.
You can fix this by inheriting your metaclass from _ctypes.StructType. Since the name of the metaclass is an implementation detail of the ctypes module, I recommend writing type(ctypes.Structure) to get the metaclass dynamically.
import ctypes
class VariableMeta(type(ctypes.Structure)):
pass
The drawback with this approach is that you limit the use of your metaclass. This might be OK if you only plan to use it for subclasses of ctypes.Structure.
Another approach is to create a intermediate metaclass that inherits from both metaclasses.
class PointsMetaClass(type(ctypes.Structure), VariableMeta):
pass
class Points(c.Structure, metaclass=PointsMetaClass):
# ...
Always make sure that you use super() instead of hard-coding type in your metaclass' __new__!
return super(VariableMeta, cls).__new__(cls, name, bases, dct)
As Guido once wrote: Writing metaclasses in Python will cause your head to explode!
This is an unusual question, but I'd like to dynamically generate the __slots__ attribute of the class based on whatever attributes I happened to have added to the class.
For example, if I have a class:
class A(object):
one = 1
two = 2
__slots__ = ['one', 'two']
I'd like to do this dynamically rather than specifying the arguments by hand, how would I do this?
At the point you're trying to define slots, the class hasn't been built yet, so you cannot define it dynamically from within the A class.
To get the behaviour you want, use a metaclass to introspect the definition of A and add a slots attribute.
class MakeSlots(type):
def __new__(cls, name, bases, attrs):
attrs['__slots__'] = attrs.keys()
return super(MakeSlots, cls).__new__(cls, name, bases, attrs)
class A(object):
one = 1
two = 2
__metaclass__ = MakeSlots
One very important thing to be aware of -- if those attributes stay in the class, the __slots__ generation will be useless... okay, maybe not useless -- it will make the class attributes read-only; probably not what you want.
The easy way is to say, "Okay, I'll initialize them to None, then let them disappear." Excellent! Here's one way to do that:
class B(object):
three = None
four = None
temp = vars() # get the local namespace as a dict()
__slots__ = temp.keys() # put their names into __slots__
__slots__.remove('temp') # remove non-__slots__ names
__slots__.remove('__module__') # now remove the names from the local
for name in __slots__: # namespace so we don't get read-only
del temp[name] # class attributes
del temp # and get rid of temp
If you want to keep those initial values it takes a bit more work... here's one possible solution:
class B(object):
three = 3
four = 4
def __init__(self):
for key, value in self.__init__.defaults.items():
setattr(self, key, value)
temp = vars()
__slots__ = temp.keys()
__slots__.remove('temp')
__slots__.remove('__module__')
__slots__.remove('__init__')
__init__.defaults = dict()
for name in __slots__:
__init__.defaults[name] = temp[name]
del temp[name]
del temp
As you can see, it is possible to do this without a metaclass -- but who wants all that boilerplate? A metaclass could definitely help us clean this up:
class MakeSlots(type):
def __new__(cls, name, bases, attrs):
new_attrs = {}
new_attrs['__slots__'] = slots = attrs.keys()
slots.remove('__module__')
slots.remove('__metaclass__')
new_attrs['__weakref__'] = None
new_attrs['__init__'] = init = new_init
init.defaults = dict()
for name in slots:
init.defaults[name] = attrs[name]
return super(MakeSlots, cls).__new__(cls, name, bases, new_attrs)
def new_init(self):
for key, value in self.__init__.defaults.items():
setattr(self, key, value)
class A(object):
__metaclass__ = MakeSlots
one = 1
two = 2
class B(object):
__metaclass__ = MakeSlots
three = 3
four = 4
Now all the tediousness is kept in the metaclass, and the actual class is easy to read and (hopefully!) understand.
If you need to have anything else in these classes besides attributes I strongly suggest you put whatever it is in a mixin class -- having them directly in the final class would complicate the metaclass even more.