metaclass conflict with ctypes Structure - python

I'm trying to create a metaclass for the class I created here: ctypes variable length structures
I want to simplify the Points class so it looks like this (Python 3.2):
class Points(c.Structure, metaclass=VariableMeta):
_fields_ = [
('num_points', c.c_uint32),
('points', 'Point*self.num_points')
]
def __init__(self):
self.num_points = 0
self.points = [0,]*MAX_SIZE
This is the metaclass I have so far:
class VariableMeta(type):
def __new__(cls, name, bases, dct):
dct['_inner_fields'] = dct['_fields_']
dct['_fields_'] = [('_buffer', c.c_byte*MAX_PACKET_SIZE)]
return type.__new__(cls, name, bases, dct)
def parse(self):
fields = []
for name, ctype in self._inner_fields:
if type(ctype) == str:
ctype = eval(ctype)
fields.append((name, ctype))
class Inner(c.Structure, PrettyPrinter):
_fields_ = fields
inner = Inner.from_address(c.addressof(self._buffer))
setattr(self, name, getattr(inner, name))
self = inner
return self
def pack(self):
fields = []
for name, ctype in self._inner_fields:
if type(ctype) == str:
ctype = eval(ctype)
fields.append((name, ctype))
class Inner(c.Structure, PrettyPrinter):
_fields_ = fields
inner = Inner()
for name, ctype in self._inner_fields:
value = getattr(self, name)
if type(value) == list:
l = getattr(inner, name)
for i in range(len(l)):
l[i] = getattr(self, name)[i]
else:
setattr(inner, name, value)
return inner
It looks like it should work, but when I run it I get the error: TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases.
I searched for hints to the solution of this problem, but ctypes Structure looks to be implemented in a c library. I am not sure how to fix this, any help or the specific solution is appreciated!

The problem is that ctypes.Structure uses its own custom metaclass: _ctypes.StructType. Since you inherit the metaclass from Structure, Python does not know which metaclass to use when constructing your class.
You can fix this by inheriting your metaclass from _ctypes.StructType. Since the name of the metaclass is an implementation detail of the ctypes module, I recommend writing type(ctypes.Structure) to get the metaclass dynamically.
import ctypes
class VariableMeta(type(ctypes.Structure)):
pass
The drawback with this approach is that you limit the use of your metaclass. This might be OK if you only plan to use it for subclasses of ctypes.Structure.
Another approach is to create a intermediate metaclass that inherits from both metaclasses.
class PointsMetaClass(type(ctypes.Structure), VariableMeta):
pass
class Points(c.Structure, metaclass=PointsMetaClass):
# ...
Always make sure that you use super() instead of hard-coding type in your metaclass' __new__!
return super(VariableMeta, cls).__new__(cls, name, bases, dct)
As Guido once wrote: Writing metaclasses in Python will cause your head to explode!

Related

How to set `__class__.__name__` as class attribute?

I need to get class name from class:
class Cls:
notation = None
def __init__(self):
notation = self.__class__.__name__
print(Cls.notation) prints None but I need 'Cls'
How to fix it or how to define class attribute which returns a name of class?
You are assigning to a local variable, not the class attribute:
def __init__(self):
Cls.notation = self.__class__.__name__
Note that self.__class__ isn't necessarily Cls, if there is a subclass of Cls involved. You might want to use
def __init__(self):
type(self).notation = self.__class__.__name__
depending on your use case.
Assigning to self.notation won't work, because that creates an instance attribute that shadows the class attribute.
If you want Cls.notation == "Cls" immediately after the class is defined, you may as well just hard-code it:
class Cls:
notation = "Cls"
or
class Cls:
pass
Cls.notation = Cls.__name__
though you can also write
class Cls:
notation = __qualname__
to set its value based on the name used in the first line of the statement, though __qualname__ takes into account nesting as well:
class Cls1:
class Cls2:
notation = __qualname__ # "Cls1.Cls2", not "Cls2"

Python format and validate class variable without instantiating it

I'm trying to validate and format a class variable. The class extends a class with ABCMeta as its __metaclass__ and I can't yet instantiate my child class. So when I run this below code it prints property object and not
the output I want, which I understand why. But then how do I do it?
from abc import ABCMeta, abstractproperty
class RuleBase(object):
__metaclass__ = ABCMeta
id = None
id = abstractproperty(id)
#property
def format_id(self):
try:
_id = 'R%02d' % self.id
except TypeError:
raise TypeError("ID must be an integer")
return _id
#classmethod
def verbose(cls):
print cls.format_id
class Rule(RuleBase):
id = 1
Rule.verbose() # prints property object and not R01
In my theory, I think this would work.
class FormattingABCMeta(ABCMeta):
def __new__(mcl, classname, bases, dictionary):
# Format here, remove format_id property from RuleBase,
# and use FormattingABCMeta as metaclass for RuleBase.
return super(C, mcl).__new__(mcl, classname, bases, dictionary)
But then I feel I'm twisting it too much. So, how to do this? And is my understanding correct? Any help
is appreciated.
Not entirely sure what your goal is but you would need to use fget on the property object passing cls to get the _id:
print cls.format_id.fget(cls)

Augmenting class attribute when inheriting in Python

I have a chain of inheritance in Python, and I want each child class to be able to add on new custom parameters. Right now I'm doing this:
class A(object):
PARAM_NAMES = ['blah1']
...
class B(A):
PARAM_NAMES = A.PARAM_NAMES + ['blah2']
...
I'm wondering if there's a slicker method, though, without referencing A twice? Can't use super() because it's not within a method definition, afaik. I suppose I could use a class method, but that'd be annoying (since I really would want a property).
What's the right way to do this?
of coarse there is always black magic you can do ... but the question is just because you can ... should you?
class MyMeta(type):
items = []
def __new__(meta, name, bases, dct):
return super(MyMeta, meta).__new__(meta, name, bases, dct)
def __init__(cls, name, bases, dct):
MyMeta.items.extend(cls.items)
cls.items = MyMeta.items[:]
super(MyMeta, cls).__init__(name, bases, dct)
class MyKlass(object):
__metaclass__ = MyMeta
class A(MyKlass):
items=["a","b","c"]
class B(A):
items=["1","2","3"]
print A.items
print B.items
since this creates a copy it will not suffer from the same problem as the other solution
(please note that I dont really recommend doing this ... its just to show you can)
This may or may not be smart, but it's technically possible to use a metaclass for this. Unlike Joran's method, I use a property, so that it retains full dynamic nature (that is, if you modify any class's private _PARAM_NAMES list after defining the class, the corresponding PARAM_NAME property of every other derived class reflects that change). For this reason I put an add_param method on the base class.
Python 3 is assumed here, and the PARAM_NAMES property returns a set to avoid duplicate items.
class ParamNameBuilderMeta(type):
def __new__(mcl, name, bases, dct):
names = dct.get("PARAM_NAMES", [])
names = {names} if isinstance(names, str) else set(names)
dct["_PARAM_NAMES"] = names
dct["PARAM_NAMES"] = property(lambda s: type(s).PARAM_NAMES)
return super().__new__(mcl, name, bases, dct)
#property
def PARAM_NAMES(cls):
# collect unique list items ONLY from our classes in the MRO
return set().union(*(c._PARAM_NAMES for c in reversed(cls.__mro__)
if isinstance(c, ParamNameBuilderMeta)))
Usage:
class ParamNameBuilderBase(metaclass=ParamNameBuilderMeta):
#classmethod
def add_param(self, param_name):
self._PARAM_NAMES.add(param_name)
class A(ParamNameBuilderBase):
PARAM_NAMES = 'blah1'
class B(A):
PARAM_NAMES = 'blah1', 'blah2'
class C(B):
pass
Check to make sure it works on both classes and instances thereof:
assert C.PARAM_NAMES == {'blah1', 'blah2'}
assert C().PARAM_NAMES == {'blah1', 'blah2'}
Check to make sure it's still dynamic:
C.add_param('blah3')
assert C.PARAM_NAMES == {'blah1', 'blah2', 'blah3'}
The behavior you've described is actually quite specific. You've said that you
want each child class to be able to add on new custom paramters
But the way you've implemented it, this will result in unpredictable behaviour. Consider:
class A(object):
PARAM_NAMES = ['blah1']
class B(A):
PARAM_NAMES = A.PARAM_NAMES + ['blah2']
class C(A):pass
print(A.PARAM_NAMES)
print(B.PARAM_NAMES)
print(C.PARAM_NAMES)
A.PARAM_NAMES.append('oops')
print(C.PARAM_NAMES)
What we notice is that the classes that choose to add new parameters have a new reference to the parameter list, while ones that do not add new parameters have the same reference as their parent. Unless carefully controlled, this is unsafe behaviour.
It is more reliable to only use constants as class properties, or to redefine the list entirely each time (make it a tuple), which is not "slicker". Otherwise, I'd reccomend class methods, as you suggest, and making the property an instance variable

Python: class static member pointing to itself? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Can I get a reference to the 'owner' class during the init method of a descriptor?
Code is worth a thousand words:
>>> class ShortRib(object):
>>> def __init__(self, owner):
>>> self.owner = owner
>>>
>>> ... some more methods and stuff ...
>>>
>>>
>>> class Cow(object):
>>> shortRib = ShortRib(self)
>>>
>>>
>>> class BrownCow(Cow):
>>> pass
>>>
>>> BrownCow.shortRib.owner
<class '__main__.BrownCow'>
This doesn't work, though i wish it would. Basically, I want each class to have some static/class variables (i'm not sure which it is in this case?) but need each of those guys to know who (which class) it belongs to. Unfortunately, I can't "get" at the class in the body of the class declaration. Of course, I could always do this using a decorator:
>>> def vars(**kwargs):
>>> def wrap(cls):
>>> for k, w in kwargs.items():
>>> setattr(cls, k, w(cls))
>>> return cls
>>> return wrap
>>>
>>> #vars(shortRib=lambda cls: ShortRib(cls)
>>> class BrownCow(Cow):
>>> ...
>>>
>>> BrownCow.shortRib.owner
which would work. Another way would to have a class decorator that goes through all the shortRibs and similar static variables and sets their owner after the class declaration is complete. However, this seems like an incredibly roundabout and unintuitive way of doing what should be a pretty simple operation: having the static/class members of a class know who they belong to.
Is there a "proper" way of doing this?
Clarification:
I want these members to belong to the class, not to the instances. I'm trying to go for a almost-purely-functional style, using classes only for inheritance of shared behavior, and not creating instances of them at all. Instances would tend to give my functions access to arbitrary instance data shared across all functions, which would break the pure-functioness I am trying for. I could just use empty instances which I don't touch, but I think using pure classes would be cleaner.
You can easily do this in __new__:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class Cow(object):
shortRib = None
def __new__(cls, *args, **kwargs):
if cls.shortRib == None:
cls.shortRib = ShortRib(cls)
return super(Cow, cls).__new__(cls, *args, **kwargs)
Cow()
Cow.shortRib.owner
Or even __init__, if you don't mind referencing self.__class___.
You can also do it with a metaclass:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class MetaCow(type):
def __new__(cls, name, base, attrs):
attrs['shortRib'] = ShortRib(cls)
return super(MetaCow, cls).__new__(cls, name, base, attrs)
class Cow(object):
__metaclass__ = MetaCow
Cow.shortRib.owner
Why not let the instances of the Cow class have shortRibs, instead of the class itself?:
class ShortRib(object):
def __init__(self,owner):
self.owner=owner
class Cow(object):
def __init__(self):
self.shortRib=ShortRib(self)
class BrownCow(Cow):
pass
print(BrownCow().shortRib.owner)
# <__main__.BrownCow object at 0xb76a8d6c>
(Otherwise, you'll need a class decorator or metaclass -- as you've already mentioned. But simple is better than complex, so why not choose simple?)
By the way, if you really do want to use classes instead of instances:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class MetaCow(type):
def __init__(cls, name, base, attrs):
super(MetaCow, cls).__init__(name, base, attrs)
cls.shortRib = ShortRib(cls)
class Cow(object):
__metaclass__ = MetaCow
class BrownCow(Cow):
pass
print(Cow.shortRib.owner)
# <class '__main__.Cow'>
print(BrownCow.shortRib.owner)
# <class '__main__.BrownCow'>
Using
class MetaCow(type):
def __new__(cls, name, base, attrs):
is incorrect. The signature for type.__new__ is
class MetaCow(type):
def __new__(meta, name, base, attrs):
Since you want to modify the attributes of cls, not meta, use the MetaCow.__init__ not MetaCow__new__.
Two methods to to do what you want:
You can override the __getattr__ method in any class to return anything you desire when you ask for the value of an attribute.
You can use a property, which has a getter that returns the object you want it to return.
Both __getattr__ methods and properties are inherited.

Finding the static attributes of a class in Python

This is an unusual question, but I'd like to dynamically generate the __slots__ attribute of the class based on whatever attributes I happened to have added to the class.
For example, if I have a class:
class A(object):
one = 1
two = 2
__slots__ = ['one', 'two']
I'd like to do this dynamically rather than specifying the arguments by hand, how would I do this?
At the point you're trying to define slots, the class hasn't been built yet, so you cannot define it dynamically from within the A class.
To get the behaviour you want, use a metaclass to introspect the definition of A and add a slots attribute.
class MakeSlots(type):
def __new__(cls, name, bases, attrs):
attrs['__slots__'] = attrs.keys()
return super(MakeSlots, cls).__new__(cls, name, bases, attrs)
class A(object):
one = 1
two = 2
__metaclass__ = MakeSlots
One very important thing to be aware of -- if those attributes stay in the class, the __slots__ generation will be useless... okay, maybe not useless -- it will make the class attributes read-only; probably not what you want.
The easy way is to say, "Okay, I'll initialize them to None, then let them disappear." Excellent! Here's one way to do that:
class B(object):
three = None
four = None
temp = vars() # get the local namespace as a dict()
__slots__ = temp.keys() # put their names into __slots__
__slots__.remove('temp') # remove non-__slots__ names
__slots__.remove('__module__') # now remove the names from the local
for name in __slots__: # namespace so we don't get read-only
del temp[name] # class attributes
del temp # and get rid of temp
If you want to keep those initial values it takes a bit more work... here's one possible solution:
class B(object):
three = 3
four = 4
def __init__(self):
for key, value in self.__init__.defaults.items():
setattr(self, key, value)
temp = vars()
__slots__ = temp.keys()
__slots__.remove('temp')
__slots__.remove('__module__')
__slots__.remove('__init__')
__init__.defaults = dict()
for name in __slots__:
__init__.defaults[name] = temp[name]
del temp[name]
del temp
As you can see, it is possible to do this without a metaclass -- but who wants all that boilerplate? A metaclass could definitely help us clean this up:
class MakeSlots(type):
def __new__(cls, name, bases, attrs):
new_attrs = {}
new_attrs['__slots__'] = slots = attrs.keys()
slots.remove('__module__')
slots.remove('__metaclass__')
new_attrs['__weakref__'] = None
new_attrs['__init__'] = init = new_init
init.defaults = dict()
for name in slots:
init.defaults[name] = attrs[name]
return super(MakeSlots, cls).__new__(cls, name, bases, new_attrs)
def new_init(self):
for key, value in self.__init__.defaults.items():
setattr(self, key, value)
class A(object):
__metaclass__ = MakeSlots
one = 1
two = 2
class B(object):
__metaclass__ = MakeSlots
three = 3
four = 4
Now all the tediousness is kept in the metaclass, and the actual class is easy to read and (hopefully!) understand.
If you need to have anything else in these classes besides attributes I strongly suggest you put whatever it is in a mixin class -- having them directly in the final class would complicate the metaclass even more.

Categories