We Keep Coding

How to define a class with dynamic attributes?

In my project, I need to create a class with attributes passed by a dict, something like this:
class_attributes = {"sensor": Nested(Sensor),
"serial_interface": Nested(SerialInterface)}
class MSchema(marshmallow.ModelSchema):
class Meta:
model = cls
attr = class_attributes
I need that "sensor" and "serial_interface" to be in the class, and can be access using MSchema.sensor or MSchema.serial_interface.

You can call the metaclass of ModelSchema directly, rather than defining the class declaratively using a class statement.
m = marshmallow.ModelSchema
class_attributes = {
"sensor": Nested(Sensor),
"serial_interface": Nested(SerialInterface)
}
m = marshmallow.ModelSchema
mc = type(m)
MSchema = mc('MSchema', (m,), {
'Meta': type('Meta', (), {'model': cls}),
**class_attributes
})
In case you aren't aware, a class statement is just a declarative syntax for calling type (or some other metaclass) with 3 arguments: the name of the class, a tuple of parent classes, and a dict of class attributes. The class statement evaluates its body to produce the dict, then calls type (or another given metaclass), and binds the return value to the name. Some simpler examples:
# Foo = type('Foo', (), {})
class Foo:
pass
# Foo = Bar('Foo', (), {})
class Foo(metaclass=Bar):
pass
# Foo = Bar('Foo', (Parent,), {'x': 3})
class Foo(Parent, metaclass=Bar):
x = 3
# def foo_init(self, x):
# self.x = x
# Foo = Bar('Foo', (), {'__init__': foo_init})
class Foo(metaclass=Bar):
def __init__(self, x):
self.x = x

Not entirely sure I understand the question to 100%, but have you tried using setattr()?
Example code would look like the following:
m_schema = MSchema()
for key, value in class_attributes.items():
setattr(m_schema, key, value)
setattr(object, string, value) takes an object to set attributes on, a string for the attribute name, and an arbitrary value as the attribute value.

How to create Python Enum class from existing dict with additional methods?

Let's say, I have a pre-existing mapping as a dictionary:
value_map = {'a': 1, 'b': 2}
I can create an enum class from this like so:
from enum import Enum
MyEnum = Enum('MyEnum', value_map)
and use it like so
a = MyEnum.a
print(a.value)
>>> 1
print(a.name)
>>> 'a'
But then I want to define some methods to my new enum class:
def double_value(self):
return self.value * 2
Of course, i can do this:
class MyEnum(Enum):
a = 1
b = 2
#property
def double_value(self):
return self.value * 2
But as I said, I have to use a pre-defined value mapping dictionary, so I cannot do this.
How can this be achieved? I tried to inherit from another class defining this method like a mixin, but I could'nt figure it out.

You can pass in a base type with mixin methods into the functional API, with the type argument:
>>> import enum
>>> value_map = {'a': 1, 'b': 2}
>>> class DoubledEnum:
... #property
... def double_value(self):
... return self.value * 2
...
>>> MyEnum = enum.Enum('MyEnum', value_map, type=DoubledEnum)
>>> MyEnum.a.double_value
2
For a fully functional approach that never uses a class statement, you can create the base mix-in with the type() function:
DoubledEnum = type('DoubledEnum', (), {'double_value': property(double_value)})
MyEnum = enum.Enum('MyEnum', value_map, type=DoubledEnum)
You can also use enum.EnumMeta() metaclass the same way, the way Python would when you create a class MyEnum(enum.Enum): ... subclass:
Create a class dictionary using the metaclass __prepare__ hook
Call the metaclass, passing in the class name, the bases ((enum.Enum,) here), and the class dictionary created in step 1.
The custom dictionary subclass that enum.EnumMeta uses isn't really designed for easy reuse; it implements a __setitem__ hook to record metadata, but doesn't override the dict.update() method, so we need to use a little care when using your value_map dictionary:
import enum
def enum_with_extras(name, value_map, bases=enum.Enum, **extras):
if not isinstance(bases, tuple):
bases = bases,
if not any(issubclass(b, enum.Enum) for b in bases):
bases += enum.Enum,
classdict = enum.EnumMeta.__prepare__(name, bases)
for key, value in {**value_map, **extras}.items():
classdict[key] = value
return enum.EnumMeta(name, bases, classdict)
Then pass in double_value=property(double_value) to that function (together with the enum name and value_map dictionary):
>>> def double_value(self):
... return self.value * 2
...
>>> MyEnum = enum_with_extras('MyEnum', value_map, double_value=property(double_value))
>>> MyEnum.a
<MyEnum.a: 1>
>>> MyEnum.a.double_value
2
You are otherwise allowed to create subclasses of an enum without members (anything that's a descriptor is not a member, so functions, properties, classmethods, etc.), so you can define an enum without members first:
class DoubledEnum(enum.Enum):
#property
def double_value(self):
return self.value * 2
which is an acceptable base class for both in the functional API (e.g. enum.Enum(..., type=DoubledEnum)) and for the metaclass approach I encoded as enum_with_extras().

You can create a new meta class (Either using a meta-metaclass or a factory function, like I do below) that derives from enum.EnumMeta (The metaclass for enums) and just adds the members before creating the class
import enum
import collections.abc
def enum_metaclass_with_default(default_members):
"""Creates an Enum metaclass where `default_members` are added"""
if not isinstance(default_members, collections.abc.Mapping):
default_members = enum.Enum('', default_members).__members__
default_members = dict(default_members)
class EnumMetaWithDefaults(enum.EnumMeta):
def __new__(mcs, name, bases, classdict):
"""Updates classdict adding the default members and
creates a new Enum class with these members
"""
# Update the classdict with default_members
# if they don't already exist
for k, v in default_members.items():
if k not in classdict:
classdict[k] = v
# Add `enum.Enum` as a base class
# Can't use `enum.Enum` in `bases`, because
# that uses `==` instead of `is`
bases = tuple(bases)
for base in bases:
if base is enum.Enum:
break
else:
bases = (enum.Enum,) + bases
return super(EnumMetaWithDefaults, mcs).__new__(mcs, name, bases, classdict)
return EnumMetaWithDefaults
value_map = {'a': 1, 'b': 2}
class MyEnum(metaclass=enum_metaclass_with_default(value_map)):
#property
def double_value(self):
return self.value * 2
assert MyEnum.a.double_value == 2
A different solution was to directly try and update locals(), as it is replaced with a mapping that creates enum values when you try to assign values.
import enum
value_map = {'a': 1, 'b': 2}
def set_enum_values(locals, value_map):
# Note that we can't use `locals.update(value_map)`
# because it's `locals.__setitem__(k, v)` that
# creates the enum value, and `update` doesn't
# call `__setitem__`.
for k, v in value_map:
locals[k] = v
class MyEnum(enum.Enum):
set_enum_values(locals(), value_map)
#property
def double_value(self):
return self.value * 2
assert MyEnum.a.double_value == 2
This seems well defined enough, and a = 1 is most likely going to be the same as locals()['a'] = 1, but it might change in the future. The first solution is more robust and less hacky (And I haven't tested it in other Python implementations, but it probably works the same)

PLUS: Adding more stuff (a dirt hack) to #Artyer's answer. 🤗
Note that you can also provide "additional" capabilities to an Enum if you create it from a dict, see...
from enum import Enum
_colors = {"RED": (1, "It's the color of blood."), "BLUE": (2, "It's the color of the sky.")}
def _set_members_colors(locals: dict):
for k, v in colors.items():
locals[k] = v[0]
class Colors(int, Enum):
_set_members_colors(locals())
#property
def description(self):
return colors[self.name][1]
print(str(Colors.RED))
print(str(Colors.RED.value))
print(str(Colors.RED.description))
Output...
Colors.RED
1
It's the color of blood.
Thanks! 😉

How to inherit a class attribute that is a dict and make it unique?

This works as I expected;
class Foo(object):
name = ''
class Bar1(Foo):
pass
class Bar2(Foo):
pass
Bar1.name == Bar2.name # returns True
Bar1.name = 'Bar1'
Bar1.name == Bar2.name # returns False, which is what I want.
This doesn't work the same, but I want it to;
class Foo(object):
fields = {'name':''}
class Bar1(Foo):
pass
class Bar2(Foo):
pass
Bar1.fields['name'] == Bar2.fields['name'] # returns True
Bar1.fields['name'] = 'Bar1'
Bar1.fields['name'] == Bar2.fields['name'] # returns True, which isn't what I want.
It seems the subclasses are still pointing at the same dict object specified in the main class, but I want them to have unique dicts. How can I do this without writing fields = {'name':''} in each of the subclasses?
PS- I definitely do want to use class level attributes, not instance attributes, as all of the instances I create will use this 'shared' dict.

The simplest way to do this is using a meta-class (I've assumed Python 2.x syntax):
class FieldsMeta(type):
def __new__(mcs, name, bases, dict):
"""This controls the creation of each new class."""
dict.update(fields={'name': ''})
return type.__new__(mcs, name, bases, dict)
class Parent(object):
__metaclass__ = FieldsMeta
class Child(Parent):
pass
In use:
>>> Parent.fields
{'name': ''}
>>> Child.fields
{'name': ''}
>>> Child.fields is Parent.fields
False
>>> Child.fields['name'] = 'Child'
>>> Child.fields
{'name': 'Child'}
>>> Parent.fields
{'name': ''}
See e.g. the data-model documentation for __new__:
[__new__] is also commonly overridden in custom metaclasses in order
to customize class creation.
and the section on customizing class creation.

You can create fields setter, which will copy dict before assign new value
class Foo(object):
fields = {'name':''}
def set_field(self, field, value):
self.fields = dict(self.fields)
self.fields[field] = value
class Bar1(Foo):
pass
class Bar2(Foo):
pass
Bar1.fields['name'] == Bar2.fields['name'] # returns True
# Bar1.fields['name'] = 'Bar1'
Bar1.set_field(Bar1, 'name', 'Bar1')
print(Bar1.fields['name'] == Bar2.fields['name']) # returns False

Defining __getattr__ and __setattr__ for Python class variables [duplicate]

Currently __setattr__ only works for instance. Is there any similar method for class? I am asking this question because I want to collect the list of defined attribute in order when user define it in class as below:
class CfgObj(object):
_fields = []
def __setattr__(self, name, value):
self._fields.append([name, value])
object.__setattr__(self, name, value)
class ACfg(CfgObj):
setting1 = Field(str, default='set1', desc='setting1 ...')
setting2 = Field(int, default=5, desc='setting2...')
I know the above code will not work as expected because the __setattr__ only called by instance as below:
acfg = ACfg()
acfg.c = 1
acfg._fields == [['c', 1]]
So, is there any equivalent __setattr__ for python class? The main purpose is to collect the define attribute in order when user define it in class.

Yes, but that's not how you want to do it.
class MC(type):
def __init__(cls, name, bases, dct):
print dct
super(MC, cls).__init__(name, bases, dct)
class C(object):
__metaclass__ = MC
foo = 42

If you define __setattr__() on the metaclass of a class, it will be called when setting attributes on the class, but only after creating the class:
>>> class Meta(type):
... def __setattr__(cls, name, value):
... print "%s=%r" % (name, value)
...
>>> class A(object):
... __metaclass__ = Meta
...
>>> A.a = 1
a=1
But it won't work at the time of class definition, so it's probably not what you want.
Getting the class attributes in the metaclass __init__() works, but you loose the order of definition (and multiple definitions as well).

What I would do to solve your problem - but not your question - is to set the timestamp of the field creation create a counter of Field objects and set the current value of the counter to the created one:
class Field(object):
count = 0
def __init__(self, value, default=None, desc=None):
self.value = value
self.default = default
self.desc = desc
# Here comes the magic
self.nth = Field.count
Field.count += 1
# self.created_at = time.time()
Then I would create a method for returning all fields ordered by its counter value:
class CfgObj(object):
def params(self):
ns = dir(self)
fs = [getattr(self, field)
for field in ns
if isinstance(getattr(self, field), Field)]
# fs = sorted(fs, key=lambda f: f.created_at)
fs = sorted(fs, key=lambda f: f.nth)
return fs
Its usage is intuitive:
class ACfg(CfgObj):
setting1 = Field(str, default='set1', desc='setting1 ...')
setting2 = Field(int, default=5, desc='setting2...')
print ACfg().params()
Clearly the fields are ordered by time of object creation, not field creation, but it can be enough for you. Is it?

Finding the static attributes of a class in Python

This is an unusual question, but I'd like to dynamically generate the __slots__ attribute of the class based on whatever attributes I happened to have added to the class.
For example, if I have a class:
class A(object):
one = 1
two = 2
__slots__ = ['one', 'two']
I'd like to do this dynamically rather than specifying the arguments by hand, how would I do this?

At the point you're trying to define slots, the class hasn't been built yet, so you cannot define it dynamically from within the A class.
To get the behaviour you want, use a metaclass to introspect the definition of A and add a slots attribute.
class MakeSlots(type):
def __new__(cls, name, bases, attrs):
attrs['__slots__'] = attrs.keys()
return super(MakeSlots, cls).__new__(cls, name, bases, attrs)
class A(object):
one = 1
two = 2
__metaclass__ = MakeSlots

One very important thing to be aware of -- if those attributes stay in the class, the __slots__ generation will be useless... okay, maybe not useless -- it will make the class attributes read-only; probably not what you want.
The easy way is to say, "Okay, I'll initialize them to None, then let them disappear." Excellent! Here's one way to do that:
class B(object):
three = None
four = None
temp = vars() # get the local namespace as a dict()
__slots__ = temp.keys() # put their names into __slots__
__slots__.remove('temp') # remove non-__slots__ names
__slots__.remove('__module__') # now remove the names from the local
for name in __slots__: # namespace so we don't get read-only
del temp[name] # class attributes
del temp # and get rid of temp
If you want to keep those initial values it takes a bit more work... here's one possible solution:
class B(object):
three = 3
four = 4
def __init__(self):
for key, value in self.__init__.defaults.items():
setattr(self, key, value)
temp = vars()
__slots__ = temp.keys()
__slots__.remove('temp')
__slots__.remove('__module__')
__slots__.remove('__init__')
__init__.defaults = dict()
for name in __slots__:
__init__.defaults[name] = temp[name]
del temp[name]
del temp
As you can see, it is possible to do this without a metaclass -- but who wants all that boilerplate? A metaclass could definitely help us clean this up:
class MakeSlots(type):
def __new__(cls, name, bases, attrs):
new_attrs = {}
new_attrs['__slots__'] = slots = attrs.keys()
slots.remove('__module__')
slots.remove('__metaclass__')
new_attrs['__weakref__'] = None
new_attrs['__init__'] = init = new_init
init.defaults = dict()
for name in slots:
init.defaults[name] = attrs[name]
return super(MakeSlots, cls).__new__(cls, name, bases, new_attrs)
def new_init(self):
for key, value in self.__init__.defaults.items():
setattr(self, key, value)
class A(object):
__metaclass__ = MakeSlots
one = 1
two = 2
class B(object):
__metaclass__ = MakeSlots
three = 3
four = 4
Now all the tediousness is kept in the metaclass, and the actual class is easy to read and (hopefully!) understand.
If you need to have anything else in these classes besides attributes I strongly suggest you put whatever it is in a mixin class -- having them directly in the final class would complicate the metaclass even more.