We Keep Coding

Is it possible in Pydantic to dynamically change representation of object?

I'm interested if there is any way to set an attribute on the Pydantic Model class the will only be used when constructing the output representation of the object.
Something that works like Django Rest Framework SerializerMethodField. It is highly useful when you need to construct output depending on the environment or many other conditions.

It's possible to have a similar mechanism as django's SerializerMethodField by overriding pydantic BaseModel class
from copy import deepcopy
from pydantic import BaseModel, Field
class Person:
def __init__(self, name, likes_cake):
self.name = name
self.likes_cake = likes_cake
class CustomPydanticModel(BaseModel):
#classmethod
def from_orm(cls, obj, getter_binding=None):
getter_binding = getter_binding or {}
obj = deepcopy(obj)
for field in cls.__fields__:
method = getter_binding.get(field)
if method is None:
method = getattr(cls, f"get_{field}", None)
if method is not None and callable(method):
setattr(obj, field, method(obj))
return super().from_orm(obj)
class PersonModel(CustomPydanticModel):
name: str
status: str | None = None
#staticmethod
def get_status(obj):
return "I like cake" if obj.likes_cake else "Hello"
class Config:
orm_mode = True
obj = Person("Patrick", True)
pydantic_obj = PersonModel.from_orm(obj)
Note that the get_status method should be static or classmethod

python - error happens when using super()

from flask_sqlalchemy import SQLAlchemy
import datetime
db = SQLAlchemy()
class BaseModel(db.Model):
"""Base data model for all objects"""
__abstract__ = True
def __init__(self, *args):
super(self).__init__(*args)
def __repr__(self):
"""Define a base way to print models"""
return '%s(%s)' % (self.__class__.__name__, {
column: value
for column, value in self._to_dict().items()
})
def json(self):
"""Define a base way to jsonify models, dealing with datetime objects"""
return {
column: value if not isinstance(value, datetime.date) else value.strftime('%Y-%m-%d')
for column, value in self._to_dict().items()
}
class Station(BaseModel, db.Model):
"""Model for the stations table"""
__tablename__ = 'stations'
id = db.Column(db.Integer, primary_key = True)
lat = db.Column(db.Float)
lng = db.Column(db.Float)
TypeError: super() argument 1 must be type, not Station
I know super with no argument is only used in python3. but in my case what should i fill in the super()? Also is it okay to put the super() in init of the father class(basemodel)?

Since you're not doing anything in BaseModel.__init__, the correct approach is to not implement it at all. Without __init__ defined in BaseModel, you'll go to the super class's __init__ automatically, and more efficiently.
That said, if you are doing something meaningful in BaseModel.__init__, you can't use one-argument super like that. One-argument super is of very limited use (basically, only for classmethods, where you're passing the type but not an instance of the type as the argument). You need two-argument super, explicitly providing the name of the current class you're trying to bypass looking for superclasses, then self, e.g:
class BaseModel(db.Model):
"""Base data model for all objects"""
__abstract__ = True
def __init__(self, *args):
super(BaseModel, self).__init__(*args)

How to write to an abstract property in Python 3.4+

In Python 3.6, Let's say I have an abstract class MyAbstractClass
from abc import ABC, abstractmethod
class MyAbstractClass(ABC):
#property
#abstractmethod
def myProperty(self):
pass
and a class MyInstantiatableClass inherit from it. So how do I write to the property myProperty on instantiation of an object from this class? I'd like to be able to both set and get myProperty. Below doesn't work.
from MyAbstractClass import MyAbstractClass
class MyInstantiatableClass(MyAbstractClass):
def __init__(self, desiredValueOfMyProperty):
????
#myProperty.setter
def myProperty(self, desiredValueOfMyProperty): # value coming from __init__
self._myProperty = desiredValueOfMyProperty
And a main function, say,
from MyInstantiatableClass import MyInstantiatableClass
def main():
MyInstantiatableClass(3) # 3 is the desiredValueOfMyProperty for this instantiation
MyInstantiatableClass(5) # 5 is the desiredValueOfMyProperty for this instantiation

It seems there's a discrepancy here; using #property along with #abstractmethod doesn't seem to enforce classes that inherit from your abc to need to define both setter and getter. Using this:
#property
#abstractmethod
def myProperty(self):
pass
#myProperty.setter
#abstractmethod
def myProperty(self):
pass
and then providing an implementation only for the getter in the class works and allows for instantiation:
#property
def myProperty(self):
return self._myProperty
This is due to the fact that only one name (myProperty) appears in the namespace of the ABC, when you override in the base class, you only need to define this one name.
There's a way around that enforces it. You can create separate abstract methods and pass them on to property directly:
class MyAbstractClass(ABC):
#abstractmethod
def getProperty(self):
pass
#abstractmethod
def setProperty(self, val):
pass
myAbstractProperty = property(getProperty, setProperty)
Providing an implementation for this abc now requires both getter and setter to have an implementation (both names that have been listed as abstractmethods in MyAbstractClass namespace need to have an implementation):
class MyInstantiatableClass(MyAbstractClass):
def getProperty(self):
return self._Property
def setProperty(self, val):
self._Property = val
myAbstractProperty = property(getProperty, setProperty)
Implementing them is exactly the same as any old property. There's no difference there.

For example, you can define the abstract getter, setter and deleter in Person abstract class, override them in Student class which extends Person abstract class as shown below. *#abstractmethod must be the innermost decorator otherwise error occurs:
from abc import ABC, abstractmethod
class Person(ABC):
#property
#abstractmethod # The innermost decorator
def name(self): # Abstract getter
pass
#name.setter
#abstractmethod # The innermost decorator
def name(self, name): # Abstract setter
pass
#name.deleter
#abstractmethod # The innermost decorator
def name(self): # Abstract deleter
pass
class Student(Person):
def __init__(self, name):
self._name = name
#property
def name(self): # Overrides abstract getter
return self._name
#name.setter
def name(self, name): # Overrides abstract setter
self._name = name
#name.deleter
def name(self): # Overrides abstract deleter
del self._name
Then, you can instantiate Student class and call the getter, setter and deleter as shown below:
obj = Student("John") # Instantiates "Student" class
print(obj.name) # Getter
obj.name = "Tom" # Setter
print(obj.name) # Getter
del obj.name # Deleter
print(hasattr(obj, "name"))
Output:
John
Tom
False
You can see my answer which explains more about abstract property.

Extract assigned variable name

See the update below
I even don't know how to make a short title for my problem.
In a class I have some class attributes of StringField class:
class Authors(Table):
# id field is already present
first_name = StringField(maxLength=100)
last_name = StringField(maxLength=100)
StringField constructor may receive an argument called name. If it's not given, i want it to be equal to class attribute's name (first_name, last_name in the example above).
Is it possible to extract the name of the variable the created instance is going to be assigned to?
I guess i have to use inspect module?
I see Django does this:
Each field type, except for ForeignKey, ManyToManyField and
OneToOneField, takes an optional first positional argument -- a
verbose name. If the verbose name isn't given, Django will
automatically create it using the field's attribute name, converting
underscores to spaces.
In this example, the verbose name is "person's first name":
first_name = models.CharField("person's first name", max_length=30)
In this example, the verbose name is "first name":
first_name = models.CharField(max_length=30)
But i don't find in Django 1.3.1 source code the part which is doing what i need.
UPDATE:
To simplify:
class Field():
def __init__(self, field_name=None):
if not field_name:
field_name = ??? # some magic here to determine the name
print(field_name)
class Table():
first_name = Field()
last_name = Field()
Running this should print first_name and last_name
SOLUTION:
class Field():
def __init__(self, name=None):
self._name = name
class Table():
first_name = Field()
last_name = Field()
for attrName, attr in Table.__dict__.items():
if isinstance(attr, Field):
if attr._name is None:
attr._name = attrName
print(Table.first_name._name)
print(Table.last_name._name)

I don't know how Django does it. But you could do it this way:
class WantFixup(object):
def new_instance(self, name, derived_name):
cls = type(self)
if name is None:
name = derived_name.replace('_', ' ')
return cls(name)
class Container(WantFixup):
def __init__(self, name=None):
self.name = name
def __repr__(self):
return "Container('%s')" % str(self.name)
class WillFixup(object):
def __init__(self):
cls = type(self)
for name in cls.__dict__:
o = cls.__dict__[name] # look up object from name
if not isinstance(o, WantFixup):
continue
print("calling %s.new_instance('%s', '%s')" % (o, o.name, name))
self.__dict__[name] = o.new_instance(o.name, name)
class Name(WillFixup):
first_name = Container("given name")
last_name = Container()
Here is an example of the above code in action:
>>> import auto_name
>>> n = auto_name.Name()
calling Container('None').new_instance('None', 'last_name')
calling Container('given name').new_instance('given name', 'first_name')
>>> print(n.__dict__)
{'first_name': Container('given name'), 'last_name': Container('last name')}
>>> print(auto_name.Name.__dict__)
{'__module__': 'auto_name', 'last_name': Container('None'), 'first_name': Container('given name'), '__doc__': None}
>>>
The class WantFixup serves two purposes. First, all classes that inherit from it can be detected using isinstance(); if our object instance is named o, we can test it like isinstance(o, WantFixup). Second, it provided the .new_instance() method function to any class that inherits from it.
The class Container is an example of a container that might need fixup. Note that it inherits from WantFixup.
The class WillFixup contains a .__init__() method that performs fixup on all classes that inherit from it. This simply loops over everything in the class dictionary, and calls the .new_instance() method function for each one, passing in the name.
Finally, class Name inherits from WillFixup and contains two instances of Container. Because it inherits from WillFixup, the method WillFixup.__init__() is called. As you can see from the example, first_name has a .name attribute set to 'given name' but last_name wasn't set, so it is patched to have its .name attribute set to 'last name'.
The .__init__() function is supposed to set up the new class instance. As long as all the special WantFixup class instances are in the parent class, the .__init__() method will automatically loop over them and set them up.
The confusing part here is that the instance has first_name set to an instance of Container that has the name patched, and will actually be used to store stuff. But the class Name contains an instance of Container that is just used to store the name of the class, and as a marker for the .__init__() method to find.
The good part is that the magic is hidden away in the base classes. The Container and Name classes just need to inherit from them, but are not themselves cluttered with stuff.
There might be a slicker way to solve the problem using metaprogramming.
http://www.ibm.com/developerworks/linux/library/l-pymeta/index.html
This solution isn't metaclass programming, but it is tested, working code.
EDIT: This is a changed version of the code. The original code was intended to show the general idea, but didn't actually init the Name object. It's not hard to actually do the init, so I changed it.

In order for the magic to happen as in the sample, Python would need to be a context-sensitive language (which is isn't, as far as I know, which isn't that far). Django uses the ModelBase meta-class to (among other tasks) set verbose names for the fields. Basically, the metaclass's __new__ loops over the class attributes to get the attribute names, adding them to the options. You can be a little more direct and alter the fields directly. Here's a Python 2 example:
class Field(object):
def __init__(self, name=None):
self.name = name
def __str__(self):
if self.name:
return self.name
return type(self).__name__
def __repr__(self):
return '%s(%s)' % (type(self).__name__, repr(self.name))
class MetaContainer(type):
#classmethod
def dub(metacls, name):
return name.replace('_', ' ').capitalize()
def __new__(cls, name, bases, attrs):
for attr in attrs:
if issubclass(type(attrs[attr]), Field) and attrs[attr].name is None:
attrs[attr].name = MetaContainer.dub(attr)
return super(MetaContainer, cls).__new__(cls, name, bases, attrs)
class Container(object):
__metaclass__ = MetaContainer
first_name = Field()
foo = Field('Foobar')
cntr = Container()
cntr.first_name
Python 3 is almost the same, but you use the metaclass class argument* rather than the __metaclass__ property:
class Container(object, metaclass=MetaContainer):
first_name = Field()
foo = Field('Foobar')
You can write a version that works with metaclasses in in Python 2 and 3 by creating an intermediate base class for the container using the metaclass directly, rather than the metaclass argument or __metaclass__ property:
ContainerBase = MetaContainer('ContainerBase', (object,), {})
class Container(ContainerBase):
first_name = Field()
foo = Field('Foobar')
* For the reason for the change, see PEP 3115: Metaclasses in Python 3000.

How to create abstract properties in python abstract classes?

In the following code, I create a base abstract class Base. I want all the classes that inherit from Base to provide the name property, so I made this property an #abstractmethod.
Then I created a subclass of Base, called Base_1, which is meant to supply some functionality, but still remain abstract. There is no name property in Base_1, but nevertheless python instatinates an object of that class without an error. How does one create abstract properties?
from abc import ABCMeta, abstractmethod
class Base(object):
# class Base(metaclass = ABCMeta): <- Python 3
__metaclass__ = ABCMeta
def __init__(self, str_dir_config):
self.str_dir_config = str_dir_config
#abstractmethod
def _do_stuff(self, signals):
pass
#property
#abstractmethod
def name(self):
"""This property will be supplied by the inheriting classes
individually.
"""
pass
class Base1(Base):
__metaclass__ = ABCMeta
"""This class does not provide the name property and should
raise an error.
"""
def __init__(self, str_dir_config):
super(Base1, self).__init__(str_dir_config)
# super().__init__(str_dir_config) <- Python 3
def _do_stuff(self, signals):
print "Base_1 does stuff"
# print("Base_1 does stuff") <- Python 3
class C(Base1):
#property
def name(self):
return "class C"
if __name__ == "__main__":
b1 = Base1("abc")

Since Python 3.3 a bug was fixed meaning the property() decorator is now correctly identified as abstract when applied to an abstract method.
Note: Order matters, you have to use #property above #abstractmethod
Python 3.3+: (python docs):
from abc import ABC, abstractmethod
class C(ABC):
#property
#abstractmethod
def my_abstract_property(self):
...
Python 2: (python docs)
from abc import ABC, abstractproperty
class C(ABC):
#abstractproperty
def my_abstract_property(self):
...

Until Python 3.3, you cannot nest #abstractmethod and #property.
Use #abstractproperty to create abstract properties (docs).
from abc import ABCMeta, abstractmethod, abstractproperty
class Base(object):
# ...
#abstractproperty
def name(self):
pass
The code now raises the correct exception:
Traceback (most recent call last):
File "foo.py", line 36, in
b1 = Base_1('abc')
TypeError: Can't instantiate abstract class Base_1 with abstract methods name

Based on James answer above
def compatibleabstractproperty(func):
if sys.version_info > (3, 3):
return property(abstractmethod(func))
else:
return abstractproperty(func)
and use it as a decorator
#compatibleabstractproperty
def env(self):
raise NotImplementedError()

In python 3.6+, you can also anotate a variable without providing a default. I find this to be a more concise way to make it abstract.
class Base():
name: str
def print_name(self):
print(self.name) # will raise an Attribute error at runtime if `name` isn't defined in subclass
class Base_1(Base):
name = "base one"
it may also be used to force you to initialize the variable in the __new__ or __init__ methods
As another example, the following code will fail when you try to initialize the Base_1 class
class Base():
name: str
def __init__(self):
self.print_name()
class Base_1(Base):
_nemo = "base one"
b = Base_1()
AttributeError: 'Base_1' object has no attribute 'name'

Using the #property decorator in the abstract class (as recommended in the answer by James) works if you want the required instance level attributes to use the property decorator as well.
If you don't want to use the property decorator, you can use super(). I ended up using something like the __post_init__() from dataclasses and it gets the desired functionality for instance level attributes:
import abc
from typing import List
class Abstract(abc.ABC):
"""An ABC with required attributes.
Attributes:
attr0
attr1
"""
#abc.abstractmethod
def __init__(self):
"""Forces you to implement __init__ in 'Concrete'.
Make sure to call __post_init__() from inside 'Concrete'."""
def __post_init__(self):
self._has_required_attributes()
# You can also type check here if you want.
def _has_required_attributes(self):
req_attrs: List[str] = ['attr0', 'attr1']
for attr in req_attrs:
if not hasattr(self, attr):
raise AttributeError(f"Missing attribute: '{attr}'")
class Concrete(Abstract):
def __init__(self, attr0, attr1):
self.attr0 = attr0
self.attr1 = attr1
self.attr2 = "some value" # not required
super().__post_init__() # Enforces the attribute requirement.

For example, you can define the abstract getter, setter and deleter with #abstractmethod and #property, #name.setter or #name.deleter in Person abstract class as shown below. *#abstractmethod must be the innermost decorator otherwise error occurs:
from abc import ABC, abstractmethod
class Person(ABC):
#property
#abstractmethod # The innermost decorator
def name(self): # Abstract getter
pass
#name.setter
#abstractmethod # The innermost decorator
def name(self, name): # Abstract setter
pass
#name.deleter
#abstractmethod # The innermost decorator
def name(self): # Abstract deleter
pass
Then, you can extend Person abstract class with Student class, override the abstract getter, setter and deleter in Student class, instantiate Student class and call the getter, setter and deleter as shown below:
class Student(Person):
def __init__(self, name):
self._name = name
#property
def name(self): # Overrides abstract getter
return self._name
#name.setter
def name(self, name): # Overrides abstract setter
self._name = name
#name.deleter
def name(self): # Overrides abstract deleter
del self._name
obj = Student("John") # Instantiates "Student" class
print(obj.name) # Getter
obj.name = "Tom" # Setter
print(obj.name) # Getter
del obj.name # Deleter
print(hasattr(obj, "name"))
Output:
John
Tom
False
Actually, even if you don't override the abstract setter and deleter in Student class and instantiate Student class as shown below:
class Student(Person): # Extends "Person" class
def __init__(self, name):
self._name = name
#property
def name(self): # Overrides only abstract getter
return self._name
# #name.setter
# def name(self, name): # Overrides abstract setter
# self._name = name
# #name.deleter
# def name(self): # Overrides abstract deleter
# del self._name
obj = Student("John") # Instantiates "Student" class
# ...
No error occurs as shown below:
John
Tom
False
But, if you don't override the abstract getter, setter and deleter in Student class and instantiate Student class as shown below:
class Student(Person): # Extends "Person" class
def __init__(self, name):
self._name = name
# #property
# def name(self): # Overrides only abstract getter
# return self._name
# #name.setter
# def name(self, name): # Overrides abstract setter
# self._name = name
# #name.deleter
# def name(self): # Overrides abstract deleter
# del self._name
obj = Student("John") # Instantiates "Student" class
# ...
The error below occurs:
TypeError: Can't instantiate abstract class Student with abstract methods name
And, if you don't override the abstract getter in Student class and instantiate Student class as shown below:
class Student(Person): # Extends "Person" class
def __init__(self, name):
self._name = name
# #property
# def name(self): # Overrides only abstract getter
# return self._name
#name.setter
def name(self, name): # Overrides abstract setter
self._name = name
#name.deleter
def name(self): # Overrides abstract deleter
del self._name
obj = Student("John") # Instantiates "Student" class
# ...
The error below occurs:
NameError: name 'name' is not defined
And, if #abstractmethod is not the innermost decorator as shown below:
from abc import ABC, abstractmethod
class Person(ABC):
#abstractmethod # Not the innermost decorator
#property
def name(self): # Abstract getter
pass
#name.setter
#abstractmethod # The innermost decorator
def name(self, name): # Abstract setter
pass
#name.deleter
#abstractmethod # The innermost decorator
def name(self): # Abstract deleter
pass
The error below occurs:
AttributeError: attribute 'isabstractmethod' of 'property' objects is not writable

Another possible solution is to use metaclasses.
A minimal example can look like this:
class BaseMetaClass(type):
def __new__(mcls, class_name, bases, attrs):
required_attrs = ('foo', 'bar')
for attr in required_attrs:
if not attr in attrs:
raise RunTimeError(f"You need to set {attr} in {class_name}")
return super().__new__(mcls, class_name, bases, attrs)
class Base(metaclass=BaseMeta):
foo: str
bar: int
One advantage of this approach is that the check will happen at definition time (not instantiation).
Also, setting class attributes in child classes is a bit easier than declaring properties (as long as they are simple values known in advance) and your final classes will look more concise