I understand that when a class inherits from an Abstract Django Model, it will not inherit the meta attribute abstract = True, which makes sense.
However in the below example nothing has inherited from it, but yet it's Meta.abstract is False even though its defined to be True:
from django.db import models
from django.db.models.base import ModelBase
class MyMeta(ModelBase):
def __new__(cls, name, bases, attrs, **kwargs):
"""Check that all implemented (not abstract) classes have a foo attribute"""
Class = super().__new__(cls, name, bases, attrs, **kwargs)
if not Class.Meta.abstract:
print(Class)
print('Class.Meta.ordering:', Class.Meta.ordering) # Sanity check
print('Class.Meta.abstract:', Class.Meta.abstract)
if not hasattr(Class, 'foo'):
raise NotImplementedError('Please add a foo attribute')
return Class
class MyAbstractModel(models.Model, metaclass=MyMeta):
name = models.CharField(max_length=250)
class Meta:
abstract = True
ordering = ('-name',)
Prints:
<class 'myapp.models.base.MyAbstractModel'>
Class.Meta.ordering: -name
Class.Meta.abstract: False
Add raises (even though it should not raise because it is an abstract class):
NotImplementedError: Please add a foo attribute
You are accessing the wrong attribute, namely Class.Meta. Since this attribute will be inherited by any child classes Django actually modifies [GitHub] it for abstract models:
if abstract:
# Abstract base models can't be instantiated and don't appear in
# the list of models for an app. We do the final setup for them a
# little differently from normal models.
attr_meta.abstract = False
new_class.Meta = attr_meta
return new_class
The attribute Django actually uses behind the scenes (All of the attributes are properly copied over to it) is _meta. Hence if you write the following you'll get the expected result:
if not Class._meta.abstract:
...
Abstract is False because children classes don't become abstract classes themselves. You need to explicitly set it True.
There is info about that in the docs.
https://docs.djangoproject.com/en/4.1/topics/db/models/
Related
I want to raise an exception if a user has mistyped an abstract class attribute name and a new class attribute is added instead of overriding one:
class FooAbstract:
default_value = 'spam' # This could be overriden
class MyFoo(FooAbstract):
defualt_value = 'bar' # A user mistyped the name and he should be notified
I guess difflib.SequenceMatcher could be used for validating newly added variable names:
from difflib import SequenceMatcher
orig_names = set([a for a in dir(FooAbstract) if not a.startswith('__')])
new_names = set(dir(MyFoo)) - set(dir(FooAbstract))
for orig_name in orig_names:
for new_name in new_names:
if 0.9 < SequenceMatcher(a=orig_name, b=new_name).ratio() < 1:
message = f'You probably misstyped "{new_name}" (use "{orig_name}" instead)'
raise AttributeError(message)
My question is where should I put this condition in order to execute it during loading of the subclass MyFoo?
Class FooAbstract should define __init_subclass__. This init subclas hook is the best place for tests like yours. It is as documented as a tool for class creation customization.
This method is called whenever the containing class is subclassed. cls
is then the new subclass.
I read that it is considered bad practice to create a variable in the class namespace and then change its value in the class constructor.
(One of my sources: SoftwareEngineering SE: Is it a good practice to declare instance variables as None in a class in Python.)
Consider the following code:
# lib.py
class mixin:
def __init_subclass__(cls, **kwargs):
cls.check_mixin_subclass_validity(cls)
super().__init_subclass__(**kwargs)
def check_mixin_subclass_validity(subclass):
assert hasattr(subclass, 'necessary_var'), \
'Missing necessary_var'
def method_used_by_subclass(self):
return self.necessary_var * 3.14
# app.py
class my_subclass(mixin):
necessary_var = None
def __init__(self, some_value):
self.necessary_var = some_value
def run(self):
# DO SOME STUFF
self.necessary_var = self.method_used_by_subclass()
# DO OTHER STUFF
To force its subclass to declare the variable necessary_var, the class mixin uses the metaclass subclass_validator.
And the only way I know to makes it work on app.py side, is to initialized necessary_var as a class variable.
I am missing something or is it the only way to do so?
Short answer
You should check that attributes and methods exist at instantiation of a class, not before. This is what the abc module does and it has good reasons to work like this.
Long answer
First, I would like to point out that it seems what you want to check is that an instance attribute exists.
Due to Python dynamic nature, it is not possible to do so before an instance is created, that is after the call to __init__. We could define Mixin.__init__, but we would then have to rely on the users of your API to have perfect hygiene and to always call super().__init__.
One option is thus to create a metaclass and add a check in its __call__ method.
class MetaMixin(type):
def __call__(self, *args, **kwargs):
instance = super().__call__(*args, **kwargs)
assert hasattr(instance, 'necessary_var')
class Mixin(metaclass=MetaMixin):
pass
class Foo(Mixin):
def __init__(self):
self.necessary_var = ...
Foo() # Works fine
class Bar(Mixin):
pass
Bar() # AssertionError
To convince yourself that it is good practice to do this at instantiation, we can look toward the abc module which uses this behaviour.
from abc import abstractmethod, ABC
class AbstractMixin(ABC):
#abstractmethod
def foo(self):
...
class Foo(AbstractMixin):
pass
# Right now, everything is still all good
Foo() # TypeError: Can't instantiate abstract class Foo with abstract methods foo
As you can see the TypeError was raise at instantiation of Foo() and not at class creation.
But why does it behave like this?
The reason for that is that not every class will be instantiated, consider the example where we want to inherit from Mixin to create a new mixin which checks for some more attributes.
class Mixin:
def __init_subclass__(cls, **kwargs):
assert hasattr(cls, 'necessary_var')
super().__init_subclass__(**kwargs)
class MoreMixin(Mixin):
def __init_subclass__(cls, **kwargs):
assert hasattr(cls, 'other_necessary_var')
super().__init_subclass__(**kwargs)
# AssertionError was raised at that point
class Foo(MoreMixin):
necessary_var = ...
other_necessary_var = ...
As you see, the AssertionError was raised at the creation of the MoreMixin class. This is clearly not the desired behaviour since the Foo class is actually correctly built and that is what our mixin was supposed to check.
In conclusion, the existence of some attribute or method should be done at instantiation, Otherwise, you are preventing a whole lot of helpful inheritance techniques. This is why the abc module does it like that and this is why we should.
I'm trying to get a python2 program working in python3, it has the following Meta class definition. Which works just fine on Py2. What's the "best" way to have this be compatible with both py2 and py3?
It's failing in the unit test where it does:
try:
raise Actor.DoesNotExist
except Actor.DoesNotExist:
pass
Failure is:
AttributeError: type object 'Actor' has no attribute 'DoesNotExist'
The base meta class definition is:
class MetaDocument(type):
def __new__(meta,name,bases,dct):
class DoesNotExist(BaseException):
pass
class MultipleDocumentsReturned(BaseException):
pass
dct['DoesNotExist'] = DoesNotExist
dct['MultipleDocumentsReturned'] = MultipleDocumentsReturned
class_type = type.__new__(meta, name, bases, dct)
if not class_type in document_classes:
if name == 'Document' and bases == (object,):
pass
else:
document_classes.append(class_type)
return class_type
class Document(object):
__metaclass__ = MetaDocument
You could use the MetaDocument() metaclass as a factory to produce a class replacing your Document class, re-using the class attributes:
class Document(object):
# various and sundry methods and attributes
body = vars(Document).copy()
body.pop('__dict__', None)
body.pop('__weakref__', None)
Document = MetaDocument(Document.__name__, Document.__bases__, body)
This doesn't require you to build the 3rd argument, the class body, manually.
You can turn this into a class decorator:
def with_metaclass(mcls):
def decorator(cls):
body = vars(cls).copy()
# clean out class body
body.pop('__dict__', None)
body.pop('__weakref__', None)
return mcls(cls.__name__, cls.__bases__, body)
return decorator
then use as:
#with_metaclass(MetaDocument)
class Document(object):
# various and sundry methods and attributes
Alternatively, use the six library for this:
#six.add_metaclass(MetaDocument)
class Document(object):
where the #six.add_metaclass() decorator also takes care of any __slots__ you may have defined; my simpler version above doesn't.
six also has a six.with_metaclass() base-class factory:
class Document(six.with_metaclass(MetaDocument)):
which injects an extra base class into the MRO.
six has a utility for this.
class Document(six.with_metaclass(MetaDocument, object)):
# class definition, without the __metaclass__
The only side effect is that the class hierarchy changes from
>>> Document.__mro__
(<class 'test.Document'>, <type 'object'>)
to
>>> Document.__mro__
(<class 'test.Document'>, <class 'test.NewBase'>, <type 'object'>)
because with_metaclass actually returns a new class with the appropriate metaclass.
I created the empty abstract class AbstractStorage and inherited the Storage class from it:
import abc
import pymongo as mongo
host = mongo.MongoClient()
print(host.alive()) # True
class AbstractStorage(metaclass=abc.ABCMeta):
pass
class Storage(AbstractStorage):
dbh = host
def __init__(self):
print('__init__')
Storage()
I expected the output to be
True
__init__
however, the one I'm getting is
True
Traceback (most recent call last):
File "/home/vaultah/run.py", line 16, in <module>
Storage()
TypeError: Can't instantiate abstract class Storage with abstract methods dbh
The problem (apparently) goes away if I remove metaclass=abc.ABCMeta (so that AbstractStorage becomes an ordinary class) and/or if I set dbh to some other value.
What's going on here?
This isn't really a problem with ABCs, it's a problem with PyMongo. There is an issue about it here. It seems that pymongo overrides __getattr__ to return some sort of database class. This means that host.__isabstractmethod__ returns a Database object, which is true in a boolean context. This cause ABCMeta to believe that host is an abstract method:
>>> bool(host.__isabstractmethod__)
True
The workaround described in the issue report is to manually set host.__isabstractmethod__ = False on your object. The last comment on the issue suggests a fix has been put in for pymongo 3.0.
Short Version
mongo.MongoClient returns an object that appears to be (is?) an abstract method, which you then assign to the dbh field in Storage. This makes Storage an abstract class, so instantiating it raises a TypeError.
Note that I don't have pymongo, so I can't tell you anything more about MongoClient than how it gets treated by ABCMeta.
Long Version
The ABCMeta.__new__ method looks inside each field of the new class it's creating. Any field that itself has a True (or "true-like") __isabstractmethod__ field is considered an abstract method. If a class has any non-overridden abstract methods, the whole class is considered abstract, so any attempt to instantiate it is an error.
From an earlier version of the standard library's abc.py:
def __new__(mcls, name, bases, namespace):
cls = super().__new__(mcls, name, bases, namespace)
# Compute set of abstract method names
abstracts = {name
for name, value in namespace.items()
if getattr(value, "__isabstractmethod__", False)}
# ...
cls.__abstractmethods__ = frozenset(abstracts)
# ...
This is not mentioned in the abc.ABCMeta class docs, but a bit lower, under the #abc.abstractmethod decorator:
In order to correctly interoperate with the abstract base class machinery, the descriptor must identify itself as abstract using __isabstractmethod__. In general, this attribute should be True if any of the methods used to compose the descriptor are abstract.
Example
I created a bogus "abstract-looking" class with an __isabstractmethod__ attribute, and two supposedly-concrete subclasses of AbstractStorage. You'll see that one produces the exact error you're getting:
#!/usr/bin/env python3
import abc
# I don't have pymongo, so I have to fake it. See CounterfeitAbstractMethod.
#import pymongo as mongo
class CounterfeitAbstractMethod():
"""
This class appears to be an abstract method to the abc.ABCMeta.__new__
method.
Normally, finding an abstract method in a class's namespace means
that class is also abstract, so instantiating that class is an
error.
If a class derived from abc.ABCMeta has an instance of
CounterfeitAbstractMethod as a value anywhere in its namespace
dictionary, any attempt to instantiate that class will raise a
TypeError: Can't instantiate abstract class <classname> with
abstract method <fieldname>.
"""
__isabstractmethod__ = True
class AbstractStorage(metaclass=abc.ABCMeta):
def __init__(self):
"""
Do-nothing initializer that prints the name of the (sub)class
being initialized.
"""
print(self.__class__.__name__ + ".__init__ executing.")
return
class ConcreteStorage(AbstractStorage):
"""
A concrete class that also _appears_ concrete to abc.ABCMeta. This
class can be instantiated normally.
"""
whatever = "Anything that doesn't appear to be an abstract method will do."
class BogusStorage(AbstractStorage):
"""
This is (supposedly) a concrete class, but its whatever field appears
to be an abstract method, making this whole class abstract ---
abc.ABCMeta will refuse to construct any this class.
"""
#whatever = mongo.MongoClient('localhost', 27017)
whatever = CounterfeitAbstractMethod()
def main():
"""
Print details of the ConcreteStorage and BogusStorage classes.
"""
for cls in ConcreteStorage, BogusStorage:
print(cls.__name__ + ":")
print(" whatever field: " + str(cls.whatever))
print(" abstract methods: " + str(cls.__abstractmethods__))
print(" Instantiating...")
print(" ", end="")
# KABOOM! Instantiating BogusStorage will raise a TypeError,
# because it appears to be an _abstract_ class.
instance = cls()
print(" instance: " + str(instance))
print()
return
if "__main__" == __name__:
main()
Running this produces:
$ ./storage.py
ConcreteStorage:
whatever field: Anything that doesn't appear to be an abstract method will do.
abstract methods: frozenset()
Instantiating...
ConcreteStorage.__init__ executing.
instance: <__main__.ConcreteStorage object at 0x253afd0>
BogusStorage:
whatever field: <__main__.CounterfeitAbstractMethod object at 0x253ad50>
abstract methods: frozenset({'whatever'})
Instantiating...
Traceback (most recent call last):
File "./storage.py", line 75, in <module>
main()
File "./storage.py", line 68, in main
instance = cls()
TypeError: Can't instantiate abstract class BogusStorage with abstract methods whatever
I need to create a business query model, in which I need to create a circular dependency, I am using a look a like design of django models to implement it,
#Modeule a.py
import b
class A:
b_obj = B()
a_property_1 = ObjectAttribute(b_obj.b_property_1) # a_property_1 is dependent on b_property_1
a_property_2 = ObjectAttribute(b_obj.b_property_2)
#Module b.py
import a
class B:
a_obj = A()
b_property_1 = ObjectAttribute(a_obj.a_property_1)
b_property_2 = ObjectAttribute(a_obj.a_property_2)
When I execute the above program, it will throw an error, name 'B' is not defined on executing a.py and viceversa.
After that, I did a bit research on this to figure out and findout django models already implemented something like this via ForeignKey
https://docs.djangoproject.com/en/dev/ref/models/fields/#foreignkey
All I need to implement the my ForeignKey module, can some one please help me in understanding the logic and writing the code in below format.
#Modeule a.py
import b
class A:
b_obj = MyForeignKey('B')
a_property_1 = ObjectAttribute(b_obj.b_property_1) # a_property_1 is dependent on b_property_1
a_property_2 = ObjectAttribute(b_obj.b_property_2)
#Module b.py
import a
class B:
a_obj = MyForeignKey('A')
b_property_1 = ObjectAttribute(a_obj.a_property_1)
b_property_2 = ObjectAttribute(a_obj.a_property_2)
There are some ways to do that. One of which would be for your foreign Key to be made as proxy classes to the actuall classes, that on instantiating, just annotate the class model, and forhe next subsequent attribute access instantiate the proxied-to class, and keep its reference, Subsequent attributes would just be redirected to the underlying class.
One mechanism that allows such hooks to be executed on attribute fecth (remebering that in Pyhton a class "method" is just a callable attribute - so it works for methods as well), is to implement the __getattribute__ method.
Let's supose you have a "models" module (or other kind of registry) wher all your models are referenced, after creation -- your code could look more or less like this:
import models
class MyForeignKey(objec):
def __init__(self, model_name, *args, **kw):
self._model_name = model_name
self._args = args
self._kw = kw
def _instantiate(self):
self._object = getattr(models, self._model_name)(*self._args, **self._kw)
def __getattribute__(self, attr):
if attr in ("_model_name", "_args", "_kw", "_object", "_instantiate"):
return object.__getattribute__(self, attr)
if not hasattr(self, "_object"):
self._instantiate()
return getattr(self._object, attr)
def __setattr__(self, attr, value):
if attr in ("_model_name", "_args", "_kw", "_object"):
return object.__setattr__(self, attr, value)
if not hasattr(self, "_object"):
self._instantiate()
return setattr(self._object, attr, value)
Note that (a) your models have to inherit from "object" like I commented in the question and (b) - this is ot complete if you implement "dunder" methods (python double underscore methods) to override behavior on any of the models - in that case, you have to set the appropriate te dunder methods to do the proxying as well.