We Keep Coding

class decorator disables __init_subclass__

I have searched around for an answer to this question but couldn't find anything. My apologies if this was already asked before.
Of the 3-4 methods I know for enforcing from a parent class a given method on a child class (editing the __new__ method of a metaclass, hooking into builtins.__build_class__, use of __init_subclass__ or using abc.abstractmethod) I usually end up using the __init_subclass__, basically because of ease of use and, unlike #abc.abstractmethod, the constraint on the child class is checked upon child class definition and not class instantiation. Example:
class Par():
def __init_subclass__(self, *args, **kwargs):
must_have = 'foo'
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
class Chi(Par):
def __init__(self):
super().__init__()
This example code will obviously throw an error, since Chi does not have a foo method. Nevertheless, I kind of just came across the fact that this constraint from the upstream class can be by-passed by using a simple class decorator:
def add_hello_world(Cls):
class NewCls(object):
def __init__(self, *args, **kwargs):
self.instance = Cls(*args, **kwargs)
def hello_world(self):
print("hello world")
return NewCls
#add_hello_world
class Par:
def __init_subclass__(self, *args, **kwargs):
must_have = "foo"
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
class Chi(Par):
def __init__(self):
super().__init__()
c = Chi()
c.hello_world()
The above code runs without a problem. Now, disregarding the fact that the class I have decorated is Par (and, of course, if Par is library code I might not even have access to it as a user code developer), I cannot really explain this behavior. It is obvious to me that one could use a decorator to add a method or functionality to an existing class, but I had never seen an unrelated decorator (just prints hello world, doesn't even mess with class creation) disable a method already present in the class.
Is this an intended Python behavior? Or is this some kind of bug? To be honest, in my understanding, this might present some security concerns.
Does this happen only to the __init_subclass__ data model? Or also to others?

Remember, decorator syntax is just function application:
class Par:
def __init_subclass__(...):
...
Par = add_hello_world(Par)
The class originally bound to Par defined __init_subclass__; the new class defined inside add_hello_world does not, and that's the class that the post-decoration name Par refers to, and the class that you are subclassing.
Incidentally, you can still access the original class Par via __init__.
Calling the decorator explicitly:
class Par:
def __init_subclass__(self, *args, **kwargs):
must_have = "foo"
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
Foo = Par # Keep this for confirmation
Par = add_hello_world(Par)
we can confirm that the closure keeps a reference to the original class:
>>> Par.__init__.__closure__[0].cell_contents
<class '__main__.Par'>
>>> Par.__init__.__closure__[0].cell_contents is Par
False
>>> Par.__init__.__closure__[0].cell_contents is Foo
True
And if you did try to subclass it, you would get the expected error:
>>> class Bar(Par.__init__.__closure__[0].cell_contents):
... pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tmp.py", line 16, in __init_subclass__
raise AttributeError(f"Must have {must_have}")
AttributeError: Must have foo

Inherit Variables from Parent Class

I am trying to work out how to inherit variables from a parent class.
I have two classes (simplified but same principle):
class Database(object):
def __init__(self, post, *args, **kwargs):
self.post = post
self.report()
def report(self):
#... obtain variables from post ...
self.database_id = self.post['id']
#... save data to database
class PDF(Database):
def __init__(self, post, *args, **kwargs):
Database.__init__(self, post, *args, **kwargs)
#... if i try to access self.database_id now, it returns an error ...
print(self.database_id)
instantiating script:
Database(request.POST)
PDF(request.POST)
I have tried just instantiating CreatePDF, as i thought the Database.__init__(self, post, *args, **kwargs) line would the Database class, but this does not work either.
I am trying to find the most pythonic way to do inherit. I can obviously obtain self.database_id from the post dict passed to PDF(), however I do not see the point in doing this twice, if I can use inheritance.
Thanks

Use:
class PDF(Database):
def __init__(self, post, *args, **kwargs):
# Stuff
super().__init__(post, *args, **kwargs)
The correct approach to instantiated an inherited class is to call super().init(args), which in this case calls Database.init because of method resolution order.
See http://amyboyle.ninja/Python-Inheritance

Django inheritance and polymorphism with proxy models

I'm working on a Django project that I did not start and I am facing a problem of inheritance.
I have a big model (simplified in the example) called MyModel that is supposed to represents different kind of items.
All the instance objects of MyModel should have the same fields but the methods behaviours varies a lot depending on the item type.
Up to this moment this has been designed using a single MyModel field called item_type.
Then methods defined in MyModel check for this field and perform different logic using multiple if:
def example_method(self):
if self.item_type == TYPE_A:
do_this()
if self.item_type == TYPE_B1:
do_that()
Even more, some of the sub-types have many things in common, so let's say the subtypes B and C represents a 1st level of inheritance.
Then these types have sub-types being for example B1, B2, C1, C2 (better explained in the example code below).
I would say that's not the best approach to perform polymorphism.
Now I want to change these models to use real inheritance.
Since all submodels have the same fields I think multi-table inheritance is not necessary. I was thinking to use proxy models because only their behaviour should change depending on their types.
This a pseudo-solution I came up to:
ITEM_TYPE_CHOICES = (
(TYPE_A, _('Type A')),
(TYPE_B1, _('Type B1')),
(TYPE_B2, _('Type B2')),
(TYPE_C1, _('Type C1')),
(TYPE_C2, _('Type C2')))
class MyModel(models.Model):
item_type = models.CharField(max_length=12, choices=ITEM_TYPE_CHOICES)
def common_thing(self):
pass
def do_something(self):
pass
class ModelA(MyModel):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_A
def do_something(self):
return 'Hola'
class ModelB(MyModel):
class Meta:
proxy = True
def common_thing(self):
pass
class ModelB1(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B1
def do_something(self):
pass
class ModelB2(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B2
def do_something(self):
pass
This might work if we already know the type of the object we are working on.
Let's say we want to instantiate a MyModel object of type C1 then we could simply instantiate a ModelC1 and the item_type would be set up correctly.
The problem is how to get the correct proxy model from the generic MyModel instances?
The most common case is when we get a queryset result: MyModel.objects.all(), all these objects are instances of MyModel and they don't know anything about the proxies.
I've seen around different solution like django-polymorphic but as I've understood that relies on multi-table inheritance, isn't it?
Several SO answers and custom solutions I've seen:
https://stackoverflow.com/a/7526676/1191416
Polymorphism in Django
http://anthony-tresontani.github.io/Python/2012/09/11/django-polymorphism/
https://github.com/craigds/django-typed-models
Creating instances of Django proxy models from their base class
but none of them convinced me 100%..
Considering this might be a common scenario did anyone came up with a better solution?

When you use django-polymorphic in your base model, you'll get this casting behavior for free:
class MyModel(PolymorphicModel):
pass
Each model that extends from it (proxy model or concrete model), will be casted back to that model when you do a MyModel.objects.all()

I have few experience with model proxies so I can't tell if this would properly work (without bearking anything I mean) nor how complicated this might be, but you could use an item_type:ProxyClass mapping and override your model's queryset (or provide a second manager with custom queryset etc) that actually lookup this mapping and instanciates the correct proxy model.
BTW you may want at django.models.base.Model.from_db, which (from a very quick glance at the source code) seems to be the method called by QuerySet.populate() to instanciate models. Just overriding this method might possibly be enough to solve the problem - but here again it might also breaks something...

I came up with a custom solution inspired by this SO answer and this blog post:
from django.db import models
from django.dispatch.dispatcher import receiver
ITEM_TYPE_CHOICES = (
(TYPE_A, _('type_a')),
(TYPE_B1, _('type_b1')),
(TYPE_B2, _('type_b2')),
(TYPE_C1, _('type_c1')),
(TYPE_C2, _('type_c2')),
)
class MyModel(models.Model):
item_type = models.CharField(max_length=12, choices=ITEM_TYPE_CHOICES)
description = models.TextField(blank=True, null=True)
def common_thing(self):
pass
def do_something(self):
pass
# ****************
# Hacking Django *
# ****************
PROXY_CLASS_MAP = {} # We don't know this yet
#classmethod
def register_proxy_class(cls, item_type):
"""Class decorator for registering subclasses."""
def decorate(subclass):
cls.PROXY_CLASS_MAP[item_type] = subclass
return subclass
return decorate
def get_proxy_class(self):
return self.PROXY_CLASS_MAP.get(self.item_type, MyModel)
# REGISTER SUBCLASSES
#MyModel.register_proxy_class(TYPE_A)
class ModelA(MyModel):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_A
def do_something(self):
pass
# No need to register this, it's never instantiated directly
class ModelB(MyModel):
class Meta:
proxy = True
def common_thing(self):
pass
#MyModel.register_proxy_class(TYPE_B1)
class ModelB1(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B1
def do_something(self):
pass
#MyModel.register_proxy_class(TYPE_B2)
class ModelB2(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B2
def do_something(self):
pass
# USING SIGNAL TO CHANGE `__class__` at runtime
#receiver(models.signals.post_init, sender=MyModel)
def update_proxy_object(sender, **kwargs):
instance = kwargs['instance']
if hasattr(instance, "get_proxy_class") and not instance._meta.proxy:
proxy_class = instance.get_proxy_class()
if proxy_class is not None:
instance.__class__ = proxy_class
I'm using the decorator register_proxy_class to register each subclass after MyModel has been declared otherwise I would have needed to explicitly declare a map of {type: subclass} inside MyModel.
This would have been bad:
because at declaration we can't reference any of the proxy subclasses from MyModel (we could solve these with string names)
the parent would be aware of its subclasses which breaks OOP principles.
How it works:
Using the #register_proxy_class(type) decorator each subclass register itself, in fact creating an entry into MyModel.PROXY_CLASS_MAP dict when the module is loaded.
Then update_proxy_object is executed whenever MyModel dispatch a post_init signal. It change the __class__ of MyModel instances at runtime to select the right proxy subclass.
So basically:
# a1: MyModel dispatch a post_init signal -> `update_proxy_object` set the proper instance __class__ = ModelA
# Do NOT call ModelA.__init__
a1 = MyModel(item_type=TYPE_A)
isinstance(a1, MyModel) # True
isinstance(a1, ModelA) # True
# a2: calls ModelA.__init__ that call the parent MyModel.__init__ then it sets up the item_type for us
a2 = ModelA() # <- no need to pass item_type
isinstance(a2,MyModel) # True
isinstance(a2, ModelA) #True
# Using custom managers of MyModel return all objects having item_type == 'TYPE_B1'
b1 = MyModel.objects.b1()[0] # get the first one
isinstance(b1, ModelB1) # True
isinstance(b1, ModelB) # True
isinstance(b1, MyModel) # True
isinstance(b1, ModelA) # False
It seems to work so far but I will experiment a bit more for possible problems I haven't think about.
Cool!

super(type, obj): obj must be an instance or subtype of type

Why do I get the following error, and how do I resolve it?
TypeError: super(type, obj): obj must be an instance or subtype of type

Another way this error can occur is when you reload the module with the class in a Jupiter notebook.
Easy solution is to restart the kernel.
http://thomas-cokelaer.info/blog/2011/09/382/
Check out #Mike W's answer for more detail.

You should call super using the UrlManager class as first argument not the URL model. super cannot called be with an unrelated class/type:
From the docs,
super(type[, object-or-type]):
Return a proxy object that delegates method calls to a parent or
sibling class of type.
So you cannot do:
>>> class D:
... pass
...
>>> class C:
... def __init__(self):
... super(D, self).__init__()
...
>>> C()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __init__
TypeError: super(type, obj): obj must be an instance or subtype of type
You should do:
qs_main = super(UrlManager, self).all(*args, **kwargs)
Or in Python 3:
qs_main = super().all(*args, **kwargs)

Elaborating in #Oğuz Şerbetci's answer, in python3 (not necessary only in Jupyter), when there is the need to reload a library, for example we have class Parent and class Child defined as
class Parent(object):
def __init__(self):
# do something
class Child(Parent):
def __init__(self):
super(Child, self).__init__(self)
then if you do this
import library.Child
reload(library)
Child()
you will get TypeError: super(type, obj): obj must be an instance or subtype of type, the solution is just to re import the class after the reload
import library.Child
reload(library)
import library.Child
Child()

For Jupyter only
You can get his issue in because reload logic have some bugs (issue)
Here is a simple solution/workaround that works for me until issue is not fixed
Add typo like 1001xx at the bottom of the file which you call in the cell
Run your cell - you will see some exception, just skip it
Remove typo which was added on step 1
Run the cell
Profit

Another interesting way is if a merge of branches has duplicated the class, so that in the file you have two definitions for the same name, e.g.
class A(Foo):
def __init__(self):
super(A, self).__init__()
#...
class A(Foo):
def __init__(self):
super(A, self).__init__()
#...
If you try to create an instance from a static reference to the first definition of A, once it tries to call super, inside the __init__ method, A will refer to the second definition of A, since it has been overwritten. The solution - ofcourse - is to remove the duplicate definition of the class, so it doesn't get overwritten.
This may seem like something that would never happen, but it just happened to me, when I wasn't paying close enough attention to the merge of two branches. My tests failed with the error message described in the question, so I thought I'd leave my findings here, even though it doesn't exactly answer the specific question.

The best solution that I have found for this problem is only available using python 3. You then don't need to specify the arguments of "super", then you won't have the error any more writing your class like this :
class D:
pass
class C(D):
def __init__(self):
super().__init__()# no arguments given to super()

This error also pops out when you simply do not instantiate child class
, and try to call a method on a class itself, like in :
class Parent:
def method():
pass
class Child(Parent):
def method():
super().method()
P = Parent()
C = Child
C.method()

Similar to #Eldamir, I solved it by realizing I had written two classes with the same name, and the second one was overwriting the first.
If that's the case, change the name of one of the classes.

So I just pasted in a form in forms.py.
I just made a fast look to see if I needed to change anything, but I didn't see that.
Then I got this super(type, obj): obj must be an instance or subtype of type error, so I searched for it on the browser, but before I checked any of the answers I looked one more time and this time I spotted the issue.
As you can see, many answers on this question says it was wrong with the super. Yes it was the same issue for me.
make sure that you look if you have any super and see if the class added matches with the class. At least that's what I did.
Before and After
Where I spotted it in my code
forms.py
Before:
class userProfileForm(ModelForm):
class Meta:
model = user_profile
fields = ("user", "rating", )
def __init__(self, *args, **kwargs):
# https://stackoverflow.com/a/6866387/15188026
hide_condition = kwargs.pop('hide_condition',None)
super(ProfileForm, self).__init__(*args, **kwargs)
if hide_condition:
self.fields['user'].widget = HiddenInput()
After:
class userProfileForm(ModelForm):
class Meta:
model = user_profile
fields = ("user", "rating", )
def __init__(self, *args, **kwargs):
# https://stackoverflow.com/a/6866387/15188026
hide_condition = kwargs.pop('hide_condition',None)
super(userProfileForm, self).__init__(*args, **kwargs)
if hide_condition:
self.fields['user'].widget = HiddenInput()
You see that the super got changed to the class name

Define field in ndb using strings

This is my code:
class SocialNodeSubscription(model.Model):
def __init__(self, *args, **kwargs):
permissions=["post","read","reply","admin"]
for p in permissions:
self.__dict__["can_"+p]=model.BooleanProperty(default=True)
I need to dynamically define fields in my model but this doesn't seem to work because dict is not the right place where to put my fields.
For who don't know about ndb, this is how it would look like going the easier way.
class SocialNodeSubscription(model.Model):
def __init__(self, *args, **kwargs):
self.can_write=model.BooleanProperty(default=True)
self.can_read=model.BooleanProperty(default=True)
...
Edit:
Now my code looks like this:
def __init__(self, *args, **kwargs):
permissions=["post","read","reply","admin"]
for p in permissions:
self._properties["can_"+p]=model.BooleanProperty(default=True)
self._fix_up_properties()
But still i get this error.
File "C:\Program Files
(x86)\Google\google_appengine\google\appengine\ext\ndb\model.py", line
972, in _store_value
entity._values[self._name] = value TypeError: 'NoneType' object does not support item assignment
What does it mean?

It's _properties,
just have a look at its metaclass MetaModel and class method _fix_up_properties.
Definition of _properties:
# Class variables updated by _fix_up_properties()
_properties = None
Method:
#classmethod
def _fix_up_properties(cls):
"""Fix up the properties by calling their _fix_up() method.
Note: This is called by MetaModel, but may also be called manually
after dynamically updating a model class.
"""

Use an expando model for a model with dynamic properties.