In my Django project, I want all my model fields to have an additional argument called documentation. (It would be similar to verbose_name or help_text, but instead for internal documentation.)
This seems straightforward: just subclass and override the field's __init__:
def __init__(self, verbose_name=None, name=None, documentation=None, **kwargs):
self.documentation = documentation
super(..., self).__init__(verbose_name, name, **kwargs)
The question is how do I make this apply to all of the 20-something field classes in django.db.models (BooleanField, CharField, PositiveIntegerField, etc.)?
The only way I see is to use metaprogramming with the inspect module:
import inspect
import sys
from django.db.models import *
current_module = sys.modules[__name__]
all_field_classes = [Cls for (_, Cls) in inspect.getmembers(current_module,
lambda m: inspect.isclass(m) and issubclass(m, Field))]
for Cls in all_field_classes:
Cls.__init__ = <???>
I am not used to seeing code like this, and don't even know if it will work. I wish I could just add the attribute to the base Field class, and have it inherit to all the child classes, but I don't see how that could be done.
Any ideas?
Indeed - you are ont he right track.
In python, introspection is a normal thing, and and you don even need to use the inspect module just because it "I am using introspection and meta programing, I must need inspect ) :-)
One thing that is not considered that much of a good practice, though, is Monkey patching - that is, if you change the classes as they are in the django.db.models itself, so that other modules will import the modified classes from there and use the modified version. (Note that in this case: not recommended != will not work) - so you would be better creating all the new model classes in your own module, and importing them from your own module, instead of from django.db.models
So, something along:
from django.db import models
# A decorator to implement the behavior you want for the
# __init__ method
def new_init(func):
def __init__(self, *args, **kw):
self.documentation = kw.pop("documentation", None)
return func(self, *args, **kw)
for name, obj in models.__dict__.items():
#check if obj is a class:
if not isinstance(obj, type):
continue
# creates a new_init, retrieving the original one -
# taking care for not to pick it as an unbound method -
# check: http://pastebin.com/t1SAusPS
new_init_method = new_init(obj.__dict__.get("__init__", lambda s:None))
# dynamically creates a new sublass of obj, overriding just the __init__ method:
new_class = type(name, (obj,), {"__init__": new_init_method})
# binds the new class to this module's scope:
globals().__setitem__(name, new_class)
Or if you prefer using monkey patching, as it is easier :-p
from django.db import models
def new_init(func):
def __init__(self, *args, **kw):
self.documentation = kw.pop("documentation", None)
return func(self, *args, **kw)
for name, obj in models.__dict__.items():
#check if obj is a class:
if not isinstance(obj, type):
continue
obj.__init__ = new_init(obj.__dict__["__init__"])
Related
I decorated some methods with #bot_thinking, which stores some information about the decorated method in the functions attribute.
One piece of information is 'class_name', but my program needs the class type as a variable, e.g. RandomBot. I would like to get this class.
Here is some sample code:
class DepthPrunedMinimaxAgent(Agent):
#bot_thinking(associated_name="minimax profondeur")
def select_move(self, game_state: GameState):
Above is the decorated part of the code.
The decorator:
functions = {}
def bot_thinking(associated_name, active=True):
def _(func):
if active:
class_name = func.__qualname__.rsplit('.')[-2]
import sys
# class_name_2=getattr(sys.modules[__name__], class_name)
# module=importlib.import_module('sources.agent')
functions[associated_name] = (associated_name, class_name,
globals()[class_name], func)
else:
functions.pop(associated_name)
return _
bot_thinking isn't a real decorator, it's a decorator factory.
From the func function, I get the class_name, but I can't use the accepted answer by #m.kocikowski, to find the correct class because this class is decorated, so it already imports the annotation module, so importing from the module of the annotation the annotated module would result in a cyclic import, which python does not seem to permit.
Do you see a method to get the class from its name?
ps:
ps:
to be clearer : the annotation part of the code need an import to the annotated classes(to retrieve the class from its name), which also need an importation of the annotation (for the annotation to work).
You can do what you want if you use a descriptor class, rather than a function, as the decorator, at least if you're using Python 3.6 or newer. That's because there's a new method added to the descriptor protocol, __set_name__. It gets called when the descriptor object is saved as a class variable. While most descriptors will use it to record the name they're being saved as, you can use it to get the class you're in.
You do need to make your decorator object wrap the real function (implementing calling and descriptor lookup methods), rather than being able to return the unmodified function you were decorating. Here's my attempt at a quick and dirty implementation. I don't really understand what you're doing with functions, so I may not have put the right data in it, but it should be close enough to get the idea across (owner is the class the method stored in).
functions = {}
def bot_thinking(associated_name, active=True):
class decorator:
def __init__(self, func):
self.func = func
def __set_name__(self, owner, name):
if active:
functions[associated_name] = (associated_name, owner.__name__,
owner, self.func)
else:
functions.pop(associated_name)
def __get__(self, obj, owner):
return self.func.__get__(obj, owner)
def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)
return decorator
The problem is the class hasn't been defined yet when the bot_thinking() decorator factory (and decorator itself) are executing. The only workaround I can think of would be to patch things up after the class is defined, as illustrated below:
from pprint import pprint, pformat
functions = {}
def bot_thinking(associated_name, active=True):
def _(func):
if active:
class_name = func.__qualname__.split(".")[-2]
functions[associated_name] = (associated_name, class_name, class_name, func)
else:
functions.pop(associated_name, None)
return func # Decorators must return a callable.
return _
class Agent: pass
class GameState: pass
class DepthPrunedMinimaxAgent(Agent):
#bot_thinking(associated_name="minimax profondeur")
def select_move(self, game_state: GameState):
pass
# After class is defined, update data put into functions dictionary.
for associated_name, info in functions.items():
functions[associated_name] = (info[0], info[1], globals()[info[2]], info[3])
pprint(functions)
Output:
{'minimax profondeur': ('minimax profondeur',
'DepthPrunedMinimaxAgent',
<class '__main__.DepthPrunedMinimaxAgent'>,
<function DepthPrunedMinimaxAgent.select_move at 0x00F158A0>)}
The specific use case I need it for is to deprecate class names.
Suppose we have class A in an earlier version and we want to deprecate its name but keep backwards compatibility:
class A(B):
def __init__(self, *args, **kwargs):
warnings.warn('deprecation!')
super(A, self).__init__(*args, **kwargs)
... and B now has the correct implementation.
When we create a class A, we will run into a deprecation warning here. We can also use the deprecated module for decorators on __init__.
However, I want to skip this process and write less code, and hopefully achieve something like:
#deprecated_alias('A')
class B:
# ... do something
Can I somehow inject the classname into the module-level namespace so that I can use A like this?
Can I somehow inject the classname into the module-level namespace so that I can use A like this?
Yes. The class decorator should:
create a new type, with overridden __init__ method, using the 3-argument invocation of type
get the module of the original class, sys.modules[original_class.__module__]
bind the new class in the module namespace, using setattr
return the original class unchanged
Example:
import sys
def deprecated_alias(name):
def decorator(class_):
mod = sys.modules[class_.__module__]
if hasattr(mod, name):
raise Exception('uhoh, name collision')
NewClass = type(name, (class_,), {'__init__': ...})
setattr(mod, name, NewClass)
return class_
return decorator
#deprecated_alias('A')
class B:
pass
I don't recommend this approach - too much magic. It will confuse IDEs and break autocompletion.
A less magical approach, perhaps? This could also be made into a decorator, and use __subclasscheck__/__subclasshook__ if you need to control the finer details of inheritance.
class A(B):
def __init__(self, *args, **kwargs):
warnings.warn('deprecation!')
return B(*args, **kwargs)
While this is not exactly what you asked for, it is substantially less magical and ultimately the same number of lines of code. It is also far more explicit:
import warnings
def deprecated(DeprecatedByClass):
class Deprecated(DeprecatedByClass):
def __new__(cls, *args, **kwargs):
warnings.warn("deprecation!")
return super(Deprecated, cls).__new__(cls, *args, **kwargs)
return Deprecated
You can then use this like so:
class B:
pass
A = deprecated(B)
I would like to have in the code underneath that when i type instance_of_A = A(, that the name of the supposed arguments is init_argumentA and not *meta_args, **meta_kwargs. But unfortunatally, the arguments of the __call__ method of the metaclass are shown.
class Meta(type):
def __call__(cls,*meta_args,**meta_kwargs):
# Something here
return super().__call__(*meta_args, **meta_kwargs)
class A(metaclass = Meta):
def __init__(self,init_argumentA):
# something here
class B(metaclass = Meta):
def __init__(self,init_argumentB):
# something here
I have searched for a solution and found the question How to dynamically change signatures of method in subclass?
and Signature-changing decorator: properly documenting additional argument. But none, seem to be completely what I want. The first link uses inspect to change the amount of variables given to a function, but i can't seem to let it work for my case and I think there has to be a more obvious solution.
The second one isn't completely what I want, but something in that way might be a good alternative.
Edit: I am working in Spyder. I want this because I have thousands of classes of the Meta type and each class have different arguments, which is impossible to remember, so the idea is that the user can remember it when seeing the correct arguments show up.
Using the code you provided, you can change the Meta class
class Meta(type):
def __call__(cls, *meta_args, **meta_kwargs):
# Something here
return super().__call__(*meta_args, **meta_kwargs)
class A(metaclass=Meta):
def __init__(self, x):
pass
to
import inspect
class Meta(type):
def __call__(cls, *meta_args, **meta_kwargs):
# Something here
# Restore the signature of __init__
sig = inspect.signature(cls.__init__)
parameters = tuple(sig.parameters.values())
cls.__signature__ = sig.replace(parameters=parameters[1:])
return super().__call__(*meta_args, **meta_kwargs)
Now IPython or some IDE will show you the correct signature.
I found that the answer of #johnbaltis was 99% there but not quite what was needed to ensure the signatures were in place.
If we use __init__ rather than __call__ as below we get the desired behaviour
import inspect
class Meta(type):
def __init__(cls, clsname, bases, attrs):
# Restore the signature
sig = inspect.signature(cls.__init__)
parameters = tuple(sig.parameters.values())
cls.__signature__ = sig.replace(parameters=parameters[1:])
return super().__init__(clsname, bases, attrs)
def __call__(cls, *args, **kwargs):
super().__call__(*args, **kwargs)
print(f'Instanciated: {cls.__name__}')
class A(metaclass=Meta):
def __init__(self, x: int, y: str):
pass
which will correctly give:
In [12]: A?
Init signature: A(x: int, y: str)
Docstring: <no docstring>
Type: Meta
Subclasses:
In [13]: A(0, 'y')
Instanciated: A
Ok - even though the reason for you to want that seems to be equivocated, as any "honest" Python inspecting tool should show the __init__ signature, what is needed for what you ask is that for each class you generate a dynamic metaclass, for which the __call__ method has the same signature of the class's own __init__ method.
For faking the __init__ signature on __call__ we can simply use functools.wraps. (but you might want to check the answers at
https://stackoverflow.com/a/33112180/108205 )
And for dynamically creating an extra metaclass, that can be done on the __metaclass__.__new__ itself, with just some care to avoud infinite recursion on the __new__ method - threads.Lock can help with that in a more consistent way than a simple global flag.
from functools import wraps
creation_locks = {}
class M(type):
def __new__(metacls, name, bases, namespace):
lock = creation_locks.setdefault(name, Lock())
if lock.locked():
return super().__new__(metacls, name, bases, namespace)
with lock:
def __call__(cls, *args, **kwargs):
return super().__call__(*args, **kwargs)
new_metacls = type(metacls.__name__ + "_sigfix", (metacls,), {"__call__": __call__})
cls = new_metacls(name, bases, namespace)
wraps(cls.__init__)(__call__)
del creation_locks[name]
return cls
I initially thought of using a named parameter to the metaclass __new__ argument to control recursion, but then it would be passed to the created class' __init_subclass__ method (which will result in an error) - so the Lock use.
Not sure if this helps the author but in my case I needed to change inspect.signature(Klass) to inspect.signature(Klass.__init__) to get signature of class __init__ instead of metaclass __call__.
I need to create a business query model, in which I need to create a circular dependency, I am using a look a like design of django models to implement it,
#Modeule a.py
import b
class A:
b_obj = B()
a_property_1 = ObjectAttribute(b_obj.b_property_1) # a_property_1 is dependent on b_property_1
a_property_2 = ObjectAttribute(b_obj.b_property_2)
#Module b.py
import a
class B:
a_obj = A()
b_property_1 = ObjectAttribute(a_obj.a_property_1)
b_property_2 = ObjectAttribute(a_obj.a_property_2)
When I execute the above program, it will throw an error, name 'B' is not defined on executing a.py and viceversa.
After that, I did a bit research on this to figure out and findout django models already implemented something like this via ForeignKey
https://docs.djangoproject.com/en/dev/ref/models/fields/#foreignkey
All I need to implement the my ForeignKey module, can some one please help me in understanding the logic and writing the code in below format.
#Modeule a.py
import b
class A:
b_obj = MyForeignKey('B')
a_property_1 = ObjectAttribute(b_obj.b_property_1) # a_property_1 is dependent on b_property_1
a_property_2 = ObjectAttribute(b_obj.b_property_2)
#Module b.py
import a
class B:
a_obj = MyForeignKey('A')
b_property_1 = ObjectAttribute(a_obj.a_property_1)
b_property_2 = ObjectAttribute(a_obj.a_property_2)
There are some ways to do that. One of which would be for your foreign Key to be made as proxy classes to the actuall classes, that on instantiating, just annotate the class model, and forhe next subsequent attribute access instantiate the proxied-to class, and keep its reference, Subsequent attributes would just be redirected to the underlying class.
One mechanism that allows such hooks to be executed on attribute fecth (remebering that in Pyhton a class "method" is just a callable attribute - so it works for methods as well), is to implement the __getattribute__ method.
Let's supose you have a "models" module (or other kind of registry) wher all your models are referenced, after creation -- your code could look more or less like this:
import models
class MyForeignKey(objec):
def __init__(self, model_name, *args, **kw):
self._model_name = model_name
self._args = args
self._kw = kw
def _instantiate(self):
self._object = getattr(models, self._model_name)(*self._args, **self._kw)
def __getattribute__(self, attr):
if attr in ("_model_name", "_args", "_kw", "_object", "_instantiate"):
return object.__getattribute__(self, attr)
if not hasattr(self, "_object"):
self._instantiate()
return getattr(self._object, attr)
def __setattr__(self, attr, value):
if attr in ("_model_name", "_args", "_kw", "_object"):
return object.__setattr__(self, attr, value)
if not hasattr(self, "_object"):
self._instantiate()
return setattr(self._object, attr, value)
Note that (a) your models have to inherit from "object" like I commented in the question and (b) - this is ot complete if you implement "dunder" methods (python double underscore methods) to override behavior on any of the models - in that case, you have to set the appropriate te dunder methods to do the proxying as well.
I'm wondering what the best way of accessing a parent variable from a nested subclass is, currently I'm using a decorator.
Is that the only/best way???
I don't want to have to directly access the parent variable (eg. ComponentModel.origin (see below)) as that would require more code in the "config" file, so I'm wondering whether I could assign parent variable in a class which the subclass in question inherits from?
Trivial example of my current solution:
# defined in a big library somewhere:
class LibrarySerialiser(object):
pass
# defined in my module:
class ModelBase:
pass
class SerialiserBase(LibrarySerialiser):
def __init__(self, *args, **kwargs):
# could i some how get hold of origin here without the decorator?
print self.origin
super(SerialiserBase, self).__init__(*args, **kwargs)
def setsubclasses(cls):
cls.Serialiser.origin = cls.origin
return cls
# written by "the user" for the particular application as the
# configuration of the module above:
#setsubclasses
class ComponentModel(ModelBase):
origin = 'supermarket'
class Serialiser(SerialiserBase):
pass
ser = ComponentModel.Serialiser()
This is obviously a trival example that misses all the real logic hence lots of the classes appear void but are really necessary.
FYI, the accepted terminology used when nesting classes as you've done is inner/outer, not parent/child or super/subclass. The parent/child or super/sub relationship refers to inheritance. This makes your decorator's name, setsubclasses, confusing, since there are no subclasses involved!
The unusual thing you're doing here is using the class as a namespace without instantiating it. Normally you would instantiate your ComponentModel and at that time, it is trivial to give your Serialiser inner class a copy of an attribute from its outer class. E.g.:
class ModelBase(object):
def __init__(self):
self.Serialiser.origin = self.origin
# ... then
cm = ComponentModel()
ser = cm.Serialiser()
Better yet, have the outer class instantiate the inner class and pass it a reference to the outer class; then it can grab any attributes it wants itself, whenever it needs them:
class ModelBase(object):
def __init__(self, *args, **kwargs):
serialiser = self.Serialiser(self, *args, **kwargs)
class SerialiserBase(LibrarySerialiser):
def __init__(self, outer, *args, **kwargs):
self.outer = outer
print self.outer.origin
super(SerialiserBase, self).__init__(*args, **kwargs)
# ...
cm = ComponentModel()
ser = cm.serialiser
However, if you insist on being able to get this attribute without instantiating the outer class, you can use a metaclass to set the attribute:
class PropagateOuter(type):
def __init__(cls, name, bases, dct):
type.__init__(cls, name, bases, dct)
if "Serialiser" in dct:
cls.Serialiser.outer = cls
class ModelBase(object):
__metaclass__ = PropagateOuter
# Python 3 version of the above
# class ModelBase(metaclass=PropagateOuter):
# pass
class SerialiserBase(LibrarySerialiser):
def __init__(self, *args, **kwargs):
print self.outer.origin
super(SerialiserBase, self).__init__(*args, **kwargs)
class ComponentModel(ModelBase):
origin = 'supermarket'
class Serialiser(SerialiserBase):
pass
ser = ComponentModel.Serialiser()
This isn't doing anything your decorator isn't, but the user gets it automatically through inheritance rather than having to specify it manually. The Zen of Python says "explicit is better than implicit" so tomato, tomato.
You could even write the metaclass so that it introspects the outer class and puts a reference to that class into every inner class regardless of their name.
By the way, one of the pitfalls of the way you're doing this is that all your model classes must subclass SerialiserBase. If a user of your class just wants the default serialiser, they can't just write Serialiser = SerialiserBase in their class definition, they must write class Serialiser(SerialiserBase): pass. This is because there's only one SerialiserBase and it obviously can't contain a reference to multiple outer classes. Of course, you could write your metaclass to deal with this (e.g. by automatically making a subclass of the specified serialiser if it already has an outer attribute).