I've written a Python 3 metaclass containing a nested metaclass (with abc), like:
class A_M(object, metaclass=abc.ABCMeta):
class A_nested_M(object, metaclass=abc.ABCMeta):
def ... # some methods
Now, implementing like
class A(A_M):
class A_nested(A_nested_M):
def ...
doesn't work. So, did i miss something about usage of metaclasses or is this type of implementation with nested metaclasses not working at all?
First thing:
Nesting class declarations is of near no use for anything in Python. Unless you are using the nested class hierarchy itself as a hard-coded namespace to keep attributes, you probably are doing the wrong thing already.
You did not tell what your (actual) problem is and what you are trying to achieve there, nor why you are using the ABCmeta metaclass. So it is hard to suggest any actually useful answers - but we can try clarifying some things:
First: you are not writting a metaclass, as you suggest in the text "I've written a Python 3 metaclass containing a nested metaclass..." - you are creating ordinary classes that have the ABCmeta as its metaclass. But you are not creating new metaclasses - You would if you were inheriting from type or from ABCMeta itself - them your new class would be used in the metaclass= parameter of subsequent (ordinary) classes. That is not the case.
Now, second, everything that is defined inside the body of your outermost A_M class will be only "visible" as attributes of A_M itself. That is the source of your error - when you try to inherit from A_nested_M you should actually write:
class A_M(object, metaclass=abc.ABCMeta):
class A_nested_M(object, metaclass=abc.ABCMeta):
def ... # some methods
class A(A_M):
class A_nested(A_M.A_nested_M):
def ...
See - A_M.A_nested_M will make Python find the superclass for A_nested: there is no reference in the local or global namespaces for A_nested_M as it only exists as an attribute of A_M outside the body of the class A_M... statement.
That said, this is still useless. If you want to have instances of A_nested referenced by instances of A class, you have to create these instances inside A.__init__() call - at which point it makes no difference if A_nested is declared inside a class body or at the module level:
class A_M(object, metaclass=abc.ABCMeta):
pass
class A_nested_M(object, metaclass=abc.ABCMeta):
def ... # some methods
class A_nested(A_nested_M):
...
class A(A_M):
def __init__(self):
self.nested = A_nested()
Now, that can be of some use. You can also declare the classes actually nested, but the only way they can be useful is by creating instances of them anyway. And unlike nested functions, nested classes do not have access to attributes or variables declared on the "nesting" class namespace (but for referring to them by their qualified name. I.e. in your example, if the A class would contain a b classmethod, a method inside A_nested that would call this method would have to call A.b(), not b())
You should implement your class like this:
class A(A_M):
class A_nested(A_M.A_nested_M):
def ...
Because A_nested_M is an inner class, you should access it just like you would access any of the class attributes, i.e. A_M.A_nested_M. See this link.
Related
Is it possible to get the the namespace parent, or encapsulating type, of a class?
class base:
class sub:
def __init__(self):
# self is "__main__.extra.sub"
# want to create object of type "__main__.extra" from this
pass
class extra(base):
class sub(base.sub):
pass
o = extra.sub()
The problem in base.sub.__init__ is getting extra from the extra.sub.
The only solutions I can think of at the moment involve having all subclasses of base provide some link to their encapsulating class type or turning the type of self in base.sub.__init__ into a string an manipulating it into a new type string. Both a bit ughly.
It's clearly possible to go the other way, type(self()).sub would give you extra.sub from inside base.sub.__init__ for a extra type object, but how do I do .. instead of .sub ? :)
The real answer is that there is no general way to do this. Python classes are normal objects, but they are created a bit differently. A class does not exist until well after its entire body has been executed. Once a class is created, it can be bound to many different names. The only reference it has to where it was created are the __module__ and __qualname__ attributes, but both of these are mutable.
In practice, it is possible to write your example like this:
class Sub:
def __init__(self):
pass
class Base:
Sub = Sub
Sub.__qualname__ = 'Base.Sub'
class Sub(Sub):
pass
class Extra(Base):
Sub = Sub
Sub.__qualname__ = 'Extra.Sub'
del Sub # Unlink from global namespace
Barring the capitalization, this behaves exactly as your original example. Hopefully this clarifies which code has access to what, and shows that the most robust way to determine the enclosing scope of a class is to explicitly assign it somewhere. You can do this in any number of ways. The trivial way is just to assign it. Going back to your original notation:
class Base:
class Sub:
def __init__(self):
print(self.enclosing)
Base.Sub.enclosing = Base
class Extra(Base):
class Sub(Base.Sub):
pass
Extra.Sub.enclosing = Extra
Notice that since Base does not exist when it body is being executed, the assignment has to happen after the classes are both created. You can bypass this by using a metaclass or a decorator. That will allow you to mess with the namespace before the class object is assigned to a name, making the change more transparent.
class NestedMeta(type):
def __init__(cls, name, bases, namespace):
for name, obj in namespace.items():
if isinstance(obj, type):
obj.enclosing = cls
class Base(metaclass=NestedMeta):
class Sub:
def __init__(self):
print(self.enclosing)
class Extra(Base):
class Sub(Base.Sub):
pass
But this is again somewhat unreliable because not all metaclasses are an instance of type, which takes us back to the first statement in this answer.
In many cases, you can use the __qualname__ and __module__ attributes to get the name of the surrounding class:
import sys
cls = type(o)
getattr(sys.modules[cls.__module__], '.'.join(cls.__qualname__.split('.')[:-1]))
This is a very literal answer to your question. It just shows one way of getting the class in the enclosing scope without addressing the probably design flaws that lead to this being necessary in the first place, or any of the many possible corner cases that this would not cover.
Could someone explain what is the difference between the following class definitions derived from BaseClass and in what cases it would matter how they are defined:
class BaseClass(object):
def __init__(self):
# ...
def as_dict(self):
# ...
class SomeClass(BaseClass):
def as_dict(self):
# Does this somehow change the method compared to 'AnotherClass.as_dict()' below
return super(SomeClass, self).as_dict()
class AnotherClass(BaseClass): pass
SomeOtherClass = BaseClass
Since you add an as_dict method to SomeClass that contains a single super call, there's no difference in the end behavior for that class. There's a difference in the fact that, overall, you've added a couple more function calls that are not necessary.
AnotherClass behaves just like SomeClass since SomeClass doesn't do anything different in as_dict. It inherits the methods of BaseClass as usual.
SomeOtherClass is simply another name for BaseClass, you aren't creating a subclassing relationship there, just attaching another name by which you can refer to that class.
This is using BaseClass with a different name.
class AnotherClass(BaseClass): pass
This is using BaseClass but modifying the method "as_dict". Inside the as_dict method, you can do anything (ie.modify the parameters sent to this method) then run the usual function of the as_dict method with super(SomeClass, self).as_dict()
class SomeClass(BaseClass):
def as_dict(self):
# Does this somehow change the method compared to 'AnotherClass.as_dict()' below
return super(SomeClass, self).as_dict()
This is simply assigning BaseClass to SomeOtherClass, which means they can use BaseClass through both keywords.
SomeOtherClass = BaseClass
In Python (2.7) one can use nested class declarations which is sometimes handy to organise local classes.
However, I can't figure out how to reference a class in a parent class so I can derive from it.
A minimal example is this:
class A(object):
class B(object):
pass
class C(object):
class D(A.B): # <-- fails with "NameError: name 'A' is not defined"
pass
How can I make class D derive from class B given the nested structure of the class declaration?
You can't. You can't reference it as A.B, because A is not yet defined (you are in the middle of the definition), and you can't reference it as B because as per PEP 227, names in class scope are not accessible:
Names in class scope are not accessible. Names are resolved in
the innermost enclosing function scope. If a class definition
occurs in a chain of nested scopes, the resolution process skips
class definitions. This rule prevents odd interactions between
class attributes and local variable access. If a name binding
operation occurs in a class definition, it creates an attribute on
the resulting class object. To access this variable in a method,
or in a function nested within a method, an attribute reference
must be used, either via self or via the class name.
An alternative would have been to allow name binding in class
scope to behave exactly like name binding in function scope. This
rule would allow class attributes to be referenced either via
attribute reference or simple name. This option was ruled out
because it would have been inconsistent with all other forms of
class and instance attribute access, which always use attribute
references. Code that used simple names would have been obscure.
That said, even if it was possible, this kind of definition looks really obscure and probably can be refactored into something simpler.
Edit: if you really, really want your class hierarchy look like this, you can just "monkey patch" A:
class A(object):
class B(object):
pass
class _C(object):
class D(A.B):
pass
A.C = _C
This is a question I tried to avoid several times, but I finally couldn't escape the subject on a recent project. I tried various solutions and decided to use one of them and would like to share it with you. Many solutions on internet simply don't work and I think it could help people not very fluent with classes and metaclasses.
I have hierarchy of classes, each with some class variables which I need to read when I instantiate objects. However, either these variables will be overwritten, or their name would be mangled if it has the form __variable. I can perfectly deal with the mangled variables, but I don't know, with an absolute certainty, which attribute I should look in the namespace of my object. Here are my definitions, including the class variables.
class BasicObject(object):
__attrs = 'size, quality'
...
class BasicDBObject(BasicObject):
__attrs = 'db, cursor'
...
class DbObject(BasicDBObject):
__attrs = 'base'
...
class Splits(DbObject):
__attrs = 'table'
...
I'd like to collect all values stored in __attrs of each class when Instantiate the Splits class. The method __init__() is only defined in the class BasicObject and nowhere else. Though, I need to scan self.__dict__ for mangled __attrs attributes. Since other attributes have the pattern attrs in these objects, I can't filter out the dictionary for everything with the pattern __attrs in it ! Therefore, I need to collect the class hierarchy for my object, and search for the mangled attributes for all these classes.
Hence, I will use a metaclass to catch each class which calls __new__() method which is being executed when a class definition is encountered when loading a module. By defining my own __new__() method in the base class, I'll be able to catch classes when each class is instantiated (instantiation of the class, not an object instantiation).
Here is the code :
import collections
class BasicObject(object) :
class __metaclass__(type) :
__parents__ = collections.defaultdict(list)
def __new__(cls, name, bases, dct) :
klass = type.__new__(cls, name, bases, dct)
mro = klass.mro()
for base in mro[1:-1] :
cls.__parents__[name] = mro[1]
return klass
def __init__(self, *args, **kargs) :
"""
Super class initializer.
"""
this_name = self.__class__.__name__
parents = self.__metaclass__.__parents__
hierarchy = [self.__class__]
while this_name in parents :
try :
father = parents[this_name]
this_name = father.__name__
hierarchy.append(father)
except :
break
print(hierarchy)
...
I could have access attributes using the class definition, but all these classes are defined in three different modules and the main one (init.py) doesn't know anything about the other modules.
This code works well in Python 2.7 and should also work in Python 3.. However, Python 3. have some new features which may help write a simpler code for this kind of introspection, but I haven't had the time to investigate it in Python 3.0.
I hope this short explanation and example will save some of your (precious) time :-)
Yes, the question is the answer; simply because I couldn't find anything other than the "Ask Question" button on the site. Did I miss something ?
I have a class sysprops in which I'd like to have a number of constants. However, I'd like to pull the values for those constants from the database, so I'd like some sort of hook any time one of these class constants are accessed (something like the getattribute method for instance variables).
class sysprops(object):
SOME_CONSTANT = 'SOME_VALUE'
sysprops.SOME_CONSTANT # this statement would not return 'SOME_VALUE' but instead a dynamic value pulled from the database.
Although I think it is a very bad idea to do this, it is possible:
class GetAttributeMetaClass(type):
def __getattribute__(self, key):
print 'Getting attribute', key
class sysprops(object):
__metaclass__ = GetAttributeMetaClass
While the other two answers have a valid method. I like to take the route of 'least-magic'.
You can do something similar to the metaclass approach without actually using them. Simply by using a decorator.
def instancer(cls):
return cls()
#instancer
class SysProps(object):
def __getattribute__(self, key):
return key # dummy
This will create an instance of SysProps and then assign it back to the SysProps name. Effectively shadowing the actual class definition and allowing a constant instance.
Since decorators are more common in Python I find this way easier to grasp for other people that have to read your code.
sysprops.SOME_CONSTANT can be the return value of a function if SOME_CONSTANT were a property defined on type(sysprops).
In other words, what you are talking about is commonly done if sysprops were an instance instead of a class.
But here is the kicker -- classes are instances of metaclasses. So everything you know about controlling the behavior of instances through the use of classes applies equally well to controlling the behavior of classes through the use of metaclasses.
Usually the metaclass is type, but you are free to define other metaclasses by subclassing type. If you place a property SOME_CONSTANT in the metaclass, then the instance of that metaclass, e.g. sysprops will have the desired behavior when Python evaluates sysprops.SOME_CONSTANT.
class MetaSysProps(type):
#property
def SOME_CONSTANT(cls):
return 'SOME_VALUE'
class SysProps(object):
__metaclass__ = MetaSysProps
print(SysProps.SOME_CONSTANT)
yields
SOME_VALUE