In Python (2.7) one can use nested class declarations which is sometimes handy to organise local classes.
However, I can't figure out how to reference a class in a parent class so I can derive from it.
A minimal example is this:
class A(object):
class B(object):
pass
class C(object):
class D(A.B): # <-- fails with "NameError: name 'A' is not defined"
pass
How can I make class D derive from class B given the nested structure of the class declaration?
You can't. You can't reference it as A.B, because A is not yet defined (you are in the middle of the definition), and you can't reference it as B because as per PEP 227, names in class scope are not accessible:
Names in class scope are not accessible. Names are resolved in
the innermost enclosing function scope. If a class definition
occurs in a chain of nested scopes, the resolution process skips
class definitions. This rule prevents odd interactions between
class attributes and local variable access. If a name binding
operation occurs in a class definition, it creates an attribute on
the resulting class object. To access this variable in a method,
or in a function nested within a method, an attribute reference
must be used, either via self or via the class name.
An alternative would have been to allow name binding in class
scope to behave exactly like name binding in function scope. This
rule would allow class attributes to be referenced either via
attribute reference or simple name. This option was ruled out
because it would have been inconsistent with all other forms of
class and instance attribute access, which always use attribute
references. Code that used simple names would have been obscure.
That said, even if it was possible, this kind of definition looks really obscure and probably can be refactored into something simpler.
Edit: if you really, really want your class hierarchy look like this, you can just "monkey patch" A:
class A(object):
class B(object):
pass
class _C(object):
class D(A.B):
pass
A.C = _C
Related
I have a class with a private constant _BAR = object().
In a child class, outside of a method (no access to self), I want to refer to _BAR.
Here is a contrived example:
class Foo:
_BAR = object()
def __init__(self, bar: object = _BAR):
...
class DFoo(Foo):
"""Child class where I want to access private class variable from parent."""
def __init__(self, baz: object = super()._BAR):
super().__init__(baz)
Unfortunately, this doesn't work. One gets an error: RuntimeError: super(): no arguments
Is there a way to use super outside of a method to get a parent class attribute?
The workaround is to use Foo._BAR, I am wondering though if one can use super to solve this problem.
Inside of DFoo, you cannot refer to Foo._BAR without referring to Foo. Python variables are searched in the local, enclosing, global and built-in scopes (and in this order, it is the so called LEGB rule) and _BAR is not present in any of them.
Let's ignore an explicit Foo._BAR.
Further, it gets inherited: DFoo._BAR will be looked up first in DFoo, and when not found, in Foo.
What other means are there to get the Foo reference? Foo is a base class of DFoo. Can we use this relationship? Yes and no. Yes at execution time and no at definition time.
The problem is when the DFoo is being defined, it does not exist yet. We have no start point to start following the inheritance chain. This rules out an indirect reference (DFoo -> Foo) in a def method(self, ....): line and in a class attribute _DBAR = _BAR.
It is possible to work around this limitation using a class decorator. Define the class and then modify it:
def deco(cls):
cls._BAR = cls.__mro__[1]._BAR * 2 # __mro__[0] is the class itself
return cls
class Foo:
_BAR = 10
#deco
class DFoo(Foo):
pass
print(Foo._BAR, DFoo._BAR) # 10 20
Similar effect can be achieved with a metaclass.
The last option to get a reference to Foo is at execution time. We have the object self, its type is DFoo, and its parent type is Foo and there exists the _BAR. The well known super() is a shortcut to get the parent.
I have assumed only one base class for simplicity. If there were several base classes, super() returns only one of them. The example class decorator does the same. To understand how several bases are sorted to a sequence, see how the MRO works (Method Resolution Order).
My final thought is that I could not think up a use-case where such access as in the question would be required.
Short answer: you can't !
I'm not going into much details about super class itself here. (I've written a pure Python implementation in this gist if you like to read.)
But now let's see how we can call super:
1- Without arguments:
From PEP 3135:
This PEP proposes syntactic sugar for use of the super type to
automatically construct instances of the super type binding to the
class that a method was defined in, and the instance (or class object
for classmethods) that the method is currently acting upon.
The new syntax:
super()
is equivalent to:
super(__class__, <firstarg>)
...and <firstarg> is the first parameter of the method
So this is not an option because you don't have access to the "instance".
(Body of the function/methods is not executed unless it gets called, so no problem if DFoo doesn't exist yet inside the method definition)
2- super(type, instance)
From documentation:
The zero argument form only works inside a class definition, as the
compiler fills in the necessary details to correctly retrieve the
class being defined, as well as accessing the current instance for
ordinary methods.
What were those necessary details mentioned above? A "type" and A "instance":
We can't pass neither "instance" nor "type" which is DFoo here. The first one is because it's not inside the method so we don't have access to instance(self). Second one is DFoo itself. By the time the body of the DFoo class is being executed there is no reference to DFoo, it doesn't exist yet. The body of the class is executed inside a namespace which is a dictionary. After that a new instance of type type which is here named DFoo is created using that populated dictionary and added to the global namespaces. That's what class keyword roughly does in its simple form.
3- super(type, type):
If the second argument is a type, issubclass(type2, type) must be
true
Same reason mentioned in above about accessing the DFoo.
4- super(type):
If the second argument is omitted, the super object returned is
unbound.
If you have an unbound super object you can't do lookup(unless for the super object's attributes itself). Remember super() object is a descriptor. You can turn an unbound object to a bound object by calling __get__ and passing the instance:
class A:
a = 1
class B(A):
pass
class C(B):
sup = super(B)
try:
sup.a
except AttributeError as e:
print(e) # 'super' object has no attribute 'a'
obj = C()
print(obj.sup.a) # 1
obj.sup automatically calls the __get__.
And again same reason about accessing DFoo type mentioned above, nothing changed. Just added for records. These are the ways how we can call super.
Consider the following class
class Foo(object):
#staticmethod
def __is():
print('__is')
def m(self):
Foo.__is() # executes
Foo.__is() # fails because of mangling
print(Foo.__dict__.keys())
Foo.__is() when it is run after the class is defined, it fails because of name mangaling. How is python interpreter able to resolve Foo.__is() inside methods but not outside the class?
Name mangling for names starting with __ inside a class, is implemented by rewriting this names to the mangled form, inside the class ONLY. So your Foo.__is inside the class gets replaced by _Foo__is that is now present in the class __dict__. That attribute is accessible either inside or outside the class, so no private protection. But after replacement the __is name does not exist anywhere (I think), that's why it does not work from outside.
From the Python help:
"__*"
Class-private names. Names in this category, when used within the
context of a class definition, are re-written to use a mangled form
to help avoid name clashes between “private” attributes of base and
derived classes. See section Identifiers (Names).
Also see my comment to errors in your code snippet.
You can think of classes as of namespaces with some whistles.
One of whistles is ability to resolve __methods inside themselves without _class__method. This is no different from "magic" of automatically passing instances as first argument to methods (your regular self)
To illustrate "namespace" idea, you can do some weird things like
class A:
def insane():
return 42
for _ in range(5):
print(insane()
I've written a Python 3 metaclass containing a nested metaclass (with abc), like:
class A_M(object, metaclass=abc.ABCMeta):
class A_nested_M(object, metaclass=abc.ABCMeta):
def ... # some methods
Now, implementing like
class A(A_M):
class A_nested(A_nested_M):
def ...
doesn't work. So, did i miss something about usage of metaclasses or is this type of implementation with nested metaclasses not working at all?
First thing:
Nesting class declarations is of near no use for anything in Python. Unless you are using the nested class hierarchy itself as a hard-coded namespace to keep attributes, you probably are doing the wrong thing already.
You did not tell what your (actual) problem is and what you are trying to achieve there, nor why you are using the ABCmeta metaclass. So it is hard to suggest any actually useful answers - but we can try clarifying some things:
First: you are not writting a metaclass, as you suggest in the text "I've written a Python 3 metaclass containing a nested metaclass..." - you are creating ordinary classes that have the ABCmeta as its metaclass. But you are not creating new metaclasses - You would if you were inheriting from type or from ABCMeta itself - them your new class would be used in the metaclass= parameter of subsequent (ordinary) classes. That is not the case.
Now, second, everything that is defined inside the body of your outermost A_M class will be only "visible" as attributes of A_M itself. That is the source of your error - when you try to inherit from A_nested_M you should actually write:
class A_M(object, metaclass=abc.ABCMeta):
class A_nested_M(object, metaclass=abc.ABCMeta):
def ... # some methods
class A(A_M):
class A_nested(A_M.A_nested_M):
def ...
See - A_M.A_nested_M will make Python find the superclass for A_nested: there is no reference in the local or global namespaces for A_nested_M as it only exists as an attribute of A_M outside the body of the class A_M... statement.
That said, this is still useless. If you want to have instances of A_nested referenced by instances of A class, you have to create these instances inside A.__init__() call - at which point it makes no difference if A_nested is declared inside a class body or at the module level:
class A_M(object, metaclass=abc.ABCMeta):
pass
class A_nested_M(object, metaclass=abc.ABCMeta):
def ... # some methods
class A_nested(A_nested_M):
...
class A(A_M):
def __init__(self):
self.nested = A_nested()
Now, that can be of some use. You can also declare the classes actually nested, but the only way they can be useful is by creating instances of them anyway. And unlike nested functions, nested classes do not have access to attributes or variables declared on the "nesting" class namespace (but for referring to them by their qualified name. I.e. in your example, if the A class would contain a b classmethod, a method inside A_nested that would call this method would have to call A.b(), not b())
You should implement your class like this:
class A(A_M):
class A_nested(A_M.A_nested_M):
def ...
Because A_nested_M is an inner class, you should access it just like you would access any of the class attributes, i.e. A_M.A_nested_M. See this link.
I need to access a variable from a nested class. The objective is to create a Schema using the Marshmallow library. The code looks like this :
class ParserScheme(Schema):
class Meta:
# Here I meed to access the value of my_variable :
result = func(my_variable)
my_variable = 'foo'
my_parser = ParserScheme()
If I manage to pass my variable as a class attribute of the outer class (ParserScheme), then it is easy to get it into the inner class (Meta).
class ParserScheme(Schema):
class_attribute = my_variable
class Meta:
result = func(ParserScheme.class_attribute)
my_variable = 'foo'
my_parser = ParserScheme()
However I can't seem to find a way to dynamically set a class attribute. And if I set a "classic" attribute (I mean an attribute of instances of the class, not of the class itself), then I can't access it from the inner class.
I also thought of using a global variable, but that does not really quite satisfy me. Is there another way of doing this ?
I am rather new to OOP, and I am not sure I understand really well the concept of class attribute. I fear that there is an easy way to do that but I did not see it as I am to focused on the way I think this should work...
Your first example breaks because name my_variable is not yet defined when the class Meta statement's body is executed.
You second example won't work either for the same reason (my_variable is not yet defined when the class ParserScheme statement's body is executed), and if it was it would still break when executing the class Meta statement's body because it will be executed as part of the class ParserScheme statement's body hence before the name ParserScheme is defined.
What you have to understand here is that class and def are executable statements which (if at the top level of a module) are executed sequentially when the module is first imported into the current process. In the case of a class statement, the statement's body is first sequentially executed in a dedicated namespace then this namespace is passed to the metaclass constructor and used to create the class object's attributes (YourClass.__dict__).
To make a long story short: in a class statement body, you just cannot reference names that are not yet defined in the current or enclosing scope. Period.
The obvious solution here would be to define my_variable before the class statement, but I assume you want it to be more dynamic ? If yes, you'll have to define your class in a function:
def create_parser_class(my_variable):
class ParserScheme(Schema):
class Meta:
result = func(my_variable)
return ParserScheme
my_variable = 42
cls = create_parser_class(my_variable)
my_parser = cls()
But I can't garantee it will work out of the box (nor even work at all FWIW) with Marshmallow (which I never used but probably has some metaclass stuff happening). Also depending on what my_variable is used for, you may want to make sure you don't have two calls to create_parser_class with the same value as argument.
As a last note : you perhaps have a XY problem here - that's sometimes the case when someone asks how to do something a bit non-obvious or unusual. Perhaps you should edit your post to explain the "problem behind" - that is, the problem you are actually trying to solve with this "solution".
Oh and yes:
I am rather new to OOP, and I am not sure I understand really well the concept of class attribute
In Python, classes are objects too (instances of their metaclass, by default the type object), and as such they have their own attributes. Every name you define (with an assignment, a def statement, a class statement or an import statement) at the top-level of the class statement becomes an attribute of the class object (unless a custom metaclass makes some transformations on the way, that is).
Class attributes are accessible from the instances too (unless shadowed by an eponym instance variable), and are shared between all instances.
According to my understanding, the first argument passed in a class method is the class itself where that class method is defined. So for example, consider the following code:
class A(object):
__x=10
#classmethod
def clam(cls,*args):
print(cls.__x)
class B(A):
__x=50
And when i called :
B.clam()
the output was 10 which is OK as per my understanding because the class method being called is defined in class A, so class A will be passed implicitly to clam() and the value of __x there is 10.
But when i ran the following code:
class A(object):
x=10
#classmethod
def clam(cls,*args):
print(cls.x)
class B(A):
x=50
and when i called:
B.clam()
My life was suddenly ruined. The output is 50.
The only difference between both the cases is that x was private in former one.
What happened exactly? Why the output from the later one is 50 ? Was there any scope change or suddenly the first parameter passed to class method defined in A became class B ?
The first parameter in the class method is the class on which you are calling the method, not (necessarily) the class that defines the method. (Having a variable that always holds the same class would probably not be that useful.)
In the first case, name mangling distinguishes the two fields, to protect you from accidentally shadowing private variables in a subclass. The field A.__x becomes A._A__x and B.__x becomes B._B__x. This ensures that you can't accidentally pick up a field in a subclass with a similar name to your private field in A. That's part of the reason name mangling exists.
In the second case, there is no name mangling: you get the x field as defined in B, the class you are calling the method on.