I have a class with a private constant _BAR = object().
In a child class, outside of a method (no access to self), I want to refer to _BAR.
Here is a contrived example:
class Foo:
_BAR = object()
def __init__(self, bar: object = _BAR):
...
class DFoo(Foo):
"""Child class where I want to access private class variable from parent."""
def __init__(self, baz: object = super()._BAR):
super().__init__(baz)
Unfortunately, this doesn't work. One gets an error: RuntimeError: super(): no arguments
Is there a way to use super outside of a method to get a parent class attribute?
The workaround is to use Foo._BAR, I am wondering though if one can use super to solve this problem.
Inside of DFoo, you cannot refer to Foo._BAR without referring to Foo. Python variables are searched in the local, enclosing, global and built-in scopes (and in this order, it is the so called LEGB rule) and _BAR is not present in any of them.
Let's ignore an explicit Foo._BAR.
Further, it gets inherited: DFoo._BAR will be looked up first in DFoo, and when not found, in Foo.
What other means are there to get the Foo reference? Foo is a base class of DFoo. Can we use this relationship? Yes and no. Yes at execution time and no at definition time.
The problem is when the DFoo is being defined, it does not exist yet. We have no start point to start following the inheritance chain. This rules out an indirect reference (DFoo -> Foo) in a def method(self, ....): line and in a class attribute _DBAR = _BAR.
It is possible to work around this limitation using a class decorator. Define the class and then modify it:
def deco(cls):
cls._BAR = cls.__mro__[1]._BAR * 2 # __mro__[0] is the class itself
return cls
class Foo:
_BAR = 10
#deco
class DFoo(Foo):
pass
print(Foo._BAR, DFoo._BAR) # 10 20
Similar effect can be achieved with a metaclass.
The last option to get a reference to Foo is at execution time. We have the object self, its type is DFoo, and its parent type is Foo and there exists the _BAR. The well known super() is a shortcut to get the parent.
I have assumed only one base class for simplicity. If there were several base classes, super() returns only one of them. The example class decorator does the same. To understand how several bases are sorted to a sequence, see how the MRO works (Method Resolution Order).
My final thought is that I could not think up a use-case where such access as in the question would be required.
Short answer: you can't !
I'm not going into much details about super class itself here. (I've written a pure Python implementation in this gist if you like to read.)
But now let's see how we can call super:
1- Without arguments:
From PEP 3135:
This PEP proposes syntactic sugar for use of the super type to
automatically construct instances of the super type binding to the
class that a method was defined in, and the instance (or class object
for classmethods) that the method is currently acting upon.
The new syntax:
super()
is equivalent to:
super(__class__, <firstarg>)
...and <firstarg> is the first parameter of the method
So this is not an option because you don't have access to the "instance".
(Body of the function/methods is not executed unless it gets called, so no problem if DFoo doesn't exist yet inside the method definition)
2- super(type, instance)
From documentation:
The zero argument form only works inside a class definition, as the
compiler fills in the necessary details to correctly retrieve the
class being defined, as well as accessing the current instance for
ordinary methods.
What were those necessary details mentioned above? A "type" and A "instance":
We can't pass neither "instance" nor "type" which is DFoo here. The first one is because it's not inside the method so we don't have access to instance(self). Second one is DFoo itself. By the time the body of the DFoo class is being executed there is no reference to DFoo, it doesn't exist yet. The body of the class is executed inside a namespace which is a dictionary. After that a new instance of type type which is here named DFoo is created using that populated dictionary and added to the global namespaces. That's what class keyword roughly does in its simple form.
3- super(type, type):
If the second argument is a type, issubclass(type2, type) must be
true
Same reason mentioned in above about accessing the DFoo.
4- super(type):
If the second argument is omitted, the super object returned is
unbound.
If you have an unbound super object you can't do lookup(unless for the super object's attributes itself). Remember super() object is a descriptor. You can turn an unbound object to a bound object by calling __get__ and passing the instance:
class A:
a = 1
class B(A):
pass
class C(B):
sup = super(B)
try:
sup.a
except AttributeError as e:
print(e) # 'super' object has no attribute 'a'
obj = C()
print(obj.sup.a) # 1
obj.sup automatically calls the __get__.
And again same reason about accessing DFoo type mentioned above, nothing changed. Just added for records. These are the ways how we can call super.
Related
I have a problem understanding some concepts of data structures in Python, in the following code.
class Stack(object): #1
def __init__(self): #2
self.items=[]
def isEmpty(self):
return self.items ==[]
def push(self,item):
self.items.append(item)
def pop(self):
self.items.pop()
def peak(self):
return self.items[len(self.items)-1]
def size(self):
return len(self.items)
s = Stack()
s.push(3)
s.push(7)
print(s.peak())
print (s.size())
s.pop()
print (s.size())
print (s.isEmpty())
I don't understand what is this object argument
I replaced it with (obj) and it generated an error, why?
I tried to remove it and it worked perfectly, why?
Why do I have __init__ to set a constructor?
self is an argument, but how does it get passed? and which object does it represent, the class it self?
Thanks.
object is a class, from which class Stack inherits. There is no
class obj, hence error. However, you can define a class that does
not inherit from anything (at least, in Python 2).
self represents an object on which the method is called; for
example when you do s.pop(), self inside method pop refers to
the same object as s - it is not a class, it is an instance of the class.
1
object here is the class your new class inherits from. There is already a base class named object, but there is no class named obj which is why replacing object with obj would cause an error. Anyway in your example code it is not needed at all since all classes in python 3 implicitly extends the object class.
2
__init__ is the constructor of the object and self there represents the object that you are creating itself, not the class, just like in the other methods you made.
Point 1:
Some history required here... Originally Python had two distinct kind of types, those implemented in C (whether in the stdlib or C extensions) and those implemented in Python with the class statement. Python 2.2 introduced a new object model (known as "new-style classes") to unify both, but kept the "classic" (aka "old-style") model for compatibility. This new model also introduced quite a lot of goodies like support for computed attributes, cooperative super calls via the super() object, metaclasses etc, all of which coming from the builtin object base class.
So in Python 2.2.x to 2.7.x, you can either create a new-style class by inheriting from object (or any subclass of object) or an old-style one by not inheriting from object (nor - obviously - any subclass of object).
In Python 2.7., since your example Stack class does not use any feature of the new object model, it works as well as an 'old-style' or as a 'new-style' class, but try to add a custom metaclass or a computed attribute and it will break in one way or another.
Python 3 totally removed old-style classes support and object is the defaut base class if you dont explicitely specify one, so whatever you do your class WILL inherit from object and will work as well with or without explicit parent class.
You can read this for more details.
Point 2.1 - I'm not sure I understand the question actually, but anyway:
In Python, objects are not fixed C-struct-like structures with a fixed set of attributes, but dict-like mappings (well there are exceptions but let's ignore them for the moment). The set of attributes of an object is composed of the class attributes (methods mainly but really any name defined at the class level) that are shared between all instances of the class, and instance attributes (belonging to a single instance) which are stored in the instance's __dict__. This imply that you dont define the instance attributes set at the class level (like in Java or C++ etc), but set them on the instance itself.
The __init__ method is there so you can make sure each instance is initialised with the desired set of attributes. It's kind of an equivalent of a Java constructor, but instead of being only used to pass arguments at instanciation, it's also responsible for defining the set of instance attributes for your class (which you would, in Java, define at the class level).
Point 2.2 : self is the current instance of the class (the instance on which the method is called), so if s is an instance of your Stack class, s.push(42) is equivalent to Stack.push(s, 42).
Note that the argument doesn't have to be called self (which is only a convention, albeit a very strong one), the important part is that it's the first argument.
How s get passed as self when calling s.push(42) is a bit intricate at first but an interesting example of how to use a small feature set to build a larger one. You can find a detailed explanation of the whole mechanism here, so I wont bother reposting it here.
This question already has answers here:
How does `super` interacts with a class's `__mro__` attribute in multiple inheritance?
(2 answers)
Closed 4 years ago.
From Python3's documentation super() "returns a proxy object that delegates method calls to a parent or sibling class of type." What does that mean?
Suppose I have the following code:
class SuperClass():
def __init__(self):
print("__init__ from SuperClass.")
print("self object id from SuperClass: " + str(id(self)))
class SubClass(SuperClass):
def __init__(self):
print("__init__ from SubClass.")
print("self object id from SubClass: " + str(id(self)))
super().__init__()
sc = SubClass()
The output I get from this is:
__init__ from SubClass.
self object id from SubClass: 140690611849200
__init__ from SuperClass.
self object id from SuperClass: 140690611849200
This means that in the line super().__init__(), super() is returning the current object which is then implicitly passed to the superclass' __init__() method. Is this accurate or am I missing something here?
To put it simply, I want to understand the following:
When super().__init__() is run,
What exactly is being passed to __init__() and how? We are calling it on super() so whatever this is returning should be getting passed to the __init__() method from what I understand about Python so far.
Why don't we have to pass in self to super().__init__()?
returns a proxy object that delegates method calls to a parent or
sibling class of type.
This proxy is an object that acts as the method-calling portion of the parent class. It is not the class itself; rather, it's just enough information so that you can use it to call the parent class methods.
If you call __init__(), you get your own, local, sub-class __init__ function. When you call super(), you get that proxy object, which will redirect you to the parent-class methods. Thus, when you call super().__init__(), that proxy redirects the call to the parent-class __init__ method.
Similarly, if you were to call super().foo, you would get the foo method from the parent class -- again, re-routed by that proxy.
Is that clear to you?
Responses to OP comments
But that must mean that this proxy object is being passed to
__init__() when running super().__init__() right?
Wrong. The proxy object is like a package name, such as calling math.sqrt(). You're not passing math to sqrt, you're using it to denote which sqrt you're using. If you wanted to pass the proxy to __init__, the call would be __init__(super()). That call would be semantically ridiculous, of course.
When we have to actually pass in self which is the sc object in my example.
No, you are not passing in sc; that is the result of the object creation call (internal method __new__), which includes an invocation of init. For __init__, the self object is a new item created for you by the Python run-time system. For most class methods, that first argument (called self out of convention, this in other languages) is the object that invoked the method.
This means that in the line super().__init__(), super() is returning the current object which is then implicitly passed to the superclass' __init__() method. Is this accurate or am I missing something here?
>>> help(super)
super() -> same as super(__class__, <first argument>)
super call returns a proxy/wrapper object which remembers:
The instance invoking super()
The class of the calling object
The class that's invoking super()
This is perfectly sound. super always fetches the attribute of the next class in the hierarchy ( really the MRO) that has the attribute that you're looking for. So it's not returning the current object, but rather and more accurately, it returns an object that remembers enough information to search for attributes higher in the class hierarchy.
What exactly is being passed to __init__() and how? We are calling it on super() so whatever this is returning should be getting passed to the __init__() method from what I understand about Python so far.
You're almost right. But super loves to play tricks on us. super class defines __getattribute__, this method is responsible for attribute search. When you do something like: super().y(), super.__getattribute__ gets called searching for y. Once it finds y it passes the instance that's invoking the super call to y. Also, super has __get__ method, which makes it a descriptor, I'll omit the details of descriptors here, refer to the documentation to know more. This answers your second question as well, as to why self isn't passed explicitly.
*Note: super is a little bit different and relies on some magic. Almost for all other classes, the behavior is the same. That is:
a = A() # A is a class
a.y() # same as A.y(a), self is a
But super is different:
class A:
def y(self):
return self
class B(A):
def y(self)
return super().y() # equivalent to: A.y(self)
b = B()
b.y() is b # True: returns b not super(), self is b not super()
I wrote a simple test to investigate what CPython does for super:
class A:
pass
class B(A):
def f(self):
return super()
#classmethod
def g(cls):
return super()
def h(selfish):
selfish = B()
return super()
class C(B):
pass
c = C()
for method in 'fgh':
super_object = getattr(c, method)()
print(super_object, super_object.__self__, super_object.__self_class__, super_object.__thisclass__) # (These methods were found using dir.)
The zero-argument super call returns an object that stores three things:
__self__ stores the object whose name matches the first parameter of the method—even if that name has been reassigned.
__self_class__ stores its type, or itself in the case of a class method.
__thisclass__ stores the class in which the method is defined.
(It is unfortunate that __thisclass__ was implemented this way rather than fetching an attribute on the method because it makes it impossible to use the zero-argument form of super with meta-programming.)
The object returned by super implements getattribute, which forwards method calls to the type found in the __mro__ of __self_class__ one step after __thisclass__.
I was searching for the meaning of default parameters object,self that are present as default class and function parameters, so moving away from it, if we are calling an attribute of a class should we use Foo (class reference) or should we use Foo() (instance of the class).
If you are reading a normal attribute then it doesn't matter. If you are binding a normal attribute then you must use the correct one in order for the code to work. If you are accessing a descriptor then you must use an instance.
The details of python's class semantics are quite well documented in the data model. Especially the __get__ semantics are at work here. Instances basically stack their namespace on top of their class' namespace and add some boilerplate for calling methods.
There are some large "it depends on what you are doing" gotchas at work here. The most important question: do you want to access class or instance attributes? Second, do you want attribute or methods?
Let's take this example:
class Foo(object):
bar = 1
baz = 2
def __init__(self, foobar="barfoo", baz=3):
self.foobar = foobar
self.baz = baz
def meth(self, param):
print self, param
#classmethod
def clsmeth(cls, param):
print cls, param
#staticmethod
def stcmeth(param):
print param
Here, bar is a class attribute, so you can get it via Foo.bar. Since instances have implicit access to their class namespace, you can also get it as Foo().bar. foobar is an instance attribute, since it is never bound to the class (only instances, i.e. selfs) - you can only get it as Foo().foobar. Last, baz is both a class and an instance attribute. By default, Foo.baz == 2 and Foo().baz == 3, since the class attribute is hidden by the instance attribute set in __init__.
Similarly, in an assignment there are slight differences whether you work on the class or an instance. Foo.bar=2 will set the class attribute (also for all instances) while Foo().bar=2 will create an instance attribute that shadows the class attribute for this specific instance.
For methods, it is somewhat similar. However, here you get the implicit self parameter for instance method (what a function is if defined for a class). Basically, the call Foo().meth(param=x) is silently translated to Foo.meth(self=Foo(), param=x). This is why it is usually not valid to call Foo.meth(param=x) - meth is not "bound" to an instance and thus lacks the self parameter.
Now, sometimes you do not need any instance data in a method - for example, you have strict string transformation that is an implementation detail of a larger parser class. This is where #classmethod and #staticmethod come into play. A classmethod's first parameter is always the class, as opposed to the instance for regular methods. Foo().clsmeth(param=x) and Foo.clsmeth(param=x) result in a call of clsmethod(cls=Foo, param=x). Here, the two are equivalent. Going one step further, a staticmethod doesn't get any class or instance information - it is like a raw function bound to the classes namespace.
What is the difference between these two class declarations? What does "object" do?
class className(object):
pass
class className:
pass
Why do I get this error when I run the below code: "Takes no arguments (1 given)"
class Hobbs():
def represent():
print "Hobbs represent!"
represent = classmethod(represent)
Hobbs.represent()
Why does "Foo.class_foo()" give no error even though I did not pass an argument to the function.
class Foo(object):
#staticmethod
def static_foo():
print "static method"
#classmethod
def class_foo(cls):
print "Class method. Automatically passed the class: %s" % cls
Foo.static_foo()
Foo.class_foo()
Why do I get this error when I run the below code?
class Foo(object):
def static_foo():
print "static method"
static_foo = staticmethod(static_foo)
def class_foo(cls):
print "Class method. Automatically passed the class: %s" % cls
class_foo = classmethod(class_foo)
Foo.static_foo()
Foo.class_foo()
"TypeError: unbound method static_foo() must be called with Foo
instance as first argument (got nothing instead)"
Using object as the base class for new classes has been convention since at least Python 2.2, and is called "New-Style Classes" - see this question for more details. Old style classes (i.e.: ones that don't inherit from object) are set to be deprecated in Python 3.0. The reasons for these changes are somewhat obscure, and have to do with low-level class resolution and inheritance patterns.
Python instance methods, by convention, take self as their first argument. This argument is passed implicitly - so if your method definition doesn't take self, then the interpreter will complain that the method you're trying to call doesn't accept the argument that's being automatically passed to it. This works exactly the same for classmethods, only instead of taking self, they usually take cls. (Just a naming convention.) A quick fix:
class Hobbs():
def represent(cls):
print "Hobbs represent!"
represent = classmethod(represent)
Hobbs.represent()
Calling Foo.class_foo() doesn't cause any issues, as Python automatically passes the class object to the class_foo method whenever you call it. These methods are called bound methods - meaning that they are regular functions, but bound to a class or instance object. Bound methods automatically take the class or instance object that they're bound to as their first argument.
Indentation level matters in Python. I've tried executing the code sample you've provided, but both the static_foo = and class_foo = lines must be within the Foo class definition, rather than below it or within other methods. When indented properly, the code runs fine:
class Foo(object):
def static_foo():
print "static method"
static_foo = staticmethod(static_foo)
def class_foo(cls):
print "Class method. Automatically passed the class: %s" % cls
class_foo = classmethod(class_foo)
Foo.static_foo()
Foo.class_foo()
The last two are identical - empty brackets is the same as omitting them. The first inherits from the builtin class object, making it a "new style class". The reason for new and old style classes is historical, and old-style are only kept around for backward compatibility - essentially, in Python 2, the advice is to always inherit from object if you don't inherit from anything else, because some of the fancy tricks you will learn eventually rely on it. If you upgrade to Python 3, this becomes the default behaviour, and all three class declarations are equivalent.
A classmethod needs to take a first argument similar to self - when you call Hobbs.represent(), Python end up passing Hobbs in as that first argument. This is the fundamental difference between classmethod and staticmethod - a classmethod takes a first argument (being the class it was called on), a staticmethod doesn't.
Same as 2 - class is passed in to the classmethod in place of the usual self.
This one appears to be an indentation issue - your code works as written if it is indented as:
def static_foo():
print "static method"
static_foo = staticmethod(staticfoo)
but not as
def static_foo():
print "static method"
static_foo = staticmethod(staticfoo)
Because the line reassigning static_foo needs to be in the class body, not part of the function itself. In the latter, that line isn't executed until the function is run (which means it isn't run, since the function errors) - and it assigns a staticmethod to a local variable rather than to the method itself. This type of error is one of the reasons it is good to use the decorator syntax:
class Hobbs:
#staticmethod
def static_foo():
print "static method"
works.
all class functions must take self as the first argument
class A:
def my_func(self):
print "In my func"
static methods are classes that are pretty much just a function in a namespace (and are rarely used in python)
class methods are functions in the class namespace that should be called on the class itself rather than an instance
Most of your questions aren't really about object orientation per se, but Python's specific implementation of it.
Python 2.x has undergone some evolution, with new features being added. So there are two ways to define classes, resulting in a "new-style class", and and "old-style class". Python 3.x has only "new style class".
The base object of new-style classes is called object. If you inherit from that you have a new-style class. It gives you some of the extra features such as certain decorators. If you have a bare (no inheritance) definition in Python 2.x you have an old-style class. This exists for backwards compatibility. In Python 3.x you will also get a new-style class (so inheriting from object is optional there).
You have made represent() and "class method". So it will get the class object as implicit first argument when it is called. But those only work with new-style classes. you have tried to use it with an old-style class. So it won't work.
Python automatically inserts the class object as argument zero for a class method. So that is the correct pattern and it works.
the method somehow didn't get make into a class method, maybe because the indentation is wrong.
class ClassName(OtherClass): means that ClassName inherits from
OtherClass. inheritance is a big subject but basically it means that
ClassName has at least the same functions and fields as OtherClass.
In python, Everything is an object and therefor, all classes inherit
implicitely or explicitely from object. This being said
the class ClassName(): declaration is an old syntax and should be avoided.
class ClassName: is equivalent to class ClassName(object):
A class method is not a static method. It is like any other instance method except it is passed the class as parameter rather than the instance.
Your class method declaration is wrong. It should have a cls parameter.
A static method in the other hand, is a method that is called out of context. Meaning it has no relation with any instance. It can be thought of as a independent function that is simply put in a class for semantic reasons.
This is why it does not require a self parameter and one is never passed to it.
You have an indentation error. That might be causing the error.
Yet another question on what the 'self' is for, what happens if you don't use 'self' and what's 'cls' for.
I "have done my homework", I just want to make sure I got it all.
self - To access an attribute of an object, you need to prefix the attribute name with the object name (objname.attributename). The same way self is used to access an attribute inside the object (class) itself. So if you didn't prefix a variable with self in a class method, you wouldn't be able to access that variable in other methods of the class, or outside of the class. So you could omit it if you wanted to make the variable local to that method only. The same way if you had a method and you didn't have any variable you wanted to share with other methods, you could omit the self from the method arguments.
cls - Each instance creates it's own "copy" of the attributes, so if you wanted all the instances of a class to share the same variable, you would prefix that variable name with 'cls' in the class declaration.
Is this all right? Thanks.
The same way self is used to access an attribute inside the object (class) itself.
Not inside the object / class, just inside the class' instance methods. self is just a convention, you could call it whatever you wanted, even something different in each method.
So if you didn't prefix a variable with self in a class method, you wouldn't be able to access that variable in other methods of the class, or outside of the class.
self is used in instance methods, cls is often used in class methods. Otherwise, correct.
So you could omit it if you wanted to make the variable local to that method only.
Yes, inside a method a variable name is like inside any other function -- the interpreter looks for the name locally, then in closures, then in the globals / module level, then in the Python built-ins.
The same way if you had a method and you didn't have any variable you wanted to share with other methods, you could omit the self from the method arguments.
No, you can't just omit "self" from the method arguments. You have to tell Python you want a staticmethod, which won't automatically get passed the instance of the class, ether by doing #staticmethod above the def line, or mymethod = staticmethod(mymethod) below the method body.
Each instance creates it's own "copy" of the attributes, so if you wanted all the instances of a class to share the same variable, you would prefix that variable name with 'cls' in the class declaration.
Inside the class definition, but outside any methods, names are bound to the class -- that's how you define methods etc. You don't prefix them with cls or anything else.
cls is generally used in the __new__ special staticmethod, or in classmethods, which you make similarly to staticmethods. These are methods that only need access to the class, but not to things specific to each instance of the class.
Inside a classmethod, yes, you'd use this to refer to attributes you wanted all instances of the class, and the class itself, to share.
Like self, cls is just a convention, and you could call it whatever you wanted.
A brief example:
class Foo(object):
# you couldn't use self. or cls. out here, they wouldn't mean anything
# this is a class attribute
thing = 'athing'
def __init__(self, bar):
# I want other methods called on this instance of Foo
# to have access to bar, so I create an attribute of self
# pointing to it
self.bar = bar
#staticmethod
def default_foo():
# static methods are often used as alternate constructors,
# since they don't need access to any part of the class
# if the method doesn't have anything at all to do with the class
# just use a module level function
return Foo('baz')
#classmethod
def two_things(cls):
# can access class attributes, like thing
# but not instance attributes, like bar
print cls.thing, cls.thing
You use self as the first argument in regular methods where the instance is passed automatically through this argument. So whatever the first argument is in a method - it points to the current instance
When a method is decorated with #classmethod it gets the class passed as its first argument so the most common name for it is cls as it points to the class.
You usually do not prefix any variables (hungarian notation is bad).
Here's an example:
class Test(object):
def hello(self):
print 'instance %r says hello' % self
#classmethod
def greet(cls):
print 'class %r greet you' % cls
Output:
>>> Test().hello()
instance <__main__.Test object at 0x1f19650> says hello
>>> Test.greet()
class <class '__main__.Test'> greet you