As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
It's often stated that super should be avoided in Python 2. I've found in my use of super in Python 2 that it never acts the way I expect unless I provide all arguments such as the example:
super(ThisClass, self).some_func(*args, **kwargs)
It seems to me this defeats the purpose of using super(), it's neither more concise, or much better than TheBaseClass.some_func(self, *args, **kwargs). For most purposes method resolution order is a distant fairy tale.
Other than the fact that 2.7 is the last major release to Python 2, why does super remain broken in Python 2?
How and why has Python 3's super changed? Are there any caveats?
When and why should I use super going forward?
super() is not broken -- it just should not be considered the standard way of calling a method of the base class. This did not change with Python 3.x. The only thing that changed is that you don't need to pass the arguments self, cls in the standard case that self is the first parameter of the current function and cls is the class currently being defined.
Regarding your question when to actually use super(), my answer would be: hardly ever. I personally try to avoid the kind of multiple inheritance that would make super() useful.
Edit: An example from real life that I once ran into: I had some classes defining a run() method, some of which had base classes. I used super() to call the inherited constructors -- I did not think it mattered because I was using single inheritance only:
class A(object):
def __init__(self, i):
self.i = i
def run(self, value):
return self.i * value
class B(A):
def __init__(self, i, j):
super(B, self).__init__(i)
self.j = j
def run(self, value):
return super(B, self).run(value) + self.j
Just imagine there were several of these classes, all with individual constructor prototypes, and all with the same interface to run().
Now I wanted to add some additional functionality to all of these classes, say logging. The additional functionality required an additional method to be defined on all these classes, say info(). I did not want to invade the original classes, but rather define a second set of classes inheriting from the original ones, adding the info() method and inheriting from a mix-in providing the actual logging. Now, I could not use super() in the constructor any more, so I used direct calls:
class Logger(object):
def __init__(self, name):
self.name = name
def run_logged(self, value):
print "Running", self.name, "with info", self.info()
return self.run(value)
class BLogged(B, Logger):
def __init__(self, i, j):
B.__init__(self, i, j)
Logger.__init__("B")
def info(self):
return 42
Here things stop working. The super() call in the base class constructor suddenly calls Logger.__init__(), and BLogged can't do anything about it. There is actually no way to make this work, except for removing the super() call in B itself.
[Another Edit: I don't seem to have made my point, judging from all the comments here and below the other answers. Here is how to make this code work using super():
class A(object):
def __init__(self, i, **kwargs):
super(A, self).__init__(**kwargs)
self.i = i
def run(self, value):
return self.i * value
class B(A):
def __init__(self, j, **kwargs):
super(B, self).__init__(**kwargs)
self.j = j
def run(self, value):
return super(B, self).run(value) + self.j
class Logger(object):
def __init__(self, name, **kwargs):
super(Logger,self).__init__(**kwargs)
self.name = name
def run_logged(self, value):
print "Running", self.name, "with info", self.info()
return self.run(value)
class BLogged(B, Logger):
def __init__(self, **kwargs):
super(BLogged, self).__init__(name="B", **kwargs)
def info(self):
return 42
b = BLogged(i=3, j=4)
Compare this with the use of explicit superclass calls. You decide which version you prefer.]
This and similar stories are why I think that super() should not be considered the standard way of calling methods of the base class. It does not mean super() is broken.
super() is not broken, in Python 2 or Python 3.
Let's consider the arguments from the blog post:
It doesn't do what it sounds like it does.
OK, you may agree or disagree on that, it's pretty subjective. What should it have been called then? super() is a replacement for calling the superclass directly, so the name seems fine to me. It does NOT call the superclass directly, because if that was all it did, it would be pointless, as you could do that anyway. OK, admittedly, that may not be obvious, but the cases where you need super() are generally not obvious. If you need it, you are doing some pretty hairy multiple inheritance. It's not going to be obvious. (Or you are doing a simple mixin, in which case it will be pretty obvious and behave as you expect even if you didn't read the docs).
If you can call the superclass directly, that's probably what you'll end up doing. That's the easy and intuitive way of doing it. super() only comes into play when that doesn't work.
It doesn't mesh well with calling the superclass directly.
Yes, because it's designed to solve a problem with doing that. You can call the superclass directly if, and only if, you know exactly what class that is. Which you don't for mixins, for example, or when your class hierarchy is so messed up that you actually are merging two branches (which is the typical example in all examples of using super()).
So as long as every class in your class hierarchy has a well defined place, calling the superclass directly works. If you don't, then it does not work, and in that case you must use super() instead. That's the point of super() that it figures out what the "next superclass" is according to the MRO, without you explicitly having to specify it, because you can't always do that because you don't always know what it is, for example when using mixins.
The completely different programming language Dylan, a sort of lisp-thingy, solves this in another way that can't be used in Python because it's very different.
Eh. OK?
super() doesn't call your superclass.
Yeah, you said that.
Don't mix super() and direct calling.
Yeah, you said that too.
So, there is two arguments against it: 1. The name is bad. 2. You have to use it consistently.
That does not translate to it being "broken" or that it should be "avoided".
You seem to imply in your post that
def some_func(self, *args, **kwargs):
self.__class__.some_func(self, *args, **kwargs)
is not an infinite recursion. It is, and super would be more correct.
Also, yes, you are required to pass all arguments to super(). This is a bit like complaining that max() doesn't work like expected unless you pass it all the numbers you want to check.
In 3.x, however, fewer arguments are needed: you can do super().foo(*args, **kwargs) instead of super(ThisClass, self).foo(*args, **kwargs).
Anyway, I'm unsure as to any situations when super should be avoided. Its behavior is only "weird" when MI is involved, and when MI is involved, super() is basically your only hope for a correct solution. In Single-Inheritance it's just slightly wordier than SuperClass.foo(self, *args, **kwargs), and does nothing different.
I think I agree with Sven that this sort of MI is worth avoiding, but I don't agree that super is worth avoiding. If your class is supposed to be inherited, super offers users of your class hope of getting MI to work, if they're weird in that way, so it makes your class more usable.
Did you read the article that you link it? It doesn't conclude that super should be avoided but that you should be wary of its caveats when using it. These caveats are summarized by the article, though I would disagree with their suggestions.
The main point of the article is that multiple inheritance can get messy, and super doesn't help as much as the author would want. However doing multiple inheritance without super is often even more complicated.
If you're not doing multiple inheritance, super gives you the advantage that anyone inheriting from your class can add simple mixins and their __init__ would be properly called. Just remember to always call the __init__ of the superclass, even when you're inheriting from object, and to pass all the remaining arguments (*a and **kw) to it. When you're calling other methods from the parent class also use super, but this time use their proper signature that you already know (i.e. ensure that they have the same signature in all classes).
If you're doing multiple inheritance you'd have to dig deeper than that, and probably re-read the same article more carefully to be aware of the caveats. And it's also only during multiple inheritance when you might a situation where an explicit call to the parent might be better than super, but without a specific scenario nobody can tell you whether super should be used or not.
The only change in super in Python 3.x is that you don't need to explicitly pass the current class and self to it. This makes super more attractive, because using it would mean no hardcoding of either the parent class or the current class.
#Sven Marnach:
The problem with your example is that you mix explicit superclass calls B.__init__ and Logger.__init__ in Blogged with super() in B. That won't work. Either you use all explicit superclass calls or use super() on all classes. When you use super() you need to use it on all classes involved, including A I think. Also in your example I think you could use explicit superclass calls in all classes, i.e use A.__init__ in class B.
When there is no diamond inheritance I think super() doesn't have much advantage. The problem is, however, that you don't know in advance if you will get into any diamond inheritance in the future so in that case it would be wise to use super() anyway (but then use it consistently). Otherwise you would end up having to change all classes at a later time or run into problems.
Related
Sometimes self can denote the instance of the class and sometimes the class itself. So why don't we use inst and klass instead of self? Wouldn't that make things easier?
How things are now
class A:
#classmethod
def do(self): # self refers to class
..
class B:
def do(self): # self refers to instance of class
..
How I think they should be
class A:
#classmethod
def do(klass): # no ambiguity
..
class B:
def do(inst): # no ambiguity
..
So how come we don't program like this when in the zen of Python it is stated that explicit is better than implicit? Is there something that I am missing?
Class method support was added much later to Python, and the convention to use self for instances had already been established. Keeping that convention stable has more value than to switch to a longer name like instance.
The convention for class methods is to use the name cls:
class A:
#classmethod
def do(cls):
In other words, the conventions are already there to distinguish between a class object and the instance; never use self for class methods.
Also see PEP 8 - Function and method arguments:
Always use self for the first argument to instance methods.
Always use cls for the first argument to class methods.
I think it would be better to use "cls":
class A:
#classmethod
def do(cls): # cls refers to class
..
class B:
def do(self): # self refers to instance of class
..
It's requirement of PEP8:
http://legacy.python.org/dev/peps/pep-0008/#function-and-method-arguments
I think the point is that conventially you don't use self for methods wrapped with #classmethod. (You could write kls, cls, etc.)
There is ultimately nothing stopping you from writing inst instead of self if you so desire. So your second example would work fine and is actually the expected way to handle it (in terms of distinguishing an instance vs a class). However, you should definitely use self when dealing with instances. It's a Python convention and breaking it is strongly discouraged.
PEP8
Seeing as others have mentioned it, it's true PEP8 does say to use both self and cls in the case of instance and class methods, respectively. The only thing I'd add to this is that while there isn't any sensible reason to break this rule, changing self is significantly worse (from a semantic POV) because of its strong use inside of 99.999% of Python code. Its use is so universal that many (if not most) beginners assume it's a keyword and are confused by the idea that one can change self to anything.
This strong relationship to code and convention is not so apparent with class methods IMO. Of course I would urge anyone to follow PEP8 as much as possible, but if you felt inclined to use kls instead of cls, I feel that you'd be committing a lesser evil than if you changed self. However, whichever name you go with should remain consistent throughout your program.
I was looking into Python's super method and multiple inheritance. I read along something like when we use super to call a base method which has implementation in all base classes, only one class' method will be called even with variety of arguments. For example,
class Base1(object):
def __init__(self, a):
print "In Base 1"
class Base2(object):
def __init__(self):
print "In Base 2"
class Child(Base1, Base2):
def __init__(self):
super(Child, self).__init__('Intended for base 1')
super(Child, self).__init__()# Intended for base 2
This produces TyepError for the first super method. super would call whichever method implementation it first recognizes and gives TypeError instead of checking for other classes down the road. However, this will be much more clear and work fine when we do the following:
class Child(Base1, Base2):
def __init__(self):
Base1.__init__(self, 'Intended for base 1')
Base2.__init__(self) # Intended for base 2
This leads to two questions:
Is __init__ method a static method or a class method?
Why use super, which implicitly choose the method on it's own rather than explicit call to the method like the latter example? It looks lot more cleaner than using super to me. So what is the advantage of using super over the second way(other than writing the base class name with the method call)
super() in the face of multiple inheritance, especially on methods that are present on object can get a bit tricky. The general rule is that if you use super, then every class in the hierarchy should use super. A good way to handle this for __init__ is to make every method take **kwargs, and always use keyword arguments everywhere. By the time the call to object.__init__ occurs, all arguments should have been popped out!
class Base1(object):
def __init__(self, a, **kwargs):
print "In Base 1", a
super(Base1, self).__init__()
class Base2(object):
def __init__(self, **kwargs):
print "In Base 2"
super(Base2, self).__init__()
class Child(Base1, Base2):
def __init__(self, **kwargs):
super(Child, self).__init__(a="Something for Base1")
See the linked article for way more explanation of how this works and how to make it work for you!
Edit: At the risk of answering two questions, "Why use super at all?"
We have super() for many of the same reasons we have classes and inheritance, as a tool for modularizing and abstracting our code. When operating on an instance of a class, you don't need to know all of the gritty details of how that class was implemented, you only need to know about its methods and attributes, and how you're meant to use that public interface for the class. In particular, you can be confident that changes in the implementation of a class can't cause you problems as a user of its instances.
The same argument holds when deriving new types from base classes. You don't want or need to worry about how those base classes were implemented. Here's a concrete example of how not using super might go wrong. suppose you've got:
class Foo(object):
def frob(self):
print "frobbign as a foo"
class Bar(object):
def frob(self):
print "frobbign as a bar"
and you make a subclass:
class FooBar(Foo, Bar):
def frob(self):
Foo.frob(self)
Bar.frob(self)
Everything's fine, but then you realize that when you get down to it,
Foo really is a kind of Bar, so you change it
class Foo(Bar):
def frob(self):
print "frobbign as a foo"
Bar.frob(self)
Which is all fine, except that in your derived class, FooBar.frob() calls Bar.frob() twice.
This is the exact problem super() solves, it protects you from calling superclass implementations more than once (when used as directed...)
As for your first question, __init__ is neither a staticmethod nor a classmethod; it is an ordinary instance method. (That is, it receives the instance as its first argument.)
As for your second question, if you want to explicitly call multiple base class implementations, then doing it explicitly as you did is indeed the only way. However, you seem to be misunderstanding how super works. When you call super, it does not "know" if you have already called it. Both of your calls to super(Child, self).__init__ call the Base1 implementation, because that is the "nearest parent" (the most immediate superclass of Child).
You would use super if you want to call just this immediate superclass implementation. You would do this if that superclass was also set up to call its superclass, and so on. The way to use super is to have each class call only the next implementation "up" in the class hierarchy, so that the sequence of super calls overall calls everything that needs to be called, in the right order. This type of setup is often called "cooperative inheritance", and you can find various articles about it online, including here and here.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last month.
The community reviewed whether to reopen this question last month and left it closed:
Original close reason(s) were not resolved
Improve this question
I often find myself overwriting methods of a parent class, and can never decide if I should explicitly list given parameters or just use a blanket *args, **kwargs construct. Is one version better than the other? Is there a best practice? What (dis-)advantages am I missing?
class Parent(object):
def save(self, commit=True):
# ...
class Explicit(Parent):
def save(self, commit=True):
super(Explicit, self).save(commit=commit)
# more logic
class Blanket(Parent):
def save(self, *args, **kwargs):
super(Blanket, self).save(*args, **kwargs)
# more logic
Perceived benefits of explicit variant
More explicit (Zen of Python)
easier to grasp
function parameters easily accessed
Perceived benefits of blanket variant
more DRY
parent class is easily interchangeable
change of default values in parent method is propagated without touching other code
Liskov Substitution Principle
Generally you don't want you method signature to vary in derived types. This can cause problems if you want to swap the use of derived types. This is often referred to as the Liskov Substitution Principle.
Benefits of Explicit Signatures
At the same time I don't think it's correct for all your methods to have a signature of *args, **kwargs. Explicit signatures:
help to document the method through good argument names
help to document the method by specifying which args are required and which have default values
provide implicit validation (missing required args throw obvious exceptions)
Variable Length Arguments and Coupling
Do not mistake variable length arguments for good coupling practice. There should be a certain amount of cohesion between a parent class and derived classes otherwise they wouldn't be related to each other. It is normal for related code to result in coupling that reflects the level of cohesion.
Places To Use Variable Length Arguments
Use of variable length arguments shouldn't be your first option. It should be used when you have a good reason like:
Defining a function wrapper (i.e. a decorator).
Defining a parametric polymorphic function.
When the arguments you can take really are completely variable (e.g. a generalized DB connection function). DB connection functions usually take a connection string in many different forms, both in single arg form, and in multi-arg form. There are also different sets of options for different databases.
...
Are You Doing Something Wrong?
If you find you are often creating methods which take many arguments or derived methods with different signatures you may have a bigger issue in how you're organizing your code.
My choice would be:
class Child(Parent):
def save(self, commit=True, **kwargs):
super(Child, self).save(commit, **kwargs)
# more logic
It avoids accessing commit argument from *args and **kwargs and it keeps things safe if the signature of Parent:save changes (for example adding a new default argument).
Update : In this case, having the *args can cause troubles if a new positional argument is added to the parent. I would keep only **kwargs and manage only new arguments with default values. It would avoid errors to propagate.
If you are certain that Child will keep the signature, surely the explicit approach is preferable, but when Child will change the signature I personally prefer to use both approaches:
class Parent(object):
def do_stuff(self, a, b):
# some logic
class Child(Parent):
def do_stuff(self, c, *args, **kwargs):
super(Child, self).do_stuff(*args, **kwargs)
# some logic with c
This way, changes in the signature are quite readable in Child, while the original signature is quite readable in Parent.
In my opinion this is also the better way when you have multiple inheritance, because calling super a few times is quite disgusting when you don't have args and kwargs.
For what it's worth, this is also the preferred way in quite a few Python libs and frameworks (Django, Tornado, Requests, Markdown, to name a few). Although one should not base his choices on such things, I'm merely implying that this approach is quite widespread.
Not really an answer but more a side note: If you really, really want to make sure the default values for the parent class are propagated to the child classes you can do something like:
class Parent(object):
default_save_commit=True
def save(self, commit=default_save_commit):
# ...
class Derived(Parent):
def save(self, commit=Parent.default_save_commit):
super(Derived, self).save(commit=commit)
However I have to admit this looks quite ugly and I would only use it if I feel I really need it.
I prefer explicit arguments because auto complete allows you to see the method signature of the function while making the function call.
In addition to the other answers:
Having variable arguments may "decouple" the parent from the child, but creates a coupling between the object created and the parent, which I think is worse, because now you created a "long distance" couple (more difficult to spot, more difficult to maintain, because you may create several objects in your application)
If you're looking for decoupling, take a look at composition over inheritance
I frequently do this sort of thing:
class Person(object):
def greet(self):
print "Hello"
class Waiter(Person):
def greet(self):
Person.greet(self)
print "Would you like fries with that?"
The line Person.greet(self) doesn't seem right. If I ever change what class Waiter inherits from I'm going to have to track down every one of these and replace them all.
What is the correct way to do this is modern Python? Both 2.x and 3.x, I understand there were changes in this area in 3.
If it matters any I generally stick to single inheritance, but if extra stuff is required to accommodate multiple inheritance correctly it would be good to know about that.
You use super:
Return a proxy object that delegates
method calls to a parent or sibling
class of type. This is useful for
accessing inherited methods that have
been overridden in a class. The search
order is same as that used by
getattr() except that the type itself
is skipped.
In other words, a call to super returns a fake object which delegates attribute lookups to classes above you in the inheritance chain. Points to note:
This does not work with old-style classes -- so if you are using Python 2.x, you need to ensure that the top class in your hierarchy inherits from object.
You need to pass your own class and instance to super in Python 2.x. This requirement was waived in 3.x.
This will handle all multiple inheritance correctly. (When you have a multiple inheritance tree in Python, a method resolution order is generated and the lookups go through parent classes in this order.)
Take care: there are many places to get confused about multiple inheritance in Python. You might want to read super() Considered Harmful. If you are sure that you are going to stick to a single inheritance tree, and that you are not going to change the names of classes in said tree, you can hardcode the class names as you do above and everything will work fine.
Not sure if you're looking for this but you can call a parent without referring to it by doing this.
super(Waiter, self).greet()
This will call the greet() function in Person.
katrielalex's answer is really the answer to your question, but this wouldn't fit in a comment.
If you plan to go about using super everywhere, and you ever think in terms of multiple inheritance, definitely read the "super() Considered Harmful" link. super() is a great tool, but it takes understanding to use correctly. In my experience, for simple things that don't seem likely to get into complicated diamond inheritance tangles, it's actually easier and less tedious to just call the superclass directly and deal with the renames when you change the name of the base class.
In fact, in Python2 you have to include the current class name, which is usually more likely to change than the base class name. (And in fact sometimes it's very difficult to pass a reference to the current class if you're doing wacky things; at the point when the method is being defined the class isn't bound to any name, and at the point when the super call is executed the original name of the class may not still be bound to the class, such as when you're using a class decorator)
I'd like to make it more explicit in this answer with an example. It's just like how we do in JavaScript. The short answer is, do that like we initiate the constructor using super.
class Person(object):
def __init__(self, name):
self.name = name
def greet(self):
print(f"Hello, I'm {self.name}")
class Waiter(Person):
def __init__(self, name):
super().__init__(name)
# initiate the parent constructor
# or super(Waiter, self).__init__(name)
def greet(self):
super(Waiter, self).greet()
print("Would you like fries with that?")
waiter = Waiter("John")
waiter.greet()
# Hello, I'm John
# Would you like fries with that?
I am subclassing an object in order to override a method that I want to add some functionality to. I don't want to completely replace it or add a differently named method but remain compatible to the superclasses method by just adding an optional argument to the method.
Is it possible to work with *args and **kwargs to pass through all arguments to the superclass and still add an optional argument with a default?
I intuitively came up with the following but it doesn't work:
class A(object):
def foo(self, arg1, arg2, argopt1="bar"):
print arg1, arg2, argopt1
class B(A):
def foo(self, *args, argopt2="foo", **kwargs):
print argopt2
A.foo(self, *args, **kwargs)
b = B()
b.foo("a", "b", argopt2="foo")
Of course I can get it to work when I explicitly add all the arguments of the method of the superclass:
class B(A):
def foo(self, arg1, arg2, argopt1="foo", argopt2="bar"):
print argopt2
A.foo(self, arg1, arg2, argopt1=argopt1)
What's the right way to do this, do I have to know and explicitly state all of the overridden methods arguments?
class A(object):
def foo(self, arg1, arg2, argopt1="bar"):
print arg1, arg2, argopt1
class B(A):
def foo(self, *args, **kwargs):
argopt2 = kwargs.get('argopt2', default_for_argopt2)
# remove the extra arg so the base class doesn't complain.
del kwargs['argopt2']
print argopt2
A.foo(self, *args, **kwargs)
b = B()
b.foo("a", "b", argopt2="foo")
What's the right way to do this, do I
have to know and explicitly state all
of the overridden methods arguments?
If you want to cover all cases (rather than just rely on the caller to always do things your way, e.g., always call you only with the extra argument passed by-name, never by position) you do have to code (or dynamically discover) a lot of knowledge about the signature of the method you're overriding -- hardly surprising: inheritance is a strong form of coupling, and overriding methods is one way that coupling presents itself.
You could dynamically discover the superclass's method arguments via inspect.getargspec, in order to make sure you call it properly... but this introspection technique can get tricky if two classes are trying to do exactly the same thing (once you know your superclass's method accepts *a and/or **kw you can do little more than pass all the relevant arguments upwards and hope, with fingers crossed, that the upstream method chain eventually does proper housecleaning before calling a version that's not quite so tolerant).
Such prices may be worth paying when you're designing a wrapper that's meant to be applied dynamically to callables with a wide variety of signatures (especially since in a decorator setting you can arrange to pay the hefty cost of introspection just once per function you're decorating, not every time the resulting wrapper is called). It seems unlikely to be a worthwhile technique in a case such as yours, where you'd better know what you're subclassing (subclassing is strong coupling: doing it blindly is definitely not advisable!), and so you might as well spell out the arguments explicitly.
Yes, if the superclass's code changes drastically (e.g., by altering method signatures), you'll have to revise the subclass as well -- that's (part of) the price of inheritance. The overall price's hefty enough that the new Go programming language does totally without it -- forcing you to apply the Gang of 4's excellent advice to prefer composition over inheritance. In Python complete abstinence from inheritance would just be impractical, but using it soberly and in moderation (and accepting the price you'll pay in terms of coupling when you do) remains advisable.
When subclassing and overriding methods, one must always decide if using super() is a good idea, and this page is good for that.
I'm not saying that super() should be avoided, like the article author may be: I'm saying that super() has some very important prerequisits that must be followed if you don't want super() to come back and bite you.