We Keep Coding

Python's `self` argument for class defined wrapped func

There is a simplified code I have:
class A:
def foo(*args, **kwargs):
def foo_sub(*args, **kwargs):
print(f"foo args: {args}")
return foo_sub
def bar(*args, **kwargs):
print(f"bar args: {args}")
a = A()
class B:
foo = a.foo(1)
bar = a.bar
a.foo(2)()
a.bar()
B.foo()
B.bar()
B().foo()
B().bar()
And there is output:
foo args: ()
bar args: (<__main__.A object at 0x7f9763e38080>,)
foo args: ()
bar args: (<__main__.A object at 0x7f9763e38080>,)
foo args: (<__main__.B object at 0x7f9763e38828>,)
bar args: (<__main__.A object at 0x7f9763e38080>,)
I need to wrap foo func in A class and I really don't understand why does B().foo() passes self as argument? How can I prevent it?
More complicated code example: https://codehs.com/sandbox/id/python-3-2uVmcT

I don't know what you are trying to do here with all this wrappers and stuff. You might want to look at decorators or context managers to cleanly do this.
https://docs.python.org/3/glossary.html#term-decorator
https://docs.python.org/3/reference/compound_stmts.html#grammar-token-decorator
https://docs.python.org/3/library/functools.html#functools.wraps
https://docs.python.org/3/library/stdtypes.html#context-manager-types
Answering your question, one way of doing this is by defining static methods as also already pointed out by #MisterMiyagi. Generally in programming, static methods are methods of a class that you can call without needing a class instance/object. This means that such methods has no reference to a class instance's/object's state (in the form of class attributes/fields). In Python, ordinary class methods has access to the instance/object itself via this automatic self argument. If you don't want it, define it as a #staticmethod instead.
A static method does not receive an implicit first argument.
Which is by practice named self.
https://docs.python.org/3/library/functions.html#staticmethod
If you access a method (a function defined in a class namespace)
through an instance, you get a special object: a bound method (also
called instance method) object. When called, it will add the self
argument to the argument list.
"Through an instance" as how you did with B() to do B().foo().
https://docs.python.org/3/library/stdtypes.html#methods
Quick code to solve your case (note that as I pointed out earlier, there are better alternatives to this):
class A:
#staticmethod # Or use the inline-decorator stlye as #kaya3 mentioned
def foo(*args, **kwargs):
def foo_sub(*args, **kwargs):
print(f"foo args: {args}")
return foo_sub
#staticmethod
def bar(*args, **kwargs):
print(f"bar args: {args}")
a = A()
class B:
#staticmethod
def foo(*args, **kwargs):
return a.foo(1)(*args, **kwargs)
#staticmethod
def bar(*args, **kwargs):
return a.bar(*args, **kwargs)
a.foo(2)()
a.bar()
B.foo()
B.bar()
B().foo()
B().bar()
Output:
foo args: ()
bar args: ()
foo args: ()
bar args: ()
foo args: ()
bar args: ()
Otherwise if you really badly want to skip the self attribute altogether and hack all the way through, you could trigger obj.method.__func__(*args, **kwargs) as described in https://docs.python.org/3/library/stdtypes.html#methods
WARNING: This suggestion is hackish. Proceed with caution :)

Thanks #kaya3 for giving a good idea with staticmethod and also everyone who replied!
I found a solution with metaclass
class BMeta(type):
def __new__(cls, name, bases, dct):
if isinstance(dct.get("foo"), FunctionType):
dct["foo"] = staticmethod(dct["foo"])
return super().__new__(cls, name, bases, dct)
class B(metaclass=BMeta):
foo = a.foo(1)
bar = a.bar
https://codehs.com/sandbox/id/python-3-BaZXiA

how do you automate setting self attributes in init of python class?

I'd like to automate unpacking init variables once and for all. My thought process is to unpack all and use setattr with the name of the variable as the attribute name.
class BaseObject(object):
def __init__(self, *args):
for arg in args:
setattr(self, argname, arg)
class Sub(BaseObject):
def __init__(self, argone, argtwo, argthree, argfour):
super(Sub, self).__init__(args*)
Then you should be able to do:
In [3]: mysubclass = Sub('hi', 'stack', 'overflow', 'peeps')
In [4]: mysubclass.argtwo
Out[4]: 'stack'
Based on the param for 'stack' having been named argtwo.
This way you'll automatically have access but you could still override like below:
class Sub(BaseObject):
def __init__(self, argone, argtwo, argthree, argfour):
super(Sub, self).__init__(arglist)
clean_arg_three = process_arg_somehow(argthree)
self.argthree = clean_arg_three
Clearly I'm stuck on how to pass the actual name of the param (argone, argtwo, etc) as a string to setattr, and also how to properly pass the args into the super init (the args of super(Sub, self).__init__(args*))
Thank you

Use kwargs instead of args
class BaseObject(object):
def __init__(self, **kwargs):
for argname, arg in kwargs.iteritems():
setattr(self, argname, arg)
class Sub(BaseObject):
def __init__(self, argone, argtwo, argthree, argfour):
super(Sub, self).__init__(argone=argone, argtwo=argtwo, argthree=argthree)
Then you can do:
s = Sub('a', 'b', 'c')

BaseObject.__init__() needs to discover the names of the arguments to Sub.__init__() somehow. The explicit approach would be to pass a dictionary of arguments to BaseObject.__init__(), as Uri Shalit suggested. Alternatively, you can use the inspect module to do this magically, with no extra code in your subclass. That comes with the usual downsides of magic ("how'd that happen?").
import inspect
class BaseObject(object):
def __init__(self):
frame = inspect.stack()[1][0]
args, _, _, values = inspect.getargvalues(frame)
for key in args:
setattr(self, key, values[key])
class Sub(BaseObject):
def __init__(self, argone, argtwo, argthree, argfour):
super(Sub, self).__init__()
This works fine as written, but if you don't define Sub.__init__(), this will grab the arguments from the wrong place (i.e., from the function where you call Sub() to create an object). You might be able to use inspect to doublecheck that the caller is an __init__() function of a subclass of BaseObject. Or you could just move this code to a separate method, e.g., set_attributes_from_my_arguments() and call that from your subclasses' __init__() methods when you want this behavior.

import inspect
class BaseObject(object):
def __init__(self, args):
del args['self']
self.__dict__.update(args)
class Sub(BaseObject):
def __init__(self, argone, argtwo, argthree, argfour):
args = inspect.getargvalues(inspect.currentframe()).locals
super(Sub, self).__init__(args)
s = Sub(1, 2, 3, 4)

Method Resolution Order in case of Base Classes Having different init params

I am trying to understand MRO in Python. Although there are various posts here, I am not particularly getting what I want. Consider two classes A and B derived from BaseClass, each having an __init__ taking different params.
class BaseClass(object):
def __init__(self):
print "I am the base class"
class A(BaseClass):
def __init__(self, something, anotherthing):
super(A, self).__init__()
self.something = something
self.anotherthing = anotherthing
def methodsA(self):
...
class B(BaseClass):
def __init__(self, someOtherThing):
super(B, self).__init__()
self.someOtherThing = someOtherThing
def methodsB(self):
...
The question is, if I need to derive a Third Class C from both A and B, how do I initialise the __init__, if I have to? I can safely derive C from either B or A.
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
super(C, self).__init__(something, anotherthing, someOtherThing)
The above implementation gives me an error.

As jonrsharpe mentioned at the end of his post, the best way I've come across
for handling this type of situation is accepting **kwargs and extracting
named arguments explicitly.
class BaseClass(object):
def __init__(self, **kwargs):
print("BaseClass.__init__({},{})".format('', kwargs))
super(BaseClass,self).__init__(**kwargs)
class A(BaseClass):
def __init__(self, **kwargs):
print("A.__init__({},{})".format('', kwargs))
a = kwargs.pop('something')
super(A,self).__init__(**kwargs)
class B(BaseClass):
def __init__(self, **kwargs):
print("B.__init__({},{})".format('', kwargs))
b = kwargs.pop('anotherthing')
super(B,self).__init__(**kwargs)
class C(A, B):
def __init__(self, **kwargs):
print("C.__init__({},{})".format('', kwargs))
super(C,self).__init__(**kwargs)
c = C(something=1,anotherthing='a')
Arguments that need to be extracted should be passed in named, so they appear in kwargs.
You can even explicitly accept only named arguments by ommitting the *args as in the example, so you catch yourself with a TypeError if you forget.
EDIT:
After thinking on it a while I realize that my example is very specific to your example, and if you introduce another class or change inheritance it may break. There are two things that should be addressed to make this more general:
BaseClass does not call super.
For the example this doesn't matter, but if another class is introduced the MRO might change such that there is a class after BaseClass and it should therefore call super. This leads to the second issue:
object.__init__() takes no parameters
If we want to make the classes (BaseClass specifically) safe to put into a generic multiple inheritance structure where its super call might be dispatched to another class or object, we need to pop arguments off kwargs when we consume them.
This adds another complication, though, in that it requires that no two __init__ functions share the same parameter name. I guess the takeaway is that making multiple inheritance work in a general way is difficult.
Here is an interesting article (found through google) about some of the details: article

I believe you can't use super for this. You'll have to use the "old style":
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
A.__init__(self, something, anotherthing)
B.__init__(self, someOtherThing)

To understand this, try without any arguments:
class BaseClass(object):
def __init__(self):
print("BaseClass.__init__")
class A(BaseClass):
def __init__(self):
print("A.__init__")
super(A, self).__init__()
class B(BaseClass):
def __init__(self):
print("B.__init__")
super(B, self).__init__()
class C(A, B):
def __init__(self):
print("C.__init__")
super(C, self).__init__()
When we run this:
>>> c = C()
C.__init__
A.__init__
B.__init__
BaseClass.__init__
This is what super does: it makes sure everything gets called, in the right order, without duplication. C inherits from A and B, so both of their __init__ methods should get called, and they both inherit from BaseClass, so that __init__ should also be called, but only once.
If the __init__ methods being called take different arguments, and can't deal with extra arguments (e.g. *args, **kwargs), you get the TypeErrors you refer to. To fix this, you need to make sure that all the methods can handle the appropriate arguments.

While bj0's answer is mostly right, manually extracting the arguments from kwargs is more complicated and awkward than is necessary. It also means that you won't detect when extra arguments are passed in to one of the class constructors.
The best solution is to accept **kwargs, but only use it to pass on any unknown arguments. When this reaches object (BaseClass's base), it will raise an error if there were unnecessary arguments:
class BaseClass(object):
def __init__(self, **kwargs):
super(BaseClass, self).__init__(**kwargs) # always try to pass on unknown args
class A(BaseClass):
def __init__(self, something, anotherthing, **kwargs): # take known arguments
super(A, self).__init__(**kwargs) # pass on the arguments we don't understand
self.something = something
self.anotherthing = anotherthing
class B(BaseClass):
def __init__(self, someOtherThing, **kwargs): # same here
super(B, self).__init__(**kwargs) # and here
self.someOtherThing = someOtherThing
class C(A, B): # this will work, with someOtherThing passed from A.__init__ to B.__init__
pass
class D(B, A): # this will also work, with B.__init__ passing on A.__init__'s arguments
pass
import threading
class E(C, threading.Thread): # keyword arguments for Thread.__init__ will work!
def run(self):
print(self.something, self.anotherthing, self.someOtherThing)
If one of your classes modifies (or provides a default for) an argument that is also used by one of its base classes, you can both take a specific parameter and pass it on by keyword:
class F(C):
def __init__(self, something, **kwargs):
super(F, self).__init__(something="foo"+something, **kwargs)
You do need to be calling all your constructors with only keyword arguments, no positional ones. For instance:
f = F(something="something", anotherthing="bar", someOtherThing="baz")
It's possible to support something similar for positional arguments, but usually its a bad idea because you can't count on the argument order. If you had just one class that took positional arguments (perhaps an unknown number of them in *args), you could probably make that work by passing *args into and out of each __init__ method, but multiple classes taking different positional arguments is asking for trouble due to the order they appear in possibly changing as you do multiple inheritance.

Thanks all for helping me understand MRO. Below is my complete Code together with output. I hope this will also help other's.
class BaseClass(object):
def __init__(self, *args, **kwargs):
self.name = kwargs.get('name')
def printName(self):
print "I am called from BaseClass"
print self.name
def setName(self, givenName):
print "I am called from BaseClass"
self.name=givenName
def CalledFromThirdGen(self):
print "I am called from BaseClass and invoked from Third Generation Derived Class"
class FirstGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(FirstGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.FamilyName = kwargs.get('FamilyName')
def printFullName(self):
print "I am called from FirstDerivedClass"
print self.name + ' ' + self.FamilyName
def printName(self):
print "I am called from FirstDerivedClass, although I was present in BaseClass"
print "His Highness " + self.name + ' ' + self.FamilyName
class SecondGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(SecondGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.middleName = kwargs.get('middleName')
self.FamilyName = kwargs.get('FamilyName')
def printWholeName(self):
print "I am called from SecondDerivedClass"
print self.name + ' ' + self.middleName + ' ' + self.FamilyName
def printName(self):
print "I am called from SecondDerivedClass, although I was present in BaseClass"
print "Sir " + self.name + ' ' + self.middleName + ' ' + self.FamilyName
class ThirdGenDerived(FirstGenDerived, SecondGenDerived):
def __init__(self, *args, **kwargs):
super(ThirdGenDerived, self).__init__(*args, **kwargs)
if name == "main":
print "Executing BaseClass"
BaseClass(name='Robin').printName()
print "Executing Instance of BaseClass with SetName \n"
Instance = BaseClass()
Instance.setName("Little John")
Instance.printName()
print "################################################\n"
print "Executing FirstGenDerived with printName and printFullName\n"
FirstGenDerived(name='Robin', FamilyName='Hood').printFullName()
FirstGenDerived(name='Robin', FamilyName='Hood').printName()
print "################################################\n"
print "Executing FirstGenderived with instance\n"
Instance2 = FirstGenDerived(name=None, FamilyName="Hood")
Instance2.setName("Edwards")
Instance2.printFullName()
print "################################################\n"
print "Executing SecondGenDerived with printName and printWholeName\n"
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printWholeName()
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
print "Executing ThirdGenDerived\n"
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').CalledFromThirdGen()
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
Output:
Executing BaseClass
I am called from BaseClass
Robin
Executing Instance of BaseClass with SetName
I am called from BaseClass
I am called from BaseClass
Little John
Executing FirstGenDerived with printName and printFullName
I am called from FirstDerivedClass
Robin Hood
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood
Executing FirstGenderived with instance
I am called from BaseClass
I am called from FirstDerivedClass
Edwards Hood
Executing SecondGenDerived with printName and printWholeName
I am called from SecondDerivedClass
Robin Williams Hood
I am called from SecondDerivedClass, although I was present in BaseClass
Sir Robin Williams Hood
Executing ThirdGenDerived
I am called from BaseClass and invoked from Third Generation Derived Class
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood

Passing an argument to a decorator inside a list accessing self vars?

How can I modify a self variable with a decorator?
Ex.
class Foo:
def __init__(self,a):
self.a = a
self.li = []
def afunction(self):
pass
I want to add the function object afunction to the list self.li so I can call it in a list. Ex. Have a list of functions defined by the class. How would I do that?
Thanks

I don't think you need a decorator. Functions are first-class objects in Python:
class Foo:
def __init__(self,a):
self.a = a
self.li = [self.afunction]
def afunction(self):
pass

If your intention is to mark certain functions of a class as a special type so that you can identify them later for some other purpose, you could use a decorator, or you could just use a naming convention.
def marked(function):
function.marked = 1
return function
class MarkAware(object):
def run_marked(self, *args, **kwargs):
for name in dir(self):
meth = getattr(self, name)
if hasattr(meth, 'marked'):
meth(*args, **kwargs)
def foo(self):
pass
#marked
def bar(self):
pass
Alternative:
class NameConvention(object):
def run_batchable(self, *args, **kwargs):
for name in dir(self):
if name.startswith('batchable_'):
getattr(self, name)(*args, **kwargs)
def foo(self):
pass
def batchable_bar(self):
pass

As Lattyware explains in a comment to unutbu's answer, you can't directly do what you're asking, because any decorator on afunction will be run while the class itself is being created, not when each instance is created.
If all you really want is "a list of functions defined by the class", you don't need anything fancy at all for that. Just create that list in __init__:
def __init__(self, a):
self.a = a
self.li = [f for f in dir(self) if inspect.ismethod(f)]
If you want a list of certain specific functions, the easiest way is the way unutbu suggests, which still doesn't require a decorator.
If you want the decorator just to mark "this method should go into li", see sr2222's answer.
None of these are what you asked for, but they are probably what you want. There are a few ways to actually use a decorator to add the function to self.li, but they're all pretty horrible, and you probably don't want them. For example:
class Foo:
def __init__(self,a):
self.a = a
self.li = []
def mydecorator(f):
self.li.append(f)
return f
#mydecorator
def afunction(self):
print('a')
self.afunction = new.instancemethod(afunction, self, Foo)

Avoid specifying all arguments in a subclass

I have a class:
class A(object):
def __init__(self,a,b,c,d,e,f,g,...........,x,y,z)
#do some init stuff
And I have a subclass which needs one extra arg (the last W)
class B(A):
def __init__(self.a,b,c,d,e,f,g,...........,x,y,z,W)
A.__init__(self,a,b,c,d,e,f,g,...........,x,y,z)
self.__W=W
It seems dumb to write all this boiler-plate code, e.g passing all the args from B's Ctor to the inside call to A's ctor, since then every change to A's ctor must be applied to two other places in B's code.
I am guessing python has some idiom to handle such cases which I am unaware of. Can you point me in the right direction?
My best hunch, is to have a sort of Copy-Ctor for A and then change B's code into
class B(A):
def __init__(self,instanceOfA,W):
A.__copy_ctor__(self,instanceOfA)
self.__W=W
This would suit my needs since I always create the subclass when given an instance of the father class, Though I am not sure whether it's possible...

Considering that arguments could be passed either by name or by position, I'd code:
class B(A):
def __init__(self, *a, **k):
if 'W' in k:
w = k.pop('W')
else:
w = a.pop()
A.__init__(self, *a, **k)
self._W = w

Edit: based on Matt's suggestion, and to address gnibbler's concern re a positional-argument approach; you might check to make sure that the additional subclass-specific argument is being specified—similar to Alex's answer:
class B(A):
def __init__(self, *args, **kwargs):
try:
self._w = kwargs.pop('w')
except KeyError:
pass
super(B,self).__init__(*args, **kwargs)
>>> b = B(1,2,w=3)
>>> b.a
1
>>> b.b
2
>>> b._w
3
Original answer:
Same idea as Matt's answer, using super() instead.
Use super() to call superclass's __init__() method, then continue initialising the subclass:
class A(object):
def __init__(self, a, b):
self.a = a
self.b = b
class B(A):
def __init__(self, w, *args):
super(B,self).__init__(*args)
self.w = w

In situations where some or all of the arguments passed to __init__ have default values, it can be useful to avoid repeating the __init__ method signature in subclasses.
In these cases, __init__ can pass any extra arguments to another method, which subclasses can override:
class A(object):
def __init__(self, a=1, b=2, c=3, d=4, *args, **kwargs):
self.a = a
self.b = b
# …
self._init_extra(*args, **kwargs)
def _init_extra(self):
"""
Subclasses can override this method to support extra
__init__ arguments.
"""
pass
class B(A):
def _init_extra(self, w):
self.w = w

Are you wanting something like this?
class A(object):
def __init__(self, a, b, c, d, e, f, g):
# do stuff
print a, d, g
class B(A):
def __init__(self, *args):
args = list(args)
self.__W = args.pop()
A.__init__(self, *args)