Weak reference callback is not called - python

The callback assigned to a weak reference object is not called when the strong reference is destroyed, the .callback attribute confirms the function was assigned. Why is it not being called?
It seems the weak reference to the function Foo.bar is not destroyed when the strong reference is destroyed, why?
def is_dead(wr):
print("Callback function called!")
class Foo(object):
def bar(self): pass
method = Foo.bar
wr = weakref.ref(method, is_dead)
method = None
print(wr.__callback__)

Related

Destructor being called multiple times

i am trying to check how new , init and del works and was running below mentioned code , what caught my attention is destructor is being called twice and i don't know the exact reason of it, Can anyone help with it?
class Example:
# creation of new instance
def __new__(cls):
print("example created", object.__new__(cls))
return object.__new__(cls)
# Initializing
def __init__(self):
print("Example Instance.")
# Calling destructor
def __del__(self):
print("Destructor called, Example deleted.")
obj = Example()
del obj
From geeksforgeeks:
The __del__() method is a known as a destructor method in Python. It is called when all references to the object have been deleted i.e when an object is garbage collected. Note : A reference to objects is also deleted when the object goes out of reference or when the program ends.
Python automatically manages memory for you using reference counting and garbage collector. That is, references unnecessary are being automatically deleted.
So you need to create it explicitly to avoid this behavior.
class Example:
instance = ""
# creation of new instance
def __new__(cls):
instance = object.__new__(cls)
print("example created", instance)
return instance
# Initializing
def __init__(self):
print("Example Instance.")
# Calling destructor
def __del__(self):
print("Destructor called, Example deleted.")
result:
>>> obj = Example()
example created <__main__.Example object at 0x7fd0b3efa5b0>
Example Instance.
>>> del obj
Destructor called, Example deleted.
>>>

What exactly is a method in Python?

In the code below, the object a has two functions as attributes: one is a class attribute while one is its own instance attribute.
class A:
def foo(*args):
pass
a = A()
def bar(*args):
pass
a.bar = bar
print(a.foo)
print(a.bar)
What I expected was that both bar() and foo() would be methods of the object a, but, from the output of this code, it turns out that this is not the case—only foo is a method of a.
<bound method A.foo of <__main__.A object at 0x0000016F5F579AF0>>
<function bar at 0x0000016F5F5845E0>
So what exactly is a method in Python? Is it a class attribute that holds a function definition, which seems to be the case.
And why isn't the attribute bar considered a method by Python? What exactly is the idea behind this behaviour of Python?
A method is an instance of the class method returned by, among other things, the __get__ method of a function-valued class attribute.
a.bar is an instance attribute, not a class attribute.
When looking for a.foo, Python first looks for A.foo. Having found it, it next checks if its value has a __get__ method (which, as a function value, it does.) Because A.foo is a descriptor (i.e, has a __get__ method), its __get__ method is called: a.foo is the same as A.foo.__get__(a, A). The return value is a method object, whose __call__ method calls the underlying function with the object and its own arguments. That is,
a.foo(x) == A.foo.__get__(a, A)(x)
== A.foo(a, x)
Because a.bar is an instance attribute, the descriptor protocol is not invoked, so a.bar is the exact same object as bar.
(This is an extremely condensed version of the contents of the Descriptor HowTo Guide, in particular the section on methods. I highly recommended reading it.)
The lookup for a.bar proceeds by first looking for bar in A.__dict__. When it isn't found, Python looks in a.__dict__, and finds a function to call: end of story.
This is because you are setting the attribute to an instance of the class, not itself.
Following your example, let's set:
A.bar = bar
You'll see that the output is:
print(A.bar)
# <function bar at 0x0000021821D57280>
c = A()
print(c.bar)
# <bound method bar of <__main__.A object at 0x0000021821EEA400>>
Bound methods are methods that require the actual instance of the class itself. This is why we typically have self as the first parameter: the instance knows to pass itself to the function when it gets called. By setting the function as an attribute to the instance, the method is not bound, and the first parameter will not be itself.
We can see this behavior if we set our function bar as the following:
def bar(*args):
print("Argument types:", *map(type, args))
a.bar = bar
a.bar()
# Argument types:
A.bar = bar
c = A()
c.bar()
# Argument types: <class '__main__.A'>
We see that with a bound method, it passes itself to the function. This is the difference between setting a method as an attribute versus actually setting it to the class itself.
a.bar is not a BOUND method. It is a callable that has been set to an attribute. This means, among other things, that it doesn't automatically get the self.
class A:
def __init__(self):
self.baz = 'baz'
def foo(self):
print(self.baz)
def bar(obj):
print(obj.baz)
a = A()
a.bar = bar
a.foo()
# prints baz
a.bar()
# raises TypeError: bar() missing 1 required positional argument: 'obj'
a.bar(a)
# prints baz
You can make a method into a bound method with types.MethodType:
a.bar = types.MethodType(bar, a)
a.bar()
# prints baz
This only binds it to the instance; other instances won't have this attribute
a2 = A()
a2.bar()
# AttributeError: 'A' object has no attribute 'bar'

Different way to create an instance method object in Python

The Python 3.x language reference described two ways to create a method object:
User-defined method objects may be created when getting an attribute of a class (perhaps via an instance of that class), if that attribute is a user-defined function object or a class method object.
When an instance method object is created by retrieving a user-defined function object from a class via one of its instances, its self attribute is the instance, and the method object is said to be bound. The new method’s func attribute is the original function object.
When a user-defined method object is created by retrieving another method object from a class or instance, the behaviour is the same as for a function object, except that the func attribute of the new instance is not the original method object but its func attribute.
When an instance method object is called, the underlying function (func) is called, inserting the class instance (self) in front of the argument list. For instance, when C is a class which contains a definition for a function f(), and x is an instance of C, calling x.f(1) is equivalent to calling C.f(x, 1).
When an instance method object is derived from a class method object, the “class instance” stored in self will actually be the class itself, so that calling either x.f(1) or C.f(1) is equivalent to calling f(C,1) where f is the underlying function.
In different ways, they both have different __func__ and __self__ values, but I'm not very aware of these two different ways, could someone explain it to me?
Python Language Reference | The Standard Type Hierarchy:
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarchy
I'm not 100% sure that I completely understand your question, but maybe looking at an example will be helpful. To start, lets create a class which has a function in the definition:
>>> class Foo(object):
... def method(self):
... pass
...
>>> f = Foo()
User-defined method objects may be created when getting an attribute of a class (perhaps via an instance of that class), if that attribute is a user-defined function object or a class method object.
Ok, so we can create a method object by just accessing the attribute on an instance (if the attribute is a function). In our setup, f is an instance of the class Foo:
>>> type(f.method)
<class 'method'>
Compare that with accessing the method attribute on the class:
>>> type(Foo.method)
<class 'function'>
When an instance method object is created by retrieving a user-defined function object from a class via one of its instances, its __self__ attribute is the instance, and the method object is said to be bound. The new method’s __func__ attribute is the original function object.
This is just telling us what attributes exist on instance methods. Let's check it out:
>>> instance_method = f.method
>>> instance_method.__func__ is Foo.method
True
>>> instance_method.__self__ is f
True
So we see that the method object has a __func__ attribute which is just a reference to the actual Foo.method function. It also has a __self__ attribute that is a reference to the instance.
When an instance method object is called, the underlying function (func) is called, inserting the class instance (self) in front of the argument list. For instance, when C is a class which contains a definition for a function f(), and x is an instance of C, calling x.f(1) is equivalent to calling C.f(x, 1).
Basically, in reference to our example above, this is just saying that If:
instance_method = f.method
Then:
instance_method(arg1, arg2)
executes the following:
instance_method.__func__(instance_method.__self__, arg1, arg2)
For completeness and as an addendum to the excellent answer provided by #mgilson, I wanted to explain the remaining 2 paragraphs referenced in the original question.
First let's create a class with a classmethod:
>>> class Foo(object):
... #classmethod
... def cmethod(cls):
... pass
...
>>> f = Foo()
Now for the 3rd paragraph:
When a user-defined method object is created by retrieving another method object from a class or instance, the behaviour is the same as for a function object, except that the func attribute of the new instance is not the original method object but its func attribute.
This means:
>>> class_method = f.cmethod
>>> class_method.__func__ is Foo.cmethod.__func__
True
>>> class_method.__self__ is Foo
True
Note that __self__ is a reference to the Foo class. Finally, the last paragraph:
When an instance method object is derived from a class method object, the “class instance” stored in self will actually be the class itself, so that calling either x.f(1) or C.f(1) is equivalent to calling f(C,1) where f is the underlying function.
This just says that all of the following are equivalent:
>>> f.cmethod(arg1, arg2)
>>> Foo.cmethod(arg1, arg2)
>>> f.cmethod.__func__(Foo, arg1, arg2)
>>> Foo.cmethod.__func__(Foo, arg1, arg2)
>>> f.cmethod.__func__(f.cmethod.__self__, arg1, arg2)
>>> Foo.cmethod.__func__(Foo.cmethod.__self__, arg1, arg2)

Python - why can I call a class method with an instance?

New to Python and having done some reading, I'm making some methods in my custom class class methods rather than instance methods.
So I tested my code but I hadn't changed some of the method calls to call the method in the class rather than the instance, but they still worked:
class myClass:
#classmethod:
def foo(cls):
print 'Class method foo called with %s.'%(cls)
def bar(self):
print 'Instance method bar called with %s.'%(self)
myClass.foo()
thing = myClass()
thing.foo()
thing.bar()
This produces:
class method foo called with __main__.myClass.
class method foo called with __main__.myClass.
instance method bar called with <__main__.myClass instance at 0x389ba4>.
So what I'm wondering is why I can call a class method (foo) on an instance (thing.foo), (although it's the class that gets passed to the method)? It kind of makes sense, as 'thing' is a 'myClass', but I was expecting Python to give an error saying something along the lines of 'foo is a class method and can't be called on an instance'.
Is this just an expected consequence of inheritance with the 'thing' object inheriting the foo method from its superclass?
If I try to call the instance method via the class:
myClass.bar()
then I get:
TypeError: unbound method bar() must be called with myClass instance...
which makes perfect sense.
You can call it on an instance because #classmethod is a decorator (it takes a function as an argument and returns a new function).
Here is some relavent information from the Python documentation
It can be called either on the class (such as C.f()) or on an instance
(such as C().f()). The instance is ignored except for its class. If a
class method is called for a derived class, the derived class object
is passed as the implied first argument.
There's also quite a good SO discussion on #classmethod here.
Let's start with a quick overview of the descriptor protocol. If a class defines a __get__ method, an instance of that class is a descriptor. Accessing a descriptor as the attribute of another object produces the return value of the __get__ method, not the descriptor itself.
A function is a descriptor; this is how instance methods are implemented. Given
class myClass:
#classmethod:
def foo(cls):
print('Class method foo called with %s.' % (cls,))
def bar(self):
print 'Instance method bar called with %s.'%(self)
c = myClass()
the expression c.bar is equivalent to
myClass.__dict__['bar'].__get__(c, myClass)
while the expression myClass.bar is equivalent to the expression
myClass.__dict__['bar'].__get__(None, myClass)
Note the only difference is in the object passed as the first argument to __get__. The former returns a new method object, which when called passes c and its own arguments on to the function bar. This is why c.bar() and C.bar(c) are equivalent. The latter simply returns the function bar itself.
classmethod is a type that provides a different implementation of __get__. This means that c.foo() and myClass.foo() call __get__ as before:
# c.foo
myClass.__dict__['foo'].__get__(c, myClass)
# myClass.foo
myClass.__dict__['foo'].__get__(None, myClass)
Now, however, both calls return the same method object, and this method object, when called, passes myClass as the first argument to the original function object. That is, c.foo() is equivalent to myClass.foo(), which
is equivalent to x(myClass) (where x is the original function defined before the decoration bound the name foo to an instance of classmethod).

Python function pointer in class __init__

In the code below, class A has a member function which is set to point to a function defined outside the class.
in class B, the same function is set to the external pointer in the class definition.
Calling the function for an object of type A will fail, because the self does not get passed to the function. But for B, the self gets passed.
Why does the self get passed for B, but not for A?
def f1(s,a):
print s
print a
class A(object):
def __init__(self):
self.fp1 = f1
class B(object):
fp1 = f1
a=A()
b=B()
try:a.fp1("blaa")
except Exception, e: print `e`
try:b.fp1("bluu")
except Exception, e: print `e`
Output:
TypeError('f1() takes exactly 2 arguments (1 given)',)
<__main__.B object at 0x2ab0dacabed0>
bluu
When you did self.fp1 = f1 you just assigned a function to an instance variable of the class A. So when you call it you have to pass two arguments.
When you did:
class B(object):
fp1 = f1
during creation process of the class B python found a function fp1 in the class scope and created an instancemethod from it (replaced the variable with name fp1 with an instancemethod created from the function that it held before). When you call an instancemethod on an object self gets automatically passed as the first argument.
You can check this by typing:
>>> a = A()
>>> b = B()
>>> type(a.fp1)
function
>>> type(b.fp1)
instancemethod
In class A you bind a function to an instance. This could be really considered as "function pointer" so all arguments must be passed explicitly. In class B you bind the function to the class which will cause the function to work as method. You could modify class definition A to
class A(object):
def __init__(self):
A.fp1 = f1
which will give the same behavior has class B, i.e. fp1 of all instances points to f1, or you could wrap f1.
class A(object):
def __init__(self):
self.fp1 = lambda a: f1(self, a)
This will allow to change fp1 for each instance individually. The latter variant is probably what you were looking for.
The magic that makes instance.method(...) equivalent to Class.method(instance, ...) is dependent on the function object being an attribute of the class. The details vary (and with them, the ugly workaround by which you can create such a method). In Python 3, all functions are descriptors. In Python 2, there are special unbound method objects which are implicitly created to wrap functions stored as class attributes and do roughly what all functions do by themselves in Python 3.
In either case, accessing it through an instance creates a bound method which passes the instance along as first argument when called. In either case, a function accessed through an instance attribute is in no way special, it's just another object which can be passed around and used.
You can achieve similar behavior by either using partial (a bit of a leaky abstraction):
from functools import partial
# in __init__
self.fp1 = partial(f1, self)
or by creating a method which delegates:
def __init__(self):
self._fp1 = f1
def fp1(*args, **kwds):
return self._fp1(self, *args, **kwds)
In the first case you create a field in the class, that has a method object stored in it. So "a.fp1" is not a method call and therefore "a" is not put as the first argument. It's a retrieval of a method object, and then calling it.
For the second case, you can refer to the documentation:
Any function object that is a class attribute defines a method for
instances of that class.
So, for b "fp1" becomes a method for instances of class b.
You can find more detailed explanation here: method objects vs function objects , Python class instances vs class

Categories