python how to "free" an object after callback with "self" is used - python

please review the following code…
class Client(MqC):
def __init__(self, tmpl=None):
self.ConfigSetBgError(self.BgError)
MqC.__init__(self)
def BgError(self):
... do some stuff……
I can add the callback BgError as class or as object callback…
class = self.ConfigSetBgError(Client.BgError)
object = self.ConfigSetBgError(self.BgError)
both cases are working because the callback code can handle this
the problem is the refCount of the self object… the case (2) increment the refCount by ONE… so the following code shows the difference code…
cl = Client()
print("refCount=" + str(sys.getrefcount(cl)))
cl = None
.tp_dealloc is called… because refCount=2
.tp_dealloc is NOT called because refCount=3
→ so… question… ho to solve this refCount cleanup issue ?

If you are worried about the callback keeping the instance alive, then don't pass in a bound method. self.BgError creates a method object (through the descriptor protocol), which references the instance object because it needs to have access to that instance when you call it; that's how the self parameter is passed in in the first place.
If you don't need to reference the instance state and the callback API can handle unbound methods or class method or static methods, then pass one of those in instead.
For example, you can make BgError a class method or static method:
#classmethod
def BgError(cls):
# ...
Now both Client.BgError and self.BgError (instance_of_Client.BgError) produce a method object bound to the class instead of the instance, giving you consistent behaviour. No additional reference to the instance is made.
If you do need the instance state, pass in a wrapper function with a weak reference to your instance. When invoked, check if the weak reference is still available before using the instance. Also see using python WeakSet to enable a callback functionality for a more in-depth post about callbacks and weak references. There the callback registry takes care of producing and storing weak references, but the same principles apply.

Related

Why using self keyword when calling parent method from child method in Python?

Why it's required to use self keyword as an argument when calling the parent method from the child method?
Let me give an example,
class Account:
def __init__(self,filepath):
self.filepath = filepath
with open(self.filepath,"r") as file:
self.blanace = int(file.read())
def withDraw(self,amount):
self.blanace = self.blanace - amount
self.commit()
def deposite(self,amount):
self.blanace = self.blanace + amount
self.commit()
def commit(self):
with open(self.filepath,"w") as file:
file.write(str(self.blanace))
class Checking(Account):
def __init__(self,filepath):
Account.__init__(sellf,filepath) ######## I'm asking about this line.
Regarding this code,
I understand that self is automatically passed to the class when declaring a new object, so,
I expect when I declare new object, python will set self = the declared object, so now the self keyword will be available in the "'init'" child method, so no need to write it manually again like
Account.__init__(sellf,filepath) ######## I'm asking about this line.
All instance methods are just function-valued class attributes. If you access the attribute via an instance, some behind-the-scenes "magic" (known as the descriptor protocol) takes care of changing foo.bar() to type(foo).bar(foo). __init__ itself is also just another instance method, albeit one you usually only call explicitly when overriding __init__ in a child.
In your example, you are explicitly invoking the parent class's __init__ method via the class, so you have to pass self explicitly (self.__init__(filepath) would result in infinite recursion).
One way to avoid this is to not refer to the parent class explicitly, but to let a proxy determine the "closest" parent for you.
super().__init__(filepath)
There is some magic here: super with no arguments determines, with some help from the Python implementation, which class it statically occurs in (in this case, Checking) and passes that, along with self, as the implicit arguments to super. In Python 2, you always had to be explicit: super(Checking, self).__init__(filepath). (In Python 3, you can still pass argument explicitly, because there are some use cases, though rare, for passing arguments other than the current static class and self. Most commonly, super(SomeClass) does not get self as an implicit second argument, and handles class-level proxying.)
Specifically, the function class defines a __get__ method; if the result of an attribute lookup defines __get__, the return value of that method is returned instead of the attribute value itself. In other words,
foo.bar
becomes
foo.__dict__['bar'].__get__(foo, type(foo))
and that return value is an object of type method. Calling a method instance simply causes the original function to be called, with its first argument being the instance that __get__ took as its first argument, and its remaining arguments are whatever other arguments were passed to the original method call.
Generally speaking, I would tally this one up to the Zen of Python -- specifically, the following statements:
Explicit is better than implicit.
Readability counts.
In the face of ambiguity, refuse the temptation to guess.
... and so on.
It's the mantra of Python -- this, along with many other cases may seem redundant and overly simplistic, but being explicit is one of Python's key "goals." Perhaps another user can give more explicit examples, but in this case, I would say it makes sense to not have arguments be explicitly defined in one call, then vanish -- it might make things unclear when looking at a child function without also looking at its parent.

Define #property on function

In JavaScript, we can do the following to any object or function
const myFn = () => {};
Object.defineProperties(myFn, {
property: {
get: () => console.log('property accessed')
}
});
This will allow for a #property like syntax by defining a getter function for the property property.
myFn.property
// property accessed
Is there anything similar for functions in Python?
I know we can't use property since it's not a new-style class, and assigning a lambda with setattr will not work since it'll be a function.
Basically what I want to achieve is that whenever my_fn.property is to return a new instance of another class on each call.
What I currently have with setattr is this
setattr(my_fn, 'property', OtherClass())
My hopes are to design an API that looks like this my_fn.property.some_other_function().
I would prefer using a function as my_fn and not an instance of a class, even though I realize that it might be easier to implement.
Below is the gist of what I'm trying to achieve
def my_fn():
pass
my_fn = property('property', lambda: OtherClass())
my_fn.property
// will be a new instance of OtherClass on each call
It's not possible to do exactly what you want. The descriptor protocol that powers the property built-in is only invoked when:
The descriptor is defined on a class
The descriptor's name is accessed on an instance of said class
Problem is, the class behind functions defined in Python (aptly named function, exposed directly as types.FunctionType or indirectly by calling type() on any function defined at the Python layer) is a single shared, immutable class, so you can't add descriptors to it (and even if you could, they'd become attributes of every Python level function, not just one particular function).
The closest you can get to what you're attempting would be to define a callable class (defining __call__) that defines the descriptor you're interested in as well. Make a single instance of that class (you can throw away the class itself at this point) and it will behave as you expect. Make __call__ a staticmethod, and you'll avoid changing the signature to boot.
For example, the behavior you want could be achieved with:
class my_fn:
# Note: Using the name "property" for a property has issues if you define
# other properties later in the class; this is just for illustration
#property
def property(self):
return OtherClass()
#staticmethod
def __call__(...whatever args apply; no need for self...):
... function behavior goes here ...
my_fn = my_fn() # Replace class with instance of class that behaves like a function
Now you can call the "function" (really a functor, to use C++ parlance):
my_fn(...)
or access the property, getting a brand new OtherClass each time:
>>> type(my_fn.property) is type(my_fn.property)
True
>>> my_fn.property is my_fn.property
False
No, this isn't what you asked for (you seem set on having a plain function do this for you), but you're asking for a very JavaScript specific thing which doesn't exist in Python.
What you want is not currently possible, because the property would have to be set on the function type to be invoked correctly. And you are not allowed to monkeypatch the function type:
>>> type(my_fn).property = 'anything else'
TypeError: can't set attributes of built-in/extension type 'function'
The solution: use a callable class instead.
Note: What you want may become possible in Python 3.8 if PEP 575 is accepted.

Test if a python instance method is the same function as a provided by a class

I have a variable that points to a specific class instance's method.
Lets say there is a class called Client that implements a method get.
A single instance called client is created for Client, and a variable get_func is assigned with client's get.
For example, lets assume I have the following simplified code:
class Client:
def get(self):
print("This is the get function!")
client = Client()
get_func = client.get
I have little actual control over get_func and how it's used, I cannot change that.
I would now want to make sure get_func has the Client.get.
The trivial test get_func == Client.get does not work, as get_func is a bound method of a specific Client instance.
I also cannot get the client instance directly (but a way to get the self of a bound method is a valid option, if only I knew how to do that)
Client.get is either a function object (Python 3), or an unbound method (Python 2). Client().get on the other hand, is a bound method. Methods are wrappers around a function object, recording the instance they are bound to (if there is an instance) to pass in as the self argument. See the Python descriptor How-to as to how Python produces methods from functions.
You can unwrap both a bound method and an unbound method to get the underlying function object that they wrap, and test that, with the __func__ attribute:
get_func.__func__ is Client.get # Python 3
get_func.__func__ is Client.get.__func__ # Python 2, unwrap the unbound method
If you need to make your code compatible with both Python 2 and 3, you could create a simple helper function:
def unwrap_method(f):
return getattr(f, '__func__', f)
and use that:
unwrap_method(get_func) is unwrap_method(Client.get)

get fully qualified method name from inspect stack

I have trouble completing the following function:
def fullyQualifiedMethodNameInStack(depth=1):
"""
The function should return <file>_<class>_<method> for the method in the
stack at specified depth.
"""
fileName=inspect.stack()[depth][1]
methodName=inspect.stack()[depth][3]
class= """ please help!!! """
baseName=os.path.splitext( os.path.basename( fileName ) )[0]
return '{0}_{1}_{2}'.format( baseName, className, methodName )
As you can see I want the class name of the method being executed. The stack that inspect returns only has the method name, and I do not know how to find the class belonging to the method.
You are always looking at a function context; the method context is already gone by the time the function executes. Technically speaking, functions act as descriptors when accessed as attributes on an instance (instance.method_name), which return a method object. The method object then, when called, in turn calls the original function with the instance as the first argument. Because this all happens in C code, no trace of the original method object remains on the Python stack.
The stack only needs to keep track of namespaces and the code to be executed for the local namespace. The original function object is no longer needed for these functions, only the attached code object still retains the name of the original function definition for traceback purposes.
If you know the function to be a method, you could search for a self local name. If present, type(self) is the class of the instance, which is not necessarily the class the method was defined on.
You would then have to search the class and it's bases (looping over the .__mro__ attribute) to try and locate what class defined that method.
There are a few more snags you need to take into account:
Naming the first argument of a method self is only a convention. If someone picked a different name, you won't be able to figure that out without parsing the Python source yourself, or by going up another step in the stack and trying to deduce from that line how the function was called, then retrieve the method and it's .im_self attribute.
You only are given the original name of the function, under which it was defined. That is not necessarily the name under which it was invoked. Functions are first-class objects and can be added to classes under different names.
Although a function was passed in self and was defined as part of a class definition, it could have been invoked by manually passing in a value for self instead of the usual route of the method (being the result of the function acting as a descriptor) passing in the self reference for the caller.
In Python 3.3 and up, the situation is marginally better; function objects have a new __qualname__ attribute, which would include the class name. The problem is then that you still need to locate the function object from the parent stack frame.

How does a python method automatically receive 'self' as the first argument?

Consider this example of a strategy pattern in Python (adapted from the example here). In this case the alternate strategy is a function.
class StrategyExample(object):
def __init__(self, strategy=None) :
if strategy:
self.execute = strategy
def execute(*args):
# I know that the first argument for a method
# must be 'self'. This is just for the sake of
# demonstration
print locals()
#alternate strategy is a function
def alt_strategy(*args):
print locals()
Here are the results for the default strategy.
>>> s0 = StrategyExample()
>>> print s0
<__main__.StrategyExample object at 0x100460d90>
>>> s0.execute()
{'args': (<__main__.StrategyExample object at 0x100460d90>,)}
In the above example s0.execute is a method (not a plain vanilla function) and hence the first argument in args, as expected, is self.
Here are the results for the alternate strategy.
>>> s1 = StrategyExample(alt_strategy)
>>> s1.execute()
{'args': ()}
In this case s1.execute is a plain vanilla function and as expected, does not receive self. Hence args is empty. Wait a minute! How did this happen?
Both the method and the function were called in the same fashion. How does a method automatically get self as the first argument? And when a method is replaced by a plain vanilla function how does it not get the self as the first argument?
The only difference that I was able to find was when I examined the attributes of default strategy and alternate strategy.
>>> print dir(s0.execute)
['__cmp__', '__func__', '__self__', ...]
>>> print dir(s1.execute)
# does not have __self__ attribute
Does the presence of __self__ attribute on s0.execute (the method), but lack of it on s1.execute (the function) somehow account for this difference in behavior? How does this all work internally?
You can read the full explanation here in the python reference, under "User defined methods". A shorter and easier explanation can be found in the python tutorial's description of method objects:
If you still don’t understand how methods work, a look at the implementation can perhaps clarify matters. When an instance attribute is referenced that isn’t a data attribute, its class is searched. If the name denotes a valid class attribute that is a function object, a method object is created by packing (pointers to) the instance object and the function object just found together in an abstract object: this is the method object. When the method object is called with an argument list, a new argument list is constructed from the instance object and the argument list, and the function object is called with this new argument list.
Basically, what happens in your example is this:
a function assigned to a class (as happens when you declare a method inside the class body) is... a method.
When you access that method through the class, eg. StrategyExample.execute you get an "unbound method": it doesn't "know" to which instance it "belongs", so if you want to use that on an instance, you would need to provide the instance as the first argument yourself, eg. StrategyExample.execute(s0)
When you access the method through the instance, eg. self.execute or s0.execute, you get a "bound method": it "knows" which object it "belongs" to, and will get called with the instance as the first argument.
a function that you assign to an instance attribute directly however, as in self.execute = strategy or even s0.execute = strategy is... just a plain function (contrary to a method, it doesn't pass via the class)
To get your example to work the same in both cases:
either you turn the function into a "real" method: you can do this with types.MethodType:
self.execute = types.MethodType(strategy, self, StrategyExample)
(you more or less tell the class that when execute is asked for this particular instance, it should turn strategy into a bound method)
or - if your strategy doesn't really need access to the instance - you go the other way around and turn the original execute method into a static method (making it a normal function again: it won't get called with the instance as the first argument, so s0.execute() will do exactly the same as StrategyExample.execute()):
#staticmethod
def execute(*args):
print locals()
You need to assign an unbound method (i.e. with a self parameter) to the class or a bound method to the object.
Via the descriptor mechanism, you can make your own bound methods, it's also why it works when you assign the (unbound) function to a class:
my_instance = MyClass()
MyClass.my_method = my_method
When calling my_instance.my_method(), the lookup will not find an entry on my_instance, which is why it will at a later point end up doing this: MyClass.my_method.__get__(my_instance, MyClass) - this is the descriptor protocol. This will return a new method that is bound to my_instance, which you then execute using the () operator after the property.
This will share method among all instances of MyClass, no matter when they were created. However, they could have "hidden" the method before you assigned that property.
If you only want specific objects to have that method, just create a bound method manually:
my_instance.my_method = my_method.__get__(my_instance, MyClass)
For more detail about descriptors (a guide), see here.
The method is a wrapper for the function, and calls the function with the instance as the first argument. Yes, it contains a __self__ attribute (also im_self in Python prior to 3.x) that keeps track of which instance it is attached to. However, adding that attribute to a plain function won't make it a method; you need to add the wrapper. Here is how (although you may want to use MethodType from the types module to get the constructor, rather than using type(some_obj.some_method).
The function wrapped, by the way, is accessible through the __func__ (or im_func) attribute of the method.
When you do self.execute = strategy you set the attribute to a plain method:
>>> s = StrategyExample()
>>> s.execute
<bound method StrategyExample.execute of <__main__.StrategyExample object at 0x1dbbb50>>
>>> s2 = StrategyExample(alt_strategy)
>>> s2.execute
<function alt_strategy at 0x1dc1848>
A bound method is a callable object that calls a function passing an instance as the first argument in addition to passing through all arguments it was called with.
See: Python: Bind an Unbound Method?

Categories