I built a little decorator for logging purposes.
def func_detail(func):
def func_wrapper(*args,**kwargs):
log(func.__name__+' ARGS: {}'.format(str(args)))
return func(*args,**kwargs)
return func_wrapper
This works for both object methods and normal methods. I want to use it in multithreading. I have a class which contains pid as an object attribute. Is it possible to change the decorator to log pid if it detects that the method belongs to some class and this class contains attribute pid?
I've tried:
def func_detail(func):
def func_wrapper(*args,**kwargs):
log('PID: '+self.pid if self.pid is not None else ' '+func.__name__+' ARGS: {}'.format(str(args)))
return func(*args,**kwargs)
return func_wrapper
But this not works at all. Could you help me?
ABSTRACT:
I want to be able to call attribute pid from the class where the method (func) belongs without passing self as an argument to the wrapper because in that case it would not works for methods which aren't inside a classes.
The self argument to methods is not magically made available to your func_wrapper function in your decorator. Rather, it will be the first of the position arguments you're capturing with *args. If you want to make use of it, you'll need to examine args[0] and see if it has a pid attribute.
Try this, which checks first that a first argument exists, then that if it has a pid attribute:
log('{}FUNC: {} ARGS: {}'.format('PID: {} '.format(args[0].pid)
if args and hasattr(args[0], "pid") else '',
func.__name__, args))
If you are calling a method, the object itself will be the first parameter, the self in the method implementation.
If your decorator was only applied to methods (defined as def methodName(self, ...):) you could capture the first parameter in self.
Then you could try to print self.pid and catch the exception if there isn't any such attribute.
Since there is little distinction between free functions and method, I think that you should define two decorators one for free function, and another for method, or define a decorator taking a parameter saying wether it is a method or not.
Another solution is to check if the args isn't empty and print args[0].pid if it exists.
Related
Why it's required to use self keyword as an argument when calling the parent method from the child method?
Let me give an example,
class Account:
def __init__(self,filepath):
self.filepath = filepath
with open(self.filepath,"r") as file:
self.blanace = int(file.read())
def withDraw(self,amount):
self.blanace = self.blanace - amount
self.commit()
def deposite(self,amount):
self.blanace = self.blanace + amount
self.commit()
def commit(self):
with open(self.filepath,"w") as file:
file.write(str(self.blanace))
class Checking(Account):
def __init__(self,filepath):
Account.__init__(sellf,filepath) ######## I'm asking about this line.
Regarding this code,
I understand that self is automatically passed to the class when declaring a new object, so,
I expect when I declare new object, python will set self = the declared object, so now the self keyword will be available in the "'init'" child method, so no need to write it manually again like
Account.__init__(sellf,filepath) ######## I'm asking about this line.
All instance methods are just function-valued class attributes. If you access the attribute via an instance, some behind-the-scenes "magic" (known as the descriptor protocol) takes care of changing foo.bar() to type(foo).bar(foo). __init__ itself is also just another instance method, albeit one you usually only call explicitly when overriding __init__ in a child.
In your example, you are explicitly invoking the parent class's __init__ method via the class, so you have to pass self explicitly (self.__init__(filepath) would result in infinite recursion).
One way to avoid this is to not refer to the parent class explicitly, but to let a proxy determine the "closest" parent for you.
super().__init__(filepath)
There is some magic here: super with no arguments determines, with some help from the Python implementation, which class it statically occurs in (in this case, Checking) and passes that, along with self, as the implicit arguments to super. In Python 2, you always had to be explicit: super(Checking, self).__init__(filepath). (In Python 3, you can still pass argument explicitly, because there are some use cases, though rare, for passing arguments other than the current static class and self. Most commonly, super(SomeClass) does not get self as an implicit second argument, and handles class-level proxying.)
Specifically, the function class defines a __get__ method; if the result of an attribute lookup defines __get__, the return value of that method is returned instead of the attribute value itself. In other words,
foo.bar
becomes
foo.__dict__['bar'].__get__(foo, type(foo))
and that return value is an object of type method. Calling a method instance simply causes the original function to be called, with its first argument being the instance that __get__ took as its first argument, and its remaining arguments are whatever other arguments were passed to the original method call.
Generally speaking, I would tally this one up to the Zen of Python -- specifically, the following statements:
Explicit is better than implicit.
Readability counts.
In the face of ambiguity, refuse the temptation to guess.
... and so on.
It's the mantra of Python -- this, along with many other cases may seem redundant and overly simplistic, but being explicit is one of Python's key "goals." Perhaps another user can give more explicit examples, but in this case, I would say it makes sense to not have arguments be explicitly defined in one call, then vanish -- it might make things unclear when looking at a child function without also looking at its parent.
Trying to determine if I should try to use decorator or some other Pythonic way to reduce code that many of my functions are doing. I would like these functions to be able to call just maybe one other function at the start of each function or somehow "decorate" the start of each function. I have never used decorator before and am struggling to implement this decorate idea in a pythonic way to reduce the common set of share code at each function.
I have many functions that will perform the same set of steps at the start of the function. However, there is some structure concerns of the common code that makes this "decorator" idea difficult :
The functions are all in child class of a parent class.
The common commands between the functions reference variable names
that are specific to the function (but a subset of the function
name).
The common commands need to return to the caller and not execute any
more of the child function if a certain condition is met. ("if jobj : " block in the sample code)
For variable/attribute examples, child function get_nas_server(self) will utilize "nas_server" variable variants in the common set of code. Subtracting the get_ from the function name reveals the base of the variable name to be used in the common set of code. Example variables names and object attributes derived from "get_nas_server" function name:
nas_server
nas_server.json
self.nas_server (attribute)
Here is the common code from one of the functions:
####################################################################
def get_nas_server(self):
####################################################################
"""\nGets COMMAND nas_server and places data into self.nas_server"""
try:
self.nas_server
return self.nas_server
except AttributeError:
pass
self.get_file_cmd('nas_server')
jobj = self.fresh_json('nas_server.json')
if jobj :
self.nas_server = jobj
return self.nas_server
self.get_file_cmd('get_nas_server')
Everything below that code above in the function is specific to the function purpose and not appropriate for discussion here. Basically I am trying to make all that code above reusable in my functions, but the code has to have the variables and attribute changed depending on the function name.
Thanks for reading if you got this far and thanks for all help.
Seems like something you could define as a helper method in the parent class:
class Parent(object):
def _get_something(name):
try:
return getattr(self, name)
except AttributeError:
pass
self.get_file_cmd(name)
jobj = self.fresh_json(name+'.json')
if jobj :
setattr(self, name, jobj)
return jobj
self.get_file_cmd('get_'+name)
As this snippet demonstrates, you can use the getattr(), setattr() and hasattr() functions to reference object attributes by name.
You can use a decorator inside your class, if the initial bookkeeping stuff is too specific to that class. Everytime you need to do that initial check inside a class method, you can simply decorate it with the auto_nas decorator I have added below. The scope of this wrapper, I am assuming, is specific to this class alone. Otherwise, you can move it outside the class for generality, but remember to change (self=None) if you're going to do so.
class YourMainClass():
# This is your main class inside which all the methods are defined.
def auto_nas(func):
# This wrapper function will be called instead of your class methods.
def wrap(self):
# All your bookkeeping stuff goes here.
try:
self.nas_server
return self.nas_server
except AttributeError:
pass
self.get_file_cmd('nas_server')
jobj = self.fresh_json('nas_server.json')
if jobj :
self.nas_server = jobj
return self.nas_server
self.get_file_cmd('get_nas_server')
# Things went well - we shall call the actual method.
return func(self)
return wrap
#auto_nas
def get_nas_server(self):
# This is one of your methods that require bookkeeping
pass
Also, notice the self inside the wrapper function. It is automatically passed as an argument by Python, and thus you can access all the instance specific attributes from inside that decorator. Once again, this is appropriate if your decorator must do jobs specific to each instances of your class. If that is not the case, you can simply put all of that code in a function and call it each time whenever you want.
Much thanks to Augurar for pointing in the proper direction. I ended up created a stub method in the parent class. Just needed to call the method with the proper technique to "extend" the parent method. Then calling that stub method from the child class with super() There must be a proper name for this technique? method override? method extend?
Here is the parent class method mostly as Augurar suggested:
###########################################################################
def get_stub(self,fcmd):
###########################################################################
try:
return getattr(self, fcmd)
except AttributeError:
pass
jobj = self.fresh_json(fcmd + '.json')
if jobj :
setattr(self, fcmd, jobj)
return jobj
if not self.get_file_cmd(fcmd):
self.log.error("{} not found".format(fcmd))
return False
Here is the manner in which the child method uses this parent method:
####################################################################
def get_nas_server(self):
####################################################################
super().get_stub('nas_server')
#bunch of child method specific code below here
How to use instance as a parameter in the python member function decorators.
The following is an example.
def foo(func):
def wrap(s):
func()
s.ma()
return wrap
class A:
def ma(self):
print "this is ma"
#foo(self) #error.name 'self' is not defined
def mb(self):
print "this is mb"
It's not clear what you're looking for, but if you want to be able to use the reference to the instance inside your decorator:
def foo(func):
def wrap(s): # I'd call this 'self' instead of 's' to remind us it's a reference to an instance
func(s) # This is a function, not a method yet - so we need to pass in the reference
s.ma() # This is a method, because you use attribute lookup on the object s to get it
return wrap
class A:
def ma(self):
print "this is ma"
#foo # if the way foo wraps mb doesn't depend on some arg, don't use args here
def mb(self):
print "this is mb"
I think you're confused here about the difference between methods and functions in Python – you seem to expect func will work like a method, when in fact it will still be a function when it is being decorated. It is the decorated function that will, at attribute lookup on the instance, be turned into a method; this means you still need the explicit self when you call func in your wrapper function.
See the terrific answer to How to make a chain of function decorators? for a better explanation of what's going on.
[Updated]: Answer inline below question
I have an inspecting program and one objective is for logic in a decorator to know whether the function it is decorating is a class method or regular function. This is failing in a strange way. Below is code run in Python 2.6:
def decorate(f):
print 'decorator thinks function is', f
return f
class Test(object):
#decorate
def test_call(self):
pass
if __name__ == '__main__':
Test().test_call()
print 'main thinks function is', Test().test_call
Then on execution:
decorator thinks function is <function test_call at 0x10041cd70>
main thinks function is <bound method Test.test_call of <__main__.Test object at 0x100425a90>>
Any clue on what's going wrong, and if it is possible for #decorate to correctly infer that test_call is a method?
[Answer]
carl's answer below is nearly perfect. I had a problem when using the decorator on a method that subclasses call. I adapted his code to include a im_func comparison on superclass members:
ismethod = False
for item in inspect.getmro(type(args[0])):
for x in inspect.getmembers(item):
if 'im_func' in dir(x[1]):
ismethod = x[1].im_func == newf
if ismethod:
break
else:
continue
break
As others have said, a function is decorated before it is bound, so you cannot directly determine whether it's a 'method' or 'function'.
A reasonable way to determine if a function is a method or not is to check whether 'self' is the first parameter. While not foolproof, most Python code adheres to this convention:
import inspect
ismethod = inspect.getargspec(method).args[0] == 'self'
Here's a convoluted way that seems to automatically figure out whether the method is a bound or not. Works for a few simple cases on CPython 2.6, but no promises. It decides a function is a method if the first argument to is an object with the decorated function bound to it.
import inspect
def decorate(f):
def detect(*args, **kwargs):
try:
members = inspect.getmembers(args[0])
members = (x[1].im_func for x in members if 'im_func' in dir(x[1]))
ismethod = detect in members
except:
ismethod = False
print ismethod
return f(*args, **kwargs)
return detect
#decorate
def foo():
pass
class bar(object):
#decorate
def baz(self):
pass
foo() # prints False
bar().baz() # prints True
No, this is not possible as you have requested, because there is no inherent difference between bound methods and functions. A method is simply a function wrapped up to get the calling instance as the first argument (using Python descriptors).
A call like:
Test.test_call
which returns an unbound method, translates to
Test.__dict__[ 'test_call' ].__get__( None, spam )
which is an unbound method, even though
Test.__dict__[ 'test_call' ]
is a function. This is because functions are descriptors whose __get__ methods return methods; when Python sees one of these in the lookup chain it calls the __get__ method instead of continuing up the chain.
In effect, the 'bound-methodiness' of a function is determined at runtime, not at define-time!
The decorator simply sees the function as it is defined, without looking it up in a __dict__, so cannot tell whether it is looking at a bound method.
It might be possible to do this with a class decorator that modifies __getattribute__, but that's a particularly nasty hack. Why must you have this functionality? Surely, since you have to place the decorator on the function yourself, you could pass it an argument that says whether said function is defined within a class?
class Test:
#decorate( method = True )
def test_call:
...
#decorate( method = False )
def test_call:
...
Your decorator is run before the function becomes a method. def keyword inside a class defines a function line in any other place, then the functions defined in the body of a class are added to the class as methods. Decorator operates on the function before it is processed by the class that is why your code 'fails'.
There is no way for the #decorate to see the function is actually a method. A workaround for that would be to decorate the function whatever it is (e.g. adding an attribute do_something_about_me_if_I_am_a_method ;-)) and then process it again after the class is computed (iterating over the class members and doing whatever you want with those decorated).
I tried a slightly different example, with one decorated method and one undecorated method.
def decorate(f):
print 'decorator thinks function is', f
return f
class Test(object):
#decorate
def test_call(self):
pass
def test_call_2(self):
pass
if __name__ == '__main__':
print 'main thinks function is', Test.test_call
print 'main thinks function 2 is', Test.test_call_2
Then the output is:
decorator thinks function is <function test_call at 0x100426b18>
main thinks function is <unbound method Test.test_call>
main thinks function 2 is <unbound method Test.test_call_2>
Thus, the decorator saw a different type than the main function did, but the decorator did not change the function's type, or it would be different from the undecorated function.
I'm trying to get the name of all methods in my class.
When testing how the inspect module works, i extraced one of my methods by obj = MyClass.__dict__['mymethodname'].
But now inspect.ismethod(obj) returns False while inspect.isfunction(obj) returns True, and i don't understand why. Is there some strange way of marking methods as methods that i am not aware of? I thought it was just that it is defined in the class and takes self as its first argument.
You are seeing some effects of the behind-the-scenes machinery of Python.
When you write f = MyClass.__dict__['mymethodname'], you get the raw implementation of "mymethodname", which is a plain function. To call it, you need to pass in an additional parameter, class instance.
When you write f = MyClass.mymethodname (note the absence of parentheses after mymethodname), you get an unbound method of class MyClass, which is an instance of MethodType that wraps the raw function you obtained above. To call it, you need to pass in an additional parameter, class instance.
When you write f = MyClass().mymethodname (note that i've created an object of class MyClass before taking its method), you get a bound method of an instance of class MyClass. You do not need to pass an additional class instance to it, since it's already stored inside it.
To get wrapped method (bound or unbound) by its name given as a string, use getattr, as noted by gnibbler. For example:
unbound_mth = getattr(MyClass, "mymethodname")
or
bound_mth = getattr(an_instance_of_MyClass, "mymethodname")
Use the source
def ismethod(object):
"""Return true if the object is an instance method.
Instance method objects provide these attributes:
__doc__ documentation string
__name__ name with which this method was defined
__func__ function object containing implementation of method
__self__ instance to which this method is bound"""
return isinstance(object, types.MethodType)
The first argument being self is just by convention. By accessing the method by name from the class's dict, you are bypassing the binding, so it appears to be a function rather than a method
If you want to access the method by name use
getattr(MyClass, 'mymethodname')
Well, do you mean that obj.mymethod is a method (with implicitly passed self) while Klass.__dict__['mymethod'] is a function?
Basically Klass.__dict__['mymethod'] is the "raw" function, which can be turned to a method by something called descriptors. This means that every function on a class can be both a normal function and a method, depending on how you access them. This is how the class system works in Python and quite normal.
If you want methods, you can't go though __dict__ (which you never should anyways). To get all methods you should do inspect.getmembers(Klass_or_Instance, inspect.ismethod)
You can read the details here, the explanation about this is under "User-defined methods".
From a comment made on #THC4k's answer, it looks like the OP wants to discriminate between built-in methods and methods defined in pure Python code. User defined methods are of types.MethodType, but built-in methods are not.
You can get the various types like so:
import inspect
import types
is_user_defined_method = inspect.ismethod
def is_builtin_method(arg):
return isinstance(arg, (type(str.find), type('foo'.find)))
def is_user_or_builtin_method(arg):
MethodType = types.MethodType
return isinstance(arg, (type(str.find), type('foo'.find), MethodType))
class MyDict(dict):
def puddle(self): pass
for obj in (MyDict, MyDict()):
for test_func in (is_user_defined_method, is_builtin_method,
is_user_or_builtin_method):
print [attr
for attr in dir(obj)
if test_func(getattr(obj, attr)) and attr.startswith('p')]
which prints:
['puddle']
['pop', 'popitem']
['pop', 'popitem', 'puddle']
['puddle']
['pop', 'popitem']
['pop', 'popitem', 'puddle']
You could use dir to get the name of available methods/attributes/etc, then iterate through them to see which ones are methods. Like this:
[ mthd for mthd in dir(FooClass) if inspect.ismethod(myFooInstance.__getattribute__(mthd)) ]
I'm expecting there to be a cleaner solution, but this could be something you could use if nobody else comes up with one. I'd like if I didn't have to use an instance of the class to use getattribute.