Listing all class members with Python `inspect` module - python

What is the "optimal" way to list all class methods of a given class using inspect? It works if I use the inspect.isfunction as predicate in getmembers like so
class MyClass(object):
def __init(self, a=1):
pass
def somemethod(self, b=1):
pass
inspect.getmembers(MyClass, predicate=inspect.isfunction)
returns
[('_MyClass__init', <function __main__.MyClass.__init>),
('somemethod', <function __main__.MyClass.somemethod>)]
But isn't it supposed to work via ismethod?
inspect.getmembers(MyClass, predicate=inspect.ismethod)
which returns an empty list in this case. Would be nice if someone could clarify what's going on. I was running this in Python 3.5.

As described in the documentation, inspect.ismethod will show bound methods. This means you have to create an instance of the class if you want to inspect its methods. Since you are trying to inspect methods on the un-instantiated class you are getting an empty list.
If you do:
x = MyClass()
inspect.getmembers(x, predicate=inspect.ismethod)
you would get the methods.

Related

How to pass function pointers from class methods

I need to pass some 3rd party code a list of functions that accept a single parameter, e.g. list.
run_funcs([m1, m2])
My code (methods) are part of a class, therefore they take another self parameter.
class MyClass():
def m1(self, o):
return <something>
def m2(self, o):
return <something>
Can I somehow extract from an object instance a list of functions runnable by run_funcs?
I need to have the methods in the class because I need some initialization (build some lists) which the methods access.
I (think) I should be able to create a list of function from an instance (which will then have the self), but I am not sure how ...
If you have an instance of the class, you can access the methods on the instance:
mc = MyClass()
run_funcs([mc.m1, mc.m2])
If you want to inspect a class and print a name and
and function reference to each method which has an argspec
of two args see the code block below.
When inspecting the methods' argspec, if it has two args,
the first one in a class method is 'self', thus when calling,
it would only require one arg.
In the loop below: method[0] is the name and
method[1] is a method reference.
I haven't actually tried it but if you need to generate functions
from a class at runtime, I think this is down the correct path
to victory.
import inspect
class MyClass():
def m1(self, o):
return 'method1'
def m2(self, o):
return 'method2'
mc = MyClass()
for method in inspect.getmembers(mc):
if isinstance(method[1], type(mc.m1)):
if len(inspect.getfullargspec(method[1]).args) == 2:
print(f'mc.{method[0]}')
print(f' mc.{method[1]}')
>>> mc.m1
>>> mc.<bound method MyClass.m1 of <funcs_from_class.MyClass object at 0x11181e6d8>>
>>>mc.m2
>>> mc.<bound method MyClass.m2 of <funcs_from_class.MyClass object at 0x11181e6d8>>

How can I refer to the currently being defined class? [duplicate]

For a recursive function we can do:
def f(i):
if i<0: return
print i
f(i-1)
f(10)
However is there a way to do the following thing?
class A:
# do something
some_func(A)
# ...
If I understand your question correctly, you should be able to reference class A within class A by putting the type annotation in quotes. This is called forward reference.
class A:
# do something
def some_func(self, a: 'A')
# ...
See ref below
https://github.com/python/mypy/issues/3661
https://www.youtube.com/watch?v=AJsrxBkV3kc
In Python you cannot reference the class in the class body, although in languages like Ruby you can do it.
In Python instead you can use a class decorator but that will be called once the class has initialized. Another way could be to use metaclass but it depends on what you are trying to achieve.
You can't with the specific syntax you're describing due to the time at which they are evaluated. The reason the example function given works is that the call to f(i-1) within the function body is because the name resolution of f is not performed until the function is actually called. At this point f exists within the scope of execution since the function has already been evaluated. In the case of the class example, the reference to the class name is looked up during while the class definition is still being evaluated. As such, it does not yet exist in the local scope.
Alternatively, the desired behavior can be accomplished using a metaclass like such:
class MetaA(type):
def __init__(cls):
some_func(cls)
class A(object):
__metaclass__=MetaA
# do something
# ...
Using this approach you can perform arbitrary operations on the class object at the time that the class is evaluated.
Maybe you could try calling __class__.
Right now I'm writing a code that calls a class method from within the same class.
It is working well so far.
I'm creating the class methods using something like:
#classmethod
def my_class_method(cls):
return None
And calling then by using:
x = __class__.my_class_method()
It seems most of the answers here are outdated. From python3.7:
from __future__ import annotations
Example:
$ cat rec.py
from __future__ import annotations
class MyList:
def __init__(self,e):
self.data = [e]
def add(self, e):
self.data.append(e)
return self
def score(self, other:MyList):
return len([e
for e in self.data
if e in other.data])
print(MyList(8).add(3).add(4).score(MyList(4).add(9).add(3)))
$ python3.7 rec.py
2
Nope. It works in a function because the function contents are executed at call-time. But the class contents are executed at define-time, at which point the class doesn't exist yet.
It's not normally a problem because you can hack further members into the class after defining it, so you can split up a class definition into multiple parts:
class A(object):
spam= 1
some_func(A)
A.eggs= 2
def _A_scramble(self):
self.spam=self.eggs= 0
A.scramble= _A_scramble
It is, however, pretty unusual to want to call a function on the class in the middle of its own definition. It's not clear what you're trying to do, but chances are you'd be better off with decorators (or the relatively new class decorators).
There isn't a way to do that within the class scope, not unless A was defined to be something else first (and then some_func(A) will do something entirely different from what you expect)
Unless you're doing some sort of stack inspection to add bits to the class, it seems odd why you'd want to do that. Why not just:
class A:
# do something
pass
some_func(A)
That is, run some_func on A after it's been made. Alternately, you could use a class decorator (syntax for it was added in 2.6) or metaclass if you wanted to modify class A somehow. Could you clarify your use case?
If you want to do just a little hacky thing do
class A(object):
...
some_func(A)
If you want to do something more sophisticated you can use a metaclass. A metaclass is responsible for manipulating the class object before it gets fully created. A template would be:
class AType(type):
def __new__(meta, name, bases, dct):
cls = super(AType, meta).__new__(meta, name, bases, dct)
some_func(cls)
return cls
class A(object):
__metaclass__ = AType
...
type is the default metaclass. Instances of metaclasses are classes so __new__ returns a modified instance of (in this case) A.
For more on metaclasses, see http://docs.python.org/reference/datamodel.html#customizing-class-creation.
If the goal is to call a function some_func with the class as an argument, one answer is to declare some_func as a class decorator. Note that the class decorator is called after the class is initialized. It will be passed the class that is being decorated as an argument.
def some_func(cls):
# Do something
print(f"The answer is {cls.x}")
return cls # Don't forget to return the class
#some_func
class A:
x = 1
If you want to pass additional arguments to some_func you have to return a function from the decorator:
def some_other_func(prefix, suffix):
def inner(cls):
print(f"{prefix} {cls.__name__} {suffix}")
return cls
return inner
#some_other_func("Hello", " and goodbye!")
class B:
x = 2
Class decorators can be composed, which results in them being called in the reverse order they are declared:
#some_func
#some_other_func("Hello", "and goodbye!")
class C:
x = 42
The result of which is:
# Hello C and goodbye!
# The answer is 42
What do you want to achieve? It's possible to access a class to tweak its definition using a metaclass, but it's not recommended.
Your code sample can be written simply as:
class A(object):
pass
some_func(A)
If you want to refer to the same object, just use 'self':
class A:
def some_func(self):
another_func(self)
If you want to create a new object of the same class, just do it:
class A:
def some_func(self):
foo = A()
If you want to have access to the metaclass class object (most likely not what you want), again, just do it:
class A:
def some_func(self):
another_func(A) # note that it reads A, not A()
Do remember that in Python, type hinting is just for auto-code completion therefore it helps IDE to infer types and warn user before runtime. In runtime, type hints almost never used(except in some cases) so you can do something like this:
from typing import Any, Optional, NewType
LinkListType = NewType("LinkList", object)
class LinkList:
value: Any
_next: LinkListType
def set_next(self, ll: LinkListType):
self._next = ll
if __name__ == '__main__':
r = LinkList()
r.value = 1
r.set_next(ll=LinkList())
print(r.value)
And as you can see IDE successfully infers it's type as LinkList:
Note: Since the next can be None, hinting this in the type would be better, I just didn't want to confuse OP.
class LinkList:
value: Any
next: Optional[LinkListType]
It's ok to reference the name of the class inside its body (like inside method definitions) if it's actually in scope... Which it will be if it's defined at top level. (In other cases probably not, due to Python scoping quirks!).
For on illustration of the scoping gotcha, try to instantiate Foo:
class Foo(object):
class Bar(object):
def __init__(self):
self.baz = Bar.baz
baz = 15
def __init__(self):
self.bar = Foo.Bar()
(It's going to complain about the global name 'Bar' not being defined.)
Also, something tells me you may want to look into class methods: docs on the classmethod function (to be used as a decorator), a relevant SO question. Edit: Ok, so this suggestion may not be appropriate at all... It's just that the first thing I thought about when reading your question was stuff like alternative constructors etc. If something simpler suits your needs, steer clear of #classmethod weirdness. :-)
Most code in the class will be inside method definitions, in which case you can simply use the name A.

Methods defined outside class which itself is subsequently imported

Is it OK to define functions outside a particular class, use them in the class, and then import that class elsewhere and use it?
Are there any risks associated with doing that, rather than making all functions methods of the class?
I'm writing this code in python 2.7
For example, make a class like this:
def func(a):
return a
class MyClass():
def class_func(self, thing):
return func(thing)
Then import MyClass into another python script and use its class_func method.
Doing this is okay, and in fact a language feature of python. Functions have access to names of the scope they are defined in, regardless of where they are called from.
For example, you can also do something like this:
factor = 2
def multiply(num):
return num*factor
See this post for some background information.
The "risk" associated with this is that the outside name is explicitly not under your control. It can be freely modified by other parts of your program, without the implication being clear.
Consider this example:
def func(a):
return a
class MyClass(object): # note: you should inherit from object in py2.X!
def class_func(self, thing):
return func(thing)
myinstance = MyClass()
foo = myinstance.class_func(1)
def func(a):
return str(a)
bar = myinstance.class_func(1)
Here, foo and bar will be different, namely the integer 1 and the string "1".
Usually, making this possible is the entire point of using such a structure, however.
It's ok, but if func uses only in MyClass it can be helpful to make it staticmethod and place inside MyClass near class_func:
class MyClass(object):
#staticmethod
def _func(a):
return a
def class_func(self, thing):
return type(self)._func(thing)

Find class in which a method is defined

I want to figure out the type of the class in which a certain method is defined (in essence, the enclosing static scope of the method), from within the method itself, and without specifying it explicitly, e.g.
class SomeClass:
def do_it(self):
cls = enclosing_class() # <-- I need this.
print(cls)
class DerivedClass(SomeClass):
pass
obj = DerivedClass()
# I want this to print 'SomeClass'.
obj.do_it()
Is this possible?
If you need this in Python 3.x, please see my other answer—the closure cell __class__ is all you need.
If you need to do this in CPython 2.6-2.7, RickyA's answer is close, but it doesn't work, because it relies on the fact that this method is not overriding any other method of the same name. Try adding a Foo.do_it method in his answer, and it will print out Foo, not SomeClass
The way to solve that is to find the method whose code object is identical to the current frame's code object:
def do_it(self):
mro = inspect.getmro(self.__class__)
method_code = inspect.currentframe().f_code
method_name = method_code.co_name
for base in reversed(mro):
try:
if getattr(base, method_name).func_code is method_code:
print(base.__name__)
break
except AttributeError:
pass
(Note that the AttributeError could be raised either by base not having something named do_it, or by base having something named do_it that isn't a function, and therefore doesn't have a func_code. But we don't care which; either way, base is not the match we're looking for.)
This may work in other Python 2.6+ implementations. Python does not require frame objects to exist, and if they don't, inspect.currentframe() will return None. And I'm pretty sure it doesn't require code objects to exist either, which means func_code could be None.
Meanwhile, if you want to use this in both 2.7+ and 3.0+, change that func_code to __code__, but that will break compatibility with earlier 2.x.
If you need CPython 2.5 or earlier, you can just replace the inpsect calls with the implementation-specific CPython attributes:
def do_it(self):
mro = self.__class__.mro()
method_code = sys._getframe().f_code
method_name = method_code.co_name
for base in reversed(mro):
try:
if getattr(base, method_name).func_code is method_code:
print(base.__name__)
break
except AttributeError:
pass
Note that this use of mro() will not work on classic classes; if you really want to handle those (which you really shouldn't want to…), you'll have to write your own mro function that just walks the hierarchy old-school… or just copy it from the 2.6 inspect source.
This will only work in Python 2.x implementations that bend over backward to be CPython-compatible… but that includes at least PyPy. inspect should be more portable, but then if an implementation is going to define frame and code objects with the same attributes as CPython's so it can support all of inspect, there's not much good reason not to make them attributes and provide sys._getframe in the first place…
First, this is almost certainly a bad idea, and not the way you want to solve whatever you're trying to solve but refuse to tell us about…
That being said, there is a very easy way to do it, at least in Python 3.0+. (If you need 2.x, see my other answer.)
Notice that Python 3.x's super pretty much has to be able to do this somehow. How else could super() mean super(THISCLASS, self), where that THISCLASS is exactly what you're asking for?*
Now, there are lots of ways that super could be implemented… but PEP 3135 spells out a specification for how to implement it:
Every function will have a cell named __class__ that contains the class object that the function is defined in.
This isn't part of the Python reference docs, so some other Python 3.x implementation could do it a different way… but at least as of 3.2+, they still have to have __class__ on functions, because Creating the class object explicitly says:
This class object is the one that will be referenced by the zero-argument form of super(). __class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super. This allows the zero argument form of super() to correctly identify the class being defined based on lexical scoping, while the class or instance that was used to make the current call is identified based on the first argument passed to the method.
(And, needless to say, this is exactly how at least CPython 3.0-3.5 and PyPy3 2.0-2.1 implement super anyway.)
In [1]: class C:
...: def f(self):
...: print(__class__)
In [2]: class D(C):
...: pass
In [3]: D().f()
<class '__main__.C'>
Of course this gets the actual class object, not the name of the class, which is apparently what you were after. But that's easy; you just need to decide whether you mean __class__.__name__ or __class__.__qualname__ (in this simple case they're identical) and print that.
* In fact, this was one of the arguments against it: that the only plausible way to do this without changing the language syntax was to add a new closure cell to every function, or to require some horrible frame hacks which may not even be doable in other implementations of Python. You can't just use compiler magic, because there's no way the compiler can tell that some arbitrary expression will evaluate to the super function at runtime…
If you can use #abarnert's method, do it.
Otherwise, you can use some hardcore introspection (for python2.7):
import inspect
from http://stackoverflow.com/a/22898743/2096752 import getMethodClass
def enclosing_class():
frame = inspect.currentframe().f_back
caller_self = frame.f_locals['self']
caller_method_name = frame.f_code.co_name
return getMethodClass(caller_self.__class__, caller_method_name)
class SomeClass:
def do_it(self):
print(enclosing_class())
class DerivedClass(SomeClass):
pass
DerivedClass().do_it() # prints 'SomeClass'
Obviously, this is likely to raise an error if:
called from a regular function / staticmethod / classmethod
the calling function has a different name for self (as aptly pointed out by #abarnert, this can be solved by using frame.f_code.co_varnames[0])
Sorry for writing yet another answer, but here's how to do what you actually want to do, rather than what you asked for:
this is about adding instrumentation to a code base to be able to generate reports of method invocation counts, for the purpose of checking certain approximate runtime invariants (e.g. "the number of times that method ClassA.x() is executed is approximately equal to the number of times that method ClassB.y() is executed in the course of a run of a complicated program).
The way to do that is to make your instrumentation function inject the information statically. After all, it has to know the class and method it's injecting code into.
I will have to instrument many classes by hand, and to prevent mistakes I want to avoid typing the class names everywhere. In essence, it's the same reason why typing super() is preferable to typing super(ClassX, self).
If your instrumentation function is "do it manually", the very first thing you want to turn it into an actual function instead of doing it manually. Since you obviously only need static injection, using a decorator, either on the class (if you want to instrument every method) or on each method (if you don't) would make this nice and readable. (Or, if you want to instrument every method of every class, you might want to define a metaclass and have your root classes use it, instead of decorating every class.)
For example, here's an easy way to instrument every method of a class:
import collections
import functools
import inspect
_calls = {}
def inject(cls):
cls._calls = collections.Counter()
_calls[cls.__name__] = cls._calls
for name, method in cls.__dict__.items():
if inspect.isfunction(method):
#functools.wraps(method)
def wrapper(*args, **kwargs):
cls._calls[name] += 1
return method(*args, **kwargs)
setattr(cls, name, wrapper)
return cls
#inject
class A(object):
def f(self):
print('A.f here')
#inject
class B(A):
def f(self):
print('B.f here')
#inject
class C(B):
pass
#inject
class D(C):
def f(self):
print('D.f here')
d = D()
d.f()
B.f(d)
print(_calls)
The output:
{'A': Counter(),
'C': Counter(),
'B': Counter({'f': 1}),
'D': Counter({'f': 1})}
Exactly what you wanted, right?
You can either do what #mgilson suggested or take another approach.
class SomeClass:
pass
class DerivedClass(SomeClass):
pass
This makes SomeClass the base class for DerivedClass.
When you normally try to get the __class__.name__ then it will refer to derived class rather than the parent.
When you call do_it(), it's really passing DerivedClass as self, which is why you are most likely getting DerivedClass being printed.
Instead, try this:
class SomeClass:
pass
class DerivedClass(SomeClass):
def do_it(self):
for base in self.__class__.__bases__:
print base.__name__
obj = DerivedClass()
obj.do_it() # Prints SomeClass
Edit:
After reading your question a few more times I think I understand what you want.
class SomeClass:
def do_it(self):
cls = self.__class__.__bases__[0].__name__
print cls
class DerivedClass(SomeClass):
pass
obj = DerivedClass()
obj.do_it() # prints SomeClass
[Edited]
A somewhat more generic solution:
import inspect
class Foo:
pass
class SomeClass(Foo):
def do_it(self):
mro = inspect.getmro(self.__class__)
method_name = inspect.currentframe().f_code.co_name
for base in reversed(mro):
if hasattr(base, method_name):
print(base.__name__)
break
class DerivedClass(SomeClass):
pass
class DerivedClass2(DerivedClass):
pass
DerivedClass().do_it()
>> 'SomeClass'
DerivedClass2().do_it()
>> 'SomeClass'
SomeClass().do_it()
>> 'SomeClass'
This fails when some other class in the stack has attribute "do_it", since this is the signal name for stop walking the mro.

How to not accidentally override a method in python?

I know there are a bunch of similar questions out there. But my question is different.
I don't want to make a method which can't be overridden.
I want to protect my newly created class to not accidentally override something.
Using underscore as a prefix is pretty good, but soon I'll get a lot of methods with a lot of underscores. and somewhere in my inherited class, I will override the grand-ancestor's class method.
What I really want is something as simple as this:
class Cat(Mammal):
def walk(self):
if ancestor_has_function('walk'):
parent.walk();
do_something_here();
If any of Cat's ancestor (Either it is Mammal, Animal, or LifeForm) has "walk" method, then the parent method should be executed first.
Is that any possibility to do this in python?
EDIT:
For instance this is the resume of answers I considered as good. Hope this will help others:
class Animal(object):
pass
#def walk(self):
# print('animal walk')
class Mammal(Animal):
def walk(self):
if hasattr(super(Mammal, self), 'walk') and callable(super(Mammal,self).walk):
super(Mammal, self).walk()
print('mammal walk')
class Cat(Mammal):
def walk(self):
super(Cat, self).walk()
print('cat walk')
if __name__ == '__main__':
cat = Cat()
cat.walk()
And here is the output:
mammal walk
cat walk
Try to uncomment Animal's walk method, and you will have it work as well too.
Generally speaking, you'll probably want to provide at least a stub method in whichever superclass is the most generic:
class Mammal(object):
def walk(self):
pass
Then, extend it in subclasses by calling super():
class Cat(Mammal):
def walk(self):
super(Cat, self).walk() # or just super().walk(), in Python 3+
do_something_here()
Making the super() call conditional is not hard, but it's probably a bad idea: it's verbose, fragile, and only encourages bad practices. If you really, really have good reason to do it, you can just use hasattr() on the super object, like you would with any other object:
class Cat(Mammal):
def walk(self):
if hasattr(super(Cat, self), 'walk'):
super(Cat, self).walk()
do_something_here()
You would only want to do this in unusual situations, though, such as subclassing classes from a third-party library where you can't rely on certain methods being present, for some reason.
Yep. hasattr checks if there is an attribute with a specific name.
and callable checks if the specific attribute is callable.
class Mammal(object):
def walk(self):
print "walking"
class Cat(Mammal):
def walk(self):
if hasattr(Mammal,'walk') and callable(Mammal.walk):
Mammal.walk(self);
print "another walking!"
and now:
>>> my_cat = Cat()
>>> my_cat.walk()
walking
another walking!
Note that you can also use super to get your parent class like that:
if hasattr(super(Cat, self),'walk'):
You can use the dir() function to get all the names declared for some module or class. Methods declared in classes higher up in the hierarchy will also be included. Note, however, that this will also include attributes, so check with callable() first.
Also, calling the parent method looks a bit different in python, see the code below.
def walk(self):
if "walk" in dir(Mammal) and callable(Mammal.walk):
Mammal.walk(self)
# do something
you can keep your original method in a field
class MyClass:
def __method(self):
pass
def __init__(self):
self.method = __method
and than check for identity and call the saved method
import inspect
class SomeClass():
def __init__(self):
...
def somefunc(self):
....
def someOtherFunc(self):
....
allmembers = inspect.getmembers(SomeClass, predicate=inspect.ismethod)
getmembers returns a list of all methods define within the given class, it is a list of tuples that contains method names and definitions:
[('__init__', <unbound method SomeClass.__init__>),
('somefunc', <unbound method SomeClass.somefunc>),
('someOtherFunc', <unbound method SomeClass.someOtherFunc>)]
Since first elements of the tuple are strings, you can use string based methods to filter base methods like __init__
allmembers = filter(lambda x: not x.startswith('__'), [x[0] for x in inspect.getmembers(SomeClass, predicate=inspect.ismethod))])
[('somefunc', <unbound method SomeClass.somefunc>),
('someOtherFunc', <unbound method SomeClass.someOtherFunc>)]
You can get a list of all methods defined within the class and check if you have a similarly named method, Sincegetmembers returns you an unbound method instance, you can also reach that function easily.

Categories