I am writing a Python package in which I have different Classes with a common method called map. That method always returns a function and is intended to be use inside a method of another package. Is there a way to lists all methods that are called map inside my package?
You're going to need to dip into a bit of reflection on this one. The steps you need are to load a reference a given module, grab the list of classes and then examine each class to make sure it has the method map:
import sys, inspect
def map_classes(module_name):
for name, obj in inspect.getmembers(sys.modules[module_name]):
if inspect.isclass(obj):
try:
if callable(getattr(obj, 'map')):
yield name
except AttributeError:
pass
This method will return a generator of all names of all classes within a module that have a map method. Note, if they have a map attribute, it will not return the name of the class.
You don't need the inspect package. Everything you need is built-in.
>>> import numpy as np
>>> for name, obj in np.__dict__.items():
... if hasattr(obj,"tolist"):
... print name, obj
Related
I have a Python file from which I would like to get all functions. I want to pass these functions to another file in order to collect data about these functions.
For Example.py:
class Example:
def method_to_extract(name: str) -> none:
print(name)
I want to extract this method as an object through the file name and pass it as an argument to another function, like
func = magic_method(Example.py) # returns method_to_extract as a callable object
collect_data(func)
Is this possible? So far, I have only been able to extract the names of functions as strings. However, that does not help me, as I want to collect data about the functions like the number of arguments, type hints, and more, and therefore need to be able to access the entire function.
I also tried getting the function using func = getattr(class, funcname), but I wasn't able to pass func as a parameter and extract the data I need.
I can think of 3 solutions, dependant on your particular needs.
Simplest one: don't use classes, just global functions:
# file1.py
def method_to_extract(name):
print(name)
And then in another file just import and use it:
# file2.py
from file2 import method_to_extract
method_to_extract()
In case you especially want to use methods inside of a class, you can make them static with #staticmethod decorator:
# file1.py
class Example:
#staticmethod
def method_to_extract(name):
print(name)
And then in another file just import and use it:
# file2.py
from file2 import Example
Example.method_to_extract()
More on staticmetod decorator here: https://www.programiz.com/python-programming/methods/built-in/staticmethod
Of course, not all of your methods can be static. Then you just have to create an instance of the class and then use its methods:
# file1.py
class Example:
def method_to_extract(name):
print(name)
And then in another file just import and use it:
# file2.py
from file2 import Example
instance = Example()
instance.method_to_extract()
Basically, the above three approaches are in general only possible ways of accessing some particular function or method in python and the fact of exporting them to another file doesn't change a thing.
Something like this:
from Example import Example.method_to_extract as mte
collect_data(mte)
The code you show is not entirely correct; you would have to make method_to_extract a #staticmethod if it's inside the Example class. Perhaps a better solution then would be to define it outside the class; or define a #classmethod which you can call to return a method of that class as an object. But both of these are generally best avoided; your class should probably simply have a method which implements collect_data, and your code should simply call that instead.
hi I just started teaching myself Python couple of days ago and right now got really confused about the usage of"." when calling Method/Function. when def is created within a class, it would be a"Method", where you would have to call it by typing "the name of the instance "."the name of the method"()
this makes sense because the first argument "self" is connected to the instance created.
however, when using Numpy I encountered a "function" np.random.rand()
How is this possible? this function already has two objectives(np,random). how can this be created when using only two elements "package(class)" and "moduel(def)". what does "." mean in this function? Thank you
The rand() function you want to use is part of the random module of the numpy package.
Without going deep into packages and namespace structure, here is a quick example of why this matters:
If you import packages, you load additional functions, classes and structures into your namespace. But maybe you want to use math and numpy, which both have a sin()-function:
import numpy as np
import math
a = np.sin()
b = math.sin()
a and b are not the same function, but have the same name, so it is useful to call them by the name of the containing package.
The same applies for the standard sum() and the np.sum().
If you explicitly want to use the rand() function without calling it by the full name, you can do that by
from numpy.random import rand
In Python, just like in Java, it is possible to define classes. Classes are representations of a concept that you want to include in your code. For example, imagine that you create a person class where you want to store name and height. A very simple implementation could be:
class person:
def __init__(self, name, height):
self.name = name
self.height = height
def grow_up(self, cm):
self.height += cm
return
In this case, __init__(self, name, height) and grow_up(self, cm) are methods: they are defined inside a class and they take as a first argument self, that is, the instance of the class itself. An instance of the class is, in this example, any single person that you associate with this class. For example
a = person("John", 120)
Now, a is an instance of the class person and the arguments "John" and 120 are passed to the special initializer method __init__. If you call a.height you will obtain 120. (Should you use a getter for that? Maybe yes, maybe no, but that's not the point of your question)
Using the method grow_up is as simple as typing
a.grow_up(10)
Calling a.height again you now obtain 130.
A function is not defined inside a class, therefore, it can't have a self variable. It is similarly defined with a def command, however, which can generate some confusion. To make things even more ambiguous for beginners, python defines modules and packages. A module is a collection of functions stored in a single file, and a package is a directory containing one or more such modules. The standard way of navigating the namespace is the same as for class functions: using a dot (.) to separate packages, modules, and functions, going from the outermost to the innermost level.
Package np, package random and function rand effectively are python objects. Objects can have attributes and you can access them with dot notation and this chain (obj.attr1.attr2...attrN) can be long.
So in your example you just access attribute rand of attribute random of package np, and since rand is a callable object, you call it with rand().
When you are using numpy or any other packages it will not create a object.
For example:
import numpy as np
It will only download this package and you can use it.
In other words you can say numpy is a folder and it contain subfolders and scripts. Random is a subfolder. Rand() is a function.
Conclusion:
np.random.rand()
You are using dots to define a path for this function.
Here is a code for numpy package: https://github.com/numpy/numpy/tree/master/numpy
I was just curious if it was possible to create a module object inside python at runtime, without loading from any python file. The purpose of this would be to create a new empty namespace where other objects can then be stored subsequently. If this is not possible, is there another way to make and pass namespaces in python without saving to disk?
You can use a class with static methods.
class Namespace:
#staticmethod
def greet():
print "hello, world!"
In Python 3 the #staticmethod decorator is not needed.
You can use a simple class:
class Namespace:
pass
Now, to create a new namespace:
n = Namespace()
To store things in the namespace:
n.foo = 1
def square(x):
return x*x
n.squared = square
To refer to things in the namespace:
print n.foo
print n.squared(12)
To pass the namespace:
def func_requiring_a_namesapce(space):
print space.foo
func_requiring_a_namespace(n)
You could use a dictionary?
Modules Are Like Dictionaries
You know how a dictionary is created and used and that it is a way to map one thing to another. That means if you have a dictionary with a key 'apple' and you want to get it then you do this:
mystuff = {'apple': "I AM APPLES!"}
print mystuff['apple']
Imagine if I have a module that I decide to name mystuff.py and I put a function in it called apple. Here's the module mystuff.py:
# this goes in mystuff.py
def apple():
print "I AM APPLES!"
Once I have that, I can use that module with import and then access the apple function:
import mystuff
mystuff.apple()
I could also put a variable in it named tangerine like this:
def apple():
print "I AM APPLES!"
# this is just a variable
tangerine = "Living reflection of a dream"
Then again I can access that the same way:
import mystuff
mystuff.apple()
print mystuff.tangerine
Refer back to the dictionary, and you should start to see how this is similar to using a dictionary, but the syntax is different. Let's compare:
mystuff['apple'] # get apple from dict
mystuff.apple() # get apple from the module
mystuff.tangerine # same thing, it's just a variable
In the case of the dictionary, the key is a string and the syntax is [key]. In the case of the module, the key is an identifier, and the syntax is .key. Other than that they are nearly the same thing.
Editied from here
tl;dr: How come property decorators work with class-level function definitions, but not with module-level definitions?
I was applying property decorators to some module-level functions, thinking they would allow me to invoke the methods by mere attribute lookup.
This was particularly tempting because I was defining a set of configuration functions, like get_port, get_hostname, etc., all of which could have been replaced with their simpler, more terse property counterparts: port, hostname, etc.
Thus, config.get_port() would just be the much nicer config.port
I was surprised when I found the following traceback, proving that this was not a viable option:
TypeError: int() argument must be a string or a number, not 'property'
I knew I had seen some precedant for property-like functionality at module-level, as I had used it for scripting shell commands using the elegant but hacky pbs library.
The interesting hack below can be found in the pbs library source code. It enables the ability to do property-like attribute lookups at module-level, but it's horribly, horribly hackish.
# this is a thin wrapper around THIS module (we patch sys.modules[__name__]).
# this is in the case that the user does a "from pbs import whatever"
# in other words, they only want to import certain programs, not the whole
# system PATH worth of commands. in this case, we just proxy the
# import lookup to our Environment class
class SelfWrapper(ModuleType):
def __init__(self, self_module):
# this is super ugly to have to copy attributes like this,
# but it seems to be the only way to make reload() behave
# nicely. if i make these attributes dynamic lookups in
# __getattr__, reload sometimes chokes in weird ways...
for attr in ["__builtins__", "__doc__", "__name__", "__package__"]:
setattr(self, attr, getattr(self_module, attr))
self.self_module = self_module
self.env = Environment(globals())
def __getattr__(self, name):
return self.env[name]
Below is the code for inserting this class into the import namespace. It actually patches sys.modules directly!
# we're being run as a stand-alone script, fire up a REPL
if __name__ == "__main__":
globs = globals()
f_globals = {}
for k in ["__builtins__", "__doc__", "__name__", "__package__"]:
f_globals[k] = globs[k]
env = Environment(f_globals)
run_repl(env)
# we're being imported from somewhere
else:
self = sys.modules[__name__]
sys.modules[__name__] = SelfWrapper(self)
Now that I've seen what lengths pbs has to go through, I'm left wondering why this facility of Python isn't built into the language directly. The property decorator in particular seems like a natural place to add such functionality.
Is there any partiuclar reason or motivation for why this isn't built directly in?
This is related to a combination of two factors: first, that properties are implemented using the descriptor protocol, and second that modules are always instances of a particular class rather than being instantiable classes.
This part of the descriptor protocol is implemented in object.__getattribute__ (the relevant code is PyObject_GenericGetAttr starting at line 1319). The lookup rules go like this:
Search through the class mro for a type dictionary that has name
If the first matching item is a data descriptor, call its __get__ and return its result
If name is in the instance dictionary, return its associated value
If there was a matching item from the class dictionaries and it was a non-data descriptor, call its __get__ and return the result
If there was a matching item from the class dictionaries, return it
raise AttributeError
The key to this is at number 3 - if name is found in the instance dictionary (as it will be with modules), then its value will just be returned - it won't be tested for descriptorness, and its __get__ won't be called. This leads to this situation (using Python 3):
>>> class F:
... def __getattribute__(self, attr):
... print('hi')
... return object.__getattribute__(self, attr)
...
>>> f = F()
>>> f.blah = property(lambda: 5)
>>> f.blah
hi
<property object at 0xbfa1b0>
You can see that .__getattribute__ is being invoked, but isn't treating f.blah as a descriptor.
It is likely that the reason for the rules being structured this way is an explicit tradeoff between the usefulness of allowing descriptors on instances (and, therefore, in modules) and the extra code complexity that this would lead to.
Properties are a feature specific to classes (new-style classes specifically) so by extension the property decorator can only be applied to class methods.
A new-style class is one that derives from object, i.e. class Foo(object):
Further info: Can modules have properties the same way that objects can?
I'm trying to get the name of all methods in my class.
When testing how the inspect module works, i extraced one of my methods by obj = MyClass.__dict__['mymethodname'].
But now inspect.ismethod(obj) returns False while inspect.isfunction(obj) returns True, and i don't understand why. Is there some strange way of marking methods as methods that i am not aware of? I thought it was just that it is defined in the class and takes self as its first argument.
You are seeing some effects of the behind-the-scenes machinery of Python.
When you write f = MyClass.__dict__['mymethodname'], you get the raw implementation of "mymethodname", which is a plain function. To call it, you need to pass in an additional parameter, class instance.
When you write f = MyClass.mymethodname (note the absence of parentheses after mymethodname), you get an unbound method of class MyClass, which is an instance of MethodType that wraps the raw function you obtained above. To call it, you need to pass in an additional parameter, class instance.
When you write f = MyClass().mymethodname (note that i've created an object of class MyClass before taking its method), you get a bound method of an instance of class MyClass. You do not need to pass an additional class instance to it, since it's already stored inside it.
To get wrapped method (bound or unbound) by its name given as a string, use getattr, as noted by gnibbler. For example:
unbound_mth = getattr(MyClass, "mymethodname")
or
bound_mth = getattr(an_instance_of_MyClass, "mymethodname")
Use the source
def ismethod(object):
"""Return true if the object is an instance method.
Instance method objects provide these attributes:
__doc__ documentation string
__name__ name with which this method was defined
__func__ function object containing implementation of method
__self__ instance to which this method is bound"""
return isinstance(object, types.MethodType)
The first argument being self is just by convention. By accessing the method by name from the class's dict, you are bypassing the binding, so it appears to be a function rather than a method
If you want to access the method by name use
getattr(MyClass, 'mymethodname')
Well, do you mean that obj.mymethod is a method (with implicitly passed self) while Klass.__dict__['mymethod'] is a function?
Basically Klass.__dict__['mymethod'] is the "raw" function, which can be turned to a method by something called descriptors. This means that every function on a class can be both a normal function and a method, depending on how you access them. This is how the class system works in Python and quite normal.
If you want methods, you can't go though __dict__ (which you never should anyways). To get all methods you should do inspect.getmembers(Klass_or_Instance, inspect.ismethod)
You can read the details here, the explanation about this is under "User-defined methods".
From a comment made on #THC4k's answer, it looks like the OP wants to discriminate between built-in methods and methods defined in pure Python code. User defined methods are of types.MethodType, but built-in methods are not.
You can get the various types like so:
import inspect
import types
is_user_defined_method = inspect.ismethod
def is_builtin_method(arg):
return isinstance(arg, (type(str.find), type('foo'.find)))
def is_user_or_builtin_method(arg):
MethodType = types.MethodType
return isinstance(arg, (type(str.find), type('foo'.find), MethodType))
class MyDict(dict):
def puddle(self): pass
for obj in (MyDict, MyDict()):
for test_func in (is_user_defined_method, is_builtin_method,
is_user_or_builtin_method):
print [attr
for attr in dir(obj)
if test_func(getattr(obj, attr)) and attr.startswith('p')]
which prints:
['puddle']
['pop', 'popitem']
['pop', 'popitem', 'puddle']
['puddle']
['pop', 'popitem']
['pop', 'popitem', 'puddle']
You could use dir to get the name of available methods/attributes/etc, then iterate through them to see which ones are methods. Like this:
[ mthd for mthd in dir(FooClass) if inspect.ismethod(myFooInstance.__getattribute__(mthd)) ]
I'm expecting there to be a cleaner solution, but this could be something you could use if nobody else comes up with one. I'd like if I didn't have to use an instance of the class to use getattribute.