Trying to determine if I should try to use decorator or some other Pythonic way to reduce code that many of my functions are doing. I would like these functions to be able to call just maybe one other function at the start of each function or somehow "decorate" the start of each function. I have never used decorator before and am struggling to implement this decorate idea in a pythonic way to reduce the common set of share code at each function.
I have many functions that will perform the same set of steps at the start of the function. However, there is some structure concerns of the common code that makes this "decorator" idea difficult :
The functions are all in child class of a parent class.
The common commands between the functions reference variable names
that are specific to the function (but a subset of the function
name).
The common commands need to return to the caller and not execute any
more of the child function if a certain condition is met. ("if jobj : " block in the sample code)
For variable/attribute examples, child function get_nas_server(self) will utilize "nas_server" variable variants in the common set of code. Subtracting the get_ from the function name reveals the base of the variable name to be used in the common set of code. Example variables names and object attributes derived from "get_nas_server" function name:
nas_server
nas_server.json
self.nas_server (attribute)
Here is the common code from one of the functions:
####################################################################
def get_nas_server(self):
####################################################################
"""\nGets COMMAND nas_server and places data into self.nas_server"""
try:
self.nas_server
return self.nas_server
except AttributeError:
pass
self.get_file_cmd('nas_server')
jobj = self.fresh_json('nas_server.json')
if jobj :
self.nas_server = jobj
return self.nas_server
self.get_file_cmd('get_nas_server')
Everything below that code above in the function is specific to the function purpose and not appropriate for discussion here. Basically I am trying to make all that code above reusable in my functions, but the code has to have the variables and attribute changed depending on the function name.
Thanks for reading if you got this far and thanks for all help.
Seems like something you could define as a helper method in the parent class:
class Parent(object):
def _get_something(name):
try:
return getattr(self, name)
except AttributeError:
pass
self.get_file_cmd(name)
jobj = self.fresh_json(name+'.json')
if jobj :
setattr(self, name, jobj)
return jobj
self.get_file_cmd('get_'+name)
As this snippet demonstrates, you can use the getattr(), setattr() and hasattr() functions to reference object attributes by name.
You can use a decorator inside your class, if the initial bookkeeping stuff is too specific to that class. Everytime you need to do that initial check inside a class method, you can simply decorate it with the auto_nas decorator I have added below. The scope of this wrapper, I am assuming, is specific to this class alone. Otherwise, you can move it outside the class for generality, but remember to change (self=None) if you're going to do so.
class YourMainClass():
# This is your main class inside which all the methods are defined.
def auto_nas(func):
# This wrapper function will be called instead of your class methods.
def wrap(self):
# All your bookkeeping stuff goes here.
try:
self.nas_server
return self.nas_server
except AttributeError:
pass
self.get_file_cmd('nas_server')
jobj = self.fresh_json('nas_server.json')
if jobj :
self.nas_server = jobj
return self.nas_server
self.get_file_cmd('get_nas_server')
# Things went well - we shall call the actual method.
return func(self)
return wrap
#auto_nas
def get_nas_server(self):
# This is one of your methods that require bookkeeping
pass
Also, notice the self inside the wrapper function. It is automatically passed as an argument by Python, and thus you can access all the instance specific attributes from inside that decorator. Once again, this is appropriate if your decorator must do jobs specific to each instances of your class. If that is not the case, you can simply put all of that code in a function and call it each time whenever you want.
Much thanks to Augurar for pointing in the proper direction. I ended up created a stub method in the parent class. Just needed to call the method with the proper technique to "extend" the parent method. Then calling that stub method from the child class with super() There must be a proper name for this technique? method override? method extend?
Here is the parent class method mostly as Augurar suggested:
###########################################################################
def get_stub(self,fcmd):
###########################################################################
try:
return getattr(self, fcmd)
except AttributeError:
pass
jobj = self.fresh_json(fcmd + '.json')
if jobj :
setattr(self, fcmd, jobj)
return jobj
if not self.get_file_cmd(fcmd):
self.log.error("{} not found".format(fcmd))
return False
Here is the manner in which the child method uses this parent method:
####################################################################
def get_nas_server(self):
####################################################################
super().get_stub('nas_server')
#bunch of child method specific code below here
Related
I am trying to unit test a block of code, and I'm running into issues with mocking the object's type to grab the right function from a dictionary.
For example:
my_func_dict = {
Foo: foo_func,
Bar: bar_func
FooBar: foobar_func
}
def generic_type_func(my_obj):
my_func = my_func_dict[type(my_obj)]
my_func()
With this code, I can swap between functions with a key lookup, and it's pretty efficient.
When I try to mock my_obj like this, I get a KeyError:
mock_obj = Mock(spec=Foo)
generic_type_func(mock_obj)
# OUTPUT:
# KeyError: <class 'unittest.mock.Mock'>
Because it's a mock type. Although, when I check isinstance(), it returns true:
is_instance_Foo = isinstance(mock_obj, Foo)
print(is_instance_foo)
# Output:
# True
Is there any way to retain the type() check, and using the dictionary lookup via a key, while still maintaining the ability to mock the input and return_type? Or perhaps a different pattern where I can retain the performance of a dictionary, but use isinstance() instead so I can mock the parameter? Looping over a list to check the type against every possible value is not preferred.
I managed to unit test this by moving the function to the parameter itself, and implicitly calling the function from the parent. I wanted to avoid this, because now the function manipulates the parent implicitly instead of explicitly from the parent itself. It looks like this now:
def generic_type_func(self, my_obj):
my_obj.my_func(self)
The function then modifies self as needed, but implicitly instead of an explicit function on the parent class.
This:
def my_func(self, parent):
self.foo_prop = parent
Rather than:
def my_foo_func(self, foo):
foo.foo_prop = self
This works fine with a mock, and I can mock that function easily. I've just hidden some of the functionality, and edit properties on the parent implicitly instead of explicitly from within the class I'm working in. Maybe this is preferable anyways, and it looks cleaner with less code on the parent class. Every instance must have my_func this way, which is enforced via an abstract base class.
I am a newbie in object oriented programming. After defining a class we simply create objects and try to access different variables and functions inside the class. In the following code I want to know that why we again have to mention class Dataset inside the function ans secondly what is the role of pass statement?
def read_data_set():
class Dataset:
pass
data_sets = Dataset()
return(data_sets)
#Function call
x=read_data_set()
print(x)
pass does nothing, it just makes Python indentations correct.
Let's say you want to create an empty function, sth like:
def empty_func():
empty_func() # throws IndentationError
Let's try to put a comment inside:
def empty_func_with_comment():
# empty
empty_func_with_comment() # throws IndentationError
To make it work we need to fix indentations by using pass:
def empty_func_with_pass():
pass
empty_func_with_pass() # it works
It basically does nothing.
It is often used as a placeholder like in your code; you'll notice that the code does not run at all without the pass there.
class SomeClass:
pass # to be filled
This is because Python expects something to be under the definition of SomeClass, but being empty it raises an IndentationError.
class SomeClass:
# to be filled <--- empty
other_function() # <--- IndentationError
It is also used with try-except when you do not want to do anything:
try:
some_stuff()
except SomeException:
pass # I don't mind this
Why do we mention class Dataset inside the function twice?
The first time
class Dataset:
pass
the class is defined,
and the second one:
data_sets = Dataset()
an instance of this class (an object) is created. Exactly as the OP has written:
After defining a class we simply create objects.
Since class is just a python statement it can be used anywhere: including function bodies, like in this case. So here the class is defined each time the function read_data_set() is called and is not defined at all if it is not called.
What is the role of pass statement?
In this example
class Dataset:
pass
the pass statement means defining a class with no members added inside it. It means that the class and its objects contain only some "default" functions and variables (aka methods and fields) that are derived from object by any class.
In general pass is used when you introduce a new block and want to leave it empty:
expression:
pass
You must include at least one instruction inside a block, that's why sometimes you need pass to say that you don't want to do anything inside the block. It is true for functions:
def do_nothing():
pass # function that does nothing
loops:
for i in collection: # just walk through the collection
pass # but do nothing at each iteration
exception handling:
try:
do_something()
except SomeException:
pass # silently ignore SomeException
context managers:
with open(filename): # open a file but do nothing with it
pass
The pass statement means that you initialise your class without defining a constructor or any attributes. Try ommitting it : The error that you see is due to the fact that python will expect the following line to belong to your class - and will consider that the indentation used is not correct.
Regarding the fact that your class name is called again inside your function : it means that you are instanciating the class you just defined. Thus, what your function returns is an object of your class.
I built a little decorator for logging purposes.
def func_detail(func):
def func_wrapper(*args,**kwargs):
log(func.__name__+' ARGS: {}'.format(str(args)))
return func(*args,**kwargs)
return func_wrapper
This works for both object methods and normal methods. I want to use it in multithreading. I have a class which contains pid as an object attribute. Is it possible to change the decorator to log pid if it detects that the method belongs to some class and this class contains attribute pid?
I've tried:
def func_detail(func):
def func_wrapper(*args,**kwargs):
log('PID: '+self.pid if self.pid is not None else ' '+func.__name__+' ARGS: {}'.format(str(args)))
return func(*args,**kwargs)
return func_wrapper
But this not works at all. Could you help me?
ABSTRACT:
I want to be able to call attribute pid from the class where the method (func) belongs without passing self as an argument to the wrapper because in that case it would not works for methods which aren't inside a classes.
The self argument to methods is not magically made available to your func_wrapper function in your decorator. Rather, it will be the first of the position arguments you're capturing with *args. If you want to make use of it, you'll need to examine args[0] and see if it has a pid attribute.
Try this, which checks first that a first argument exists, then that if it has a pid attribute:
log('{}FUNC: {} ARGS: {}'.format('PID: {} '.format(args[0].pid)
if args and hasattr(args[0], "pid") else '',
func.__name__, args))
If you are calling a method, the object itself will be the first parameter, the self in the method implementation.
If your decorator was only applied to methods (defined as def methodName(self, ...):) you could capture the first parameter in self.
Then you could try to print self.pid and catch the exception if there isn't any such attribute.
Since there is little distinction between free functions and method, I think that you should define two decorators one for free function, and another for method, or define a decorator taking a parameter saying wether it is a method or not.
Another solution is to check if the args isn't empty and print args[0].pid if it exists.
I encountered a problem with the availability of objects created within the decorator, and needed in the test_case method. My code presenting below:
def execute_results_navigation(test_case):
def wrapper(self,*args,**kwargs):
result=Result()
pagination=Pagination()
results_page_index=1
while results_page_index<=pagination.get_pages_number():
for results_object_index in range(results.get_objects_number_per_single_page()):
test_case(self,*args,**kwargs)
pagination.set_active_page_number(results_page_index)
results_page_index+=1
return wrapper
In place of test_case method is "injected" the following code (everything takes place using predefined decorator):
#execute_results_navigation
def test_check_availability_of_search_results(self):
"""
test case 2.22
"""
offer=Offer()
result.select_hotel(results_caller["button"],results_object_index)
offer_price=offer.get_offer_object_details().price
offer.verify_offer_availability(offer_price)
offer.back_to_search_results()
test_case method has no access to result,pagination objects and results_object_index variable. All objects have been initialized when calling the decorator. Maybe I'm doing something wrong with this method, but I thought that these instances exist within the wrapper method and access to them should not cause problems.
You won't be able to access local variables defined in wrapper within test_case.
Looks like test_check_availability_of_search_results is an instance method, so one way to solve your problem is to assign those variables to attributes of 'self'.
I'm trying to get the name of all methods in my class.
When testing how the inspect module works, i extraced one of my methods by obj = MyClass.__dict__['mymethodname'].
But now inspect.ismethod(obj) returns False while inspect.isfunction(obj) returns True, and i don't understand why. Is there some strange way of marking methods as methods that i am not aware of? I thought it was just that it is defined in the class and takes self as its first argument.
You are seeing some effects of the behind-the-scenes machinery of Python.
When you write f = MyClass.__dict__['mymethodname'], you get the raw implementation of "mymethodname", which is a plain function. To call it, you need to pass in an additional parameter, class instance.
When you write f = MyClass.mymethodname (note the absence of parentheses after mymethodname), you get an unbound method of class MyClass, which is an instance of MethodType that wraps the raw function you obtained above. To call it, you need to pass in an additional parameter, class instance.
When you write f = MyClass().mymethodname (note that i've created an object of class MyClass before taking its method), you get a bound method of an instance of class MyClass. You do not need to pass an additional class instance to it, since it's already stored inside it.
To get wrapped method (bound or unbound) by its name given as a string, use getattr, as noted by gnibbler. For example:
unbound_mth = getattr(MyClass, "mymethodname")
or
bound_mth = getattr(an_instance_of_MyClass, "mymethodname")
Use the source
def ismethod(object):
"""Return true if the object is an instance method.
Instance method objects provide these attributes:
__doc__ documentation string
__name__ name with which this method was defined
__func__ function object containing implementation of method
__self__ instance to which this method is bound"""
return isinstance(object, types.MethodType)
The first argument being self is just by convention. By accessing the method by name from the class's dict, you are bypassing the binding, so it appears to be a function rather than a method
If you want to access the method by name use
getattr(MyClass, 'mymethodname')
Well, do you mean that obj.mymethod is a method (with implicitly passed self) while Klass.__dict__['mymethod'] is a function?
Basically Klass.__dict__['mymethod'] is the "raw" function, which can be turned to a method by something called descriptors. This means that every function on a class can be both a normal function and a method, depending on how you access them. This is how the class system works in Python and quite normal.
If you want methods, you can't go though __dict__ (which you never should anyways). To get all methods you should do inspect.getmembers(Klass_or_Instance, inspect.ismethod)
You can read the details here, the explanation about this is under "User-defined methods".
From a comment made on #THC4k's answer, it looks like the OP wants to discriminate between built-in methods and methods defined in pure Python code. User defined methods are of types.MethodType, but built-in methods are not.
You can get the various types like so:
import inspect
import types
is_user_defined_method = inspect.ismethod
def is_builtin_method(arg):
return isinstance(arg, (type(str.find), type('foo'.find)))
def is_user_or_builtin_method(arg):
MethodType = types.MethodType
return isinstance(arg, (type(str.find), type('foo'.find), MethodType))
class MyDict(dict):
def puddle(self): pass
for obj in (MyDict, MyDict()):
for test_func in (is_user_defined_method, is_builtin_method,
is_user_or_builtin_method):
print [attr
for attr in dir(obj)
if test_func(getattr(obj, attr)) and attr.startswith('p')]
which prints:
['puddle']
['pop', 'popitem']
['pop', 'popitem', 'puddle']
['puddle']
['pop', 'popitem']
['pop', 'popitem', 'puddle']
You could use dir to get the name of available methods/attributes/etc, then iterate through them to see which ones are methods. Like this:
[ mthd for mthd in dir(FooClass) if inspect.ismethod(myFooInstance.__getattribute__(mthd)) ]
I'm expecting there to be a cleaner solution, but this could be something you could use if nobody else comes up with one. I'd like if I didn't have to use an instance of the class to use getattribute.