I am building a function to construct objects with set attributes (similar to a namedtuple); however, the output length must be variable.
I would like to build a function that allows the user to append additional attributes through a function call. Importantly, I would like to find a way to 'short-circuit' parameters and am unsure if Python is powerful enough to do this.
To explain take this trivial example:
def foo():
print("foo")
return False
def bar():
print("bar")
return True
if foo() and bar():
pass
Foo's function call returns False, and Bar short-circuits. The output console will only print foo, and bar is never executed.
Is there such a way to mimic this behavior with inspection or reflection in respect to function calls. Here is an example with my implementation is shown below:
from inspect import stack
cache = {}
def fooFormat(**kwargs):
caller = stack()[1][3]
if caller not in cache:
class fooOut(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
def optional(self, opt, **kwargs):
if (opt):
self.__dict__.update(kwargs)
return self
def __str__(self):
return caller + str(self.__dict__)
cache[caller] = iadsOut
return cache[caller](**kwargs)
def stdev(nums, avg = None):
print("\tStdev call")
if avg is None:
avg = sum(nums) / len(nums)
residuals = sum((i - avg)**2 for i in nums)
return residuals**.5
def stats(nums, verbose=False):
if verbose:
print("Stats call with verbose")
else:
print("Stats call without verbose")
total = sum(nums)
N = len(nums)
avg = total / N
return fooFormat(
avg = avg,
lowerB = min(nums),
upperB = max(nums)).optional(verbose,
stdev = stdev(nums, avg))
In the function 'stats', the return fooFormat should of course yield avg, lowerB, and upperB; additionally, it should yield std if verbose is set to True. Moreover, the function 'stdev' should NOT be called if verbose is set to False.
stats([1,2,3,4], False)
stats([1,2,3,4], True)
Of course, a way around this is:
if verbose:
return fooFormat(
avg = avg,
lowerB = min(nums),
upperB = max(nums),
stdev = stdev(nums, avg))
else:
return fooFormat(
avg = avg,
lowerB = min(nums),
upperB = max(nums))
However, I am hoping there to implement this behavior without a branch.
This doesn't quite answer the shortcutting point, but this is a more efficient way of writing it:
out_dic = { # these items will always be calculated
'avg': avg,
'lowerB':max(nums),
'upperB':min(nums)
}
if verbose: # this is calculated only if verbose
out_dic['stdev'] = stdev(nums,avg)
return fooFormat(**out_dic)
In other words you can expand a dictionary to the kwargs, and add to the dictionary dynamically.
Related
This is a continuation of the SO question asked here but with a more complicated pattern than originally requested. My main intention is to try to mock an API call based on values passed to its caller. The API call has no idea of the values passed to its caller but needs to provide the correct behavior so that the caller can be tested fully. I am using time to determine which behavior I want when I want it.
Given a an object:
# some_object.py
from some_import import someApiCall
class SomeObject():
def someFunction(time, a, b, c):
apiReturnA = someApiCall(a)
returnB = b + 1
apiReturnC = someApiCall(c)
return [apiReturnA, returnB, apiReturnC]
that is created by another object with entry point code:
# some_main.py
import some_object
class myMainObject():
def entry_point(self, time):
someObj = some_object.SomeObject()
if 'yesterday' == time:
(a, b, c) = (1, 1, 1)
elif 'today' == time:
(a, b, c) = (2, 2, 2)
elif 'later' == time:
(a, b, c) = (3, 3, 3)
elif 'tomorrow' == time:
(a, b, c) = (4, 4, 4)
else:
return "ERROR"
return someObj.someFunction(time, a, b, c)
how can I get someApiCall to change based on the time argument?
# some_import.py
def someApiCall(var):
print("I'm not mocked! I'm slow, random and hard to test")
return var + 2
Here is an example test case
# test_example.py
import some_main
def amend_someApiCall_yesterday(var):
# Reimplement api.someApiCall
return var * 2
def amend_someApiCall_today(var):
# Reimplement api.someApiCall
return var * 3
def amend_someApiCall_later(var):
# Just wrap around api.someApiCall. Call the original function afterwards. Here we can also put
# some conditionals e.g. only call the original someApiCall if the var is an even number.
import some_import
var *= 4
return some_import.someApiCall(var)
def someObject_decorator_patch(someFunction, mocker, *args):
def wrapper(time, a, b, c):
# If x imports y.z and we want to patch the calls to z, then we have to patch x.z. Patching
# y.z would still retain the original value of x.z thus still calling the original
# functionality. Thus here, we would be patching src.someApiCall and not api.someApiCall.
if time == "yesterday":
mocker.patch("some_object.someApiCall", side_effect=amend_someApiCall_yesterday)
elif time == "today":
mocker.patch("some_object.someApiCall", side_effect=amend_someApiCall_today)
elif time == "later":
mocker.patch("some_object.someApiCall", side_effect=amend_someApiCall_later)
elif time == "tomorrow":
mocker.patch("src.someApiCall", return_value=0)
else:
# Use the original api.someApiCall
pass
return someFunction(time, a, b, c)
return wrapper
def test_some_main(mocker):
results = 0
uut = some_main.myMainObject()
times = ['yesterday', 'today', 'later', 'tomorrow']
mocker.patch.object(some_main.some_object.SomeObject, 'someFunction', someObject_decorator_patch)
for time in times:
results = uut.entry_point(time)
print(results)
assert 0 != results
The test case doesn't get the result I want (it returns a function pointer).
Here is an improvised solution of https://stackoverflow.com/a/67498948/11043825 without using decorators.
As already pointed out, we still need to intercept the calls to the function that accepts the time argument which indicates how someApiCall would behave, which is either entry_point or someFunction. Here we would intercept someFunction.
Instead of implementing python decorator on someFunction which then needs to call that explicitly created decorated function, here we would amend (well this still follows the decorator design pattern) the someFunction in-place and make it available to the rest of the source code calls without explicitly changing the call to the decorated function. This is like an in-place replacement of the original functionalities, where we would replace (or rather wrap around) the original functionality with an updated one which would perform an assessment of the time before calling the original functionality.
Also for your reference, I solved it for 2 types of functions, a class method src.SomeClass.someFunction and a global function src.someFunction2.
./_main.py
import src
class MyMainClass:
def __init__(self):
self.var = 0
def entry_point(self, time):
someObj = src.SomeClass()
self.var += 1
if self.var >= 10:
self.var = 0
ret = f'\n[1]entry_point({time})-->{someObj.someFunction(time, self.var)}'
self.var += 1
ret += f'\n[2]entry_point({time})-->{src.someFunction2(time, self.var)}'
return ret
./src.py
class SomeClass:
def someFunction(self, time, var):
return f'someFunction({time},{var})-->{someSloowApiCall(var)}'
def someFunction2(time, var):
return f'someFunction2({time},{var})-->{someSloowApiCall2(var)}'
./api.py
def someSloowApiCall(var):
return f'someSloowApiCall({var})-->{special_message(var)}'
def someSloowApiCall2(var):
return f'someSloowApiCall2({var})-->{special_message(var)}'
def special_message(var):
special_message = "I'm not mocked! I'm slow, random and hard to test"
if var > 10:
special_message = "I'm mocked! I'm not slow, random or hard to test"
return special_message
./test_main.py
import _main, pytest, api
def amend_someApiCall_yesterday(var):
# Reimplement api.someSloowApiCall
return f'amend_someApiCall_yesterday({var})'
def amend_someApiCall_today(var):
# Reimplement api.someSloowApiCall
return f'amend_someApiCall_today({var})'
def amend_someApiCall_later(var):
# Just wrap around api.someSloowApiCall. Call the original function afterwards. Here we can also put
# some conditionals e.g. only call the original someSloowApiCall if the var is an even number.
return f'amend_someApiCall_later({var})-->{api.someSloowApiCall(var+10)}'
def amend_someApiCall_later2(var):
# Just wrap around api.someSloowApiCall2. Call the original function afterwards. Here we can also put
# some conditionals e.g. only call the original someSloowApiCall2 if the var is an even number.
return f'amend_someApiCall_later2({var})-->{api.someSloowApiCall2(var+10)}'
def get_amended_someFunction(mocker, original_func):
def amend_someFunction(self, time, var):
if time == "yesterday":
mocker.patch("_main.src.someSloowApiCall", amend_someApiCall_yesterday)
# or
# src.someSloowApiCall = amend_someApiCall_yesterday
elif time == "today":
mocker.patch("_main.src.someSloowApiCall", amend_someApiCall_today)
# or
# src.someSloowApiCall = amend_someApiCall_today
elif time == "later":
mocker.patch("_main.src.someSloowApiCall", amend_someApiCall_later)
# or
# src.someSloowApiCall = amend_someApiCall_later
elif time == "tomorrow":
mocker.patch("_main.src.someSloowApiCall", lambda var: f'lambda({var})')
# or
# src.someSloowApiCall = lambda var: 0
else:
pass
# or
# src.someSloowApiCall = someSloowApiCall
return original_func(self, time, var)
return amend_someFunction
def get_amended_someFunction2(mocker, original_func):
def amend_someFunction2(time, var):
if time == "yesterday":
mocker.patch("_main.src.someSloowApiCall2", amend_someApiCall_yesterday)
# or
# src.someSloowApiCall2 = amend_someApiCall_yesterday
elif time == "today":
mocker.patch("_main.src.someSloowApiCall2", amend_someApiCall_today)
# or
# src.someSloowApiCall2 = amend_someApiCall_today
elif time == "later":
mocker.patch("_main.src.someSloowApiCall2", amend_someApiCall_later2)
# or
# src.someSloowApiCall2 = amend_someApiCall_later
elif time == "tomorrow":
mocker.patch("_main.src.someSloowApiCall2", lambda var : f'lambda2({var})')
# or
# src.someSloowApiCall2 = lambda var: 0
else:
pass
# or
# src.someSloowApiCall2 = someSloowApiCall2
return original_func(time, var)
return amend_someFunction2
#pytest.mark.parametrize(
'time',
[
'yesterday',
'today',
'later',
'tomorrow',
'whenever',
],
)
def test_entrypointFunction(time, mocker):
mocker.patch.object(
_main.src.SomeClass,
"someFunction",
side_effect=get_amended_someFunction(mocker, _main.src.SomeClass.someFunction),
autospec=True, # Needed for the self argument
)
# or
# src.SomeClass.someFunction = get_amended_someFunction(mocker, src.SomeClass.someFunction)
mocker.patch(
"_main.src.someFunction2",
side_effect=get_amended_someFunction2(mocker, _main.src.someFunction2),
)
# or
# src.someFunction2 = get_amended_someFunction2(mocker, src.someFunction2)
uut = _main.MyMainClass()
print(f'\nuut.entry_point({time})-->{uut.entry_point(time)}')
Output:
$ pytest -rP
=================================== PASSES ====================================
_____________________ test_entrypointFunction[yesterday] ______________________
---------------------------- Captured stdout call -----------------------------
uut.entry_point(yesterday)-->
[1]entry_point(yesterday)-->someFunction(yesterday,1)-->amend_someApiCall_yesterday(1)
[2]entry_point(yesterday)-->someFunction2(yesterday,2)-->amend_someApiCall_yesterday(2)
_______________________ test_entrypointFunction[today] ________________________
---------------------------- Captured stdout call -----------------------------
uut.entry_point(today)-->
[1]entry_point(today)-->someFunction(today,1)-->amend_someApiCall_today(1)
[2]entry_point(today)-->someFunction2(today,2)-->amend_someApiCall_today(2)
_______________________ test_entrypointFunction[later] ________________________
---------------------------- Captured stdout call -----------------------------
uut.entry_point(later)-->
[1]entry_point(later)-->someFunction(later,1)-->amend_someApiCall_later(1)-->someSloowApiCall(11)-->I'm mocked! I'm not slow, random or hard to test
[2]entry_point(later)-->someFunction2(later,2)-->amend_someApiCall_later2(2)-->someSloowApiCall2(12)-->I'm mocked! I'm not slow, random or hard to test
______________________ test_entrypointFunction[tomorrow] ______________________
---------------------------- Captured stdout call -----------------------------
uut.entry_point(tomorrow)-->
[1]entry_point(tomorrow)-->someFunction(tomorrow,1)-->lambda(1)
[2]entry_point(tomorrow)-->someFunction2(tomorrow,2)-->lambda2(2)
______________________ test_entrypointFunction[whenever] ______________________
---------------------------- Captured stdout call -----------------------------
uut.entry_point(whenever)-->
[1]entry_point(whenever)-->someFunction(whenever,1)-->someSloowApiCall(1)-->I'm not mocked! I'm slow, random and hard to test
[2]entry_point(whenever)-->someFunction2(whenever,2)-->someSloowApiCall2(2)-->I'm not mocked! I'm slow, random and hard to test
============================== 5 passed in 0.07s ==============================
def MainCount(f):
def progFirst(*args,**kwargs):
progFirst.calls+=1
return f(*args,**kwargs)
progFirst.calls=0
return progFirst
#MainCount
def progSecond(i):
return i+1
#MainCount
def Count(i=0,j=1):
return i*j+1
print(progSecond.calls)
for n in range(5):
progSecond(n)
Count(j=0,i=1)
print(Count.calls)
Output :0
1
As per my understanding MainCount(probSecond) but I am not understant then how probSecond.calls equal to zero same in Count.calls also
As You Can See in MainCount function probFirst.Calls is attribute of function .When MainCount(probSecond) Now probSecond.calls is also attribute of MainCount function.
# A Python example to demonstrate that
# decorators can be useful attach data
# A decorator function to attach
# data to func
def attach_data(func):
func.data = 3
return func
#attach_data
def add (x, y):
return x + y
# Driver code
# This call is equivalent to attach_data()
# with add() as parameter
print(add(2, 3))
print(add.data)
I'm sorry to ask such a basic question, but what's the Pythonic way to include the same if block that can conditionally return in multiple functions? Here's my setup:
def a():
if bool:
return 'yeehaw'
return 'a'
def b():
if bool:
return 'yeehaw'
return 'b'
I'd like to factor the common conditional out of the two functions, but I'm not sure how to do so.
Use a decorator or closure
def my_yeehaw(result):
def yeehaw():
if some_bool:
return 'yeehaw'
return result
return yeehaw
a = my_yeehaw('a')
b = my_yeehaw('b')
You could use a lambda that takes in a. bool and a default value to return if the condition is false:
check = lambda condition, default: 'yeehaw' if condition else default
def a():
return check(condition, 'a')
def b():
return check(condition, 'b')
I am new to python but I think you can use a default argument to send a or b based on what is passed to the function.
def a(x='a'):
if condition: #where condition can be True or False
return 'yeehaw'
return x
(note: my naming wasn't the best, consider that same_bool function might be better called identical_if_block(...) to follow your example
And I am also assuming bool_ is a parameter, though it could work as a global. But not as bool which, like any function object, is always Truthy
>>> bool(bool)
True
)
Use a function, as long as it doesn't need to return falsies.
def same_bool(bool_):
" works for any result except a Falsy"
return "yeehaw" if bool_ else None
def a(bool_):
res = same_bool(bool_)
if res:
return res
return 'a'
def b(bool_, same_bool_func):
#you can pass in your boolean chunk function
res = same_bool_func(bool_)
if res:
return res
return 'b'
print ("a(True):", a(True))
print ("a(False):", a(False))
print ("b(True, same_bool):", b(True,same_bool))
print ("b(False, same_bool):", b(False,same_bool))
output:
a(True): yeehaw
a(False): a
b(True, same_bool): yeehaw
b(False, same_bool): b
If you do need falsies, use a special guard value
def same_bool(bool_):
" works for any result"
return False if bool_ else NotImplemented
def a(bool_):
res = same_bool(bool_)
if res is not NotImplemented:
return res
return 'a'
You could also feed in "a" and "b" since they are constant results, but I assume that's only in your simplified example.
def same_bool(bool_, val):
return "yeehaw" if bool_ else val
def a(bool_):
return same_bool(bool_, "a")
I ended up liking the decorator syntax, as the functions that include the duplicative conditional logic have a good deal else going on in them:
# `function` is the decorated function
# `args` & `kwargs` are the inputs to `function`
def yeehaw(function):
def decorated(*args, **kwargs):
if args[0] == 7: return 99 # boolean check
return function(*args, **kwargs)
return decorated
#yeehaw
def shark(x):
return str(x)
shark(7)
I have this example code, trying to demonstrate using a callback function:
def callback(a, b):
print('Sum = {0}'.format(a+b))
def main(callback=None):
print('Add any two digits.')
if callback != None:
callback
main(callback(1, 2))
I get this result:
Sum = 3
Add any two digits.
It seems that the callback function executes before the logic in main. Why? How can I make it so that the callback is not called until it is used within main?
See also: Python Argument Binders
In this code
if callback != None:
callback
callback on its own doesn't do anything; it accepts parameters - def callback(a, b):
The fact that you did callback(1, 2) first will call that function, thereby printing Sum = 3, and then main() gets called with the result of the callback function, which is printing the second line
Since callback returns no explicit value, it is returned as None.
Thus, your code is equivalent to
callback(1, 2)
main()
Solution
You could try not calling the function at first and just passing its handle.
def callback(n):
print("Sum = {}".format(n))
def main(a, b, _callback = None):
print("adding {} + {}".format(a, b))
if _callback:
_callback(a+b)
main(1, 2, callback)
Here's what you wanted to do :
def callback(a, b):
print('Sum = {0}'.format(a+b))
def main(a,b,f=None):
print('Add any two digits.')
if f is not None:
f(a,b)
main(1, 2, callback)
The problem is that you're evaluating the callback before you pass it as a callable. One flexible way to solve the problem would be this:
def callback1(a, b):
print('Sum = {0}'.format(a+b))
def callback2(a):
print('Square = {0}'.format(a**2))
def callback3():
print('Hello, world!')
def main(callback=None, cargs=()):
print('Calling callback.')
if callback is not None:
callback(*cargs)
main(callback1, cargs=(1, 2))
main(callback2, cargs=(2,))
main(callback3)
Optionally you may want to include a way to support keyword arguments.
As mentioned in the comments, your callback is called whenever it's suffixed with open and close parens; thus it's called when you pass it.
You might want to use a lambda and pass in the values.
#!/usr/bin/env python3
def main(callback=None, x=None, y=None):
print('Add any two digits.')
if callback != None and x != None and y != None:
print("Result of callback is {0}".format(callback(x,y)))
else:
print("Missing values...")
if __name__ == "__main__":
main(lambda x, y: x+y, 1, 2)
Your code is executed as follows:
main(callback(1, 2))
callback function is called with (1, 2) and it returns None (Without return statement, your function prints Sum = 3 and returns None)
main function is called with None as argument (So callback != None will always be False)
This is an old post, but perhaps the following may be additional clarification on writing and using a callback function, especially if you wonder where it gets its arguments from and whether you can access its return values (if there is no way to get it from the function that takes the callback function).
The following code defines a class CallBack that has two callback methods (functions) my_callback_sum and my_callback_multiply. The callback methods are fed into the method foo.
# understanding callback
class CallBack:
#classmethod
def my_callback_sum(cls, c_value1, c_value2):
value = c_value1 + c_value2
print(f'in my_callback_sum --> {c_value1} + {c_value2} = {value}')
cls.operator = '+'
return cls.operator, value
#classmethod
def my_callback_multiply(cls, c_value1, c_value2):
value = c_value1 * c_value2
print(f'in my_callback_multiply --> {c_value1} * {c_value2} = {value}')
cls.operator = '*'
return cls.operator, value
#staticmethod
def foo(foo_value, callback):
_, value = callback(10, foo_value)
# note foo only returns the value not the operator from callback!
return value
if __name__ == '__main__':
cb = CallBack()
value = cb.foo(20, cb.my_callback_sum)
print(f'in main --> {value} and the operator is {cb.operator}')
value = cb.foo(20, cb.my_callback_multiply)
print(f'in main --> {value} and the operator is {cb.operator}')
result:
in my_callback_sum --> 10 + 20 = 30
in main --> 30 and the operator is +
in my_callback_multiply --> 10 * 20 = 200
in main --> 200 and the operator is *
As you can see one value for the callback function c_value2 it gets from argument foo_value in foo and given in main the value 20, while c_value1 it gets internally from foo in this case the value 10 (and may be not clearly visible if foo is some method of a third party imported module, like pyaudio).
The return value of the callback function functions can be retrieved by adding it to the namespace of the class CallBack, in this case cls.operator
You can use anonymous functions
def callback(a, b):
print('Sum = {0}'.format(a+b))
def main(callback=None):
print('Add any two digits.')
if callback is not None:
callback()
tmp_func = lambda: main(lambda: callback(2,3))
tmp_func()
#OR
tmp_func = lambda x,y: main(lambda: callback(x,y))
tmp_func(2,4)
So here's an extension to this question: https://stackoverflow.com/a/37568895/2290820
on how to optionally Enable or Disable Decorator on a Function.
On those lines, I came up with something like this to make decorator get invoked on a recursive call:
def deco(f):
def fattr(attr):
f.attr = attr
def closure(*args):
f(*args)
f.unwrap = f
f.closure = closure
return f
return fattr
#deco
def printa(x):
if x > 1:
print x
return printa(x-1)
else:
print x
return
printa({1:1})(5)
# do the same call w/o deocorator
def finta(x):
if x > 1:
print x
return finta(x-1)
else:
print x
return
finta(5) # this works
to experiment with decorators on a recursive function. Clearly, printa recursive version is not behaving the way it should be.
I could do
g = printa({1:1})
g.closure(5)
to turn on the decorator option or not use that option. Anyway, regardless of good or bad design, How can I make decorator get invoked on a recursive call?
In your deco you have an assignment f.attr = attr that "eats" your argument after first recursive call. Your should modify your recursive call this way:
def deco(f):
def fattr(attr):
f.attr = attr
def closure(*args):
f(*args)
f.unwrap = f
f.closure = closure
return f
return fattr
#deco
def printa(x):
if x > 1:
print x
return printa(None)(x-1) # None will be assigned to f.attr
else:
print x
return
printa({1:1})(5)
5
4
3
2
1