Assume that I don't care about occurring exceptions at all. I have a complex function that calls multiple functions along the way.. I want to test that with certain input parameters, certain functions will be called.
So basically, I am looking for something like:
#patch(
"service.module.class.some_nested_function_1",
new_callable=AsyncMock,
)
#patch(
"service.module.class.some_nested_function_2",
new_callable=AsyncMock,
)
#pytest.mark.asyncio
async def test_complex_function(function2_mock, function1_mock):
some_param = "abc"
another_param = "xyz"
try:
call_complex_function("with_some_configuration")
except:
abort_crashing_function_and_continue_execution_in_complex_function()
assert function2_mock.assert_called_once_with(some_param)
assert function1_mock.assert_called_once_with(another_param)
EDIT: An alternative idea would be to have something like:
...
async def test_complex_function(function2_mock, function1_mock):
...
mock_every_function_call_except_complex_function_to_return_zero()
call_complex_function("with_some_configuration")
assert function2_mock.assert_called_once_with(some_param)
assert function1_mock.assert_called_once_with(another_param)
...
...
Function Call Counting Wrapper
Wrap your nested functions with this so that mocks aren't necessary and you can still see if certain functions were called or not called.
def count_calls_wrapper(fn):
call_count = 0
def _result(*args):
nonlocal call_count
call_count +=1
return fn(*args)
def _get_call_count():
return call_count
_result.get_call_count = _get_call_count
return _result
def f1():
return 10
f1 = count_calls_wrapper(f1)
assert(f1.get_call_count() == 0)
f1()
assert(f1.get_call_count() == 1)
I'm sorry to ask such a basic question, but what's the Pythonic way to include the same if block that can conditionally return in multiple functions? Here's my setup:
def a():
if bool:
return 'yeehaw'
return 'a'
def b():
if bool:
return 'yeehaw'
return 'b'
I'd like to factor the common conditional out of the two functions, but I'm not sure how to do so.
Use a decorator or closure
def my_yeehaw(result):
def yeehaw():
if some_bool:
return 'yeehaw'
return result
return yeehaw
a = my_yeehaw('a')
b = my_yeehaw('b')
You could use a lambda that takes in a. bool and a default value to return if the condition is false:
check = lambda condition, default: 'yeehaw' if condition else default
def a():
return check(condition, 'a')
def b():
return check(condition, 'b')
I am new to python but I think you can use a default argument to send a or b based on what is passed to the function.
def a(x='a'):
if condition: #where condition can be True or False
return 'yeehaw'
return x
(note: my naming wasn't the best, consider that same_bool function might be better called identical_if_block(...) to follow your example
And I am also assuming bool_ is a parameter, though it could work as a global. But not as bool which, like any function object, is always Truthy
>>> bool(bool)
True
)
Use a function, as long as it doesn't need to return falsies.
def same_bool(bool_):
" works for any result except a Falsy"
return "yeehaw" if bool_ else None
def a(bool_):
res = same_bool(bool_)
if res:
return res
return 'a'
def b(bool_, same_bool_func):
#you can pass in your boolean chunk function
res = same_bool_func(bool_)
if res:
return res
return 'b'
print ("a(True):", a(True))
print ("a(False):", a(False))
print ("b(True, same_bool):", b(True,same_bool))
print ("b(False, same_bool):", b(False,same_bool))
output:
a(True): yeehaw
a(False): a
b(True, same_bool): yeehaw
b(False, same_bool): b
If you do need falsies, use a special guard value
def same_bool(bool_):
" works for any result"
return False if bool_ else NotImplemented
def a(bool_):
res = same_bool(bool_)
if res is not NotImplemented:
return res
return 'a'
You could also feed in "a" and "b" since they are constant results, but I assume that's only in your simplified example.
def same_bool(bool_, val):
return "yeehaw" if bool_ else val
def a(bool_):
return same_bool(bool_, "a")
I ended up liking the decorator syntax, as the functions that include the duplicative conditional logic have a good deal else going on in them:
# `function` is the decorated function
# `args` & `kwargs` are the inputs to `function`
def yeehaw(function):
def decorated(*args, **kwargs):
if args[0] == 7: return 99 # boolean check
return function(*args, **kwargs)
return decorated
#yeehaw
def shark(x):
return str(x)
shark(7)
So here's an extension to this question: https://stackoverflow.com/a/37568895/2290820
on how to optionally Enable or Disable Decorator on a Function.
On those lines, I came up with something like this to make decorator get invoked on a recursive call:
def deco(f):
def fattr(attr):
f.attr = attr
def closure(*args):
f(*args)
f.unwrap = f
f.closure = closure
return f
return fattr
#deco
def printa(x):
if x > 1:
print x
return printa(x-1)
else:
print x
return
printa({1:1})(5)
# do the same call w/o deocorator
def finta(x):
if x > 1:
print x
return finta(x-1)
else:
print x
return
finta(5) # this works
to experiment with decorators on a recursive function. Clearly, printa recursive version is not behaving the way it should be.
I could do
g = printa({1:1})
g.closure(5)
to turn on the decorator option or not use that option. Anyway, regardless of good or bad design, How can I make decorator get invoked on a recursive call?
In your deco you have an assignment f.attr = attr that "eats" your argument after first recursive call. Your should modify your recursive call this way:
def deco(f):
def fattr(attr):
f.attr = attr
def closure(*args):
f(*args)
f.unwrap = f
f.closure = closure
return f
return fattr
#deco
def printa(x):
if x > 1:
print x
return printa(None)(x-1) # None will be assigned to f.attr
else:
print x
return
printa({1:1})(5)
5
4
3
2
1
I often find myself using a pattern like this:
num_repeats = 123
interval = 12
for _ in xrange(num_repeats):
result = ...
if result meets condition:
break
time.sleep(interval)
else:
raise Failed despite multiple attempts
Basically, it repeats code until the correct result is returned, or the counter expires.
Although this works, it looks too verbose to me. Is it possible to "parametrize" this loop to a reusable function or context manager, like for example
with repeat(num_repeats, interval):
code
Or maybe there's something in the standard library that would do the trick?
You can use a generator which sleeps before returning repeated results.
The advantage is that your caller is still a genuine for loop, with
all the break, continue, else semantics still in tact.
def trickle_range(num_repeats, interval):
yield 0
for k in xrange(1, num_repeats):
time.sleep(interval)
yield k
for k in trickle_range(num_repeats, interval):
... do stuff, iterate or break as you like ...
You definately won't be able to use the with statement, as python only supplies hooks before and after the code has run, but not one for invoking it, ie. You can't hide a loop within a with statement.
A nice approach is to use a lambda function:
def repeat(repeats, interval, func):
for i in xrange(repeats):
if func(i):
break
time.sleep(interval)
Which you can then use quite easily:
repeat(123, 12, lambda i: condition(i))
Or something similar
One approach would be to decorate the functions you want to repeat:
def repeats_until(num_repeats, interval, condition):
def deco(f):
def func(*args, **kwargs):
for _ in xrange(num_repeats):
result = f(*args, **kwargs)
if condition(result):
return result
time.sleep(interval)
return func
return deco
And then use it like:
#repeats_until(3, 5, lambda s: s == "hello")
def take_input():
return raw_input("Say hello: ")
Example (although I can't show the wait!)
>>> take_input()
Say hello: foo
Say hello: bar
Say hello: baz
>>> take_input()
Say hello: hello
'hello'
Alternatively, to keep the condition with the called function, something like:
def repeats(num_repeats, interval):
def deco(f):
def func(*args, **kwargs):
for _ in xrange(num_repeats):
result = f(*args, **kwargs)
if result is not None: # or e.g. False if None is valid return
return result
time.sleep(interval)
return func
return deco
#repeats(3, 5)
def take_input(condition):
s = raw_input("Say hello: ")
if condition(s):
return s
ui = take_input(lambda s: s == "hello")
This relies on the decorated function returning a value (in this case the implicit None) that tells the decorator it isn't finished yet.
Have Python iterators got a has_next method?
There's an alternative to the StopIteration by using next(iterator, default_value).
For exapmle:
>>> a = iter('hi')
>>> print next(a, None)
h
>>> print next(a, None)
i
>>> print next(a, None)
None
So you can detect for None or other pre-specified value for end of the iterator if you don't want the exception way.
No, there is no such method. The end of iteration is indicated by an exception. See the documentation.
If you really need a has-next functionality, it's easy to obtain it with a little wrapper class. For example:
class hn_wrapper(object):
def __init__(self, it):
self.it = iter(it)
self._hasnext = None
def __iter__(self): return self
def next(self):
if self._hasnext:
result = self._thenext
else:
result = next(self.it)
self._hasnext = None
return result
def hasnext(self):
if self._hasnext is None:
try: self._thenext = next(self.it)
except StopIteration: self._hasnext = False
else: self._hasnext = True
return self._hasnext
now something like
x = hn_wrapper('ciao')
while x.hasnext(): print next(x)
emits
c
i
a
o
as required.
Note that the use of next(sel.it) as a built-in requires Python 2.6 or better; if you're using an older version of Python, use self.it.next() instead (and similarly for next(x) in the example usage). [[You might reasonably think this note is redundant, since Python 2.6 has been around for over a year now -- but more often than not when I use Python 2.6 features in a response, some commenter or other feels duty-bound to point out that they are 2.6 features, thus I'm trying to forestall such comments for once;-)]]
===
For Python3, you would make the following changes:
from collections.abc import Iterator # since python 3.3 Iterator is here
class hn_wrapper(Iterator): # need to subclass Iterator rather than object
def __init__(self, it):
self.it = iter(it)
self._hasnext = None
def __iter__(self):
return self
def __next__(self): # __next__ vs next in python 2
if self._hasnext:
result = self._thenext
else:
result = next(self.it)
self._hasnext = None
return result
def hasnext(self):
if self._hasnext is None:
try:
self._thenext = next(self.it)
except StopIteration:
self._hasnext = False
else: self._hasnext = True
return self._hasnext
In addition to all the mentions of StopIteration, the Python "for" loop simply does what you want:
>>> it = iter("hello")
>>> for i in it:
... print i
...
h
e
l
l
o
Try the __length_hint__() method from any iterator object:
iter(...).__length_hint__() > 0
You can tee the iterator using, itertools.tee, and check for StopIteration on the teed iterator.
hasNext somewhat translates to the StopIteration exception, e.g.:
>>> it = iter("hello")
>>> it.next()
'h'
>>> it.next()
'e'
>>> it.next()
'l'
>>> it.next()
'l'
>>> it.next()
'o'
>>> it.next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
StopIteration docs: http://docs.python.org/library/exceptions.html#exceptions.StopIteration
Some article about iterators and generator in python: http://www.ibm.com/developerworks/library/l-pycon.html
No. The most similar concept is most likely a StopIteration exception.
I believe python just has next() and according to the doc, it throws an exception is there are no more elements.
http://docs.python.org/library/stdtypes.html#iterator-types
The use case that lead me to search for this is the following
def setfrom(self,f):
"""Set from iterable f"""
fi = iter(f)
for i in range(self.n):
try:
x = next(fi)
except StopIteration:
fi = iter(f)
x = next(fi)
self.a[i] = x
where hasnext() is available, one could do
def setfrom(self,f):
"""Set from iterable f"""
fi = iter(f)
for i in range(self.n):
if not hasnext(fi):
fi = iter(f) # restart
self.a[i] = next(fi)
which to me is cleaner. Obviously you can work around issues by defining utility classes, but what then happens is you have a proliferation of twenty-odd different almost-equivalent workarounds each with their quirks, and if you wish to reuse code that uses different workarounds, you have to either have multiple near-equivalent in your single application, or go around picking through and rewriting code to use the same approach. The 'do it once and do it well' maxim fails badly.
Furthermore, the iterator itself needs to have an internal 'hasnext' check to run to see if it needs to raise an exception. This internal check is then hidden so that it needs to be tested by trying to get an item, catching the exception and running the handler if thrown. This is unnecessary hiding IMO.
Maybe it's just me, but while I like https://stackoverflow.com/users/95810/alex-martelli 's answer, I find this a bit easier to read:
from collections.abc import Iterator # since python 3.3 Iterator is here
class MyIterator(Iterator): # need to subclass Iterator rather than object
def __init__(self, it):
self._iter = iter(it)
self._sentinel = object()
self._next = next(self._iter, self._sentinel)
def __iter__(self):
return self
def __next__(self): # __next__ vs next in python 2
if not self.has_next():
next(self._iter) # raises StopIteration
val = self._next
self._next = next(self._iter, self._sentinel)
return val
def has_next(self):
return self._next is not self._sentinel
No, there is no such method. The end of iteration is indicated by a StopIteration (more on that here).
This follows the python principle EAFP (easier to ask for forgiveness than permission). A has_next method would follow the principle of LBYL (look before you leap) and contradicts this core python principle.
This interesting article explains the two concepts in more detail.
Suggested way is StopIteration.
Please see Fibonacci example from tutorialspoint
#!usr/bin/python3
import sys
def fibonacci(n): #generator function
a, b, counter = 0, 1, 0
while True:
if (counter > n):
return
yield a
a, b = b, a + b
counter += 1
f = fibonacci(5) #f is iterator object
while True:
try:
print (next(f), end=" ")
except StopIteration:
sys.exit()
It is also possible to implement a helper generator that wraps any iterator and answers question if it has next value:
Try it online!
def has_next(it):
first = True
for e in it:
if not first:
yield True, prev
else:
first = False
prev = e
if not first:
yield False, prev
for has_next_, e in has_next(range(4)):
print(has_next_, e)
Which outputs:
True 0
True 1
True 2
False 3
The main and probably only drawback of this method is that it reads ahead one more element, for most of tasks it is totally alright, but for some tasks it may be disallowed, especially if user of has_next() is not aware of this read-ahead logic and may missuse it.
Code above works for infinite iterators too.
Actually for all cases that I ever programmed such kind of has_next() was totally enough and didn't cause any problems and in fact was very helpful. You just have to be aware of its read-ahead logic.
The way has solved it based on handling the "StopIteration" execption is pretty straightforward in order to read all iterations :
end_cursor = False
while not end_cursor:
try:
print(cursor.next())
except StopIteration:
print('end loop')
end_cursor = True
except:
print('other exceptions to manage')
end_cursor = True
I think there are valid use cases for when you may want some sort of has_next functionality, in which case you should decorate an iterator with a has_next defined.
Combining concepts from the answers to this question here is my implementation of that which feels like a nice concise solution to me (python 3.9):
_EMPTY_BUF = object()
class BufferedIterator(Iterator[_T]):
def __init__(self, real_it: Iterator[_T]):
self._real_it = real_it
self._buf = next(self._real_it, _EMPTY_BUF)
def has_next(self):
return self._buf is not _EMPTY_BUF
def __next__(self) -> _T_co:
v = self._buf
self._buf = next(self._real_it, _EMPTY_BUF)
if v is _EMPTY_BUF:
raise StopIteration()
return v
The main difference is that has_next is just a boolean expression, and also handles iterators with None values.
Added this to a gist here with tests and example usage.
With 'for' one can implement his own version of 'next' avoiding exception
def my_next(it):
for x in it:
return x
return None
very interesting question, but this "hasnext" design had been put into leetcode:
https://leetcode.com/problems/iterator-for-combination/
here is my implementation:
class CombinationIterator:
def __init__(self, characters: str, combinationLength: int):
from itertools import combinations
from collections import deque
self.iter = combinations(characters, combinationLength)
self.res = deque()
def next(self) -> str:
if len(self.res) == 0:
return ''.join(next(self.iter))
else:
return ''.join(self.res.pop())
def hasNext(self) -> bool:
try:
self.res.insert(0, next(self.iter))
return True
except:
return len(self.res) > 0
The way I solved my problem is to keep the count of the number of objects iterated over, so far. I wanted to iterate over a set using calls to an instance method. Since I knew the length of the set, and the number of items counted so far, I effectively had an hasNext method.
A simple version of my code:
class Iterator:
# s is a string, say
def __init__(self, s):
self.s = set(list(s))
self.done = False
self.iter = iter(s)
self.charCount = 0
def next(self):
if self.done:
return None
self.char = next(self.iter)
self.charCount += 1
self.done = (self.charCount < len(self.s))
return self.char
def hasMore(self):
return not self.done
Of course, the example is a toy one, but you get the idea. This won't work in cases where there is no way to get the length of the iterable, like a generator etc.