I am still learning unittest and therefore, am unable to tell if there's a something missing in my test case in test_iterators.py below. Can someone help me to understand why the ValueError failed to be raised within unittest? Here are the scripts:
iterators.py
"""
Simple class to count from zero to N
"""
class count_to(object):
def __init__(self, nber):
self.nber = nber
def __iter__(self):
return count_to_iter(self.nber)
class count_to_iter(object):
def __init__(self, nber):
self.stopat = nber
self.current_nber = 0
def __next__(self):
if self.stopat < 0:
raise ValueError
elif self.current_nber > self.stopat:
raise StopIteration
self.current_nber += 1
return self.current_nber - 1
if __name__ == '__main__':
for x in count_to(-1):
print(x)
tests/test_iterators.py
import unittest
import iterators
class TestBaseIterators(unittest.TestCase):
def setUp(self):
pass
# Can't get the negative test right yet. It returns None instead of raising a ValueError
# Calling iterators.py directly and execute main successfully raised a ValueError however
def test_negative(self):
with self.assertRaises(ValueError): iterators.count_to(-1)
if __name__ == '__main__':
unittest.main()
I have used a similar approach to test raised errors previously and it worked. However, for this particular test case, here's what I get from the test.
test_negative (test_iterators.TestBaseIterators) ... FAIL
NoneType: None
======================================================================
FAIL: test_negative (test_iterators.TestBaseIterators)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/kerwei/Git/Concepts/tests/test_iterators.py", line 19, in test_negative
with self.assertRaises(ValueError): iterators.count_to(-1)
AssertionError: ValueError not raised
----------------------------------------------------------------------
Ran 1 test in 0.004s
FAILED (failures=1)
If I were to call iterators directly from __main__, I can then successfully receive the ValueError.
(py36) Kers-MacBook-Air:Concepts kerwei$ python iterators.py
Traceback (most recent call last):
File "iterators.py", line 29, in <module>
for x in count_to(-1):
File "iterators.py", line 19, in __next__
raise ValueError
ValueError
count_to(-1) creates a new count_to instance, it does not iterate over it, but you placed the test on self.stop_at value and raise the ValueError in the count_to_iter.__next__ method, so you will obviously not get a ValueError until you iterate on the count_to instance.
The naive fix would be to force iteration, ie:
def test_negative(self):
with self.assertRaises(ValueError):
# passing the iterable to `list` will force iteration
list(iterators.count_to(-1))
But the root problem is actually more of a design issue: raising a ValueError at this point is far from optimal since it will only happen when actually consuming the iterable, so you will have to inspect the call stack up until you find where count_to has been passed a wrong value. A much better solution is to check the value and eventually raise directly at the point where count_to is instanciated so it breaks always and immediatly (instead of "eventually, when you try to use the iterator in some possibly remote part of the code):
class count_to(object):
def __init__(self, nber):
if nber < 0:
raise ValueError("count_to argument must be a positive integer")
self.nber = nber
def __iter__(self):
return count_to_iter(self.nber)
And then your current test code will work as intended.
Related
look at this:
RuntimeError: No active exception to reraise
I use raise. with out error like this:
class example:
def __getattribute__(self, attr_name):
raise # I mean: AttributeError: '...' object has no attribute '...'
This is raise statement:
raise_stmt ::= "raise" [expression ["from" expression]]
expression is OPTIONAL.
I check this, but this isn't my answer. if error says "No active exception to reraise", so I can active an error. I do not know what this error means. My question is, what is meant by "active exception" and where it is used? Does it help make the code shorter and more optimized? Is it possible to use it for the task I showed a little higher in the code?
When you use raise keyword barely, Python tries to re-raise the currently occurred exception in the current scope, If there is no exception triggered on, you will get RuntimeError: No active exception to re-raise.
To see which exception is active(being handled), you can use sys.exc_info():
import sys
try:
raise ZeroDivisionError()
except ZeroDivisionError:
type_, value, tb = sys.exc_info()
print(type_) # <class 'ZeroDivisionError'>
In the above except block you can use bare raise keyword, which re-raised the ZeroDivisionError exception for you.
If there is no active exception, the returned value of sys.exc_info() is (None, None, None). So you have to use raise keyword followed by a subclass or an instance of BaseException. This is the case in your question inside __getattribute__ method since there is no active exception.
class Example:
def __getattribute__(self, attr_name):
raise AttributeError(f'Error for "{attr_name}".')
obj = Example()
obj.foo # AttributeError: Error for "foo".
From comments:
Active exception means the exception that is currently triggered on, and is in the workflow, If you don't catch it and let it bubbles up, it will terminate the process.
I can find this for my questions:
what is meant by "active exception" and where it is used?
then using try-except and code goes to except block, error actives.
about usage we can check error and if Should not be handled, can use
raise without arg (Even In Functions). now error raised without
Reference to line. I do not know if there is another way to do this or not, but I was convinced of this method.
Example:
>>> try:
... 10 / 0
... except[ ZeroDivisionError[ as err]]:
... raise
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero
>>> def test():
... raise
...
>>> try:
... 10 / 0
... except[ ZeroDivisionError[ as err]]:
... test()
...
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero
>>>
Does it help make the code shorter and more optimized?
Yes. By using this method, the code and error are shortened. But in this method, there is less control over the exception procedure, which can sometimes be problematic.
Is it possible to use it for the task I showed a little higher in the code?
No. NotImplemented must be returned for the above code to take effect. I have full confidence in this, but there is a possibility that this alone is not enough.
I have a dummy funtion to try exceptions:
def fun(n):
try:
if n <1:
raise ValueError
return 1
except:
pass
In my unit test I use:
import unittest
class TestFibonnacci(unittest.TestCase):
def test_values(self):
self.assertEqual(fun(1),1)
self.assertRaises(ValueError, fun(-1))
However I'm unable to get an answer I actually get:
An exception occurred
E....
ERROR: test_values (main.TestFibonnacci)
Traceback (most recent call last):
The traceback here
TypeError: 'NoneType' object is not callable
Ran 1 tests in 0.001s
What I'm doing wrong?
You are calling fun(-1) immediately, rather than letting self.assertRaises call it, so that it can catch the exception.
You have to pass the function and its arguments separately.
self.assertRaises(ValueError, fun, -1)
Alternatively, you can use assertRaises as a context manager.
with self.assertRaises(ValueError):
fun(-1)
The with statement captures the exception raised by fun and provides it to the __exit__ method of the value returned by assertRaises.
I have the following mcve:
import logging
class MyGenIt(object):
def __init__(self, name, content):
self.name = name
self.content = content
def __iter__(self):
with self:
for o in self.content:
yield o
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type:
logging.error("Aborted %s", self,
exc_info=(exc_type, exc_value, traceback))
And here is sample use:
for x in MyGenIt("foo",range(10)):
if x == 5:
raise ValueError("got 5")
I would like logging.error to report the ValueError, but instead it reports GeneratorExit:
ERROR:root:Aborted <__main__.MyGenIt object at 0x10ca8e350>
Traceback (most recent call last):
File "<stdin>", line 8, in __iter__
GeneratorExit
When I catch GeneratorExit in __iter__:
def __iter__(self):
with self:
try:
for o in self.content:
yield o
except GeneratorExit:
return
nothing is logged (of course) because __exit__ is called with exc_type=None.
Why do I see GeneratorExit instead of ValueError in __exit__?
What do I do to get the desired behavior, i.e., ValueError in __exit__?
Just a quick note that you could "bring the context manager out" of the generator, and by only changing 3 lines get:
import logging
class MyGenIt(object):
def __init__(self, name, content):
self.name = name
self.content = content
def __iter__(self):
for o in self.content:
yield o
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type:
logging.error("Aborted %s", self,
exc_info=(exc_type, exc_value, traceback))
with MyGenIt("foo", range(10)) as gen:
for x in gen:
if x == 5:
raise ValueError("got 5")
A context manager that could also act as an iterator -- and would catch caller code exceptions like your ValueError.
The basic problem is that you are trying to use a with statement inside the generator to catch an exception that is raised outside the generator. You cannot get __iter__ to see the ValueError, because __iter__ is not executing at the time the ValueError is raised.
The GeneratorExit exception is raised when the generator itself is deleted, which happens when it is garbage collected. As soon as the exception occurs, the for loop terminates; since the only reference to the generator (the object obtained by calling __iter__) is in the loop expression, terminating the loop removes the only reference to the iterator and makes it available for garbage collection. It appears that here it is being garbage collected immediately, meaning that the GeneratorExit exception happens between the raising of the ValueError and the propagation of that ValueError to the enclosing code. The GeneratorExit is normally handled totally internally; you are only seeing it because your with statement is inside the generator itself.
In other words, the flow goes something like this:
Exception is raised outside the generator
for loop exits
Generator is now available for garbage collection
Generator is garbage collected
Generator's .close() is called
GeneratorExit is raised inside the generator
ValueError propagates to calling code
The last step does not occur until after your context manager has seen the GeneratorExit. When I run your code, I see the ValueError raised after the log message is printed.
You can see that the garbage collection is at work, because if you create another reference to the iterator itself, it will keep the iterator alive, so it won't be garbage collected, and so the GeneratorExit won't occur. That is, this "works":
it = iter(MyGenIt("foo",range(10)))
for x in it:
if x == 5:
raise ValueError("got 5")
The result is that the ValueError propagates and is visible; no GeneratorExit occurs and nothing is logged. You seem to think that the GeneratorExit is somehow "masking" your ValueError, but it isn't really; it's just an artifact introduced by not keeping any other references to the iterator. The fact that GeneratorExit occurs immediately in your example isn't even guaranteed behavior; it's possible that the iterator might not be garbage-collected until some unknown time in the future, and the GeneratorExit would then be logged at that time.
Turning to your larger question of "why do I see GeneratorExit", the answer is that that is the only exception that actually occurs within the generator function. The ValueError occurs outside the generator, so the generator can't catch it. This means your code can't really work in the way you seem to intend it to. Your with statement is inside the generator function. Thus it can only catch exceptions that happen in the process of yielding items from the generator; there generator has no knowledge of what happens between the times when it advances. But your ValueError is raised in the body of the loop over the generator contents. The generator is not executing at this time; it's just sitting there suspended.
You can't use a with statement in a generator to magically trap exceptions that occur in the code that iterates over the generator. The generator does not "know" about the code that iterates over it and can't handle exceptions that occur there. If you want to catch exceptions within the loop body, you need a separate with statement enclosing the loop itself.
The GeneratorExit is raised whenever a generator or coroutine is closed. Even without the context manager, we can replicate the exact condition with a simple generator function that prints out the exception information when it errors (further reducing the provided code to show exactly how and where that exception is generated).
import sys
def dummy_gen():
for idx in range(5):
try:
yield idx
except:
print(sys.exc_info())
raise
for i in dummy_gen():
raise ValueError('foo')
Usage:
(<class 'GeneratorExit'>, GeneratorExit(), <traceback object at 0x7f96b26b4cc8>)
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ValueError: foo
Note there was also an exception that was raised inside the generator itself, as noted that the except block was executed. Note that the exception was also further raise'd after the print statement but note how that isn't actually shown anywhere, because it is handled internally.
We can also abuse this fact to see if we can manipulate the flow by swallowing the GeneratorExit exception and see what happens. This can be done by removing the raise statement inside the dummy_gen function to get the following output:
(<class 'GeneratorExit'>, GeneratorExit(), <traceback object at 0x7fd1f0438dc8>)
Exception ignored in: <generator object dummy_gen at 0x7fd1f0436518>
RuntimeError: generator ignored GeneratorExit
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ValueError: foo
Note how there is an internal RuntimeError that was raised that complained about the generator ignoring the GeneratorExit function. So we from this we can clearly see that this exception is produced by the generator itself inside the generator function, and the ValueError that is raised outside that scope is never present inside the generator function.
Since a context manager will trap all exceptions as is, and the context manager is inside the generator function, whatever exception raised inside it will simply be passed to __exit__ as is. Consider the following:
class Context(object):
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type:
logging.error("Aborted %s", self,
exc_info=(exc_type, exc_value, traceback))
Modify the dummy_gen to the following:
def dummy_gen():
with Context():
for idx in range(5):
try:
yield idx
except:
print(sys.exc_info())
raise
Running the resulting code:
(<class 'GeneratorExit'>, GeneratorExit(), <traceback object at 0x7f44b8fb8908>)
ERROR:root:Aborted <__main__.Context object at 0x7f44b9032d30>
Traceback (most recent call last):
File "foo.py", line 26, in dummy_gen
yield idx
GeneratorExit
Traceback (most recent call last):
File "foo.py", line 41, in <module>
raise ValueError('foo')
ValueError: foo
The same GeneratorExit that is raised is now presented to the context manager, because this is the behavior that was defined.
I would like to use raise without printing the traceback on the screen. I know how to do that using try ..catch but doesn't find a way with raise.
Here is an example:
def my_function(self):
resp = self.resp
if resp.status_code == 404:
raise NoSuchElementError('GET'+self.url+'{}'.format(resp.status_code))
elif resp.status_code == 500:
raise ServerErrorError('GET'+self.url+'{}'.format(resp.status_code))
When executing this, if I have a 404, the traceback will print on the screen.
Traceback (most recent call last):
File "test.py", line 32, in <module>
print ins.my_function()
File "api.py", line 820, in my_function
raise NoSuchElementError('GET ' + self.url + ' {} '.format(resp.status_code))
This is an API wrapper and I don't want users to see the traceback but to see the API response codes and error messages instead.
Is there a way to do it ?
I ran into a similar problem where a parent class was using the exception value on raise to pass messages through but where I didn't want to dump the traceback. #lejlot gives a great solution using sys.excepthook but I needed to apply it with a more limited scope. Here's the modification:
import sys
from contextlib import contextmanager
#contextmanager
def except_handler(exc_handler):
"Sets a custom exception handler for the scope of a 'with' block."
sys.excepthook = exc_handler
yield
sys.excepthook = sys.__excepthook__
Then, to use it:
def my_exchandler(type, value, traceback):
print(': '.join([str(type.__name__), str(value)]))
with except_handler(my_exchandler):
raise Exception('Exceptional!')
# -> Exception: Exceptional!
That way, if an exception isn't raised in the block, default exception handling will resume for any subsequent exceptions:
with except_handler(my_exchandler):
pass
raise Exception('Ordinary...')
# -> Traceback (most recent call last):
# -> File "raise_and_suppress_traceback.py", line 22, in <module>
# -> raise Exception('Ordinary...')
# -> Exception: Ordinary...
The problem is not with raising anything, but with what python interpreter does, when your program terminates with an exception (and it simply prints the stack trace). What you should do if you want to avoid it, is to put try except block around everything that you want to "hide" the stack trace, like:
def main():
try:
actual_code()
except Exception as e:
print(e)
The other way around is to modify the exeption handler, sys.excepthook(type, value, traceback), to do your own logic, like
def my_exchandler(type, value, traceback):
print(value)
import sys
sys.excepthook = my_exchandler
you can even condition of exception type and do the particular logic iff it is your type of exception, and otherwise - backoff to the original one.
Modified #Alec answer:
#contextmanager
def disable_exception_traceback():
"""
All traceback information is suppressed and only the exception type and value are printed
"""
default_value = getattr(sys, "tracebacklimit", 1000) # `1000` is a Python's default value
sys.tracebacklimit = 0
yield
sys.tracebacklimit = default_value # revert changes
Usage:
with disable_exception_traceback():
raise AnyYourCustomException()
Use this if you only need to hide a traceback without modifying an exception message. Tested on Python 3.8
UPD: code improved by #DrJohnAStevenson comment
Catch the exception, log it and return something that indicates something went wrong to the consumer (sending a 200 back when a query failed will likely cause problems for your client).
try:
return do_something()
except NoSuchElementError as e:
logger.error(e)
return error_response()
The fake error_response() function could do anything form returning an empty response or an error message. You should still make use of proper HTTP status codes. It sounds like you should be returning a 404 in this instance.
You should handle exceptions gracefully but you shouldn't hide errors from clients completely. In the case of your NoSuchElementError exception it sounds like the client should be informed (the error might be on their end).
You can create a class that takes two values; Type and code for a custom Exception Message. Afterwards, you can just pass the class in a try/except statement.
class ExceptionHandler(Exception):
def __init__(self, exceptionType, code):
self.exceptionType = exceptionType
self.code = code
print(f"Error logged: {self.exceptionType}, Code: {self.code}")
try:
raise(ExceptionHandler(exceptionType=KeyboardInterrupt, code=101))
except Exception:
pass
Here's my exception class that is using raise:
class SCE(Exception):
"""
An error while performing SCE functions.
"""
def __init__(self, value=None):
"""
Message: A string message or an iterable of strings.
"""
if value is None:
self._values = []
elif isinstance(value, str):
self._values = [value]
else:
self._values = list(value)
def __raise__(self):
print('raising')
if not len(self._values):
return
def __str__(self):
return self.__repr__()
def __iter__(self):
return iter(self._values)
def __repr__(self):
return repr(self._values)
Currently if I raise this exception with no value I get traceback followed by:
__main__.SCE: []
Instead of what I expected which was:
raising
>>>
How do you overload raise?
As the other answer says, there is no __raise__ special method. There was a thread in 2004 on comp.lang.python where someone suggested adding such a method, but I don't think there was any followup to that. The only way I can think of to hook exception raising is either by patching the interpreter, or some kind of source or bytecode rewriting that inserts a function call next to the raise operation.
There is no such special method __raise__ (at least none that I have ever heard of or that I can find in the Python documentation).
Why do you want to do this? I can't think of any reason why you want custom code be be executed when the exception is raised (as opposed to either when the exception is constructed, which you can do with the __init__ method, or when the exception is caught, which you can do with an except block). What is your use case for this behavior, and why do you expect that Python supports it?
As others have stated, there is no such private method __raise__. Nothing prevents defining one. For example:
#!/usr/bin/env python3
class MyClass(object):
def __init__(self, raise_exceptions=False):
self.raise_exceptions = raise_exceptions
def __raise__(self, err=None):
print(err, flush=True)
if self.raise_exceptions:
raise err
def run(self):
try:
assert False, 'assertion False'
except Exception as err:
self.__raise__(err)
if __name__ == '__main__':
MyClass(raise_exceptions=False).run()
MyClass(raise_exceptions=True).run()
Here is the output:
$ python3 my_class.py
assertion False
assertion False
Traceback (most recent call last):
File "my_class.py", line 22, in <module>
MyClass(raise_exceptions=True).run()
File "my_class.py", line 17, in run
self.__raise__(err)
File "my_class.py", line 11, in __raise__
raise err
File "my_class.py", line 15, in run
assert False, 'assertion False'
AssertionError: assertion False
Process finished with exit code 1