i came across this as an interview question. This question seemed interesting. So, i am posting it here.
Consider the operation which gives semantic error like division by zero. By default, python compiler gives output like "Invalid Operation" or something. Can we control the output that is given out by Python compiler, like print some other error message, skip that division by zero operation, and carry on with rest of the instructions?
And also, how can i evaluate the cost of run-time semantic checks?
There are many python experts here. I am hoping someone will throw some light on this. Thanks in advance.
Can we control the output that is given out by Python compiler, like print some other error message, skip that division by zero operation, and carry on with rest of the instructions?
No, you cannot. You can manually wrap every dangerous command with a try...except block, but I'm assuming you're talking about an automatic recovery to specific lines within a try...except block, or even completely automatically.
By the time the error has fallen through such that sys.excepthook is called, or whatever outer scope if you catch it early, the inner scopes are gone. You can change line numbers with sys.settrace in CPython although that is only an implementation detail, but since the outer scopes are gone there is no reliable recorvery mechanism.
If you try to use the humorous goto April fools module (that uses the method I just described) to jump blocks even within a file:
from goto import goto, label
try:
1 / 0
label .foo
print("recovered")
except:
goto .foo
you get an error:
Traceback (most recent call last):
File "rcv.py", line 9, in <module>
goto .foo
File "rcv.py", line 9, in <module>
goto .foo
File "/home/joshua/src/goto-1.0/goto.py", line 272, in _trace
frame.f_lineno = targetLine
ValueError: can't jump into the middle of a block
so I'm pretty certain it's impossible.
And also, how can i evaluate the cost of run-time semantic checks?
I don't know what that is, but you're probably looking for a line_profiler:
import random
from line_profiler import LineProfiler
profiler = LineProfiler()
def profile(function):
profiler.add_function(function)
return function
#profile
def foo(a, b, c):
if not isinstance(a, int):
raise TypeError("Is this what you mean by a 'run-time semantic check'?")
d = b * c
d /= a
return d**a
profiler.enable()
for _ in range(10000):
try:
foo(random.choice([2, 4, 2, 5, 2, 3, "dsd"]), 4, 2)
except TypeError:
pass
profiler.print_stats()
output:
Timer unit: 1e-06 s
File: rcv.py
Function: foo at line 11
Total time: 0.095197 s
Line # Hits Time Per Hit % Time Line Contents
==============================================================
11 #profile
12 def foo(a, b, c):
13 10000 29767 3.0 31.3 if not isinstance(a, int):
14 1361 4891 3.6 5.1 raise TypeError("Is this what you mean by a 'run-time semantic check'?")
15
16 8639 20192 2.3 21.2 d = b * c
17 8639 20351 2.4 21.4 d /= a
18
19 8639 19996 2.3 21.0 return d**a
So the "run-time semantic check", in this case would be taking 36.4% of the time of running foo.
If you want to time specific blocks manually that are larger than you'd use timeit on but smaller than you'd want for a profiler, instead of using two time.time() calls (which is quite an inaccurate method) I suggest Steven D'Aprano's Stopwatch context manager.
I would just use an exception, this example is using python 3. For Python 2, simple remove the annotations after the function parameters. So you function signature would look like this -> f(a,b):
def f(a: int, b: int):
"""
#param a:
#param b:
"""
try:
c = a / b
print(c)
except ZeroDivisionError:
print("You idiot, you can't do that ! :P")
if __name__ == '__main__':
f(1, 0)
>>> from cheese import f
>>> f(0, 0)
You idiot, you can't do that ! :P
>>> f(0, 1)
0.0
>>> f(1, 0)
You idiot, you can't do that ! :P
>>> f(1, 1)
1.0
This is an example of how you could catch Zero Division, by making an exception case using ZeroDivisionError.
I won't go into any specific tools for making loggers, but you can indeed understand the costs associated with this kind of checking. You can put a start = time.time() at the start of the function and end = time.time() at the end. If you take the difference, you will get the execution time in seconds.
I hope that helps.
Related
I need to calculate the friction factor using the Colebrook-Equation, but unlike most cases it needs to be over a length, so the Reynolds number is a list, not a float.
I basically used what this guy this guy recommended (fixed point iteration), specially because it will be part of a bigger programm which wasn't very fast when I did it in Matlab.
For my interpretation of that code I oriented myself at the example of the fixed_point documentation, because there they also pass arguments in form of lists into the function (or actually the solver). v is a list with 1000 elements.
k = 0.000045
d = 0.03
reynolds = d/10**(-6)*v
def f(x, Re):
LHS = -2*np.log10((2.51/(Re*np.sqrt(x))) + (k/(3.71*d)))
return 1/LHS**2
x0 = [0.02]*1000
Re = np.array(reynolds)
result = fixed_point(f,x0, args = Re)
print(result)
I made sure that the starting value and the Reynolds-argument have the same length and I still get an error message.
File " ... ", line 72, in <module>
result = fixed_point(f,x0, args = Re)
...
TypeError: f() takes 2 positional arguments but 1001 were given
I feel like I'm missing something fundamentally here although I'm sitting on it longer than this problem should be taking me.
Thanks for you help,
I have a background in C and Fortran programming, however I have been trying to learn Python and object orientation. To help with some of my projects I have been trying to define some additional unit tests.
I have used the AssertAlmostEqual unit test, but I found that for large numbers it doesn't work so well, as it works to 7 decimal places (I think). When testing large exponents this becomes a bit useless. So I tried to define an assertEqualSigFig test for significant figures in stead of decimal places. This test was inspired by a stack over flow post, however I cannot find the original post I'm afraid.
This test works for integers floats and booleans however I wanted to see if it would also work with complex numbers. By splitting the numbers into the real and imaginary components and then calling itself. When this happens, no assertion Error is raised and I'm not sure why.
Here is my code:
import unittest
import math
class MyTestClass(unittest.TestCase):
"""
MyTestClass
Adds additional tests to the unit test module:
defines:
- AssertEqualSigFig
description:
- Used in place of the assertAlmostEqualTest, this tests two values
are the same to 7 significant figures (instead of decimal places)
args:
- any two integers, booleans, floats or complex number
returns:
- assertion error if not equal to defined significant figures
"""
def AssertEqualSigFig(self, expected, actual, sig_fig = 7):
if sig_fig < 1:
msg = "sig fig must be more than 1"
raise ValueError(msg)
try:
if isinstance(expected, bool):
if expected != actual:
raise AssertionError
else:
return
elif isinstance(expected, (int,float)):
pow_ex = int(math.floor(math.log(expected,10)))
pow_ac = int(math.floor(math.log(actual,10)))
tolerance = pow_ex - sig_fig + 1
tolerance = (10** tolerance)/2.0
if abs(expected - actual) > tolerance:
raise AssertionError
else:
return
elif isinstance(expected, complex):
#this part doesnt raise an error when it should
a_real = actual.real
a_imag = actual.imag
e_real = expected.real
e_imag = expected.imag
self.AssertEqualSigFig(self, a_imag, e_imag)
self.AssertEqualSigFig(self, a_real, e_real)
except AssertionError:
msg = "{0} ! = {1} to {2} sig fig".format(expected, actual, sig_fig)
raise AssertionError(msg)
This test fails when complex numbers are involved. Here are the unit tests of the unit test that it fails:
import unittest
from MyTestClass import MyTestClass
class TestMyTestClass(MyTestClass):
def test_comlex_imag_NE(self):
a = complex(10,123455)
b = complex(10,123333)
self.assertRaises(AssertionError, self.AssertEqualSigFig, a, b)
def test_complex_real_NE(self):
a = complex(2222222,10)
b = complex(1111111,10)
self.assertRaises(AssertionError, self.AssertEqualSigFig, a, b)
if __name__ == "__main__":
unittest.main()
I think it is because the self.AssertEqualSigFig call does not raise an error. I'm sure there is a silly thing I have missed, But I am still learning. Can anybody help?
I was being an idiot, I have found the solution
I should have been using
MyTestClass.assertEqualSigFig
and not
self.assertEqualSigFig
I would like Z3 to check whether it exists an integer t that satisfies my formula. I'm getting the following error:
Traceback (most recent call last):
File "D:/z3-4.6.0-x64-win/bin/python/Expl20180725.py", line 18, in <module>
g = ForAll(t, f1(t) == And(t>=0, t<10, user[t].rights == ["read"] ))
TypeError: list indices must be integers or slices, not ArithRef
Code:
from z3 import *
import random
from random import randrange
class Struct:
def __init__(self, **entries): self.__dict__.update(entries)
user = [Struct() for i in range(10)]
for i in range(10):
user[i].uid = i
user[i].rights = random.choice(["create","execute","read"])
s=Solver()
f1 = Function('f1', IntSort(), BoolSort())
t = Int('t')
f2 = Exists(t, f1(t))
g = ForAll(t, f1(t) == And(t>=0, t<10, user[t].rights == ["read"] ))
s.add(g)
s.add(f2)
print(s.check())
print(s.model())
You are mixing and matching Python and Z3 expressions, and while that is the whole point of Z3py, it definitely does not mean that you can mix/match them arbitrarily. In general, you should keep all the "concrete" parts in Python, and relegate the symbolic parts to "z3"; carefully coordinating the interaction in between. In your particular case, you are accessing a Python list (your user) with a symbolic z3 integer (t), and that is certainly not something that is allowed. You have to use a Z3 symbolic Array to access with a symbolic index.
The other issue is the use of strings ("create"/"read" etc.) and expecting them to have meanings in the symbolic world. That is also not how z3py is intended to be used. If you want them to mean something in the symbolic world, you'll have to model them explicitly.
I'd strongly recommend reading through http://ericpony.github.io/z3py-tutorial/guide-examples.htm which is a great introduction to z3py including many of the advanced features.
Having said all that, I'd be inclined to code your example as follows:
from z3 import *
import random
Right, (create, execute, read) = EnumSort('Right', ('create', 'execute', 'read'))
users = Array('Users', IntSort(), Right)
for i in range(10):
users = Store(users, i, random.choice([create, execute, read]))
s = Solver()
t = Int('t')
s.add(t >= 0)
s.add(t < 10)
s.add(users[t] == read)
r = s.check()
if r == sat:
print s.model()[t]
else:
print r
Note how the enumerated type Right in the symbolic land is used to model your "permissions."
When I run this program multiple times, I get:
$ python a.py
5
$ python a.py
9
$ python a.py
unsat
$ python a.py
6
Note how unsat is produced, if it happens that the "random" initialization didn't put any users with a read permission.
I am calling pdb on some function func i.e.,
def fun():
a = 10
c = fun2(a)
d = 40
return c+d
def fun2(a):
xyz ='str'
return a+10
Now I am running pdb using pdb.runcall(func,a,b) now It will open a pdb console for debugging, now suppose I press 2 time s(step) and q to quit in pdb console
but problem is I don't want to do this manually, I want to make some script which do something like this (automatic tell pdb first two command is s then third is q) , I am asking because there are many functions which needs atleast two time c (continue) to overall excecution of function so that it can yield/return some valid output (like say generators)
Any help will be a serious help for me.
Update after better understanding the question:
In general, I don't think this is the ideal way to test code; designing code for testability (e.g. using TDD) will often result in functions that are easier to test (e.g. using mocks/fake objects, dependency injection etc), and I would encourage you to consider refactoring the code if possible. The other issue with this approach is that the tests may become very tightly coupled to the code. However, I'll assume here that you know what you are doing, and that the above is not an option for whatever reason.
Scripting pdb
If you want to script pdb from code, this is actually possible by instantiating your own pdb.Pdb class and passing in the stdin and, at the time of writing, stdout argument (I'm not sure both should be required - I've filed https://bugs.python.org/issue33749).
Example (I just added the extra input argument to fun):
def fun(i):
a = 10 + i
c = fun2(a)
d = 40
return c+d
def fun2(a):
xyz ='str'
return a+10
import pdb
import io
output = io.StringIO()
# this contains the pdb commands we want to execute:
pdb_script = io.StringIO("p i;; i = 100;; n;; p a;; c;;")
mypdb = pdb.Pdb(stdin=pdb_script, stdout=output)
Normal result (no scripting):
In [40]: pdb.runcall(fun, 1)
...:
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb)
(Pdb) c
Out[40]: 61
Scripted pdb:
In [44]: mypdb = pdb.Pdb(stdin=pdb_script, stdout=output)
In [45]: mypdb.runcall(fun, 1)
Out[45]: 160
In [50]: print(output.getvalue())
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb) 1
> <ipython-input-1-28966c4f6e38>(3)fun()
-> c = fun2(a)
110
You may find using pdb_script.seek(0) helpful to reset the script.
Original answer - using conditional breakpoints
It sounds like what you really want is to only get into the debugger when your code is in a certain state. This can be done with conditional breakpoints (see pdb docs for details).
For example, let's say you want to break in fun2 if a > 10:
...:
In [2]: import pdb
In [3]: pdb.runcall(fun, 1)
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb) break fun2, a > 10
Breakpoint 1 at <ipython-input-1-28966c4f6e38>:6
(Pdb) c
> <ipython-input-1-28966c4f6e38>(7)fun2()
-> xyz ='str'
(Pdb) c
Out[3]: 61
In [4]: pdb.runcall(fun, -1)
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb) c
Out[4]: 59
Notice in the first case you hit the breakpoint, in the second you didn't.
Original answer - using breakpoints and executing commands when hit
You could also try setting a breakpoint and using the commands facility.
I'm looking into how to perform assert introspection in Python, in the same way that py.test does. For example...
>>> a = 1
>>> b = 2
>>> assert a == b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError # <--- I want more information here, eg 'AssertionError: 1 != 2'
I see that the py.code library has some functionality around this and I've also seen this answer, noting that sys.excepthook allows you to plug in whatever behavior you want to exceptions, but it's not clear to me how to put it all together.
You can do something like this if you want to show a detailed error message
def assertion(a,b):
try:
assert a==b
except AssertionError as e:
e.args += ('some other', 'information',)
raise
a=1
b=2
assertion(a,b)
This code will give this output:
Traceback (most recent call last):
File "tp.py", line 11, in <module>
assertion(a,b)
File "tp.py", line 4, in assertion
assert a==b
AssertionError: ('some other', 'information')
The unittest assert gives extra information (possibly more than you need). Inspired by Raymond Hettinger's talk.
This is a partial answer, only giving the values for a and b (last line of the output), not the additional introspection you are also seeking that is unique in pytest.
import unittest
class EqualTest(unittest.TestCase):
def testEqual(self, a, b):
self.assertEqual(a, b)
a, b = 1, 2
assert_ = EqualTest().testEqual
assert_(a, b)
Output
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-4-851ce0f1f668> in <module>()
9 a, b = 1, 2
10 assert_ = EqualTest().testEqual
---> 11 assert_(a, b)
<ipython-input-4-851ce0f1f668> in testEqual(self, a, b)
4
5 def testEqual(self, a, b):
----> 6 self.assertEqual(a, b)
7
8
C:\Anaconda3\lib\unittest\case.py in assertEqual(self, first, second, msg)
818 """
819 assertion_func = self._getAssertEqualityFunc(first, second)
--> 820 assertion_func(first, second, msg=msg)
821
822 def assertNotEqual(self, first, second, msg=None):
C:\Anaconda3\lib\unittest\case.py in _baseAssertEqual(self, first, second, msg)
811 standardMsg = '%s != %s' % _common_shorten_repr(first, second)
812 msg = self._formatMessage(msg, standardMsg)
--> 813 raise self.failureException(msg)
814
815 def assertEqual(self, first, second, msg=None):
AssertionError: 1 != 2
I don't think it is straightforward to reproduce pytest's assert introspection in a standalone context. The docs contain a few more details on how it works:
pytest rewrites test modules on import. It does this by using an import hook to write a new pyc files. Most of the time this works transparently. However, if you are messing with import yourself, the import hook may interfere. If this is the case, simply use --assert=reinterp or --assert=plain. Additionally, rewriting will fail silently if it cannot write new pycs, i.e. in a read-only filesystem or a zipfile.
It looks like it would require quite some hacks to make that work in arbitrary modules, so you're probably better off to use a solution suggested in the other answers.