I've got a fairly basic doctestable file:
class Foo():
"""
>>> 3+2
5
"""
if __name__ in ("__main__", "__console__"):
import doctest
doctest.testmod(verbose=True)
which works as expected when run directly through python.
However, in iPython, I get
1 items had no tests:
__main__
0 tests in 1 items.
0 passed and 0 failed.
Test passed.
Since this is part of a Django project and will need access to all of the appropriate variables and such that manage.py sets up, I can also run it through a modified command, which uses code.InteractiveConsole, one result of which is __name__ gets set to '__console__'.
With the code above, I get the same result as with iPython. I tried changing the last line to this:
this = __import__(__name__)
doctest.testmod(this, verbose=True)
and I get an ImportError on __console__, which makes sense, I guess. This has no effect on either python or ipython.
So, I'd like to be able to run doctests successfully through all three of these methods, especially the InteractiveConsole one, since I expect to be needing Django pony magic fairly soon.
Just for clarification, this is what I'm expecting:
Trying:
3+2
Expecting:
5
ok
1 items had no tests:
__main__
1 items passed all tests:
1 tests in __main__.Foo
1 tests in 2 items.
1 passed and 0 failed.
Test passed.
The root problem is that ipython plays weird tricks with __main__ (through its own FakeModule module) so that, by the time doctest is introspecting that "alleged module" through its __dict__, Foo is NOT there -- so doctest doesn't recurse into it.
Here's one solution:
class Foo():
"""
>>> 3+2
5
"""
if __name__ in ("__main__", "__console__"):
import doctest, inspect, sys
m = sys.modules['__main__']
m.__test__ = dict((n,v) for (n,v) in globals().items()
if inspect.isclass(v))
doctest.testmod(verbose=True)
This DOES produce, as requested:
$ ipython dot.py
Trying:
3+2
Expecting:
5
ok
1 items had no tests:
__main__
1 items passed all tests:
1 tests in __main__.__test__.Foo
1 tests in 2 items.
1 passed and 0 failed.
Test passed.
Python 2.5.1 (r251:54863, Feb 6 2009, 19:02:12)
[[ snip snip ]]
In [1]:
Just setting global __test__ doesn't work, again because setting it as a global of what you're thinking of as __main__ does NOT actually place it in the __dict__ of the actual object that gets recovered by m = sys.modules['__main__'], and the latter is exactly the expression doctest is using internally (actually it uses sys.modules.get, but the extra precaution is not necessary here since we do know that __main__ exists in sys.modules... it's just NOT the object you expect it to be!-).
Also, just setting m.__test__ = globals() directly does not work either, for a different reason: doctest checks that the values in __test__ are strings, functions, classes, or modules, and without some selection you cannot guarantee that globals() will satisfy that condition (in fact it won't). Here I'm selecting just classes, if you also want functions or whatnot you can use an or in the if clause in the genexp within the dict call.
I don't know exactly how you're running a Django shell that's able to execute your script (as I believe python manage.py shell doesn't accept arguments, you must be doing something else, and I can't guess exactly what!-), but a similar approach should help (whether your Django shell is using ipython, the default when available, or plain Python): appropriately setting __test__ in the object you obtain as sys.modules['__main__'] (or __console__, if that's what you're then passing on to doctest.testmod, I guess) should work, as it mimics what doctest will then be doing internally to locate your test strings.
And, to conclude, a philosophical reflection on design, architecture, simplicity, transparency, and "black magic"...:
All of this effort is basically what's needed to defeat the "black magic" that ipython (and maybe Django, though it may be simply delegating that part to ipython) is doing on your behalf for your "convenience"... any time at which two frameworks (or more;-) are independently doing each its own brand of black magic, interoperability may suddenly require substantial effort and become anything BUT convenient;-).
I'm not saying that the same convenience could have been provided (by any one or more of ipython, django and/or doctests) without black magic, introspection, fake modules, and so on; the designers and maintainers of each of those frameworks are superb engineers, and I expect they've done their homework thoroughly, and are performing only the minimum amount of black magic that's indispensable to deliver the amount of user convenience they decided they needed. Nevertheless, even in such a situation, "black magic" suddenly turns from a dream of convenience to a nightmare of debugging as soon as you want to do something even marginally outside what the framework's author had conceived.
OK, maybe in this case not quite a nightmare, but I do notice that this question has been open a while and even with the lure of the bounty it didn't get many answers yet -- though you now do have two answers to pick from, mine using the __test__ special feature of doctest, #codeape's using the peculiar __IP.magic_run feature of ironpython. I prefer mine because it does not rely on anything internal or undocumented -- __test__ IS a documented feature of doctest, while __IP, with those two looming leading underscores, scream "deep internals, don't touch" to me;-)... if it breaks at the next point release I wouldn't be at all surprised. Still, matter of taste -- that answer may arguably be considered more "convenient".
But, this is exactly my point: convenience may come at an enormous price in terms of giving up simplicity, transparency, and/or avoidance of internal/undocumented/unstable features; so, as a lesson for all of us, the least black magic &c we can get away with (even at the price of giving up an epsilon of convenience here and there), the happier we'll all be in the long run (and the happier we'll make other developers that need to leverage our current efforts in the future).
The following works:
$ ipython
...
In [1]: %run file.py
Trying:
3+2
Expecting:
5
ok
1 items had no tests:
__main__
1 items passed all tests:
1 tests in __main__.Foo
1 tests in 2 items.
1 passed and 0 failed.
Test passed.
In [2]:
I have no idea why ipython file.py does not work. But the above is at least a workaround.
EDIT:
I found the reason why it does not work. It is quite simple:
If you do not specify the module to test in doctest.testmod(), it assumes that you want to test the __main__ module.
When IPython executes the file passed to it on the command line, the __main__ module is IPython's __main__, not your module. So doctest tries to execute doctests in IPython's entry script.
The following works, but feels a bit weird:
if __name__ == '__main__':
import doctest
import the_current_module
doctest.testmod(the_current_module)
So basically the module imports itself (that's the "feels a bit weird" part). But it works. Something I do not like abt. this approach is that every module needs to include its own name in the source.
EDIT 2:
The following script, ipython_doctest, makes ipython behave the way you want:
#! /usr/bin/env bash
echo "__IP.magic_run(\"$1\")" > __ipython_run.py
ipython __ipython_run.py
The script creates a python script that will execute %run argname in IPython.
Example:
$ ./ipython_doctest file.py
Trying:
3+2
Expecting:
5
ok
1 items had no tests:
__main__
1 items passed all tests:
1 tests in __main__.Foo
1 tests in 2 items.
1 passed and 0 failed.
Test passed.
Python 2.5 (r25:51908, Mar 7 2008, 03:27:42)
Type "copyright", "credits" or "license" for more information.
IPython 0.9.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.
In [1]:
Related
When running tests that target a specific method which uses reflection, I encounter the problem that the output of tests is dependent on whether I run them with PTVS ('run all tests' in Test Explorer) or with the command line Python tool (both on Windows and Linux systems):
$ python -m unittest
I assumed from the start that it has something to do with differences in how the test runners work in PTVS and Python's unittest framework (because I've noticed other differences, too).
# method to be tested
# written in Python 3
def create_line(self):
problems = []
for creator in LineCreator.__subclasses__():
item = creator(self.structure)
cls = item.get_subtype()
line = cls(self.text)
try:
line.parse()
return line
except ParseException as exc:
problems.append(exc)
raise ParseException("parsing did not succeed", problems)
""" subclasses of LineCreator are defined in separate files.
They implement get_subtype() and return the class objects of the actual types they must instantiate.
"""
I have noticed that the subclasses found in this way will vary, depending on which modules have been loaded in the code that calls this method. This is exactly what I want (for now). Given this knowledge, I am always careful to only have access to one subclass of LineCreator in any given test module, class, or method.
However, when I run the tests from the Python command line, it is clear from the ParseException.problems attribute that both are loaded at all times. It is also easy to reproduce: inserting the following code makes all tests fail on the command line, yet they succeed on PTVS.
if len(LineCreator.__subclasses__()) > 1:
raise ImportError()
I know that my tests should run independently from each other and from any contextual factors. That is actually what I'm trying to achieve here.
In case I wasn't clear, my question is why behaviors are different, and which one is correct. And if you're feeling really generous, how to change my code to make tests succeed on all platforms.
Let's say I have two python scripts A.py and B.py. I'm looking for a way to run B from within A in such a way that:
B believes it is __main__ (so that code in an if __name__=="__main__" block in B will run)
B is not actually __main__ (so that it does not, e.g., overwrite the "__main__" entry in sys.modules)
Exceptions raised within B propagate to A (i.e., could be caught with an except clause in A).
Those exceptions, if not caught, generate a correct traceback referencing line numbers within B.
I've tried various techniques, but none seem to satisfy all my requirements.
using tools from the subprocess module means exceptions in B do not propagate to A.
execfile("B.py", {}) runs B, but it doesn't think it's main.
execfile("B.py", {'__name__': '__main__'}) makes B.py think it's main, but it also seems to screw up the exception traceback printing, so that the tracebacks refer to lines within A (i.e., the real __main__).
using imp.load_source with __main__ as the name almost works, except that it actually modifies sys.modules, thus stomping on the existing value of __main__
Is there any way to get what I want?
(The reason I'm doing this is because I'm doing some cleanup on an existing library. This library has no real test suite, just a set of "example" scripts that produce certain output. I'm trying to leverage these as tests to ensure that my cleanup doesn't affect the library's ability to execute these examples, so I want to run each example script from within my test suite. I'd like to be able to see exceptions from these scripts within the test script so the test script can report the type of failure, instead of just reporting a generic SubprocessError whenever an example script raises some exception.)
Your use case makes sense, but I still think you'd be better off refactoring the tests such that they can be run externally.
Do you test scripts have something like this?
def test():
pass
if __name__ == '__main__':
test()
If not, perhaps you should convert your tests to calling a function such as test. Then, from your main test script, you can just:
import test1
test1.test()
import test2
test2.test()
Provide a common interface to running tests, that the tests themselves use. Having a big block of code in a __main__ check is Not A Good Thing.
Sorry that I didn't answer the question you asked, but I feel this is the correct solution without deviating too far from the original test code.
Answering my own question because the result is kind of interesting and might be useful to others:
It turns out I was wrong: execfile("B.py", {'__name__': '__main__'} is the way to go after all. It does correctly produce the tracebacks. What I was seeing with incorrect line numbers weren't exceptions but warnings. These warnings were produced using warnings.warn("blah", stacklevel=2). The stacklevel=2 argument is supposed to allow for things like deprecation warnings to be raised where the deprecated thing is used, rather than at the warning call (see the documentation).
However, it seems that the execfile-d file doesn't count as a "stack level" for this purpose, and is invisible for stacklevel purposes. So if code at the top level of an execfile-d module causes a warning with stacklevel 2, the warning is not raised at the right line number in the execfile-d source file; instead it is raised at the corresponding line number of the file which is running the execfile.
This is unfortunate, but I can live with it, since these are only warnings, so they won't impact the actual performance of the tests. (I didn't notice at first that it was only the warnings that were affected by the line-number mismatches, because there were lots of warnings and exceptions intermixed in the test output.)
I think I am missing something about the sphinx extension for doctest.
The typical example in the documentation is:
.. doctest::
>>> print 1
1
Isn't there a way to let sphinx generate the output (here: 1) automatically?
As far as I understood, it is possible to run:
$ make doctest
which has the effect to test the code snippets, and compare the real output with the expected output. For example, if you have
.. doctest::
>>> print 1
3
doctest will warn you that it got 1 while it was expecting 3.
Instead, I would like sphinx to insert the real output alone in my docstring or in my .rst file. For example, if we have something like:
.. doctest::
>>> print 1
>>> print [2*x for x in range(3)]
I would like that when we run make doctest with an option, it changes the docstring to:
.. doctest::
>>> print 1
1
>>> print [2*x for x in range(3)]
[0,2,4]
I'm sure it's possible, and would be very convenient!
I have to strongly (but kindly) advise against what you're trying to do.
What you're asking is against the "test part" of the doctest module:
The doctest module searches for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown.
These tests have a reasons to be if you write the input and the expected output and let Python check if the expected output match the actual output.
If you let Python produce the expected output, well.. it will no longer be expected (by the user/author), so the doctests will never fail, hence those tests will be useless.
Note: If inside a function there's no logic (if/else, while-loops, appends, etc..) there's no need to test them. And tests must not reproduce the testing logic, otherwise they're not testing the function anymore.
I found this video about test driven development very interesting, maybe it could be of interest to you if you want to know more about this argument.
Here is a suggestion on how you could achieve what I suspect you might be looking for:
Doug Hellmann has written an interesting article called Writing Technical Documentation with Sphinx, Paver, and Cog. It has a section describing how the Cog tool can be used to automatically run code examples and capture the output for inclusion in Sphinx-built documentation.
There is also a contributed Sphinx extension called autorun that can execute code in a special
runblock directive and attach the output to the documentation.
This feature is available as part of pytest-accept, and extension of pytest: https://github.com/max-sixty/pytest-accept
Quote:
pytest-accept is a pytest plugin for automatically updating doctest
outputs. It runs doctests, observes the generated outputs, and writes
them to the doctests' documented outputs.
It's designed for a couple of use cases:
People who work with doctests and don't enjoy manually copying generated outputs from the pytest error log and pasting them into
their doctests' documented outputs. pytest-accept does the copying &
pasting for you.
People who generally find writing tests a bit annoying, and prefer to develop by "running the code and seeing whether it works". This
library aims to make testing a joyful part of that development loop.
I'm running lint as follows:
$ python -m pylint.lint m2test.py
with this code:
import M2Crypto
def f():
M2Crypto.RSA.new_pub_key("").as_pem(cipher=None).split("\n")
The lint output ends with:
Exception AttributeError: '_shutdown' in <module 'threading' from '/usr/lib/python2.7/site-packages/M2Crypto-0.21.1-py2.7-linux-x86_64.egg/M2Crypto/threading.pyc'> ignored
This code works fine when run (the above is actually a minimal test case; but the full version does work). The exception is ignored, but Bitten considers this a failure, so stops on this step.
I've tried adding 'M2Crypto.threading.init()'/'M2Crypto.threading.cleanup()' around the definition of the function, but that didn't fix the problem.
How can I prevent this problem from occurring?
I'm using M2Crypto 0.21.1, pylint 0.24 and Python 2.7 (also tried 2.7.2) on Debian Lenny x86_64.
The exception that you are seeing is caused by a bug in the astng package (presumably “Abstract Syntax Tree, Next Generation”?) which is a toolkit on which pylint depends, written by the same people. I should note in passing that I always encourage people to use pyflakes instead of pylint when possible, because it is quick, simple, fast, and predictable, whereas pylint tries to do several kinds of deep magic that are not only slow but that can get it into exactly this kind of trouble. :)
Here are the two packages on PyPI:
http://pypi.python.org/pypi/pylint
http://pypi.python.org/pypi/astng
And note that this problem had to be, necessarily, a bug in pylint and not in your code, because pylint does not run your code in order to produce its report — imagine the havoc that could be wreaked if it did (since code being linted might delete files, etcetera)! Since your code does not get run, no amount of caution, like protecting your call with threading init() or cleanup() functions, could possibly have prevented this error — unless the code snippets happened, for other reasons, to alter the behavior we are about to investigate.
So, on to your actual exception.
I had never actually heard of _shutdown before! A quick search of the Python standard library showed its definition in threading.py but not a call of the function from anywhere; only by searching the Python C source code did I discover where in pythonrun.c, during interpreter shutdown, the function is actually called:
static void
wait_for_thread_shutdown(void)
{
...
PyObject *threading = PyMapping_GetItemString(tstate->interp->modules,
"threading");
if (threading == NULL) {
/* threading not imported */
PyErr_Clear();
return;
}
result = PyObject_CallMethod(threading, "_shutdown", "");
if (result == NULL) {
PyErr_WriteUnraisable(threading);
}
...
}
Apparently it is some sort of cleanup function that the threading Standard Library module requires, and they have special-cased the Python interpreter itself to make sure that it gets called.
As you can see from the code above, Python quietly and without complaint handles the case where the threading module never gets imported during a program's run. But if threading does get imported, and still exists at shutdown time, then the interpreter looks inside for a _shutdown function and goes so far as to print an error message — and then return a non-zero exit status, the cause of your problems — if it cannot call it.
So we have to discover why the threading module exists but has no _shutdown method at the moment when pylint is done examining your program and Python is exiting. Some instrumention is called for. Can we print out what the module looks like as pylint exits? We can! The pylint/lint.py module, in its last few lines, runs its “main program” by instantiating a Run class it has defined:
if __name__ == '__main__':
Run(sys.argv[1:])
So I opened lint.py in my editor — one of the magnificent things about having each little project installed in a Python Virual Environment is that I can jump in and edit third-party code for quick experiments — and added the following print statement down at the bottom of the Run class's __init__() method:
sys.path.pop(0)
print "*****", sys.modules['threading'].__file__ # added by me!
if exit:
sys.exit(self.linter.msg_status)
I re-ran the command:
python -m pylint.lint m2test.py
And out came the __file__ string of the threading module:
***** /home/brandon/venv/lib/python2.7/site-packages/M2Crypto/threading.pyc
Well, look at that.
This is the problem!
According to this path, there actually exists an M2Crypto/threading.py module that, under all normal circumstances, should just be called M2Crypto.threading, and therefore sit in the sys.modules dictionary under the name:
sys.modules['M2Crypto.threading']
But somehow that file is also getting loaded as the main Python threading module, shadowing the official threading module that sits in the Standard Library. Because of this, the Python exit logic is quite correctly complaining that the Standard Library _shutdown() function is missing.
How could this happen? Top-level modules can only appear in paths that are listed explicitly in sys.path, not in sub-directories beneath them. This leads to a new question: is there any point during the pylint run that the …/M2Crypto/ directory itself is getting put on sys.path as though it contained top-level modules? Let's see!
We need more instrumentation: we need to have Python tell us the moment that a directory with M2Crypto in the name appears in sys.path. It will really slow things down, but let's add a trace function to pylint's __init__.py — because that is the first module that gets imported when you run -m pylint.lint — that will write an output file telling us, for every line of code executed, whether sys.path has any bad values in it:
def install_tracer():
import sys
output = open('mytracer.out', 'w')
def mytracer(frame, event, arg):
broken = any(p.endswith('M2Crypto') for p in sys.path)
output.write('{} {}:{} {}\n'.format(
broken, frame.f_code.co_filename, frame.f_lineno, event))
return mytracer
sys.settrace(mytracer)
install_tracer()
del install_tracer
Note how careful I am here: I define only one name in the module's namespace, and then carefully delete it to clean up after myself before I let pylint continue loading! And all of the resources that the trace function itself needs — namely, the sys module and the output open file — are available in the install_tracer() closure so that, from the outside, pylint looks exactly the same as always. Just in case anyone tries to introspect it, like pylint might!
This generates a file mytracer.out of about 800k lines, that each look something like this:
False /home/brandon/venv/lib/python2.7/posixpath.py:118 call
The False says that sys.path looks clean, the filename and line number are the line of code being executed, and call indicates what stage of execution the interpreter is in.
So does sys.path ever get poisoned? Let's look at just the first True or False on each line, and see how many successive lines start with each value:
$ awk '{print$1}' mytracer.out | uniq -c
607997 False
3173 True
4558 False
33217 True
4304 False
41699 True
2953 False
110503 True
52575 False
Wow! That's a problem! For runs of several thousand lines at a time, our test case is True, which means that the interpreter is running with …/M2Crypto/ — or some variant of a pathname with M2Crypto in it — on the path, where it should not be; only the directory that contains …/M2Crypto should ever be on the path. Looking for the first False to True transition in the file, I see this:
False /home/brandon/venv/lib/python2.7/site-packages/logilab/astng/builder.py:132 line
False /home/brandon/venv/lib/python2.7/posixpath.py:118 call
...
False /home/brandon/venv/lib/python2.7/posixpath.py:124 line
False /home/brandon/venv/lib/python2.7/posixpath.py:124 return
True /home/brandon/venv/lib/python2.7/site-packages/logilab/astng/builder.py:133 line
And looking at lines 132 and 133 in the builder.py file reveals our culprit:
130 # build astng representation
131 try:
132 sys.path.insert(0, dirname(path)) # XXX (syt) iirk
133 node = self.string_build(data, modname, path)
134 finally:
135 sys.path.pop(0)
Note the comment, which is part of the original code, not an addition of my own! Obviously, XXX (syt) iirk is an exclamation in this programmer's strange native language for the phrase, “put this module's parent directory on sys.path so that pylint will break mysteriously every time someone forces pylint to introspect a package with a threading sub-module.” It is, obviously, a very compact native language. :)
If you adjust the tracing module to watch sys.modules for the actual import of threading — an exercise I will leave to the reader — you will see that it happens when SocketServer, which is imported by some other Standard Library module during the analysis, in turn tries to innocently import threading.
So let us review what is happening:
pylint is dangerous magic.
As part of its magic, if it sees you import foo, then it runs off trying to find foo.py on disk, to parse it, and to predict whether you are loading valid or invalid names from its namespace.
[See my comment, below.] Because you call .split() on the return value of RSA.as_pem(), pylint tries to introspect the as_pem() method, which in turn uses the M2Crypto.BIO module, which in turn makes calls that induce pylint to import threading.
As part of loading any module foo.py, pylint throws the directory containing foo.py on sys.path, even if that directory is inside a package, and therefore gives modules in that directory the privilege of shadowing Standard Library modules of the same name during its analysis.
When Python exits, it is upset that the M2Crypto.threading library is sitting where threading belongs, because it wants to run the _shutdown() method of threading.
You should report this as a bug to the pylint / astng folks at logilab.org. Tell them I sent you.
If you decide to keep using pylint after it has done this to you, then there seem to be two solutions in this case: either don't inspect code that calls M2Crypto, or import threading during the pylint import process — by sticking import threading into the pylint/__init__.py, for example — so that the module gets the chance to grab the sys.modules['threading'] slot before pylint gets all excited and tries to let M2Crypto/threading.py grab the slot instead.
In conclusion, I think the author of astng says it best: XXX (syt) iirk. Indeed.
Many thanks to Brandon Craig Rhodes for having tracing this down and for such a detailed post.
I've removed the offending line from astng, code available from the hg repository until logilab-astng 0.23.0 is out. And I can confirm this fixes the OP's pb.
This looks more like a hack but I think it works. Copying the result of "as_pem()" and splitting it.
import M2Crypto
def f():
M2Crypto.RSA.new_pub_key("").as_pem(cipher=None)[:].split("\n")
I'm using Python 2.6.7, M2Crypto 0.21.1, pylint 0.23
I was unable to reproduce (pylint 0.24 and M2Crypto 0.21.1 on Ubuntu 11.04 64bit) but two suggestions:
Explicitly initialize threading:
import M2Crypto
def f():
M2Crypto.threading.init()
M2Crypto.RSA.new_pub_key("").as_pem(cipher=None).split("\n")
M2Crypto.threading.cleanup()
Or recompile without threading:
m2crypto = Extension(name = 'M2Crypto.__m2crypto',
sources = ['SWIG/_m2crypto.i'],
extra_compile_args = ['-DTHREADING'],
#extra_link_args = ['-Wl,-search_paths_first'], # Uncomment to build Universal Mac binaries
)
I love being able to modify the arguments the get sent to a function, using settrace, like :
import sys
def trace_func(frame,event,arg):
value = frame.f_locals["a"]
if value % 2 == 0:
value += 1
frame.f_locals["a"] = value
def f(a):
print a
if __name__ == "__main__":
sys.settrace(trace_func)
for i in range(0,5):
f(i)
And this will print:
1
1
3
3
5
What other cool stuff can you do using settrace?
I would strongly recommend against abusing settrace. I'm assuming you understand this stuff, but others coming along later may not. There are a few reasons:
Settrace is a very blunt tool. The OP's example is a simple one, but there's practically no way to extend it for use in a real system.
It's mysterious. Anyone coming to look at your code would be completely stumped why it was doing what it was doing.
It's slow. Invoking a Python function for every line of Python executed is going to slow down your program by many multiples.
It's usually unnecessary. The original example here could have been accomplished in a few other ways (modify the function, wrap the function in a decorator, call it via another function, etc), any of which would have been better than settrace.
It's hard to get right. In the original example, if you had not called f directly, but instead called g which called f, your trace function wouldn't have done its job, because you returned None from the trace function, so it's only invoked once and then forgotten.
It will keep other tools from working. This program will not be debuggable (because debuggers use settrace), it will not be traceable, it will not be possible to measure its code coverage, etc. Part of this is due to lack of foresight on the part of the Python implementors: they gave us settrace but no gettrace, so it's difficult to have two trace functions that work together.
Trace functions make for cool hacks. It's fun to be able to abuse it, but please don't use it for real stuff. If I sound hectoring, I apologize, but this has been done in real code, and it's a pain. For example, DecoratorTools uses a trace function to perform the magic feat of making this syntax work in Python 2.3:
# Method decorator example
from peak.util.decorators import decorate
class Demo1(object):
decorate(classmethod) # equivalent to #classmethod
def example(cls):
print "hello from", cls
A neat hack, but unfortunately, it meant that any code that used DecoratorTools wouldn't work with coverage.py (or debuggers, I guess). Not a good tradeoff if you ask me. I changed coverage.py to provide a mode that lets it work with DecoratorTools, but I wish I hadn't had to.
Even code in the standard library sometimes gets this stuff wrong. Pyexpat decided to be different than every other extension module, and invoke the trace function as if it were Python code. Too bad they did a bad job of it.
</rant>
I made a module called pycallgraph which generates call graphs using sys.settrace().
Of course, code coverage is accomplished with the trace function. One cool thing we haven't had before is branch coverage measurement, and that's coming along nicely, about to be released in an alpha version of coverage.py.
So for example, consider this function:
def foo(x):
if x:
y = 10
return y
if you test it with this call:
assert foo(1) == 10
then statement coverage will tell you that all the lines of the function were executed. But of course, there's a simple problem in that function: calling it with 0 raises a UnboundLocalError.
Branch measurement would tell you that there's a branch in the code that isn't fully exercised, because only one leg of the branch is ever taken.
For example, get the memory consumption of Python code line-by-line: http://pypi.python.org/pypi/memory_profiler
One latest project that uses settrace heavily is PySnooper
It helps new programmers to trace/log/monitor their program output. Cheers!
I don't have an exhaustively comprehensive answer but one thing I did with it, with the help of another user on SO, was create a program that generates the trace tables of other Python programs.
The python debugger Pdb uses sys.settrace to analyse lines to debug.
Here's an c optimization/extension for pdb that also uses sys.settrace
https://bitbucket.org/jagguli/cpdb