Basically, growl notifications (or other callbacks) when tests break or pass. Does anything like this exist?
If not, it should be pretty easy to write.. Easiest way would be to..
run python-autotest myfile1.py myfile2.py etc.py
Check if files-to-be-monitored have been modified (possibly just if they've been saved).
Run any tests in those files.
If a test fails, but in the previous run it passed, generate a growl alert. Same with tests that fail then pass.
Wait, and repeat steps 2-5.
The problem I can see there is if the tests are in a different file. The simple solution would be to run all the tests after each save.. but with slower tests, this might take longer than the time between saves, and/or could use a lot of CPU power etc..
The best way to do it would be to actually see what bits of code have changed, if function abc() has changed, only run tests that interact with this.. While this would be great, I think it'd be extremely complex to implement?
To summarise:
Is there anything like the Ruby tool autotest (part of the ZenTest package), but for Python code?
How do you check which functions have changed between two revisions of a script?
Is it possible to determine which functions a command will call? (Somewhat like a reverse traceback)
I found autonose to be pretty unreliable but sniffer seems to work very well.
$ pip install sniffer
$ cd myproject
Then instead of running "nosetests", you run:
$ sniffer
Or instead of nosetests --verbose --with-doctest, you run:
$ sniffer -x--verbose -x--with-doctest
As described in the readme, it's a good idea to install one of the platform-specific filesystem-watching libraries, pyinotify, pywin32 or MacFSEvents (all installable via pip etc)
autonose created by gfxmonk:
Autonose is an autotest-like tool for python, using the excellent nosetest library.
autotest tracks filesystem changes and automatically re-run any changed tests or dependencies whenever a file is added, removed or updated. A file counts as changed if it has iself been modified, or if any file it imports has changed.
...
Autonose currently has a native GUI for OSX and GTK. If neither of those are available to you, you can instead run the console version (with the --console option).
I just found this: http://www.metareal.org/p/modipyd/
I'm currently using thumb.py, but as my current project transitions from a small project to a medium sized one, I've been looking for something that can do a bit more thorough dependency analysis, and with a few tweaks, I got modipyd up and running pretty quickly.
Guard is an excellent tool that monitors for file changes and triggers tasks automatically. It's written in Ruby, but it can be used as a standalone tool for any task like this. There's a guard-nosetests plugin to run Python tests via nose.
Guard supports cross-platform notifications (Linux, OSX, Windows), including Growl, as well as many other great features. One of my can't-live-without dev tools.
One very useful tool that can make your life easier is entr. Written in C, and uses kqueue or inotify under the hood.
Following command runs your test suite if any *.py file in your project is changed.
ls */**.py | entr python -m unittest discover -s test
Works for BSD, Mac OS, and Linux. You can get entr from Homebrew.
Maybe buildbot would be useful http://buildbot.net/trac
For your third question, maybe the trace module is what you need:
>>> def y(a): return a*a
>>> def x(a): return y(a)
>>> import trace
>>> tracer = trace.Trace(countfuncs = 1)
>>> tracer.runfunc(x, 2)
4
>>> res = tracer.results()
>>> res.calledfuncs
{('<stdin>', '<stdin>', 'y'): 1, ('<stdin>', '<stdin>', 'x'): 1}
res.calledfuncs contains the functions that were called. If you specify countcallers = 1 when creating the tracer, you can get caller/callee relationships. See the docs of the trace module for more information.
You can also try to get the calls via static analysis, but this can be dangerous due to the dynamic nature of Python.
Django's development server has a file change monitor that watches for modifications and automatically reloads itself. You could re-use this code to launch unit tests on file modification.
Maybe Nose http://somethingaboutorange.com/mrl/projects/nose/ has a plugin http://somethingaboutorange.com/mrl/projects/nose/doc/writing_plugins.html
Found this: http://jeffwinkler.net/2006/04/27/keeping-your-nose-green/
You can use nodemon for the task, by watching .py files and execute manage.py test. The command will be: nodemon --ext py --exec "python manage.py test".
nodemon is an npm package however, I assume you have node installed.
Check out pytddmon. Here is a video demonstration of how to use it:
http://pytddmon.org/?page_id=33
Related
Unlike earlier version 3 releases nowadays building Python 3.10 from source does not seem to run the (time-consuming) tests.
I need to build Python 3.10 on an oldish platform (no, I can't change that). I would actually like to run the tests, even if they are time consuming.
Unfortunately, I can't find the way to do it. Googling shows nonsensical results (how to do testing while using Python, unittest etc), while ./configure --help doesn't show anything.
Have the tests been removed? If not, how can I enable them?
Building from source make -j 4 prefix="/usr" usually does the tests too. At least that's what I've observed.
I'm currently writing a python program and I want to distribute it to some en users (and developers). I would like to reduce the number of necessary steps to run the program to a minimum.
My use case is relatively simple. I'd like the process/tool/whatever to:
A) Download the list of packages required for the application to work.
B) Run a list of python scripts, sequentially (e.g create database and then run migrations).
I understand that distlib does some of this already. However I find the documentation kind of confusing, there seems to be an API to install scripts, but not one to execute them automatically.
Ideally I would specify a list of scripts, and a list of dependencies and have the program install them automatically.
Maybe the best way to tackle this would be to use make with a Makefile (https://www.gnu.org/software/make/).
Distlib, via the setup.py file, would help you make it more readable by giving names to some python scripts. And you could make use of make target/dependencies system to execute tasks sequentially.
If you want to stick to python, you could also use Luigi (https://luigi.readthedocs.io/en/stable/) but it seems like overkill here.
Ok, so I ended writing my own thing, based on how I wanted the interface to look. The code that installs the application looks like this:
from installtools import setup
scripts = ['create_database.py', 'run_migrations.py']
setup("Shelob", "requirements.txt", scripts)
The full script can be found here: https://gist.github.com/fdemian/808c2b95b4521cd87268235e133c563f
Since PIP doesn't have a public API(and isn't likely to have one in the near future) the script uses the subprocess API to call:
pip install -r [requirements_file_path]
After that, it calls the specified python scripts, one by one. While it is probably not a very robust, as a stopgap solution it seems to do the trick.
We have been using Django for a long time. Some old code is not being used now. How can I find which code is not being used any more and remove them.
I used coverage.py with unit tests, which works fine and shows which part of code is never used, but the test covered is very low. Is there any way to use it with WSGI server to find which code have never served any web requests?
It depends on what you mean by unused code.
For unreachable dead code, like functions are were never called, classes that are never instantiated, you can use a pure static code analyzer to find them. Pylint is a good option. Bear in mind that this is not 100% accurate, false positive is possible:
# static analysis can't detect methods called this way
func = getattr(obj, "func_name")
func()
For code that are reachable, but never reached. You have to rely on tools like coverage.py, and improve your test coverage.
On a well tested project, coverage would be ideal but with some untested legacy code I don't think there is a magical tool.
You could write a big test loading all the pages and run coverage to get some indication.
Cowboy style:
If it's not some critical code and you're fairly sure it's unused (i.e. not handling payments, etc.). Comment it out, check that the tests pass, deploy and wait a week or so before removing it definitely (or putting it back if you got a notification).
As other answers indicate coverage.py accurately finds out which parts of the code are never executed, but coverage requires your code to be actually run to perform the analysis. Vulture on the other hand, runs static analysis for finding dead (unused code) for Python Programs. Also, if you run vulture on both your library and test suite, you might be able to find untested code.
Vulture is a standard PyPI package and can be installed using pip:
$ pip install vulture
Run vulture using the command:
$ vulture apps/ tests/ --exclude settings
Bear in mind that due to Python's dynamic nature, there may be some false positives, but they can be dealt with by the means of "Whitelists" - Please refer to this answer by Rahul for more information on how to use Vulture with django and tackle false positives.
I am writing a script to automate the packaging of a 'home-made' python module and distributing it on a remote machine.
i am using Pip and have created a setup.py file but i then have to call the subprocess module to call the "python setup.py sdist" command.
i have looked at the "run_setup" method in distutils.core but i am trying to avoid using the subprocess module alltogether. (i see no point in opening a shell to run a python command if i am already in python...)
is there a way to import the distutils module into my script and pass the setup information directly to one of its methods and avoid using the shell command entirely? or any other suggestions that may help me
thanks
Just for the sake of completeness, I wanted to answer this since I came across it trying to find out how to do this myself. In my case, I wanted to be sure that the same python version was being used to execute the command, which is why using subprocess was not a good option. (Edit: as pointed out in comment, I could use sys.executable with subprocess, though programmatic execution is IMO still a cleaner approah -- and obviously pretty straightforward.)
(Using distutils.core.run_setup does not call subprocess, but uses exec in a controlled scope/environment.)
from distutils.core import run_setup
run_setup('setup.py', script_args=['sdist'])
Another option, may be to use the setuptools commands, though I have not explored this to completion. Obviously, you still have to figure out how to avoid duplicating your proj metadata.
from setuptools.dist import Distribution
from setuptools.command.sdist import sdist
dist = Distribution({'name': 'my-project', 'version': '1.0.0'}) # etc.
dist.script_name = 'setup.py'
cmd = sdist(dist)
cmd.ensure_finalized()
cmd.run() # TODO: error handling
Anyway, hopefully that will help someone in the right direction. There are plenty of valid reasons to want to perform packaging operations programmatically, after all.
If you don’t have a real reason to avoid subprocesses (i.e. lack of platform support, not just aesthetics (“I see no point”)), then I suggest you should just not care and run in a subprocess. There are a few ways to achieve what you request, but they have their downsides (like having to catch exceptions and reporting errors).
nose
is a test runner which extends PyUnit. Is it possible to write e.g
$ nosetests --with-shell myTest.py -myargs test
If not, then is there a plugin, or do i need to develop it myself.
Any suggestions ?
Nose is not a general test harness. It's specifically a Python harness which runs Python unit tests.
So, while you can write extensions for it to execute scripts and mark them as successes or failures based on the exit status or an output string, I think it's an attempt to shoehorn the harness into doing something it's not really meant to do.
You should package your tests as Python functions or classes and then have them use a library to run external scripts the output or behaviour of which is translated into something that nose can interpret rather than extend nose to directly run scripts.
Also, I've experimented with nose a bit and found it's extension mechanism quite clumsy compared to py.test. You might want to give that a shot.