I have a python program with a global function that is painful to test (it needs a large dataset to work properly). What is the best way to get around this while testing functions that call it?
I've found that the following works (but it make me feel dirty to use it).
module foo:
def PainLiesHere():
return 4; #guaranteed to be random
module test
import foo
def BlissLiesHere():
return 5
foo.PainLiesHere = BlissLiesHere
# test stuff
This is a perfectly fine way to do it. As long as you know that BlissLiesHere does not change the overall behavior of the unit you are testing...
EDIT:
This is what is being done, under all the nice extras they provide, by different kinds of mocking libraries, such as Mock, Mox, etc.
Related
I am designing python assignments for a class. I define functions, write docstrings and then I implement them. Afterward, I'd like to remove all my implementations of the functions and replace only the code (not the doc-strings, function names, and arguments) with a raise NotImplementedError.
Is there any tool (e.g. IDE) which removes all the code for me automatically, so that I don't have to replace the implemented function by myself? I was thinking about writing a small script, but I thought I might ask here before I do this ...
If anyone has written something similar or knows of a quick way how to this, I would appreciate this a lot.
Here's a minimal example of what I'd like to achieve:
test.py
def add(a,b):
"""
Adds two numbers
"""
return a+b
def multiply(a,b):
"""
Multiplies two numbers
"""
return a*b
should become in an automated fashion (and of course for much larger files):
test.py
def add(a,b):
"""
Adds two numbers
"""
raise NotImplementedEror
def multiply(a,b):
"""
Multiplies two numbers
"""
raise NotImplementedEror
I don't know of a tool to do specifically this, but Python provides great AST manipulation tools within its own standard library via the ast module. You'll need a third party module to "unparse" the result after transformation back into regular Python code, and after a quick search I found this one seems to do the trick, although there do seem to be many others.
Here's a bit of sample code to get you in the right direction. Obviously, you'll need to tweak this to get the exact behavior you want, especially if you want to provide classes instead of just top-level functions (as I've written nothing to handle that use case). But with a bit of Python knowledge, it should be possible to automate.
Also, this is Python 3 (which, as of the start of 2020, is the only supported Python version). If you're still on Python 2, it may require some modifications.
import ast
import astunparse
# Read our file using the built-in Python AST module.
with open('filename.py') as f:
data = ast.parse(f.read(), 'filename.py')
# Loop through all declarations.
for decl in data.body:
# Only modify functions
if isinstance(decl, ast.FunctionDef):
# The docstring is the first statement of the body. So we don't
# want to change it. Instead, replace the rest of the body with
# our pre-built "raise" call. Note that I figured out what "raise"
# looked like in AST form by running
#
# ast.dump(ast.parse("raise NotImplementedError()"))
#
decl.body[1:] = [ast.Raise(ast.Call(ast.Name('NotImplementedError'), [], []), None)]
# Use astunparse to pretty print the result as Python code.
print(astunparse.unparse(data))
It's definitely possible to automate, if you're willing to take the time to do it. If you're planning to do this for several assignments, or even over several semesters, you may consider making a script for it. But if you're just doing it once, it may be more worth your time to just do it by hand.
I have been writing unit tests for over a year now, and have always used patch.object for pretty much everything (modules, classes, etc).
My coworker says that patch.object should never be used to patch an object in a module (i.e. patch.object(socket, 'socket'), instead you should always use patch('socket.socket').
I much prefer the patch.object method, as it allows me to import modules and is more pythonic in my opinion. Is my coworker right?
Note: I have looked through the patch documentation and can't find any warnings on this subject. Isn't everything an object in python?
There is no such requirement, and yes, everything is an object in Python.
It is nothing more than a style choice; do you import the module yourself or does patch take care of this for you? Because that's the only difference between the two approaches; either patch() imports the module, or you do.
For the code-under-test, I prefer mock.patch() to take care of this, as this ensures that the import takes place as the test runs. This ensures I get a test error state (test failure), rather than problems while loading the test. All other modules are fair game.
Looking at the mock source code, it really doesn't look like there is a difference.
To investigate I first looked at def patch and see that it does:
getter, attribute = _get_target(target)
return _patch(
getter, attribute, new, spec, create,
spec_set, autospec, new_callable, kwargs)
wheras patch.object does the same except: getter = lambda: target
Ok, so what does this _get_target do? It pretty much splits the string and calls _importer on the first part (making an object) and uses the string the same way as get_object.
_importer is a pretty simple mechanism to import from a module (using getattr for every "component"), and pretty clearly just makes an object as well.
So fundamentally, at the source level, there is not really any difference.
Case Closed
I've really tried to start isolating my unit tests so I can pinpoint where errors occur rather than having my entire screen turn red with failures when one thing goes wrong. It's been working in all instances except when something in an initializer fails.
Check out these tests:
#setup_directory(test_path)
def test_filename(self):
flexmock(lib.utility.time).should_receive('timestamp_with_random').and_return(1234)
f = SomeFiles(self.test_path)
assert f.path == os.path.join(self.test_path, '1234.db')
#setup_directory(test_path)
def test_filename_with_suffix(self):
flexmock(lib.utility.time).should_receive('timestamp_with_random').and_return(1234)
f = SomeFiles(self.test_path, suffix='.txt')
assert f.path == os.path.join(self.test_path, '1234.txt')
I'm mocking dependent methods so that the thing I'm testing is completely isolated. What you notice is that the class needs to be instantiated for every single test. If an error is introduced in the initializer, every single test fails.
This is the offending constructor that calls the class's initializer:
SomeFiles(*args)
Is there a way to isolate or mock the initializer or object constructor?
I'm not sure what testing packages you're using, but in general, you can usually just mock the __init__() call on the class before actually attempting to use it. Something like
def my_init_mock_fn(*args, **kwargs):
print 'mock_init'
SomeFiles.__init__ = my_init_mock_fn
SomeFiles()
This isn't probably exactly what you want as from this point on SomeFiles.__init__ fn will always be the mock fn, but there are utilities like voidspace mock that provide a patch function that allow you to patch the class just for a specific scope.
from mock import patch
with patch.object(SomeFiles, '__init__', my_init_mock_fn):
SomeFiles()
..other various tests...
SomeFiles() #__init__ is reset to original __init__ fn
I'm sure there's probably similar functionality in whatever mocking package you are using.
Just realized you're using flexmock, there's a page for replace_with here.
What's causing the initialising function to fail? Maybe that's a bug that you should be looking into.
Another thing you can do, instead of mocking the object constructor, is simply mocking its return values. ie: Given this input, I expect this output -- so I'm going to use this expected output whether or not it returned correctly.
You can also stop testing on first failure. (failfast)
You also might want to reconsider how your tests are set up. If you have to recreate two files for every test, maybe ask yourself why. Could your tests be structured that you set up the two files, then run a series of tests, rinse and repeat. This would make it so only the series of tests assigned to that path fail, helping you isolate why it failed at all.
I have created a python class Test_getFileSize for use with nose
relevant sections:
def __init__(self,mytestfile="./filetest",testsize=102400):
''' Constructor'''
print " Running __init__", testsize,mytestfile
self.testsize=testsize
self.mytestfile = mytestfile
and the workhorse method:
#with_setup(setUp, tearDown)
def test_getFileSize(self):
from nose.tools import ok_, eq_,with_setup
import mp4
with open(self.mytestfile,"rb") as out:
filesize=mp4.getFileSize(out)
eq_(self.testsize,filesize,msg='Passed Test size')
print "Results ", filesize,self.testsize
If I run nosetest against the file containing this class, it correctly tests the class using the default values and the correct setUp and tearDown methods. Problem is that when I write a class to do just that, the setUp method never gets run.
What I want to be able to do is test different file sizes ( i.e. pass a filesize value).
If there is a better way to do it, I am all ears. I would prefer not to do it via the command line if possible.
Thanks
Jim
You could write a test function (not part of a class) where the test function itself is a generator, with each yield returning a new function to run with arguments to generate another test. That would work well if you had 500 different filenames/filesizes as a list you wanted to test against.
See here for a simple example/docs: http://nose.readthedocs.org/en/latest/writing_tests.html#test-generators
With a test class, things get a bit trickier - since it doesn't allow you to use this generator method for class methods. You could use a metaclass to return a class with a suitable number of functions to run your test (one per case, for example.) but that might be beyond what you want to do.
That being said, you might find it sufficient to have a single test method that iterates over a list of filenames/sizes and performs the test on each one. The work there is significantly less, but also results in a single "test" output line for the collective set of tests.
You might reference this question for an answer as to how one person did this:
nose, unittest.TestCase and metaclass: auto-generated test_* methods not discovered
I have two functions like the following:
def fitnesscompare(x, y):
if x.fitness>y.fitness:
return 1
elif x.fitness==y.fitness:
return 0
else: #x.fitness<y.fitness
return -1
that are used with 'sort' to sort on different attributes of class instances.
These are used from within other functions and methods in the program.
Can I make them visible everywhere rather than having to pass them to each object in which they are used?
Thanks
The best approach (to get the visibility you ask about) is to put this def statement in a module (say fit.py), import fit from any other module that needs access to items defined in this one, and use fit.fitnesscompare in any of those modules as needed.
What you ask, and what you really need, may actually be different...:
as I explained in another post earlier today, custom comparison functions are not the best way to customize sorting in Python (which is why in Python 3 they're not even allowed any more): rather, a custom key-extraction function will serve you much better (future-proof, more general, faster). I.e., instead of calling, say
somelist.sort(cmp=fit.fitnesscompare)
call
somelist.sort(key=fit.fitnessextract)
where
def fitnessextract(x):
return x.fitness
or, for really blazing speed,
import operator
somelist.sort(key=operator.attrgetter('fitness'))
Defining a function with def makes that function available within whatever scope you've defined it in. At module level, using def will make that function available to any other function inside that module.
Can you perhaps post an example of what is not working for you? The code you've posted appears to be unrelated to your actual problem.