I am writing an application which makes use of the current time (via datetime.datetime.now())
Is there a way, for debugging purposes, to override this call so that it returns a specific timestamp?
As a fallback I could write a function which would be called instead of datetime.datetime.now() and return whatever is needed (the actual current time in producton and the required test time when debugging) but there may be a more pythonic way to perform these kind of actions?
Broadly, your options are:
Use the unittest.mock library, which can replace the function on the fly with a dummy function that always gives the same results (or use another mocking library that does the same thing). This means you don't have to modify your function; however, reasonable people can disagree on whether monkey patching with mock is good practice, even for debugging. I think this is the most widely used solution to this problem in Python.
Modify your function to do something different depending on its environment (the actual environment variables on your system, or global state, or something else). This is the easiest way, but also the crudest and most fragile way, so you'll have to be sure to change it back after your debugging is finished.
Modify your function to accept a function itself as a parameter, and pass in datetime.datetime.now as that function in normal operation, but pass in something different (for instance a stub) for testing.
You can use the mock library to mock the datetime.datetime.now() usage:
import mock
def my_test():
my_mock = mock.Mock(return_value=your_desired_timestamp)
with mock.patch('mymodule.datetime.datetime.now', my_mock):
# Here, all calls to `datetime.datetime.now` referenced by `datetime.datetime`
# defined in `my_module` will be mocked to return `your_desired_timestamp`.
Related
I have some code I'd like to quickly test. This code includes a line that needs to query a server and obtain a True/False answer:
result = server.queryServer(data)
Is there a way to override this function call so that it just calls a local function that always returns True (so that I can debug without running the server)?
Mock is your friend. It allows you to mock entire classes or functions of them.
What you want is called mocking, replacing existing objects with temporary objects that act differently just for a test.
Use the unittest.mock library to do this. It will create the temporary objects for you, and give you the tools to replace the object, and restore the old situation, for the duration of a test.
The module provides patchers to do the replacement. For example, you could use a context manager:
from unittest import mock
with mock.patch('server.queryServer') as mocked_queryServer:
mocked_queryServer.return_value = True # always return True when called
# test your code
# ...
# afterwards, check if the mock has been called at least once.
mocked_queryServer.assert_called()
I added an assertion at the end there; mock objects not only let you replace functions transparently, they then also record what happens to them, letting you check if your code worked correctly by calling the mock.
I have been writing unit tests for over a year now, and have always used patch.object for pretty much everything (modules, classes, etc).
My coworker says that patch.object should never be used to patch an object in a module (i.e. patch.object(socket, 'socket'), instead you should always use patch('socket.socket').
I much prefer the patch.object method, as it allows me to import modules and is more pythonic in my opinion. Is my coworker right?
Note: I have looked through the patch documentation and can't find any warnings on this subject. Isn't everything an object in python?
There is no such requirement, and yes, everything is an object in Python.
It is nothing more than a style choice; do you import the module yourself or does patch take care of this for you? Because that's the only difference between the two approaches; either patch() imports the module, or you do.
For the code-under-test, I prefer mock.patch() to take care of this, as this ensures that the import takes place as the test runs. This ensures I get a test error state (test failure), rather than problems while loading the test. All other modules are fair game.
Looking at the mock source code, it really doesn't look like there is a difference.
To investigate I first looked at def patch and see that it does:
getter, attribute = _get_target(target)
return _patch(
getter, attribute, new, spec, create,
spec_set, autospec, new_callable, kwargs)
wheras patch.object does the same except: getter = lambda: target
Ok, so what does this _get_target do? It pretty much splits the string and calls _importer on the first part (making an object) and uses the string the same way as get_object.
_importer is a pretty simple mechanism to import from a module (using getattr for every "component"), and pretty clearly just makes an object as well.
So fundamentally, at the source level, there is not really any difference.
Case Closed
I am trying to mock the infamous datetime.now() function so that I have a frozen date easy to test in my unit test. I usually use unittest-mock package to do so, patching the datetime object of the interested module.
The problem is that the datetime.now() I am trying to patch is coming from a default function parameter, as follows (in views.py):
def user_as_guest(user_id,
date=datetime.now()):
So this simply doesn't work (test.py):
with patch('myapp.views.datetime') as mock_date:
frozen_date = datetime.now(pytz.utc)
mock_date.now.return_value = frozen_date
Events.user_as_guest(user_id_1)
Even if debugging the flow I can see the datetime.now() is correctly patched, I imagine the callable object used for assigning the default parameter has a different scope from the module, but I can't go further. Thank for your help!
Side Note
I realized the code I wrote was not doing exactly what I wanted. I thought the default function parameters were evaluated and late bound, that is not actually the case.
Indeed, my date object would contain a date in the past, not the one at the time of the function call. This explains more why I couldn't patch it with my approach.
I fixed everything assigning the date in the function body, so the old but gold mocking approach works. But still, the question is valid...
I've really tried to start isolating my unit tests so I can pinpoint where errors occur rather than having my entire screen turn red with failures when one thing goes wrong. It's been working in all instances except when something in an initializer fails.
Check out these tests:
#setup_directory(test_path)
def test_filename(self):
flexmock(lib.utility.time).should_receive('timestamp_with_random').and_return(1234)
f = SomeFiles(self.test_path)
assert f.path == os.path.join(self.test_path, '1234.db')
#setup_directory(test_path)
def test_filename_with_suffix(self):
flexmock(lib.utility.time).should_receive('timestamp_with_random').and_return(1234)
f = SomeFiles(self.test_path, suffix='.txt')
assert f.path == os.path.join(self.test_path, '1234.txt')
I'm mocking dependent methods so that the thing I'm testing is completely isolated. What you notice is that the class needs to be instantiated for every single test. If an error is introduced in the initializer, every single test fails.
This is the offending constructor that calls the class's initializer:
SomeFiles(*args)
Is there a way to isolate or mock the initializer or object constructor?
I'm not sure what testing packages you're using, but in general, you can usually just mock the __init__() call on the class before actually attempting to use it. Something like
def my_init_mock_fn(*args, **kwargs):
print 'mock_init'
SomeFiles.__init__ = my_init_mock_fn
SomeFiles()
This isn't probably exactly what you want as from this point on SomeFiles.__init__ fn will always be the mock fn, but there are utilities like voidspace mock that provide a patch function that allow you to patch the class just for a specific scope.
from mock import patch
with patch.object(SomeFiles, '__init__', my_init_mock_fn):
SomeFiles()
..other various tests...
SomeFiles() #__init__ is reset to original __init__ fn
I'm sure there's probably similar functionality in whatever mocking package you are using.
Just realized you're using flexmock, there's a page for replace_with here.
What's causing the initialising function to fail? Maybe that's a bug that you should be looking into.
Another thing you can do, instead of mocking the object constructor, is simply mocking its return values. ie: Given this input, I expect this output -- so I'm going to use this expected output whether or not it returned correctly.
You can also stop testing on first failure. (failfast)
You also might want to reconsider how your tests are set up. If you have to recreate two files for every test, maybe ask yourself why. Could your tests be structured that you set up the two files, then run a series of tests, rinse and repeat. This would make it so only the series of tests assigned to that path fail, helping you isolate why it failed at all.
I have two functions like the following:
def fitnesscompare(x, y):
if x.fitness>y.fitness:
return 1
elif x.fitness==y.fitness:
return 0
else: #x.fitness<y.fitness
return -1
that are used with 'sort' to sort on different attributes of class instances.
These are used from within other functions and methods in the program.
Can I make them visible everywhere rather than having to pass them to each object in which they are used?
Thanks
The best approach (to get the visibility you ask about) is to put this def statement in a module (say fit.py), import fit from any other module that needs access to items defined in this one, and use fit.fitnesscompare in any of those modules as needed.
What you ask, and what you really need, may actually be different...:
as I explained in another post earlier today, custom comparison functions are not the best way to customize sorting in Python (which is why in Python 3 they're not even allowed any more): rather, a custom key-extraction function will serve you much better (future-proof, more general, faster). I.e., instead of calling, say
somelist.sort(cmp=fit.fitnesscompare)
call
somelist.sort(key=fit.fitnessextract)
where
def fitnessextract(x):
return x.fitness
or, for really blazing speed,
import operator
somelist.sort(key=operator.attrgetter('fitness'))
Defining a function with def makes that function available within whatever scope you've defined it in. At module level, using def will make that function available to any other function inside that module.
Can you perhaps post an example of what is not working for you? The code you've posted appears to be unrelated to your actual problem.