A lot of my mock usage is in unit tests, but I am not sure if I can use the mock library for production, consider the following trivial example to get data from external source.
class Receiver(object):
def get_data(self):
return _call_api(...)
Now, can I use mock library to change the get_data() function for re-run purpose on production?
with patch('Receiver.get_data') as mock_get_data:
mock_get_data.return_values = [1, 2]
...
Some might suggest to write another Rerun receiver as a better approach, while I don't disagree but I am still raising this question for the sake of curiosity.
My questions include:
If no, what's the reason?
If yes, any caveats?
I would agree that for production use, a Receiver subclass that has an overridden get_data method would be much better.
The reason is simple -- if each type of receiver only receives data from a single source then your code will be much easier to read and maintain. If the same Reciever will end up returning data from multiple sources, then the code will be confusing and you'll end up needing to hunt down whether you were fetching data from one place of whether it's data that you explicitly set via mock, etc.
No. If a function is supposed to behave a certain way in production, then code it to behave that way. If you need fallback or retry behavior, mock is not the right way to do that.
Related
I have a custom framework which runs different code for different clients. I have monkeypatched certain methods in order to customize functionality for a client.
Here is the pattern simplified:
#import monkeypatches here
if self.config['client'] == 'cool_dudes':
from app.monkeypatches import Stuff
if self.config['client'] == 'cool_dudettes':
from app.monkeypatches import OtherStuff
Here is an example patch:
from app.framework.stuff import Stuff
def function_override(self):
return pass
Stuff.function = function_override
This works fine when the program executes as it is executed in a batch manner, spinning up from scratch every time. However, when running across unit tests, I find that the monkey patches persist across tests, causing unexpected behavior.
I realize that it would be far better to use an object oriented inheritance approach to these overrides, but I inherited this codebase and am not currently empowered to rearchitect it to that degree.
Barring properly re-architecting the program, how can I prevent these monkey patches from persisting across unit tests?
The modules, including app.framework.<whatever>, are not reloaded for every test. So, any changes in them you make persist. The same happens if your module is stateful (that's one of the reasons why global state is not such a good idea, you should rather keep state in objects).
Your options are to:
undo the monkey-patches when needed, or
change them into something more generic that would change (semi-)automatically depending on the test running, or
(preferred) Do not reinvent the wheel and use an existing, manageable, time-proven solution for your task (or at least, base your work on one if it doesn't meet your requirements completely). E.g. if you use them for mocking, see How can one mock/stub python module like urllib . Among the suggestions there is #mock.patch that does the patching for a specific test and undoes it upon its completion.
Anyone coming here looking for information about monkeypatching, might want to have a look at pytest's monkeypatch fixture. It avoids the problem of the OP by automatically undoing all modifications after the test function has finished.
I am very new to Python and I saw many projects on Github using Mock to do their test but I don't understand why.
When we use mock, we construct a Mock object with a specific return_value, I don't truely understand why we do this. I know sometimes it is difficult to build our needed resources but what is the point if we construct a object/function with a certain return value ?
Mock can help to write unit tests.
In unit tests, you want to test a small portion of your implementation. For example, as small as one function or one class.
In a moderately large software, these small parts depend on each other. Or sometimes there are external dependencies. You open files, do syscalls, or get external data in some other way.
While writing a directed unit test for a small portion of your code, you do not want to spend time to set-up everything else around it. (The files, syscalls, external data). Mock comes to your help there. With mock, you can make the other dependencies of your code behave exactly as you like. This way you can focus on testing your intended implementation.
Coming to the mock with the return value: Say you want to test func_a. And func_a calls func_b. func_b does a lot of funky processing to calculate its return value. For example, talking to an external service, doing bunch of syscalls, or some other expensive operation. Since you are testing func_a, you only care about possible return values of func_b. (So that func_a can use them) In this scenario you would mock func_b and set the return values explicitly. This can really simplify your test complexity.
I am working on a project that aims to augment the Python socket messages with partial ordering information. The library I'm building is written in Python, and needs to be interposed on an existing system's messages sent through the socket functions.
I have read some of the resources out there, namely the answer by #Omnifarious at this question python-importing-from-builtin-library-when-module-with-same-name-exist
There is an extremely ugly and horrible thing you can do that does not
involve hooking the import mechanism. This is something you should
probably not do, but it will likely work. It turns your calendar
module into a hybrid of the system calendar module and your calendar
module.
I have implemented the import mechanism solution, but we have decided this is not the direction we'd like to take, since it relies too much on the environment. The solution to merge classes into a hybrid, rather than relying on the import mechanisms, seems to be the best approach in my case.
Why has the hybrid been called an ugly and horrible solution? I'd like to start implementing it in my project but I am wary of the warnings. It does seem a bit hackish, but since it would be part of an installation script, wouldn't it be OK to run this once?
Here is a code snippet where the interposition needs to intercept the socket message before it's sent:
class vector_clock:
def __init__(self):
"""
Initiate the clock with the object
"""
self.clock = [0,0]
def sendMessage(self):
"""
Send Message to the server
"""
self.msg = "This is the test message to that will be interposed on"
self.vector_clock.increment(0) # We are clock position 0
# Some extraneous formatting details removed for brevity….
# connectAndSend needs interpositioning to include the vector clock
self.client.connectAndSend(totalMsg);
self.client.s.close()
From my understanding of your post, you wish to modify the existing socket library to inject your own functionality into it.
Yes, this is completely doable, and possibly it is even the easiest solution to your problem, but you have to consider all of the implications of what you are doing.
The most important point is that you are not just modifying socket for yourself, but for anything that is run in any part of your process which uses the socket library unless it uses it's own class loader. I understand that there is probably some existing library you are using which uses socket and you want to inject this functionality into it, but this will affect EVERYTHING.
From this you have to consider the question: is your change 100% backwards compatible. Unless you can guarantee that you know every single use case of socket by any library used by your process (hint: you can't), then you need to make sure that it completely preserves all existing functionality or else somewhere down the road stuff in some core library is going to mysteriously break and you will have no idea why and no way to debug it. An example of something 100% backwards compatible (or as close as it is possible to get) is injecting a decorator which saves timing information to one of your own modules.
If you completely understand this and still think that your solution is a good one then I say "go for it". However, have you considered any alternatives?
If you just need to inject this functionality for a specific set of libraries that you use, then I would suggest doing something like patching: https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch
You could subclass whatever core library you want to modify and then patch the library to use your class instead. At it's core, what patch does is it modifies the global bindings used in the target module to use a different class/module than the one it had originally used.
PS. I don't think yours is a situation which calls for hooking the import mechanism.
I have a class that connects three other services, specifically designed to make implementing the other services more modular, but the bulk of my unit test logic is in mock verification. Is there a way to redesign to avoid this?
Python example:
class Input(object): pass
class Output(object): pass
class Finder(object): pass
class Correlator(object): pass
def __init__(self, input, output, finder):
pass
def run():
finder.find(input.GetRows())
output.print(finder)
I then have to mock input, output and finder. Even if I do make another abstraction and return something from Correlator.run(), it will still have to be tested as a mock.
Just ask yourself: what exactly do you need to check in this particular test case? If this check does not rely on other classes not being dummy, then you are OK.
However, a lot of mocks is usually a smell in sense that you are probably trying to test integration without actually doing integration. So if you assume that if the class passes test with mocks, it will be fine working with real classes, than yes, you have to write some more tests.
Personally, I don't write many Unit tests at all. I'm web developer and I prefer functional tests, that test the whole application via HTTP requests, as users would. Your case may be different
There's no reason to only use unit test - Maybe integration tests would be more useful for this case. Initialize all the objects properly, use the main class a bit, and assert on the (possibly complex) results. That way you'll test interfaces, output predictability, and other things which are important further up the stack. I've used this before, and found that something which is difficult to integration test probably has too many attributes / parameters or too complicated/wrongly formatted output.
On a quick glance, this do look like the level of mocking becomes to large. If you're on a dynamic language (I'm assuming yes here since your example is in Python), I'd try to construct either subclasses of the production classes with the most problematic methods overridden and presenting mocked data, so you'd get a mix of production and mocked code. If your code path doesn't allow for instantiating the objects, I'd try monkey patching in replacement methods returning mock data.
Weather or not this is code smell also depends on the quality of mocked data. Dropping into a debugger and copy-pasting known correct data or sniffing it from the network is in my experience the preferred way of ensuring that.
Integration vs unit testing is also an economical question: how painful is it to replace unit tests with integration/functional tests? The larger the scale of your system, the more there is to gain with light-weight mocking, and hence, unit tests.
I'd like something equivalent to
calling method: $METHOD_NAME
args: $ARGS
output: $OUTPUT
to be automatically logged to a file (via the logging module, possibly) for every (user-defined) method call. The best solution I can come up with is to write a decorator that will do this, and then add it to every function. Is there a better way?
Thanks
You could look at the trace module in the standard library, which
allows you to trace program execution, generate annotated statement coverage listings, print caller/callee relationships and list functions executed during a program run. It can be used in another program or from the command line.
You can also log to disk:
import sys
import trace
# create a Trace object, telling it what to ignore, and whether to
# do tracing or line-counting or both.
tracer = trace.Trace(
ignoredirs=[sys.prefix, sys.exec_prefix],
trace=0,
count=1)
# run the new command using the given tracer
tracer.run('main()')
# make a report, placing output in /tmp
r = tracer.results()
r.write_results(show_missing=True, coverdir="/tmp")
One approach that might simplify things a bit would be to use a metaclass to automatically apply your decorator for you. It'd cut down on the typing at the expense of requiring you to delve into the arcane and mysterious world of metaclass programming.
It depends how exactly are you going to use it.
Most generic approach would be to follow stdlib's 'profile' module path and therefore have control over each call, but its somewhat slow.
If you know which modules you need to track before giving them control, I'd go with iterating over all their members and wrapping with tracking decorator. This way tracked code stays clean and it doesn't take too much coding to implement.
A decorator would be a simple approach for a smaller project, however with decorators you have to be careful about passing arguments to make sure that they don't get mangled on their way through. A metaclass would probably be more of the "right" way to do it without having to worry about adding decorators to every new method.