We Keep Coding

Python - Why we should use mock to do test?

I am very new to Python and I saw many projects on Github using Mock to do their test but I don't understand why.
When we use mock, we construct a Mock object with a specific return_value, I don't truely understand why we do this. I know sometimes it is difficult to build our needed resources but what is the point if we construct a object/function with a certain return value ?

Mock can help to write unit tests.
In unit tests, you want to test a small portion of your implementation. For example, as small as one function or one class.
In a moderately large software, these small parts depend on each other. Or sometimes there are external dependencies. You open files, do syscalls, or get external data in some other way.
While writing a directed unit test for a small portion of your code, you do not want to spend time to set-up everything else around it. (The files, syscalls, external data). Mock comes to your help there. With mock, you can make the other dependencies of your code behave exactly as you like. This way you can focus on testing your intended implementation.
Coming to the mock with the return value: Say you want to test func_a. And func_a calls func_b. func_b does a lot of funky processing to calculate its return value. For example, talking to an external service, doing bunch of syscalls, or some other expensive operation. Since you are testing func_a, you only care about possible return values of func_b. (So that func_a can use them) In this scenario you would mock func_b and set the return values explicitly. This can really simplify your test complexity.

Setting up environment for testing in Python

I'm writing integration tests using plain unittest in Python (import unittest) and are creating stubs for some external services. Now I want to run the same tests with a real implementation; but also keep the stubs. That way I can run the tests with and without the stubs and compare behaviour.
I'm running my tests both from SetupTools and through PyCharm. Is there some generic way for me to set/inject/bootstrap a parameter which tells my code wether to use the stub or the real implementation? Command line preferrable. Any pointers appreciated. :)

It sounds like you are looking for a mocking framework. Mocking frameworks allow you to create a 'stub' for the method from within your test. This is good because you don't want to be inserting any test specific code into your actual code.
One of the more popular mocking frameworks for python 2.* is python-mock (in fact it comes with python 3) So you can write the code as:
from mock import MagicMock
test_foo_mocked():
bar = MagicMock()
bar.return_value = 'fake_val'
assertEqual(bar(), 'fake_val')
test_foo_real():
assertEqual(bar(), 'real_val')
Side Note:
I would really recommend that you think of these as completely unrelated tests. There are many benefits to keeping your integration tests separate from your unit tests. Thinking of them as two different ways of running the 'same test' may encourage you to write bad tests. Unit tests should be able to test things that would be difficult or impossible to test through integration tests and vice versa.

Are there any python libraries to help test external python scripts

I'd like to test some python scripts.
Are there any python libraries to help test external system behaviors(running scripts, testing the contents of external files, managing input/output files, and similar actions).
Also I tried making the scripts more api like to allow imports rather then calling it directly for more unit test like tests. Changes include making scripts easier to run interactively(factor lots of stuff into functions/module values and make it less procedural, add parameter to silence stdout, passing optional args to main) also serializing results in addition to the usual output formats(even though the functions to generate the output files have a medium amount of logic in them)).
Is this a good strategy or is it better to attempt to test scripts by running them blackbox style and examining output.

Test library
I'll go ahead and suggest unittest (even though it's the top Google hit for "python unit testing" and you probably already know of it). It's a very nice, easy to use, feature-ful library for unit testing.
Test strategy
Writing testable code is hard. Testing things like side-effects, environments, and file output can take the unit right out of unit test.
What I typically try to do is structure the code so that as little of it as possible does I/O or other nasty things. Then all of that code can usually be straightforwardly unit-tested.
For the parts that are hard to break into units, such as the command-line interface, I test for file output etc.
Conclusion
use unit tests as much as possible
otherwise, use black-box tests
constantly refactor code to make writing unit tests easier & more effective

Is having a unit test that is mostly mock verification a smell?

I have a class that connects three other services, specifically designed to make implementing the other services more modular, but the bulk of my unit test logic is in mock verification. Is there a way to redesign to avoid this?
Python example:
class Input(object): pass
class Output(object): pass
class Finder(object): pass
class Correlator(object): pass
def __init__(self, input, output, finder):
pass
def run():
finder.find(input.GetRows())
output.print(finder)
I then have to mock input, output and finder. Even if I do make another abstraction and return something from Correlator.run(), it will still have to be tested as a mock.

Just ask yourself: what exactly do you need to check in this particular test case? If this check does not rely on other classes not being dummy, then you are OK.
However, a lot of mocks is usually a smell in sense that you are probably trying to test integration without actually doing integration. So if you assume that if the class passes test with mocks, it will be fine working with real classes, than yes, you have to write some more tests.
Personally, I don't write many Unit tests at all. I'm web developer and I prefer functional tests, that test the whole application via HTTP requests, as users would. Your case may be different

There's no reason to only use unit test - Maybe integration tests would be more useful for this case. Initialize all the objects properly, use the main class a bit, and assert on the (possibly complex) results. That way you'll test interfaces, output predictability, and other things which are important further up the stack. I've used this before, and found that something which is difficult to integration test probably has too many attributes / parameters or too complicated/wrongly formatted output.

On a quick glance, this do look like the level of mocking becomes to large. If you're on a dynamic language (I'm assuming yes here since your example is in Python), I'd try to construct either subclasses of the production classes with the most problematic methods overridden and presenting mocked data, so you'd get a mix of production and mocked code. If your code path doesn't allow for instantiating the objects, I'd try monkey patching in replacement methods returning mock data.
Weather or not this is code smell also depends on the quality of mocked data. Dropping into a debugger and copy-pasting known correct data or sniffing it from the network is in my experience the preferred way of ensuring that.
Integration vs unit testing is also an economical question: how painful is it to replace unit tests with integration/functional tests? The larger the scale of your system, the more there is to gain with light-weight mocking, and hence, unit tests.

Python - doctest vs. unittest [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I'm trying to get started with unit testing in Python and I was wondering if someone could explain the advantages and disadvantages of doctest and unittest.
What conditions would you use each for?

Both are valuable. I use both doctest and nose taking the place of unittest. I use doctest for cases where the test is giving an example of usage that is actually useful as documentation. Generally I don't make these tests comprehensive, aiming solely for informative. I'm effectively using doctest in reverse: not to test my code is correct based on my doctest, but to check that my documentation is correct based on the code.
The reason is that I find comprehensive doctests will clutter your documentation far too much, so you will either end up with either unusable docstrings, or incomplete testing.
For actually testing the code, the goal is to thoroughly test every case, rather than illustrate what is does by example, which is a different goal which I think is better met by other frameworks.

I use unittest almost exclusively.
Once in a while, I'll put some stuff in a docstring that's usable by doctest.
95% of the test cases are unittest.
Why? I like keeping docstrings somewhat shorter and more to the point. Sometimes test cases help clarify a docstring. Most of the time, the application's test cases are too long for a docstring.

Another advantage of doctesting is that you get to make sure your code does what your documentation says it does. After a while, software changes can make your documentation and code do different things. :-)

I work as a bioinformatician, and most of the code I write is "one time, one task" scripts, code that will be run only once or twice and that execute a single specific task.
In this situation, writing big unittests may be overkill, and doctests are an useful compromise. They are quicker to write, and since they are usually incorporated in the code, they allow to always keep an eye on how the code should behave, without having to have another file open. That's useful when writing small script.
Also, doctests are useful when you have to pass your script to a researcher that is not expert in programming. Some people find it very difficult to understand how unittests are structured; on the other hand, doctests are simple examples of usage, so people can just copy and paste them to see how to use them.
So, to resume my answer: doctests are useful when you have to write small scripts, and when you have to pass them or show them to researchers that are not computer scientists.

If you're just getting started with the idea of unit testing, I would start with doctest because it is so simple to use. It also naturally provides some level of documentation. And for more comprehensive testing with doctest, you can place tests in an external file so it doesn't clutter up your documentation.
I would suggest unittest if you're coming from a background of having used JUnit or something similar, where you want to be able to write unit tests in generally the same way as you have been elsewhere.

I don't use doctest as a replacement for unittest. Although they overlap a bit, the two modules don't have the same function:
I use unittest as a unit testing framework, meaning it helps me determine quickly the impact of any modification on the rest of the code.
I use doctest as a guarantee that comments (namely docstrings) are still relevant to current version of the code.
The widely documented benefits of test driven development I get from unittest. doctest solves the far more subtle danger of having outdated comments misleading the maintenance of the code.

I use unittest exclusively; I think doctest clutters up the main module too much. This probably has to do with writing thorough tests.

Using both is a valid and rather simple option. The doctest module provides the DoctTestSuite and DocFileSuite methods which create a unittest-compatible testsuite from a module or file, respectively.
So I use both and typically use doctest for simple tests with functions that require little or no setup (simple types for arguments). I actually think a few doctest tests help document the function, rather than detract from it.
But for more complicated cases, and for a more comprehensive set of test cases, I use unittest which provides more control and flexibility.

I almost never use doctests. I want my code to be self documenting, and the docstrings provide the documentation to the user. IMO adding hundreds of lines of tests to a module makes the docstrings far less readable. I also find unit tests easier to modify when needed.

Doctest can some times lead to wrong result. Especially when output contains escape sequences. For example
def convert():
"""
>>> convert()
'\xe0\xa4\x95'
"""
a = '\xe0\xa4\x95'
return a
import doctest
doctest.testmod()
gives
**********************************************************************
File "hindi.py", line 3, in __main__.convert
Failed example:
convert()
Expected:
'क'
Got:
'\xe0\xa4\x95'
**********************************************************************
1 items had failures:
1 of 1 in __main__.convert
***Test Failed*** 1 failures.
Also doesn't check the type of the output. It just compares the output strings. For example it have made some type rational which prints just like integer if it is a whole number. Then suppose you have function which return rational. So, a doctest won't differentiate if the output is rational whole number or a integer number.

I prefer the discovery based systems ("nose" and "py.test", using the former currently).
doctest is nice when the test is also good as a documentation, otherwise they tend to clutter the code too much.