How would one test the os module methods provided in python. For example how would you test the use of os.mkdir?
def create_folder(self):
os.mkdir("/parentFolder/newFolder")
What can be used to test this method?
This method would have test cases such as
Verifying the folder was created
Insufficient permissions to create folder
Insufficient space to create folder
When unit-testing create_folder, you don't test os.mkdir. This is for two reasons:
It is part of an external library (in this case the standard library, but the same would be the case for third-party libraries), so should be covered by the test suites for that library; and
Even if it was part of your codebase, it's a different unit to the one under test.
Additionally, it's worth noting that your testing of this, as demonstrated by user2393256's answer, would likely be using other functionality from the same external library - if the test fails, do you conclude that os.mkdir didn't work or that os.path.isdir didn't work?
From the perspective of testing create_folder, what really matters is that it interacts with that function correctly. I would mock os.mkdir out (using e.g. unittest.mock) and check that it is being called with the appropriate path. You can also change the return value and side effects of the mock, allowing you to simulate things like insufficient permissions or space, and test your app's response to that, without having to somehow set up that environment for real. When testing other units of functionality that call create_folder I would then mock out create_folder entirely, as it's a tested and trusted unit.
Beyond the unit testing, you would have a level of integration testing, which makes sure that all parts of your application work together correctly. At this point you would test overall functionality, e.g. that you can save a file then later load it back in, rather than specifics like "is the folder created?"
Finally, and specifically for standard library functionality, you have to have a certain amount of trust that the language itself is tested (even if not directly, at least by the thousands of programs out there using this already!) and working.
To check if the directory was created you can use
os.path.isdir()
As for the permission: there is a python idiom which says
It's easier to ask for forgiveness than for permission
In that case i would try to create the folder and catch the exception that could be thrown.
Related
I have a custom framework which runs different code for different clients. I have monkeypatched certain methods in order to customize functionality for a client.
Here is the pattern simplified:
#import monkeypatches here
if self.config['client'] == 'cool_dudes':
from app.monkeypatches import Stuff
if self.config['client'] == 'cool_dudettes':
from app.monkeypatches import OtherStuff
Here is an example patch:
from app.framework.stuff import Stuff
def function_override(self):
return pass
Stuff.function = function_override
This works fine when the program executes as it is executed in a batch manner, spinning up from scratch every time. However, when running across unit tests, I find that the monkey patches persist across tests, causing unexpected behavior.
I realize that it would be far better to use an object oriented inheritance approach to these overrides, but I inherited this codebase and am not currently empowered to rearchitect it to that degree.
Barring properly re-architecting the program, how can I prevent these monkey patches from persisting across unit tests?
The modules, including app.framework.<whatever>, are not reloaded for every test. So, any changes in them you make persist. The same happens if your module is stateful (that's one of the reasons why global state is not such a good idea, you should rather keep state in objects).
Your options are to:
undo the monkey-patches when needed, or
change them into something more generic that would change (semi-)automatically depending on the test running, or
(preferred) Do not reinvent the wheel and use an existing, manageable, time-proven solution for your task (or at least, base your work on one if it doesn't meet your requirements completely). E.g. if you use them for mocking, see How can one mock/stub python module like urllib . Among the suggestions there is #mock.patch that does the patching for a specific test and undoes it upon its completion.
Anyone coming here looking for information about monkeypatching, might want to have a look at pytest's monkeypatch fixture. It avoids the problem of the OP by automatically undoing all modifications after the test function has finished.
I practice TDD but I have not used mocking before.
Suppose I want to build a function that should create a folder, but only if that folder does not already exist. As part of my TDD cycle I first want to create a test to see that my function won’t delete an already existing folder.
As my function will probably use os.rm, I gather I could use mocking to see whether os.rm has been called or not. But this isn’t very satisfactory as there are many ways to delete folders. What if I change my function later on to use shutil.rmtree? os.rm would not have been called, but perhaps the function now incorrectly removes the folder.
Is it possible to use mocking in a way which is insensitive to the method? (without actually creating files on my machine and seeing whether they are deleted or not - what I have been doing until now)
I can think of 2 options:
You wrap the deletion in a function, e.g., delete_folder; in the tests, you mock the function and check whether it has been called.
You use pyfakefs to mock the entire filesystem. I haven't used it yet, but it seems to be very powerful.
The problem of "mockism" is that tests bind your code to a particular implementation. Once you have decided to test for a particular function call you have to call (or not as in your example) that function in your production code.
As you have already noticed, there is plenty of ways to remove the directory (even by running rm -rf as external process).
I think the way you are doing it already is the best - you check for an actual side-effect you are interested, no matter how it has been generated.
If you are wondering about performance, you may try to make that test optional, and run it less frequently than the rest of your test suite.
I'm writing integration tests using plain unittest in Python (import unittest) and are creating stubs for some external services. Now I want to run the same tests with a real implementation; but also keep the stubs. That way I can run the tests with and without the stubs and compare behaviour.
I'm running my tests both from SetupTools and through PyCharm. Is there some generic way for me to set/inject/bootstrap a parameter which tells my code wether to use the stub or the real implementation? Command line preferrable. Any pointers appreciated. :)
It sounds like you are looking for a mocking framework. Mocking frameworks allow you to create a 'stub' for the method from within your test. This is good because you don't want to be inserting any test specific code into your actual code.
One of the more popular mocking frameworks for python 2.* is python-mock (in fact it comes with python 3) So you can write the code as:
from mock import MagicMock
test_foo_mocked():
bar = MagicMock()
bar.return_value = 'fake_val'
assertEqual(bar(), 'fake_val')
test_foo_real():
assertEqual(bar(), 'real_val')
Side Note:
I would really recommend that you think of these as completely unrelated tests. There are many benefits to keeping your integration tests separate from your unit tests. Thinking of them as two different ways of running the 'same test' may encourage you to write bad tests. Unit tests should be able to test things that would be difficult or impossible to test through integration tests and vice versa.
I'd like to test some python scripts.
Are there any python libraries to help test external system behaviors(running scripts, testing the contents of external files, managing input/output files, and similar actions).
Also I tried making the scripts more api like to allow imports rather then calling it directly for more unit test like tests. Changes include making scripts easier to run interactively(factor lots of stuff into functions/module values and make it less procedural, add parameter to silence stdout, passing optional args to main) also serializing results in addition to the usual output formats(even though the functions to generate the output files have a medium amount of logic in them)).
Is this a good strategy or is it better to attempt to test scripts by running them blackbox style and examining output.
Test library
I'll go ahead and suggest unittest (even though it's the top Google hit for "python unit testing" and you probably already know of it). It's a very nice, easy to use, feature-ful library for unit testing.
Test strategy
Writing testable code is hard. Testing things like side-effects, environments, and file output can take the unit right out of unit test.
What I typically try to do is structure the code so that as little of it as possible does I/O or other nasty things. Then all of that code can usually be straightforwardly unit-tested.
For the parts that are hard to break into units, such as the command-line interface, I test for file output etc.
Conclusion
use unit tests as much as possible
otherwise, use black-box tests
constantly refactor code to make writing unit tests easier & more effective
I want to create a ramdisk in Python. I want to be able to do this in a cross-platform way, so it'll work on Windows XP-to-7, Mac, and Linux. I want to be able to read/write to the ramdisk like it's a normal drive, preferably with a drive letter/path.
The reason I want this is to write tests for a script that creates a directory with a certain structure. I want to create the directory completely in the ramdisk so I'll be sure it would be completely deleted after the tests are over. I considered using Python's tempfile, but if the test will be stopped in the middle the directory might not be deleted. I want to be completely sure it's deleted even if someone pulls the plug on the computer in the middle of a test.
How about PyFilesystem?
https://docs.pyfilesystem.org/en/latest/reference/memoryfs.html
https://docs.pyfilesystem.org/en/latest/reference/tempfs.html
The downside is that you have to access the filesystem with PyFilesystem API, but you can also access the real fs with PyFilesystem.
Because file and directory-handling is so low-level and OS dependent, I doubt anything like what you want exists (or is even possible). Your best bet might be to implement a "virtual" file-system-like set of functions, classes, and methods that keep track of the files and directory-hierarchy created and their content.
Callables in such an emulation would need to have the same signature and return the same value(s) as their counterparts in the various Python standard built-ins and modules your application uses.
I suspect this might not be as much work as it sounds -- emulating the standard Python file-system interface -- depending on how much of it you're actually using since you wouldn't necessarily have to imitate all of it. Also, if written in Pure Python™, it would also be portable and easy to maintain and enhance.
One option might be to inject (monkey patch) modified versions of the methods used in the os module as well as the builtins open and file that write to StringIO files instead of to disk. Obviously this substitution should only occur for the module being tested;
Please read this:
http://docs.python.org/library/tempfile.html#tempfile.TemporaryFile
"Return a file-like object that can be
used as a temporary storage area. The
file is created using mkstemp(). It
will be destroyed as soon as it is
closed (including an implicit close
when the object is garbage
collected)."
It's all handled for you. Do nothing and it already works.