We Keep Coding

Avoid type warnings when mocking objects in unit tests?

Assuming I have a function that takes a complex object and does something with it:
def foo(bar: SomeComplexObject):
...
In unit tests bar will be replaced by a mock object, but that of courses now raises type warnings. Should I simply ignore or suppress these or is there a proper way to deal with them (without changing the original function signature of course)?
Update: I've seen now that this is an open issue on mypy, but its been in that state for over two years. Has any consensus emerged on how to work around this?

I'm going to put my 2¢ in and say that you should type-check your testsuite. Its still code and static type checking will help you write better code faster.
That leaves the question of how. Ideally, if your function expects SomeComplexObject then you also pass in an instance thereof. Either by building one in your test fixtures, or by subclassing and overriding what you don't need. The best unit test is the one that operates on proper input.
That still leaves the case where this is impractical or we explicitly want to test how invalid input is handled. In that case just explicitly cast your mock to the type that mypy requires:
from typing import cast
def test_foo():
mock_bar = cast(SomeComplexObject, MockBar())
foo(mock_bar)

How to override current time?

I am writing an application which makes use of the current time (via datetime.datetime.now())
Is there a way, for debugging purposes, to override this call so that it returns a specific timestamp?
As a fallback I could write a function which would be called instead of datetime.datetime.now() and return whatever is needed (the actual current time in producton and the required test time when debugging) but there may be a more pythonic way to perform these kind of actions?

Broadly, your options are:
Use the unittest.mock library, which can replace the function on the fly with a dummy function that always gives the same results (or use another mocking library that does the same thing). This means you don't have to modify your function; however, reasonable people can disagree on whether monkey patching with mock is good practice, even for debugging. I think this is the most widely used solution to this problem in Python.
Modify your function to do something different depending on its environment (the actual environment variables on your system, or global state, or something else). This is the easiest way, but also the crudest and most fragile way, so you'll have to be sure to change it back after your debugging is finished.
Modify your function to accept a function itself as a parameter, and pass in datetime.datetime.now as that function in normal operation, but pass in something different (for instance a stub) for testing.

You can use the mock library to mock the datetime.datetime.now() usage:
import mock
def my_test():
my_mock = mock.Mock(return_value=your_desired_timestamp)
with mock.patch('mymodule.datetime.datetime.now', my_mock):
# Here, all calls to `datetime.datetime.now` referenced by `datetime.datetime`
# defined in `my_module` will be mocked to return `your_desired_timestamp`.

Good way to isolate tests that depend on an initializer

I've really tried to start isolating my unit tests so I can pinpoint where errors occur rather than having my entire screen turn red with failures when one thing goes wrong. It's been working in all instances except when something in an initializer fails.
Check out these tests:
#setup_directory(test_path)
def test_filename(self):
flexmock(lib.utility.time).should_receive('timestamp_with_random').and_return(1234)
f = SomeFiles(self.test_path)
assert f.path == os.path.join(self.test_path, '1234.db')
#setup_directory(test_path)
def test_filename_with_suffix(self):
flexmock(lib.utility.time).should_receive('timestamp_with_random').and_return(1234)
f = SomeFiles(self.test_path, suffix='.txt')
assert f.path == os.path.join(self.test_path, '1234.txt')
I'm mocking dependent methods so that the thing I'm testing is completely isolated. What you notice is that the class needs to be instantiated for every single test. If an error is introduced in the initializer, every single test fails.
This is the offending constructor that calls the class's initializer:
SomeFiles(*args)
Is there a way to isolate or mock the initializer or object constructor?

I'm not sure what testing packages you're using, but in general, you can usually just mock the __init__() call on the class before actually attempting to use it. Something like
def my_init_mock_fn(*args, **kwargs):
print 'mock_init'
SomeFiles.__init__ = my_init_mock_fn
SomeFiles()
This isn't probably exactly what you want as from this point on SomeFiles.__init__ fn will always be the mock fn, but there are utilities like voidspace mock that provide a patch function that allow you to patch the class just for a specific scope.
from mock import patch
with patch.object(SomeFiles, '__init__', my_init_mock_fn):
SomeFiles()
..other various tests...
SomeFiles() #__init__ is reset to original __init__ fn
I'm sure there's probably similar functionality in whatever mocking package you are using.
Just realized you're using flexmock, there's a page for replace_with here.

What's causing the initialising function to fail? Maybe that's a bug that you should be looking into.
Another thing you can do, instead of mocking the object constructor, is simply mocking its return values. ie: Given this input, I expect this output -- so I'm going to use this expected output whether or not it returned correctly.
You can also stop testing on first failure. (failfast)
You also might want to reconsider how your tests are set up. If you have to recreate two files for every test, maybe ask yourself why. Could your tests be structured that you set up the two files, then run a series of tests, rinse and repeat. This would make it so only the series of tests assigned to that path fail, helping you isolate why it failed at all.

Is it common/good practice to test for type values in Python?

Is it common in Python to keep testing for type values when working in a OOP fashion?
class Foo():
def __init__(self,barObject):
self.bar = setBarObject(barObject)
def setBarObject(barObject);
if (isInstance(barObject,Bar):
self.bar = barObject
else:
# throw exception, log, etc.
class Bar():
pass
Or I can use a more loose approach, like:
class Foo():
def __init__(self,barObject):
self.bar = barObject
class Bar():
pass

Nope, in fact it's overwhelmingly common not to test for type values, as in your second approach. The idea is that a client of your code (i.e. some other programmer who uses your class) should be able to pass any kind of object that has all the appropriate methods or properties. If it doesn't happen to be an instance of some particular class, that's fine; your code never needs to know the difference. This is called duck typing, because of the adage "If it quacks like a duck and flies like a duck, it might as well be a duck" (well, that's not the actual adage but I got the gist of it I think)
One place you'll see this a lot is in the standard library, with any functions that handle file input or output. Instead of requiring an actual file object, they'll take anything that implements the read() or readline() method (depending on the function), or write() for writing. In fact you'll often see this in the documentation, e.g. with tokenize.generate_tokens, which I just happened to be looking at earlier today:
The generate_tokens() generator requires one argument, readline, which must be a callable object which provides the same interface as the readline() method of built-in file objects (see section File Objects). Each call to the function should return one line of input as a string.
This allows you to use a StringIO object (like an in-memory file), or something wackier like a dialog box, in place of a real file.
In your own code, just access whatever properties of an object you need, and if it's the wrong kind of object, one of the properties you need won't be there and it'll throw an exception.

I think that it's good practice to check input for type. It's reasonable to assume that if you asked a user to give one data type they might give you another, so you should code to defend against this.
However, it seems like a waste of time (both writing and running the program) to check the type of input that the program generates independent of input. As in a strongly-typed language, checking type isn't important to defend against programmer error.
So basically, check input but nothing else so that code can run smoothly and users don't have to wonder why they got an exception rather than a result.

If your alternative to the type check is an else containing exception handling, then you should really consider duck typing one tier up, supporting as many objects with the methods you require from the input, and working inside a try.
You can then except (and except as specifically as possible) that.
The final result wouldn't be unlike what you have there, but a lot more versatile and Pythonic.
Everything else that needed to be said about the actual question, whether it's common/good practice or not, I think has been answered excellently by David's.

I agree with some of the above answers, in that I generally never check for type from one function to another.
However, as someone else mentioned, anything accepted from a user should be checked, and for things like this I use regular expressions. The nice thing about using regular expressions to validate user input is that not only can you verify that the data is in the correct format, but you can parse the input into a more convenient form, like a string into a dictionary.

What is your strategy to avoid dynamic typing errors in Python (NoneType has no attribute x)?

I'm not sure if I like Python's dynamic-ness. It often results in me forgetting to check a type, trying to call an attribute and getting the NoneType (or any other) has no attribute x error. A lot of them are pretty harmless but if not handled correctly they can bring down your entire app/process/etc.
Over time I got better predicting where these could pop up and adding explicit type checking, but because I'm only human I miss one occasionally and then some end-user finds it.
So I'm interested in your strategy to avoid these. Do you use type-checking decorators? Maybe special object wrappers?
Please share...

forgetting to check a type
This doesn't make much sense. You so rarely need to "check" a type. You simply run unit tests and if you've provided the wrong type object, things fail. You never need to "check" much, in my experience.
trying to call an attribute and
getting the NoneType (or any other)
has no attribute x error.
Unexpected None is a plain-old bug. 80% of the time, I omitted the return. Unit tests always reveal these.
Of those that remain, 80% of the time, they're plain old bugs due to an "early exit" which returns None because someone wrote an incomplete return statement. These if foo: return structures are easy to detect with unit tests. In some cases, they should have been if foo: return somethingMeaningful, and in still other cases, they should have been if foo: raise Exception("Foo").
The rest are dumb mistakes misreading the API's. Generally, mutator functions don't return anything. Sometimes I forget. Unit tests find these quickly, since basically, nothing works right.
That covers the "unexpected None" cases pretty solidly. Easy to unit test for. Most of the mistakes involve fairly trivial-to-write tests for some pretty obvious species of mistakes: wrong return; failure to raise an exception.
Other "has no attribute X" errors are really wild mistakes where a totally wrong type was used. That's either really wrong assignment statements or really wrong function (or method) calls. They always fail elaborately during unit testing, requiring very little effort to fix.
A lot of them are pretty harmless but if not handled correctly they can bring down your entire app/process/etc.
Um... Harmless? If it's a bug, I pray that it brings down my entire app as quickly as possible so I can find it. A bug that doesn't crash my app is the most horrible situation imaginable. "Harmless" isn't a word I'd use for a bug that fails to crash my app.

If you write good unit tests for all of your code, you should find the errors very quickly when testing code.

You can also use decorators to enforce the type of attributes.
>>> #accepts(int, int, int)
... #returns(float)
... def average(x, y, z):
... return (x + y + z) / 2
...
>>> average(5.5, 10, 15.0)
TypeWarning: 'average' method accepts (int, int, int), but was given
(float, int, float)
15.25
>>> average(5, 10, 15)
TypeWarning: 'average' method returns (float), but result is (int)
15
I'm not really a fan of them, but I can see their usefulness.

One tool to try to help you keep your pieces fitting together well is interfaces. zope.interface is the most notable package in the Python world for using interfaces. Check out http://wiki.zope.org/zope3/WhatAreInterfaces and http://glyph.twistedmatrix.com/2009/02/explaining-why-interfaces-are-great.html to start to get an idea how interfaces and z.i in particular work. Interfaces can prove very useful in a large Python codebases.
Interfaces are no substitute for testing. Reasonably comprehensive testing is especially important in highly dynamic languages like Python where there are types of bugs that could not exist in a statically types language. Tests will also help you catch the sorts of bugs that are not unique to dynamic languages. Fortunately, developing in Python means that testing is easy (due to the flexibility) and you have plenty of time to write them that you saved because you're using Python.

One advantage of TDD is that you end up writing code that is easier to write tests for.
Writing code first and then the tests can result in code that superficially works the same, but is much harder to write 100% coverage tests for.
Each case is likely to be different
It might make sense to have a decorator to check whether a particular parameter is None (or some other unexpected value) if you use it in a bunch of places.
Maybe it is appropriate to use the Null pattern - if the code is blowing up because you are setting the initial value to None, you could instead set the initial value to a null version of the object.
More and more wrappers can add up to quite a performance hit though, so it's always better to write code from the start that avoids the corner cases

forgetting to check a type
With duck typing, it shouldn't be necessary to check a type. But that's theory, in reality you will often want to validate input parameters (e.g. checking a UUID with a regex). For that purpose, I created myself some handy decorators for simple type and return type checking which are called like this:
#decorators.params(0, int, 2, str) # first parameter must be integer / third a string
#decorators.returnsOrNone(int, long) # must return an int/long value or None
def doSomething(integerParam, noMatterWhatParam, stringParam):
...
For everything else I mostly use assertions. Of course one often forgets to check a parameter, so it's necessary to test and to test often.
trying to call an attribute
Happens to me very seldom. Actually I often use methods instead of direct access to attributes (the "good" old getter/setter approach sometimes).
because I'm only human I miss one occasionally and then some end-user finds it
"Software is always completed at the customers'." - An anti-pattern which you should solve with unit tests that handle all possible cases in a function. Easier said than done, but it helps...
As for other common Python mistakes (mistyped names, wrong imports, ...), I'm using Eclipse with PyDev for projects (not for small scripts). PyDev warns you about most of the simple kinds of mistakes.

I haven’t done a lot of Python programming, but I’ve done no programming at all in staticly typed languages, so I don’t tend to think about things in terms of variable types. That might explain why I haven’t come across this problem much. (Although the small amount of Python programming I’ve done might explain that too.)
I do enjoy Python 3’s revised handling of strings (i.e. all strings are unicode, everything else is just a stream of bytes), because in Python 2 you might not notice TypeErrors until dealing with unusual real world string values.

You can hint your IDE via function doc, for example: http://www.pydev.org/manual_adv_type_hints.html, in JavaScript the jsDoc helps in a similar way.
But at some point you will face errors that a typed language would avoid immediately without unit tests (via the IDE compilation and the types/inference).
Of course this does not remove the benefit of unit tests, static analysis and assertions. For larger project I tend to use statically typed languages because they have very good IDE support (excellent autocompletion, heavy refactoring...). You can still use scripting or a DSL for some sub part of the project.

Something you can use to simplify your code is using the Null Object Design Pattern (to which I was introduced in Python Cookbook).
Roughly, the goal with Null objects is to provide an 'intelligent'
replacement for the often used primitive data type None in Python or
Null (or Null pointers) in other languages. These are used for many
purposes including the important case where one member of some group
of otherwise similar elements is special for whatever reason. Most
often this results in conditional statements to distinguish between
ordinary elements and the primitive Null value.
This object just eats the lack of attribute error, and you can avoid checking for their existence.
It's nothing more than
class Null(object):
def __init__(self, *args, **kwargs):
"Ignore parameters."
return None
def __call__(self, *args, **kwargs):
"Ignore method calls."
return self
def __getattr__(self, mname):
"Ignore attribute requests."
return self
def __setattr__(self, name, value):
"Ignore attribute setting."
return self
def __delattr__(self, name):
"Ignore deleting attributes."
return self
def __repr__(self):
"Return a string representation."
return "<Null>"
def __str__(self):
"Convert to a string and return it."
return "Null"
With this, if you do Null("any", "params", "you", "want").attribute_that_doesnt_exists() it won't explode, but just silently become the equivalent of pass.
Normally you'd do something like
if obj.attr:
obj.attr()
With this, you just do:
obj.attr()
and forget about it. Beware that extensive use of the Null object can potentially hide bugs in your code.

I tend to use
if x is None:
raise ValueError('x cannot be None')
But this will only work with the actual None value.
A more general approach is to test for the necessary attributes before you try to use them. For example:
def write_data(f):
# Here we expect f is a file-like object. But what if it's not?
if not hasattr(f, 'write'):
raise ValueError('write_data requires a file-like object')
# Now we can do stuff with f that assumes it is a file-like object
The point of this code is that instead of getting an error message like "NoneType has no attribute write", you get "write_data requires a file-like object". The actual bug isn't in write_data(), and isn't really a problem with NoneType at all. The actual bug is in the code that calls write_data(). The key is to communicate that information as directly as possible.