Practical Decisions on Testing: Trading off Value with Difficulty

Recently I had reason to condense an approach to decisions around the value versus the effort of testing - specifically unit testing, but likely it applies at higher levels - and came up with the following diagram that I think illustrates it quite well.

On it we see two axis: one being the value of the tests and the being how difficult it is to write the tests. We can consider here that code that's complex, expresses business logic etc. is something that's going to be high value, but very trivial code down to the level of property getters and setters likely won't add much, if any. Code that's easy to test will have little or no dependencies that may require mocking or stubbing. Sometimes though code can be is particularly difficult to test, often when tied to platform components that prove troublesome or even impossible to treat in this way.

The graph then breaks down into four quadrants:

• Bottom right - easy to test and high value. This is the sweet-spot where we should be looking to focus and get close to 100% code coverage.
• Top right - high value, but the effort in testing is higher. We should aim to do the best we can here, but sometimes we might run into something that just proves to difficult to test to justify the effort.
• Bottom left - easy to test but not much value. We still might look to add tests here for many cases, unless the value is almost zero, given it's easy to do and so doesn't take much effort.
• Top left - difficult to test and not much value. Here's we're we'll justifiably spend the least focus.

Above the level of writing individual tests, there are two directions that we should aim to influence regarding the health of the code-base from a testing perspective - shown on the diagram with the large blue arrows.

Firstly, we improve code coverage by pushing up along on the line of the diagonal. Areas of code that sit just over the border, in the white area above the diagonal line, are where we'll get the most value for our effort.

And secondly, we should look to improve the testability of our code-base, making the parts that are difficult to test, less so. Even when tied to platforms that aren't making testing easy in places, we can still introduce wrapping classes, draw out much of the logic that is platform agnostic into more testable constructs, etc.