Thursday, May 24, 2018

On the value of code coverage

I've read lots of interesting articles and blog posts on whether a high unit test code coverage percentage is a good thing to aim for or not.  After some deliberation, my answer (as always) is "it depends."


In fact, the older I get, the more I am discovering that my first answer to almost any question within my professional / work environment these days is "it depends".

This is not (as some might think) due to the fact that I am unable to make a decision or that I don't actually know what I am talking about, but instead, because my advancing years and countless screw ups throughout those years, has instilled within me, the good habit of asking more questions.  Simple questions are generally missing the level of information that allows one to make a well informed decision.

So, when asked "Is Xamarin Forms a good technology to build this app in?", I respond with "It depends", followed by a series of clarifying questions:

  • How large is the requirements list for this application?
  • Does the app require a complex user interface with many custom controls?
  • What are the skills of the team we are intending to use to build this app?
  • Do we need a really small app with a fast download and startup time?
  • ..and so on.

Xamarin Forms gets better and closer to Native with every release and many argue that there is very little that one cannot do with it if you know the tricks of the trade.

So, back to code coverage...

I currently find myself in a new role with what might be described as a legacy organic codebase.
So, when I read articles stating that very high unit test code coverage is not going to prove that the application is bug free, then of course, I have to agree.  But then, that's not what unit testing is about anyway.

One thing that high unit test coverage DOES (in general) do, is let you know that the codebase is in pretty good shape.  It may not be great, but it is no doubt better than much of the legacy code I have worked on throughout my career.

Now, code quality is all relative, but if you've ever worked on a single class that is 10,000+ lines long, with one method alone over 2000 lines, then you should understand where I am coming from. If a team has managed to get unit test coverage upto to or beyond the 60% mark, then it suggests quite strongly that the code is inherently (mostly) unit testable.

This normally means that:

  • Classes are quite small, with small methods
  • Code is therefore relatively easy to reason about
  • Abstractions are in place to allow mocking
  • Dependency injection is likely in place

It's very hard to get coverage above 50% without all of the above being true, and if the code is indeed in that sort of shape, then it is significantly easier to work with than code at the other end of the spectrum.  Even if the unit tests are brittle or of low quality, and the structure is not as good as you'd hope, you're already in a much better place than behemoth code with few or no unit tests. 

Poor unit tests (against essentially testable code) can be re-written in minutes, whereas knocking un-testable code into such a shape that you can actually start writing unit tests, can take many careful hours or days and be a risky endeavour as you are starting by refactoring without unit tests in place.

Many developers I know balk at the idea of working on or re-shaping legacy code because it is difficult.  I rather enjoy it - there's a puzzle like quality to it, where you've got to work out a sequence of gradual improvements that will slowly unlock things until you get to a point where the task gets progressively easier. 

It's a task where you have to create a new & better design using all the tools & skills in your armoury and migrate the code into it without leaving the whole thing in a broken heap for too long. Some would suggest re-writing, but that is seldom the correct course of action or feasible from a business perspective.

So, if you find yourself questioning the value of your unit test code coverage, just remember that you're already in a pretty good place, and your good coverage is helping to keep you there.








No comments: