Counting Android project Code Coverage with JaCoCo: Part I - What is Code Coverage?

As you may not know, I am a full time developer and in my spare time, I read books and watch romantic comedies a lot.
Just kidding, I do not watch romantic comedies a lot, I almost watch superhero movies.
Recently, there have been Internet problems in Vietnam so that I have not had such a good connection to enjoy the movies. So, I have resumed writing tech blog posts.

What is Code Coverage?

In one of previous entries, I mentioned Test-driven development (TDD), a software development process in which writing test is the first step and the software is developed to pass the newly-added test cases. Your code must pass the test cases first, but if you only test a small proportion of your code, how much confidence can you have in it?

There are some ways those can be used to determine how well your tests exercise your code. The TDD proponents often tend to push Code Coverage as a useful metric for gauging how much your tests cover your source code. Code Coverage measures how many lines/blocks/arcs of your code are executed while the tests run.

Code coverage metrics

A Code Coverage metric is the just a percentage executed coverage criteria. According to this article, among the most common metrics there are:

Statement coverage

A statement is covered if it is executed.

Branch coverage

The goal of branch coverage is to ensure that whenever a program can jump, it jumps to all possible destinations.
100% branch coverage implies 100% statement coverage.

Path coverage

To achieve full path coverage they must all be taken. Note that missing elses count as paths.
100% path coverage implies 100% branch coverage.
Condition coverage
All Boolean expression to be evaluated for true and false.

Code Coverage Standard

I surfed the Internet and I read this interesting article, which states some standards you may want to reach. I will do a copy-paste work with editing:

• 100%. You might choose this because you want to be sure everything is tested. Again, this comes back to degree of confidence: If your coverage is below 100%, you know some subset of your code is untested.
• 99% (or 95%, other numbers in the high nineties.) Appropriate in cases where you want to convey a level of confidence similar to 100%, but leave yourself some margin to not worry about the occasional hard-to-test corner of code.
• 80%. Generally, the intent here is to show that most of your code is tested. This is appropriate for middle-ground cases where "well-tested" is not a high priority (you don't want to waste effort on low-value tests), but is enough of a priority that you'd still like to have some standard in place.

Is Code Coverage perfect?

When working with testing in general and code coverage in particular, there is a famous quote from Dijkstra:

Testing never proves the absence of faults, it only shows their presence.

Code coverage is indicative, but not proof, of a well-tested software. Remember, having "100% code-coverage" means every line of code is tested, it doesn't mean they are tested under every situation. Code coverage is only meaningful in the context of well-written tests. It doesn't save you from crappy tests.

So, improve your test-writing skills also!

It's enough for this post and here come superhero movies