Getting Code Coverage at your Fingertips
Based on a story of a developer’s struggle to deliver under a tight timeline:
- Developer spends a lot of time trying to get code to work as intended
- Developer spends little time writing unit tests, if any
- Developer spends little or no time writing integration and functional tests
- Developer checks in and completes feature
Everyone should write tests as part of the development lifecycle. I admit. I, too, struggle to write tests for my own good.
Who can be 100% accurate and efficient at testing code? No one. That is why we deploy tools and have a set of scans aimed at it to make sure it works fine. But, writing code is, in fact, a form of art — drawn in binary. Everyone takes a different perspective at looking at (and between) the lines of code. Every individual may take a different approach to writing the exact same function. Thus, the tests that every developer writes is from his/her perspective of what the code should do.
For example, I write a function that returns the product of 2 numbers:
function multiple(x,y) {
return (x * y);
}One developer tests for positive number as output, when inputs are positive.
assertThat(
test.multiple(
math.anyPositiveNumber(),math.anyPositiveNumber()
)
).isPositive();Another tests for even number as output, when one of the inputs is even.
assertThat(
test.multiple(
math.anyEvenNumber(),math.anyOddNumber()
)
).isEven();There are a million and one ways to test a function, and the above is just a simple example. But you get my point about looking (testing) it from different perspectives.
This is where the idea of “code coverage” help developers identify where there is a lack of testing in the code. As a quick and dirty guide, I’d like to think of measuring code coverage as (also measuring developer’s ability to test code):
(TESTED SOURCE CODE / SOURCE CODE) => CODE COVERAGE
Mindset and Approach for Writing Tests
Disclaimer: I’m not an expert of writing tests but I do know that writing low quality tests can falsely represent code coverage metrics.
As mentioned earlier, there are numerous testing frameworks and methodologies to aid one’s testing palette. The most important tests, though, are those that are meaningful to its purpose — usually business-driven tests. These comic strips on TicketMaster explain the philosophy of testing much better.
Developers should have the ability to write valid, well documented, scoped test cases for their code. Peer reviews and discussions can help with this process to come up with test cases as suppose to rushing into writing test cases because developers and managers should not be focused on getting “good” code coverage metrics as an end-goal.
For larger applications, there might even be a need to hire full-time testing / QA engineer because there are many tests to pass — unit, functional, integration, security, etc.
Some tools have plugins (like this security plugin) which are well defined, essential tests written by the industry and using these instead of writing custom tests can help save time as well as gain immediate code coverage.
Read this article by SeaLights about the benefits of measuring code coverage.
When to write tests?
The short answer is — now.
As mentioned earlier, tests should be written as part of the development lifecycle so that code coverage. One suggestion is to have writing and performing such tests as part of the Definition of “Done” when using the AGILE or SCRUM methodologies and as early as possible.
The earlier you catch a bug, the cheaper it is to fix. It costs significantly more expensive to fix a bug later in the software development lifecycle. This makes sense especially it will takes days or even weeks to make a change in production code as developers lose context and re-run all the tests for this change.
Code Coverage Metrics — Helpful?
Measuring code coverage enables visibility on whether the code was even tested or not, and gives an indication of the overall coding practice. 0% code coverage indicates no tests were written and is a sign of bad practice. Most organisations are glad to have reach 90% code coverage.
It is debatable to an outsider whether or not code coverage metrics are valuable to them, but to developers and engineers who understand what the tests are doing, metrics can make a whole lot of sense.
Do not be overly reliant on the metrics if one does not know what they mean. Code coverage metrics, along with the understanding of the written tests, can paint a better picture at telling the health status of the code.
Code Coverage Tools
SonarCloud as an Example
SonarCloud provides a free, open-source code scanning platform that helps one gain visibility on a project’s code coverage. Besides code coverage, it also displays code smells and even security vulnerabilities at a glance with intuitive graphs. The tool also measures the technical debt (in terms of minutes) to fix the issues, which is great for monitoring the overall health of a project.
Additional example of code coverage tool is the Atlassian Clover — if you are looking for seamless integration with the Atlassian suite but it currently only supports Java and Groovy.
Code scanning tools that are successful have these characteristics:
- Easy and intuitive to use
- Graphically beautiful and represent data in consumable chunks
- Lightweight and portable; platform agnostic
- Easy integration with other build/CICD tools using plugins
- Great community support and contribution
