The fraction of code that is tested is known as the test coverage. In general, higher coverage is good. For example, white box test code coverage might be the fraction of lines of code executed by tests, with 95% being a pretty good result and 98% to 99% is often the best that people do without heroic effort. (95% coverage means 5% of the code is never executed -- not even once -- during testing. Hard to believe this is a pretty good result. But as I said, executing code that handles rare cases can be a challenge.)
Although lines of code executed is the classic coverage metric, there are other possible coverage metrics that might be useful depending on your situation:
- Testing that code can correctly handle exceptions it might encounter (for example, does it handle malloc failing?)
- Testing that all entries of a lookup table are exercised (what if only one table entry is out to lunch?)
- Testing that all states and arcs of a statechart have been exercised
- Testing that algorithms have been checked for numerical stability in tricky areas where they might have problems
It's important to remember that even if you have 100% test coverage, it doesn't mean you have tested the system completely (usually "complete" testing is impossible -- there are too many possibilities). What good coverage does mean is that you haven't left out anything obvious. And that's good enough to make understanding the coverage of your testing worthwhile. Chapter 23 of my book discusses the concepts and practices of embedded software testing in more detail.
---
No comments:
Post a Comment
Please send me your comments. I read all of them, and I appreciate them. To control spam I manually approve comments before they show up. It might take a while to respond. I appreciate generic "I like this post" comments, but I don't publish non-substantive comments like that.
If you prefer, or want a personal response, you can send e-mail to comments@koopman.us.
If you want a personal response please make sure to include your e-mail reply address. Thanks!