Friday, September 22, 2017

Monday, August 28, 2017

The Spaghetti Factor -- A Software Complexity Metric Proposal


I've had to review code that has spaghetti-level complexity in control flow (too high cyclomatic complexity).  And I've had to review code that has spaghetti-level complexity its data flow (too many global variables mixed together into a single computation).  And I've had to review procedures that just go on for page after page with no end in sight. But the stuff that will really make your brain hurt is code that has all of these problems.

There are many complexity metrics out there. But I haven't seen a one that directly tries to help balance three key points of complexity into a single intuitive number: code complexity, data complexity, and module size. So here is a proposal that could help drive improvement in a lot of the terrible embedded control code I've seen:



The Spaghetti Factor metric (SF):

SF = SCC + (Globals*5) + (SLOC/20)

SCC = Strict Cyclomatic Complexity
Globals = # of read/write global variables
SLOC = # source lines of non-comment code (e.g., C statements)

Scoring:
5-10 - This is the sweet spot for most code except simple helper functions
15 - Don't go above this for most modules
20 - Look closely; review to see if refactoring makes sense
30 - Refactor the design
50 - Untestable; throw the module away and fix the design
75 - Unmaintainable; throw the module away; throw the design away; start over
100 - Nightmare; probably you need to throw the whole subsystem away and re-architect it



Notation:

SCC is Strict Cyclomatic Complexity (sometimes called CC2).  This is a variant of McCabe Cyclomatic complexity (MCC). In general terms, MCC is based on the number of branches in the program. SCC additionally considers complexity based on the number of conditions within each branch. SCC is an approximation of how many test cases it takes to exercise all the paths through code including all the different ways there are to trigger each branch. In other words, it is an estimate of how much work it is to do unit testing. Think of it as an approximation to the effort required for MC/DC testing. But in practice it is also a measure of how hard it is to understand the code.  The idea is to keep SCC low enough that it is feasible to understand and test paths through the code.

Globals is the number of read/write global variables accessed by the module. This does not include "const" values, nor file static variables.  In an ideal world you have zero or near-zero global variables. If you have inherent global state, you should encapsulated that in a state object with appropriate access functions to enforce well-disciplined writes.  Referencing an unstructured pile of dozens or hundreds of global variables can make software difficult to test, and can make subsystem testing almost impossible. Partly that is due to the test scaffolding required, but partly that is simply due to the effort of chasing down all the globals and trying to figure out what they do both inbound and outbound. Moreover, too many globals can make it nearly impossible to chase down bugs or understand the effects of changing one part of the code on the rest of the code. An important goal of this part of the metric is to discourage use of many disjoint global variables to implicitly pass data around from routine to routine instead of passing parameters along with function calls.

SLOC is the number of non-comment "Source Lines of Code."  For C programs, this is the number of programming statements. Typical guidelines are a maximum 100-225 maximum lines of code for a module, with most modules being smaller than that.

As an example calculation, if you have 100 lines of code with an SCC of 9 and 1 global reference, your score will be  SF = 9 + (1*5) + (100/20) = 19.  A score of 19 is on the upper edge of being OK. If you have a distribution of complexity across modules, you'd want most of them to be a bit lower in complexity than this example calculation.

Discussion:

The guideline values are taken primarily from MCC, which typically has a guideline of 10 for most modules, 15 as a usual bound, and 30 as limit.  To account for globals and length, based on my experience, I've changed that to 15 for most modules, 20 as a soft limit and 30 as a hard limit.  You might wish to adjust the threshold and multipliers based on your system and experience. In particular it is easy to make a case that these limits aren't strict enough for life-critical software, and a case can be made for being a little more relaxed in throw-away GUI management code.  But I think this is a good starting point for most every-day embedded software that is written by a human (as opposed to auto-generated code).

The biggest exception is usually what to do about switch statements.  If you exempt them you can end up with multiple switches in one module, or multiple switch/if/switch/if layered nesting.  (Neither is a pretty sight.) I think it is justifiable to exempt modules that have ONLY a switch and conditional logic to do sanity checking on the switch value.  But, because 30 is a pretty generous limit, you're only going to see this rarely. Generally the only legitimate reason to have a switch bigger than that is for something like processing a message type for a communication protocol.  So I think you should not blanket exempt switch statements, but rather include them in an overall case-by-case sign-off by engineering management as to which few exceptions are justifiable.

Some might make the observation that this metric discourages extensive error checking.  That's a different topic, and certainly the intent is NOT to discourage error checking. But the simple answer is that error checking has to be tested and understood, so you can't simply ignore that part of the complexity. One way to handle that situation is to put error checking into a subroutine or wrapper function to get that complexity out of the way, then have that wrapper call the actual function that does the work.  Another way is to break your overall code down into smaller pieces so that each piece is simple enough for you to understand and test both the functionality and the error checking.

Finally, any metric can be gamed, and that is surely true of simple metrics like this one.  A good metric score doesn't necessarily mean your code is fantastic. Additionally, this metric does not consider everything that's important, such as the total number of globals across your code base. On the other hand, if you score poorly on this metric, most likely your code is in need of improvement.

What I recommend is that you use this metric as a way to identify code that is needlessly complex.  It is the rare piece of code indeed that unavoidably needs to have a high score on this complexity metric. And if all your code has a good score, that means it should be that much easier to do peer review and unit testing to ensure that other aspects of the code are in good shape.

References:

A NIST paper on applying metrics is here: http://www.mccabe.com/pdf/mccabe-nist235r.pdf including an interesting discussion of the pitfalls of handling switch statements within a complexity framework.

Monday, July 24, 2017

Don't use macros for MIN and MAX



It is common to see small helper functions implemented as macros, especially in older C code. Everyone seems to do it.  But you should avoid macros, and instead use inline functions.

The motivation for using macros was originally that you needed to use a small function in many places but were worried about the overhead of doing a subroutine call. So instead, you used a macro, which expands into source code in the preprocessor phase.  That was a reasonable tradeoff 40 years ago. Not such a great idea now, because macros cause problems for no good reason.

For example, you might look on the Web and find these common macros
    #define MAX(a,b) ((a) > (b) ? a : b)
    #define MIN(a,b) ((a) < (b) ? a : b)

And you might find that it seems to work for a while.  You might get bitten by the missing "()" guards around the second copy of a and b in the above -- which version you get depends on which cut & paste code site you visit. 

But then you'll find that there are still weird situations where you get unexpected behavior. For example, what does this do?
    c = MAX(a++, b);
If a is greater than b executing the code will increment a twice, but if a is less than or equal to b it will only increment a once.  And if you start mixing types or putting complicated expressions into the macro things can get weird and buggy in a hurry.

Another related problem is that the macro will expand, increasing the cyclomatic complexity of your code. That's because a macro is equivalent to you having put the conditional branch into the source code. (Remember, macro expansion is done by the preprocessor, the so compiler itself acts as if you'd typed the conditional assignment expression every place you use the macro.) This complexity rating is justified, because there is no actual procedure that can be unit tested independently.

As it turns out, macros are evil. See the C++ FAQ: https://isocpp.org/wiki/faq/misc-technical-issues#macros-with-if  which lists 4 different types of evil behavior.  There are fancy hacks to try to get any particular macros such as MIN and MAX to be better behaved, but no matter how hard you try you're really just making a deal with the devil. 

What's the fix?

The fix is: don't use macros. Instead use inline procedure calls.

You should already have access to built-in functions for floating point such as fmin() and fmax().  If it's there, use the stuff from your compiler vendor instead of writing it yourself!

If your compiler doesn't have integer min and max, or you are worried about breaking existing macro code, convert the macros into inline functions with minimal changes to your code base:

inline int32_t MAX(int32_t a, int32_t b) { return((a) > (b) ? a : b); }
inline int32_t MIN(int32_t a, int32_t b) { return((a) < (b) ? a : b); }

If you have other types to deal with you might need different variants depending on the types, but often a piece of code uses predominantly one data type for its calculations, so in practice this is usually not a big deal. And don't forget, if your build environment has a built in min or max you can just set up the macro to call that directly.

What about performance?

The motivation for using macros back in the bad old days was efficiency. A subroutine call involved a lot of overhead. But the inline keyword tells the compiler to expand the code in-place while retaining all the advantages of a subroutine call.  Compilers are pretty good at optimization these days. So there is no overhead at run-time.  I've also seen advice to put the inline function in a header file so it will be visible to any procedure needing it, and the macro was already there anyway.

Strictly speaking, "inline" is a suggestion to the compiler. However, if you have a decent compiler it will follow the suggestion unless the inline function is so big the call overhead just doesn't matter. Some compilers have a warning flag that will let you know when the inline didn't happen.  For example, use -Winline for gcc.  If your compiler ignores "inline" for something as straightforward as MIN or MAX, get a different compiler.

What about multiple types?

A perceived advantage of the macro approach is that you can play fast and loose with types.  But playing fast and loose with types is a BAD IDEA because you'll get bugs.  

If you really hate having to match the function name to the data types then what you need is to switch to a language that can handle this by automatically picking the right function based on the operator types. In other words, switch from a to a language that is 45 years old (C) to one that is only about 35 years old (C++).  There's a paper from 1995 that explains this in the context of min and max implemented with templates:  http://www.aristeia.com/Papers/C++ReportColumns/jan95.pdf
As it turns out the rabbit hole goes a lot deeper than you might think for a generic solution.

But you don't have to go down the rabbit hole.  For most code the best answer is simply to use inline functions and pick the function name that matches your data types. You shouldn't lose any performance at all, and you'll be likely to save a lot of time chasing obscure bugs.

Monday, May 22, 2017

#define vs. const

Is your code full of "#define" statements?  If so, you should consider switching to the const keyword.

Old school C:
    #define MYVAL 7

Better approach:
   const uint32_t myVal = 7;

Here are some reasons you should use const instead of #define:
  • #define has global scope, so you're creating (read-only) global values every time you use #define. Global scope is evil, so don't do that.  (Read-only global scope for constant values is a bit less evil than global variables per se, especially if you can't use the namespace features of C++. But gratuitous global scope is always a bad idea.) A const alternative can obey scoping rules, including being purely local if defined inside a procedure, or more commonly file static with the "static" keyword.
  • Const lets you do more aggressive type checking (depending upon your compiler and static analysis tools, especially if you use a typedef more specific than built-in C data types). While C is a bit weak as a language in this area compared to other languages, a classical example is a const lets you identify a number as being in feet or meters, while the #define approach is just as if you'd typed the number 7 in with no units. The #define approach can bite you if you use the wrong value in the wrong place. Type checking is an effective way to find bugs, and using #define gives up an opportunity to let static analysis tools help you with that.
  • Const lets you use the value as if it were a variable when you need to (e.g., passing an address to the variable) without having to change how the variable is defined.
  • #define in general is so bug-prone that you should minimize its use just to avoid having to spend time asking "is this one OK?" in a peer review. Most #define uses tend to be const variables in old-school code, so getting rid of them can dramatically reduce the peer review burden of sifting through hundreds of #define statements to look for problems.
Here are some common myths about this tradeoff. (Note that on some systems these statements might be true, especially if you have and old and lame compiler.  But they don't necessarily have to be true and they often are false, especially on newer chips with newer compilers.)
  • "Const wastes memory."  False if you have a compiler that is smart enough to do the right thing. Sure, if you want to pass a pointer to the const it will actually have to live in memory somewhere, but you can't even pass a pointer to a #define at all. One of the points of "const" is to give the compiler a hint that lets it optimize memory footprint.
  • "Const won't work for X." Generally false if you have a newer compiler, and especially if you are using a mostly-C subset of the capability of a C++ compiler, as is increasingly common. And honestly, most of the time #define is just being used as a plain old integer const to get rid of magic numbers. const will work fine.  (If you have magic numbers instead of #define, then you have bigger problems than this even.) Use const for the no-brainer cases. Something is probably wrong if everything about your code is so special you need #define everywhere.
  • "Const hassles me about type conversions."  That's a feature to prevent you from being sloppy!  So strictly speaking the compiler doing this is not a myth. The myth is that this is a bad thing.
There are plenty of discussions on this topic.  You'll also see that some folks advocate using enums for some situations, which we'll get to another time. For now, if you change as many #defines as you can to consts then that is likely to improve your code quality, and perhaps flush out a few bugs you didn't realize you had.

Be careful when reading discussion group postings on this topic.  There is a lot of dis-information out there about performance and other potential tradeoff factors, usually based on statements about 20 year old versions of the C language or experiences with compilers that have poor optimization capability.  In general, you should always use const by default unless your particular compiler/system/usage presents a compelling case not to.

See also the Barr Group C coding standard rule 1.8.b which says to use const, and has a number of other very useful rules.


Monday, May 8, 2017

Optimize for V&V, not for writing code



Writing code should be made more difficult so that Verification &Validation can be made easier.

I first heard this notion years ago at a workshop in which several folks from industry who build high assurance software (think flight controls) stood up and said that V&V is what matters. You might expect that from flight control folks, but their reasoning applies to pretty much every embedded project. That's because it is a matter of economics. 

Multiple speakers at that workshop said that aviation software can require 4 or 5 hours of V&V for every 1 hour of creating software. It makes no economic sense to make life easy for the 1 hour side of the ratio at the expense of making life painful for the 5 hour side of the ratio.

Good, but non-life-critical, embedded software requires about 2 hours of V&V for every 1 hour of code creation. So the economic argument still holds, with a still-compelling multiplier of 2:1.  I don't care if you're Vee,  Agile, hybrid model or whatever. You're spending time on V&V, including at least some activities such as peer review, unit test, created automated tests, performing testing, chasing down bugs, and so on. For embedded products that aren't flaky, probably you spend more time on V&V than you do on creating the code. If you're doing TDD you're taking an approach that has the idea of starting with a testing viewpoint built in already, by starting from testing and working outward from there. But that's not the only way to benefit from this observation.

The good news is that making code writing "difficult" does not involve gratuitous pain. Rather, it involves being smart and a bit disciplined so that the code you produce is easier for others to perform V&V on. A bit of up front thought and organization can save a lot on downstream effort. Some examples include:
  • Writing concise but helpful code comments so that reviewers can understand what you meant.
  • Writing code to be obvious rather than clever, again to help reviewers.
  • Follow a style guide to make your code consistent, and thus easier to understand.
  • Writing code that compiles clean for static analysis, avoiding time wasted finding defects in test that a tool could have found, and avoiding a person having to puzzle out which warnings matter, and which don't.
  • Spending some time to make your unit interfaces easier to test, even if it requires a bit more work designing and coding the unit.
  • Spending time making it easy to trace between your design and the code. For example, if you have a statechart, make sure the statechart uses names that map directly to enum names rather than using arbitrary state variables such as "magic number" integers between 1 and 7. This makes it easier to ensure that the code and design match. (For that matter, just using statecharts to provide a guide to what the code does also helps.)
  • Spending time up front documenting module interaction so that integration testers don't have to puzzle out how things are supposed to work together. Sequence diagrams can help a lot.
  • Making the requirements both testable and easy to trace. Make every requirement idea a stand-alone sentence or paragraph and give it a number so it's easy to trace to a specific test primarily designed to test that particular requirement. Avoid having requirements in huge paragraphs of free-form text that mix lots of different concepts together.
Sure, these sound like a good idea, but many developers skip or skimp on them because they don't think they can afford the time. They don't have time to make their code clean because they're too busy writing bugs to meet a deadline. Then they, and everyone else, pay for this during the test cycle. (I'm not saying the programmers are necessarily the main culprits here, especially if they didn't get a vote on their deadline. But that doesn't change the outcome.)

I'm here to say you can't afford not to follow these basic code quality practices. That's because every hour you're saving by cutting corners up front is probably costing you double (or more) downstream by making V&V more painful than it should be. It's always hard to invest in downstream benefits when the pressure is on, but doing so is costing you dearly when you skimp on code quality.

Do you have any tricks to make code easier to understand that I missed?