Host-based Coverage Analysis

Often times, for various reasons, it seems preferable to collect source coverage analysis data on the host – i.e. on some hardware platform that is not the final hardware platform. As far as this author is concerned, it sounds like a reasonable thing. Here’s what you’ll need to ensure:

1. Make sure your pre-processor settings are the same.

You don’t want to be running source coverage analysis on different sources. Now, there will undoubtedly be code (at the very highest and lowest levels) that will be different – the typical #ifdef WINDOWS and #ifdef TARGET_OS type flags. That stuff may need to be analyzed manually. But otherwise: keep your preproc settings the same.

2. Confirm that the results of the functional tests run for SCA are identical to the results when those tests are run on the target.

To get SCA, you have to run requirements-based functional tests. Those functional tests should run with identical results between the target and the host. If they do, you will have confirmed (by inference) that you have achieved complete coverage on the target, you just haven’t measured it.

If you keep these two goals in mind, host-based SCA should be sufficient to comply with DO-178B objectives and should be much more efficient than doing full measurement and analysis on the target.

Reasonable types of verification for low-level requirements

One question that frequently comes up when discussing verification of safety critical software is exactly how certain requirements will be verified. For instance, you may have a requirement that says

The application shall load the processor’s Control Register with the value 0×12345678 during powerup initialization.

The need for the code in question is obvious. Whether such a requirement should actually be a requirement or design detail is debatable. (In my opinion it should be design detail.)

But if it is a requirement, how do you go about verifying it?

I have had verifiers suggest that the best way is to do white-box testing, to set a breakpoint in the code in a debugger and verify that the line of code is actually hit, and to then examine the contents of the register to verify that the value is present. While this is certainly a valid test, I think it’s overkill.

Assuming that the value you’re writing to the register is important for the software to run correctly, by doing any functional testing you have implicitly verified that the value was written to the register. Additionally, recall that DO-178B requires that each requirement be verified; however, it provides for multiple ways of performing that verification, including code inspection.

In my opinion, if you have a low-level software requirement similar to this one, the combination of a code inspection and an implicit functional test is sufficient verification of the requirement.