Planning Documents and Level D Software

There’s an interesting little conundrum with Table A-1 of DO-178B when it comes to Level D software. The first objective in the table requires that software development process activities are defined. However, given that A-1 #6 isn’t applicable to Level D, there is no requirement that the plans actually comply to DO-178B.

In the case of other Level D objectives, this can mean that the artifact doesn’t need to be created. For instance, none of the table A-4 objectives apply to Level D, save the one about partitioning integrity. This is usually interpreted to mean that low-level software requirements aren’t required for Level D software.

With planning documents, though, the bar is a little bit higher, since Table A-9 requires that independent assurance is obtained that the development and processes complied with the approved plans and standards.

Let’s just summarize it this way: For Level D software, you have to have a plan. And you have to follow the plan. You don’t have to show, however, that your plan complies with DO-178B. Non-intuitive, maybe – but simple enough.

How close to formal release do you fix problem reports?

This is a familiar question that comes up at the end of software release cycles. Software development has been completed. Verification is concluding. The last few tests are being run and a problem shows up. What do you do now?

If it’s a serious functional problem, the answer is obvious: you fix it (and hope that it’s not so serious that it affects your delivery date). But what if it’s a minor failure? Still an error, to be sure, but one that is unlikely to show up in any normal use case?

There is a point at which it becomes counter-productive to continue making changes to the software. If the fix for the bug is isolated and easily identified, sure, fix it. But if there is some uncertainty about the fix, or if there is concern that the fix will impact other functional areas in the software, then it is reasonable to leave it alone. The risk of causing other problems in the software with hasty last-minute fixes becomes greater than the risk of leaving the known problem in place.

Be sure to document the problem with a problem report so that it can be fixed for the next release. And be sure to document the problem report, along with the rationale for why it’s safe to leave in the build, and put that documentation in the Software Accomplishment Summary.

Sometimes it’s just safer to leave the bug there.

Host-based Coverage Analysis

Often times, for various reasons, it seems preferable to collect source coverage analysis data on the host – i.e. on some hardware platform that is not the final hardware platform. As far as this author is concerned, it sounds like a reasonable thing. Here’s what you’ll need to ensure:

1. Make sure your pre-processor settings are the same.

You don’t want to be running source coverage analysis on different sources. Now, there will undoubtedly be code (at the very highest and lowest levels) that will be different – the typical #ifdef WINDOWS and #ifdef TARGET_OS type flags. That stuff may need to be analyzed manually. But otherwise: keep your preproc settings the same.

2. Confirm that the results of the functional tests run for SCA are identical to the results when those tests are run on the target.

To get SCA, you have to run requirements-based functional tests. Those functional tests should run with identical results between the target and the host. If they do, you will have confirmed (by inference) that you have achieved complete coverage on the target, you just haven’t measured it.

If you keep these two goals in mind, host-based SCA should be sufficient to comply with DO-178B objectives and should be much more efficient than doing full measurement and analysis on the target.

Reasonable types of verification for low-level requirements

One question that frequently comes up when discussing verification of safety critical software is exactly how certain requirements will be verified. For instance, you may have a requirement that says

The application shall load the processor’s Control Register with the value 0×12345678 during powerup initialization.

The need for the code in question is obvious. Whether such a requirement should actually be a requirement or design detail is debatable. (In my opinion it should be design detail.)

But if it is a requirement, how do you go about verifying it?

I have had verifiers suggest that the best way is to do white-box testing, to set a breakpoint in the code in a debugger and verify that the line of code is actually hit, and to then examine the contents of the register to verify that the value is present. While this is certainly a valid test, I think it’s overkill.

Assuming that the value you’re writing to the register is important for the software to run correctly, by doing any functional testing you have implicitly verified that the value was written to the register. Additionally, recall that DO-178B requires that each requirement be verified; however, it provides for multiple ways of performing that verification, including code inspection.

In my opinion, if you have a low-level software requirement similar to this one, the combination of a code inspection and an implicit functional test is sufficient verification of the requirement.