My company designed and manufactured a small controller for a client that sells air conditioning products (not actually air conditioners, but they do change the properties of the ambient air).
Recently, our client wanted to sell the product to a nuclear power generation company, and although it is to be used outside of the actual power generation, the product must undergo a validation and verification (V&V) process. The V&V requirements are often difficult to understand, and will be arduous to satisfy, if that is even possible.
The hardware of our controller consists of little more than an op-amp, a voltage regulator, a current transformer, several relays, and a PIC 16F877 microcontroller. It was originally designed about 15 years ago with a major revision about six years ago. The code is written in assembler and takes up most of the 4K bytes available on the micro.
To meet the V&V requirements, there are many issues that we will need to address. We comply with a large number of the requirements in one way or another as part of our ISO9000 procedures, so we're covered for peer review, specifications, revision control, and formal functional validation. The requirements also call for information on failures, returns, customer support, and the like. There are also requirements for descriptions of operation, user interface, and communications software. All of these are to be expected, though. It's some of the other requirements that have left me wondering just what we should do, especially since this is just a simple controller. Here are some examples and my thoughts on them.
Describe the software design, including the process loops and functions managed and/or controlled.
I don’t know at what level one needs to provide this. Do you think program comments would be sufficient and would flow charts be acceptable?
Describe the design processes and organizational interrelationships during the design process.
I have never had to quantify this before. Is there a wrong answer? I understand that processes and interrelationships can affect the quality of a product, but once the product has been successfully tested and field proven, how could this information be relevant any longer?
Provide copies of the hardware and software FMEA documents.
Provide MTBF and MTBR values for the design under review.
List the five hardware components with the highest probability of failure.
Identify the design elements and/or coding methodology implemented to reduce probability of coding related failures.
We do not undertake formal FMEA (failure mode and effects analysis) at my company because we do not have the resources. I wonder how many small companies actually do? We also do not undertake MTBF (mean time between failure) studies unless requested by the customer, because they are costly. Can you actually create a MTBF for software? Given that this product has only 4K in assembly code, do I still have to try and find some kind of methodology to describe what I did, especially after the fact?
Provide a list of the code modules that implement the design specification requirements.
Identify the testing activities utilized to evaluate and approve coded modules.
Identify the black and/or white box test methodologies utilized to evaluate and approve coded modules.
Provide a copy of a black and/or white box test plan.
Provide a copy of a code module approval document
The software is tested as small, discrete, functional blocks of code.
Here they are dictating the test methodology to be used. Am I limited to these methods? I certainly did all the tests and documented them. When a product is as small as this, how do you partition the operation? There are certainly separate functions, like start a fan and activate a valve, but they all interrelate. So where are the sides of the box?
Regression testing or engineering evaluations are to be performed prior to release when code is revised.
They are quite right, of course. A small change in one area can impact some other function (as I said earlier it is all interrelated). The problem is that there are many options (the system has 10 jumpers) and some options take days or even months to check, so regression testing could take a long, long time. This is an area of ongoing concern for me.
The manufacturer maintains cyber security during product development.
This security requirement includes the development equipment, PROM programmers, and computer based test equipment including those at the subcontractor. What controls does one have in place in Asia? The requirement also includes updating of software in the field and the security in place to implement updates.
Describe the activities and timing of those activities performed during a complete process loop.
If multiple process loops are employed, describe the timing and activities performed by each loop and the relationships of the loops to one another.
Describe all conditional and non-conditional branching activities in the firmware design.
These are doozies. Any action that is timed uses a hardware timer, but to figure out execution times of each loop? There are just so many ways that the execution goes through a portion of code with all the conditional jumps, I just shake my head. Any jump instruction must be described -- in the PIC there is no long conditional jump so any conditional jump in fact consists of two branches, to say nothing of the paging techniques used. Would comments in the listing be sufficient to meet these requirements?
Describe the existence of any installed or design features (interrupts, diagnostics, manual inputs) that could impact operation of the unit and describe why the feature will not negatively impact on firmware operation.
This requirement is rather puzzling to me. Any interrupt or manual input is likely to impact the operation as per the requirements. Testing, validation, and verification are supposed to prove that there is no negative impact.
It seems to me that these V&V requirements for the nuclear industry are more arduous than those that I saw in my time in the space sector! From my perspective as a consumer, I guess I am happy that the standards are so rigorous. But trying to meet them -- especially long after the design was completed -- will be long and difficult and expensive, if it is even at all possible.
How would your designs shape up to these requirements? Would you be able to go back and address all the issues?
Tweet This
