Over the years we have had many projects where users have asked us to qualify their laboratory equipment and validate commercial off-the-shelf (COTS) software. Sounds pretty straight forward, right? But when we look at the customer’s user requirements documents, we realized that they were derived from user manuals and specification sheets that manufacturers provide for that equipment. Some requirements even read like advertising lingo and performance claims from a website. And no amount of persuasion or cautionary tales can ever be enough to avoid the ultimate request from the customer to “just validate it” based on the manufacturer’s specifications. Worse yet, some of those specifications may contradict how the customer plans to use the equipment.
The regulatory concerns here are fairly obvious. Instead of defining requirements, selecting and purchasing a product based on those requirements, and validating that product for its intended purpose, the customer chooses to purchase a product first and then try to squeeze it into a compliance box, dress it up with qualification wrapping paper, and put a validation bow on top. But besides that, this practice can lead to lengthy and costly detours for a project. For example, how does one prove that software can “process approximately XX transactions per minute”? Or that a system has “typical throughput of…”? User requirements must first of all be just that, actual user requirements. “The system shall do #1, #2, etc…” and it “…shall conform to specifications #1, #2, etc…” Requirements have to be based on how the customer will be using a product, as well as the required, quantifiable, and measurable specifications such as accuracy, precision, reproducibility, speed, capacity, etc. User requirements should not be theoretical performance specifications that may only be achievable during testing of sub-components at the manufacturing site (if the product will not be operated by the user under the same conditions).
The end results of developing system requirements directly and solely from user manuals can throw a project into a tailspin. Users may want equipment or software validated for everything that it can do, so they can figure out how they want to use it later. But this is impractical, and will undoubtedly lead to testing failures, document re-writes, and a bloated, out-of-control budget. It is our job to help our customers understand the importance of actively participating in validation activities, most importantly during requirements gathering and development. The more accurate and precise these initial building blocks of validation are, the smoother the road to successful completion.