By David Ade, MasterControl
Dated: 6/1/2008
The words “software validation” strike fear in the hearts of software users – from everyday users to chief financial officers. Software validation has been described as a “necessary evil” in the regulated world. With an execution cost of 50-100% or more of the software price, validation becomes such a burden that can prevent companies from purchasing or upgrading to software that would improve product safety, quality, and efficiency.
Why Validate?
The most common reason regulated companies validate software is because it is required by the Food and Drug Administration (FDA). Many companies spend a great amount of time and money to go through the validation effort without understanding why it is necessary, often only executing validation to satisfy FDA requirements. The FDA defines validation in “Guideline on General Principles of Process Validation” May 1987, which states: “Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its pre- determined specifications and quality attributes.”
Most companies may only look at the first three words of this statement, falsely believing that generating mounds of binders and documentation demonstrates that the software is validated correctly. According to a seasoned validation specialist’s recent comments to a validation newsgroup, this voluminous amount of paperwork may only prove that mounds of paper were generated. “In every Good Automated Manufacturing Practice (GAMP) environment in which I have worked, significant man-hours are spent in generating paperwork that is never used by the business again, imparts no useful information and is glossed over by auditors (internal and FDA) who accept the mere presence of the document as evidence that the company is doing what it is supposed to be doing, and who have no idea how to evaluate the actual content of said document.”
Instead of viewing validation as an evil necessity, companies should view it as a process that makes good business sense. Good software validation practices help increase usability and reliability of the software, reduce possible errors and risks by finding issues before the software is used, and greatly minimize recalls caused by adverse effects on patients.
In January 2002 the FDA released a new guidance, “General Principles of Software Validation; Final Guidance for Industry”, which defined validation as: “…confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.”
This statement emphasizes that validation should conform to “user needs and intended uses”, thus making validation a practical process rather than a time-consuming or cost-burdensome requirement to the company. Typically too much time and effort is put into Operational Qualification (OQ) at the expense of a thorough Performance Qualification (PQ). This is the case because PQ testing tests the application based on user requirements, and how they plan on using the system. It can be costly and detrimental to find out application does not fit your needs after it has gone “live”.
FDA Software Validation Expectations
In 2002, the FDA published a new Guidance titled, “General Principles of Software Validation.” The Guidance outlines validation expectations and illustrates how risk management can help define what and how much validation evidence is required. It states: “The level of validation effort should be commensurate with the risk posed …” In this document, the FDA also addresses the validation of off the shelf software, stating: “The vendor’s lifecycle documentation, such as testing protocols and results, source code, design specification, and requirements specification, can be useful in establishing that the software has been validated.”
In September of 2004, the FDA reemphasized the use of a risk-based approach by releasing the Guidance, “Pharmaceutical CGMPS for the 21ST Century — A Risk-Based Approach Final Report.” The FDA also sent spokespersons to conferences to inform the industry on how the FDA itself uses risk assessment, stating that industry should follow suit to determine the validation effort required.
Using Risk Assessment in Software Validation
Through the release of industry Guidances, FDA has provided informational tools leading the industry to validate software based on risk while allowing more focus to be placed on how the software will be used, and the effect risk has on patient safety and product quality. For companies wanting to take advantage of today’s technology, the utilization of risk assessment is a reasonable alternative to the traditional avenues of validation.
The FDA stated in Guidance for Industry: Part 11, Electronic Records; Electronic Signatures — Scope and Application (2003) “We recommend that you base your approach on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety, and record integrity.”
One way companies can use risk assessment to conform to these Part 11 guidelines is to evaluate/assess a vendor’s internal testing to see if it can be used to reduce their OQ testing effort. Most vendors test their software extensively before it is released and thoroughly document their test results.
Companies that use a risk assessment approach can review a vendor’s test records, and audit the vendor’s development process and leverage that information to either reduce or eliminate their OQ testing. This does not remove the PQ requirement, but significantly reduces the overall testing effort.
The risk assessment concept and approach has been supported in regulated industries in the last several years. A statement made at the IVT Conference last year affirms this: “Validation ought to be inversely proportional to the documentation created during vendor development and testing.”
If companies use vendor documentation to reduce testing effort, an audit of the vendor test records is important and it should be done before the data is accepted. If any gaps in testing or high-risk areas are identified during the audit, additional testing can be done by executing portions of the OQ test scripts again or by completing additional testing during the PQ test phase to mitigate risk.
Companies should also trace their requirements to the OQ test records to verify that their requirements have been met. Any gaps or areas of high risk should be retested or completed during the PQ phase of testing. Companies should understand the vendor’s test records and methodology in order to be able to defend the acceptance of their data. Upon acceptance of the vendor’s documentation, companies can focus on how they are going to use the software in the PQ testing phase.
Validation is also an issue with companies wanting to upgrade to a most recent software release. If the upgrade is a “major release,” companies should plan to go through a full revalidation effort. Due to the time and cost involved, many companies tend to put off upgrading until the value of the new functionality outweighs the cost of validation. This upgrade delay, which may be up to three years or more, denies the use of new functionality and enhancements that could improve the use of the software.
Using risk assessment when upgrading software versions can be very beneficial in reducing the validation effort. By using the vendor’s documentation, audit, previous history, and knowledge of the software, companies can justify testing only the new functionality as part of the PQ effort.
Savings from Vendor Documentation
Based on completed project data information from MasterControl customers’ implementation from June 2006 to Sept 2007, a comparison was made between companies that used vendor documentation to reduce their OQ effort and those that did not. Of the 80 companies surveyed, 26 companies that used the vendor documentation were able to reduce the time from the start of the project to when they went “live” by 33%. This reduction in time helped release the software sooner, saving the company money and allowing the company to focus more time and resources on the PQ effort, which gave them a better and more defendable validation package.
When software solutions are not deployed and validated properly the results can be product recalls, production downtime, business closure, or – worst-case scenario – risk to patients’ health. Using a risk-based approach to validate software allows life science companies to save effort, time, and money, without affecting the quality and safety of the software. Avoiding software validation pitfalls begins with evaluating potential risks and developing a plan to deal with hazards once they have been identified.
Companies should keep their efforts in line with FDA and industry Guidance, which suggest that the level of validation to be performed should be based on the risk of the records generated by the system. If this is followed accordingly, the cost of overall validation can be reduced, allowing companies to better take advantage of the benefits of an electronic system sooner due to less validation work and faster system implementation.
|
