Data Management: Information Integrity

Data quality is the life-blood of clinical trial data management. Poor data quality not only generates additional work for later clinical analyses but also carries the risk of eventually leading to the failure of a drug application. As more pharmaceuticals companies expand their clinical trials into Asian countries such as India and China, this issue is becoming a major concern to sponsors and regulatory agencies such as the US Food and Drug Administration (FDA).

What are the factors that one should consider when conducting trials in Asia? Is it possible to measure the quality of clinical data collected by a third party?

While there are many factors that can impact data quality, standards and training are considered the major ones when it comes to clinical trials in Asia. Although the language and cultural differences may be significant at the beginning, these can be minimized if standards are in place and adequate training is provided.

There are four steps that can lead to higher data quality: adopting a standard; training a team; measuring data quality with known criteria; and implementing a set of technologies to improve data quality.

Embracing a Common Standard
The key to data quality is in adopting a common standard, sticking to the standard practices and processes, and using technology to reduce human errors. Adopting a standard not only increases interoperability and efficiency, but also provides a foundation for data quality and its measurability.

Pharmaceutical companies that are outsourcing work to Asia should first find out if a contract research organization (CRO) adheres to industry standard practices such as Good Clinical Practice (GCP), Good Laboratory Practice (GLP), and Good Manufacturing Practice (GMP). Secondly, it needs to check if the CRO has adopted a common data standard.

International Conference On Harmonization (ICH) has harmonized GCP. The FDA has recommended the Study Data Tabulation Model (SDTM) developed by the Clinical Data Interchange Standards Consortium (CDISC), as data standards for structure, terminology and code sets, in the Federal Register, Volume 71, No 237, December 11, 2006.

The lack of standards has become a bottleneck to utilizing and improving health informatics in China, as suggested in a study by the Institute for Health Information, Fourth Military Medical University. The main barriers presented in the process of standardization not only consists of financial, technical, cultural and language problems but also entails legal and ethical concerns.

One of the ways to minimize these problems and to ease concerns is to conduct standard-based training and interchanges. While the Chinese central and local governments have taken the initiative to try to improve on the lack of standards, the private sector seems to have been more proactive.

The 12th Healthcare Industry Forum held in Beijing in October 2008, carried the theme of "One World One Standard". The CDISC Coordinating Committee in China (C3C) was formed in the same month and hosted its first China Interchange in Shanghai. C3C directly advocates and promotes the CDISC standard adoption in China.

If the industry can adhere to the three "A"s: Advocating, Adopting and Advancing, the rapid adoption of the standard may be seen in the near future. The three 'A's must start with well-planned training.

There needs to be a common vision for developing safer and more effective drugs, efficiently and productively with common standards. The team has to consist of advocates, practitioners, developers, and supporters. As successfully demonstrated in the Comprehensive International Program for Research on AIDS (CIPRA) project, it is critical to have well-planned and implemented training and team building activities.

CIPRA is an initiative sponsored by the US National Institutes of Health (NIH) that provides long-term support for laboratory and clinical studies for HIV/AIDS prevention and treatment in developing regions. China CIPRA is conducted by the Chinese Center for Disease Control and Prevention (CDC) under the leadership of the Ministry of Health, together with other domestic and international scientists. It has a five-year grant of US$14.8 million and is composed of five inter-related projects and four cores. Core A encompasses administration and training.

Experts were hired from the Statistical Center for HIV/AIDS Research and Prevention (SCHARP), Westat, Premier Research, and NIH to give training to members from all the cores in the project. Many of the experts are native Chinese who have been working in the field for over a decade. The China CIPRA project also had advocating committees and working groups and was networked with other NIH projects in China.

The coordinated effort paid off. A study on the quality of international clinical studies in China was conducted by Jason Chang from the National Institute of Clinical Drug Studies at The Xijing Hospital, Fourth Military Medical University. The study concluded that GCP adherence between US and China for the project is similar in distribution pattern. Overall, the China CIPRA program was at least equivalent to US studies from an ICH/GCP perspective.

Measuring Data Quality Based on Criteria
When outsourcing clinical trials to a CRO, there needs to be credibility in the clinical data provided by the third party and in how it is collected and processed. It is nearly impossible to measure quality that one does not have control over and where there is no common standard. The "Specifications for Study Data Tabulation Model Validation Criteria for the Janus Operational Pilot" published by FDA in 2008 makes it possible to measure data quality through a commonly defined set of compliance checks - even if one does not have direct control over the data sets.

A data standard defines a target for data structure and content for systems and people. A well-designed system can become a tool for the production of standardized products.

In clinical trails, it is necessary to produce SDTM data sets. Since many trials were started before SDTM was developed by CDISC and recommended by FDA, efforts have been made by pharmaceutical companies to convert these legacy data sets into SDTM format.

Janus is a clinical trial data repository (data warehouse) standard that is sanctioned by the FDA. It provides a data model for clinical study data warehousing and enables an integrated review environment. FDA has conducted two pilot projects to load SDTM data sets into the data repository.

Many extracting, transforming and loading (ETL) tools have been developed and tuned to conduct data integration and standardization (DIS). Some companies including CROs in China are specialized in DIS and provide data conversion services. Since there are different ways to translate data, how does one know that the data is truly in SDTM format? And how can the quality of SDTM data sets be measured?

In the specifications for SDTM validation criteria published by FDA in January 2008, a set of 109 compliance checks have been defined, and these checks are classified into three levels of severity:

• High: The error is serious and will prevent the study data from being loaded successfully into the Janus repository.
• Medium: The error may impact the reviewability of the submission, but will not prevent the data from being loaded into the repository.
• Low: The error may or may not impact the reviewability or the integrity of the submission. It will not prevent the study data from being loaded into the Janus repository. These compliance checks can be further categorized into three levels of validation:
• Structure validation: Checks the structure of the data sets including data type, column length, format, and presence of required variables.
• Integrity validation: Checks the relationship integrity of data sets including consistency (cross-column, cross-domain, external dictionary) and referential constraints.
• Value validation: Checks the value compliance in terms of limits (range, upper and lower bounds), code list, illegal values, and meta-data.

Based on these defined criteria, some companies have developed more extensive checks to validate SDTM compliance. Figure 2 shows the comparison list of the compliance checks between FDA and Octagon Research. This list is not exhaustive and the number of checks may increase in future. These checks provide a quantitative measurement for data quality against a targeted standard.

Leveraging on Technology to Improve Data Quality
Once the criteria have been clearly defined, utilizing ETL and database technologies can help to streamline the process of data conversion and validation. The model of the extended validation engine in the ETL process involves the following:

• Adding a validation process (compliance check) between transformation and loading, where the compliance check is to ensure conformance to both the data model and the business rules.
• Maping CDISC SDTM domains and variables with Janus tables and columns.
• Building the meta-database, including a code list for all the controlled variables.
• Defining dimensions, hierarchies and cubes for Online Analytical Processing (OLAP).

Figure 3 shows the implementation model for the validation engine in the entire clinical data management process. Clinical data is collected through a regular clinical data management system (CDMS) or electronic data capture (EDC) system. The data sets are converted to SDTM using an ETL tool such as Oracle Warehouse Builder (OWB), or exported to SDTM format if the EDC is SDTM compliant.

The SDTM data sets are put through a checkpoint (validation engine) and uploaded into Janus data warehouse if they pass the data quality validation. Once all the data sets from different companies or from different therapeutic indications are loaded into this single repository, users or regulators will be able to review them in the same environment, possibly with the same set of tools.

Many data conversion projects in China or other Asia countries are conducted by clinical programmers using a computer programming language such as SAS, PL/SQL, Java, or Perl. The resulting data quality depends on the programmer's skill and the enforcement of standard operating procedures for quality control by the CRO.

Implementing an industry strength ETL tool such as OWB will allow a company to enjoy the benefits of:

• Hiring non-programmers (who cost less);
• An audited environment;
• Built-in security;
• Consistency among all the users;
• Ease of management and support.

However, as it is typically expensive to deploy and maintain such systems, few CROs in Asia can afford them.

Ideally, the data that is collected in a system should be compliant to the SDTM standard. All data issues should be resolved before the data sets are pushed downstream for processing. Some EDC systems are already SDTM aware or compliant, for example, PhaseForward's InForm EDC, Octagon Research's ViewPoint Fuse, etc.

It is only beneficial to use an EDC system at the commencement of a clinical trial. The deployment of such systems has been slow in the past decade. More clinical data sets will need to be converted to SDTM and to be put through the validation engine to encourage more clinical studies to be conducted with EDC systems.

Pumping data through a validation engine to validate clinical data sets is the final step that one must take before submitting the data sets to the FDA and committing the user fee for the drug application.

Once FDA improves on its infrastructure and has the capability to load and store all clinical data in the Janus data warehouse, even the data coming from a SDTM compliant system may have to go through a validation process before it gets loaded into the clinical repository. While it is important to complete projects on time and within budget, it is also necessary to ensure quality as well. With commonly recognized standards, it is possible to achieve data quality through training, defined measures, and technologies.

The goal is to timely develop safer drugs and to enhance public health. With the collective effort of the life science industry, it is possible to build a standards-based and process-centric platform that integrates technologies, processes, and people to achieve higher data quality across the world.

-------------------------------------------------------------------------------------------

-----------------------------------------------------------------

Team Building Through Training

There are two types of standards in clinical trials: ethical standards such as GCP and data standards such as the CDISC SDTM. GCP provides a unified standard in the conduct of clinical trials. It serves as an international ethical and scientific quality standard for the design, conduct, performance, monitoring, auditing, recording, analysis, and reporting of trials that involve human subjects.

GCP has been practiced in the European Union (EU), Japan, the US, China, and other countries. Compliance with GCP provides the assurance that the data is reported, that the results are credible and accurate, and that the rights, safety and confidentiality of trial subjects are protected.

CDISC SDTM is a data submission standard and provides a stable structure for clinical studies and interoperability among systems. It has been adopted by companies in the US. In China, GCP has been exercised in certain projects but CDISC SDTM is still relatively new to Chinese investigators and clinical professionals.

The awareness and knowledge of standards come from education, training and self-motivated studies. GCP guidelines have been taught at US institutions such as the University of Southern California. The more immediate and effective way for learning the standards is via on-job training.

-----------------------------------------------------------------