Evaluation of the COBAS Hepatitis C Virus (HCV) TaqMan Analyte-Specific Reagent Assay and Comparison to the COBAS Amplicor HCV Monitor V2.0
http://www.100md.com
微生物临床杂志 2005年第5期
ARUP Institute for Clinical and Experimental Pathology
University of Utah Health Sciences Center, Salt Lake City, Utah
ABSTRACT
Performance characteristics of the COBAS hepatitis C virus (HCV) TaqMan analyte-specific reagent (TM-ASR) assay using the QIAGEN BioRobot 9604 for RNA extraction were evaluated and compared to the COBAS Amplicor HCV Monitor V2.0 (Amplicor) and Versant HCV bDNA 3.0 (Versant) assays using clinical samples. Calibration of TM-ASR using Armored RNA allowed determination of the distribution of HCV RNA in clinical samples, using 22,399 clinical samples. Limit of detection, linearity, and inter- and intraassay assay precision were determined for the TM-ASR assay using multiple clinical specimen panels across multiple determinations. Genotype specificity for the TM-ASR assay was determined using samples with different HCV RNA genotypes evaluated and compared against predetermined results. Contamination control of the TM-ASR assay was evaluated using pools of HCV RNA-positive and -negative samples tested in a checkerboard pattern over 12 runs of 96 samples. Correlation of the TM-ASR, Amplicor, and Versant assays was determined using 100 paired clinical samples and Deming regression analysis. The TM-ASR performed well with respect to linearity, precision, and contamination control. The correlation between TM-ASR and the Amplicor and Versant assays was poor, with large differences between assay results for individual samples. Calibration of the TM-ASR assay with Armored RNA allowed for a wide dynamic range and description of the distribution of HCV RNA in clinical samples.
INTRODUCTION
The role of hepatitis C virus (HCV) RNA testing has evolved from early research assays with uncertain utility to well-established production tests routinely used for patient care. Recent recommendations emphasize the use of anti-HCV testing by immunoassay or chemiluminescent immunoassay signal-to-cutoff ratios in combination with HCV RNA or anti-HCV by recombinant immunoblot analysis for the identification of infected patients. Molecular testing for HCV RNA remains an important part of the diagnostic algorithm (2). In addition, quantitative HCV RNA testing has become increasingly used for the management of chronically infected patients. Studies have demonstrated a high likelihood of nonresponsiveness to pegylated interferon-ribavirin combination therapy when a 2-log drop in HCV RNA levels is not observed during the first 12 weeks of therapy (5, 12). This finding allows the physician and patient to reassess treatment. Those that stop treatment are spared significant cost and morbidity associated with long-term HCV drug therapy.
The use of HCV kinetics to guide therapy has highlighted the issue of the dynamic range of HCV quantitative tests. The upper limit of detection, especially for PCR assays based on end-point analysis (COBAS Amplicor, Amplicor, and Superquant) is not sufficient to accurately determine the baseline viral load for a significant percentage of patients at the onset of therapy. For these samples, repeat dilution testing is required to establish an initial viral load (9). Quantitative HCV testing with Versant HCV bDNA V3.0 (Versant; Bayer Diagnostics, Tarrytown, NY) (3) offers a significantly higher upper limit of detection but at the cost of lower sensitivity and potentially specificity at its limit of detection (7, 11).
The development of real-time reverse transcription-PCR technology affords the potential for sensitive, broad-dynamic-range testing in a format that is amenable to automation and improved contamination control. However, commercially available assays that utilize real-time PCR technology have been slow to come to market, and developed "home-brew" assays have been limited. The first commercial manufacturer to market with a real-time PCR platform was Roche Diagnostics (Indianapolis, IN) with the introduction of the COBAS HCV TaqMan analyte-specific reagent (ASR) assay (1, 4). While the COBAS HCV TaqMan ASR (TM-ASR) assay potentially offers many advantages over current commercially available technologies, including expanded dynamic range, high throughput, decreased time to result, increased automation, and improved sensitivity, the user is required to calibrate this assay without manufacturer support.
We evaluated the TM-ASR assay using the QIAGEN BioRobot 9604 instrument (QIAGEN Inc., Valencia, CA) and QIAamp virus kit for RNA isolation. The TM-ASR linearity, precision, and genotype inclusion were evaluated, and the TM-ASR assay was compared to the COBAS Amplicor HCV Monitor V2.0 (Amplicor) and Versant assays. In addition, several methods of calibrating the TM-ASR assay were evaluated using both commercially available material and Armored RNA (Ambion Diagnostics, Austin, TX) referenced directly to the WHO HCV RNA second international standard 2003 (96/798).
MATERIALS AND METHODS
Sample selection. EDTA plasma and serum samples submitted to ARUP Laboratories for the determination of HCV RNA levels by Amplicor were used in this study. Samples tested or results compiled from databases were deidentified prior to use or analysis in this study.
COBAS HCV TaqMan ASR: nucleic acid extraction. The extraction of viral RNA and quantitation standard (QS) RNA was achieved by using the QIAGEN 96 virus protocol performed on the QIAGEN BioRobot 9604 (B configuration, build 12). In brief, the extraction consists of sample (220 μl) lysis by protease in a guanidium, carrier RNA, and QS containing lysis buffer (buffer AL) at an elevated temperature (60°C). Sample lysate was then added to 240 μl ethanol, the solution was transferred to the silica-containing QIAGEN column, and the lysate was removed by applying vacuum. The sample was then washed one time with the first wash buffer (AW1 buffer) and two times with the second wash buffer (AW2 buffer), with wash buffer removed by application of vacuum in all cases. Nuclease-free water (buffer AVE) and centrifugation were used to elute extracted RNA from the column. The procedure was similar to what is described in the package insert, but included the following modifications to introduce the recommended concentration of QS to each sample: 900 μl of HCV ASR QS (Roche Diagnostics Corporation, Indianapolis, IN) was added to one vial of QIAGEN carrier RNA (1,350 μg/vial), and 500 μl of the QS/carrier solution was added to 27 ml of buffer AL, which was split into four aliquots of 6.5 ml each and placed on the deck of the QIAGEN BioRobot 9604. TM-ASR master mix was reconstituted by adding 789 μl of manganese acetate (Roche Diagnostics Corporation) to 1 vial of TM-ASR master mix (lot SL1; Roche Diagnostics Corporation). TM-ASR master mix was split into four vials of 1.5 ml each, and 50 μl was aliquoted into individual K-tubes (Roche Diagnostics Corporation) using the QIAGEN BioRobot 9604. After master mix transfer, 50 μl of eluted sample RNA was distributed to the correct K-tube location on the K-tube plate using the QIAGEN BioRobot 9604.
Amplification and detection. Loaded K-tubes were placed on the COBAS TaqMan 96 analyzer (Roche Diagnostics Corporation), where the K-tubes were robotically capped, placed in K-carriers, and loaded into the correct thermocyclers. Cycling conditions were as described in the TM-ASR technical bulletin.
The Versant HCV bDNA 3.0 and COBAS Amplicor HCV Monitor assays were performed as per the manufacturers' instructions.
Samples greater than 600,000 IU/ml in the Amplicor assay were diluted 1:50 in normal human serum (NHS) and retested. Final results in the Amplicor assay were determined by multiplying the diluted HCV RNA IU/ml result by the dilution factor.
Calibration. The TM-ASR assay requires user calibration of the assay with each new lot of reagents. Calibrators are not supplied with the TM-ASR reagents and are not available directly from the manufacturer. Calibrators are processed in the same manner as patient samples, including nucleic acid extraction and amplification. After extraction and amplification, ASR external calibration software, V1.0 and V2.0 (Roche Molecular Systems, Inc., Pleasanton, CA), was used to establish a relationship between the values of the calibrators assigned by the operator and the elbow values of the QS and target probes. The result of this calibration is a set of three calibration coefficients (a, b, and c), which are then entered into the COBAS TaqMan Analyzer software (TaqLink V1.11) at the beginning of subsequent runs and are used to quantitate the unknown samples.
For this study, calibration of the TM-ASR assay was performed using standards obtained from Acrometrix Inc. (Benicia, CA) and consisted of calibrators containing the following nominal HCV RNA concentrations: 500, 5,000, 50,000, 200,000, 500,000, and 2,000,000 IU/ml. Calibration consisted of assaying each calibrator five times on three separate TM-ASR runs, calculating the calibration coefficients for each run, and then averaging a given calibration coefficient to obtain working calibration coefficients. The latest version (V2.0) of the software allows for data from multiple runs to be incorporated into a single set of calibration coefficients.
Calibration using HCV Armored RNA 1b. In order to determine if the linear range of the TM-ASR assay could be expanded by using higher concentrations of standards, Armored RNA was selected because of the large volume of high-titer material which could be readily obtained. A high-titer sample of HCV Armored RNA genotype 1b was obtained and serially diluted using normal human plasma negative for anti-HCV and HCV RNA. Armored RNA standards contained the following calculated HCV RNA IU/ml concentrations: 143,000,000, 14,300,000, 1,430,000, 143,000, 14,300, 1,430, and 717. Calibration of the TM-ASR using Armored RNA calibrators consisted of RNA extraction with the QIAGEN BioRobot 9604 and QIAamp virus kit and RNA amplification and detection using TM-ASR reagents assayed on the COBAS TaqMan analyzer, with each of the calibrators tested in replicates of eight. Four replicates of the WHO HCV RNA second international standard 2003 (96/798) reconstituted in 2 ml of molecular-grade water (1:4 dilution) were processed concurrently to ensure that the calibration coefficients obtained were directly linked to the current WHO international standard. The calibration run was performed using calibration coefficients determined using Acrometrix calibration material, and the results were normalized to the calculated concentration of HCV RNA 2nd International Standard 2003 (96/798). Actual HCV RNA concentrations of the calibrators were calculated by normalizing the average result of the calibrator with the lowest percent coefficient of variation (CV) to the average HCV RNA IU/ml value of the hepatitis C RNA 2nd International Standard 2003 (96/798) and then calculating the concentrations of the remaining calibrators. The calculated HCV RNA IU/ml concentrations of the calibrators were input into ASR external calibration software, V2.0 (Roche Molecular Systems) with the appropriate .RSF file exported from the TaqLink software V1.11 (Roche Molecular Systems), for the calibration run to generate calibration coefficients. Observed results were back-calculated by using ASR external calibration software, V2.0, and compared against the expected HCV RNA values by linear regression analysis.
Limit of detection. To determine the lowest level of HCV RNA in IU/ml that can be detected by the COBAS TaqMan HCV ASR assay, a six-level clinical specimen low-end dilution panel was obtained from Roche Molecular Systems, Inc., extracted, and amplified in replicate over five runs of 96 samples/run. The panel consisted of an HCV specimen of genotype 1a at concentrations of 0, 10, 25, 35, 50, and 100 IU/ml. Each run consisted of five different HCV panel members (n = 16/level/run), as well as a negative panel member (n = 15/run). Invalid results, defined as any result flagged by the COBAS TaqMan Analyzer as "invalid," were excluded prior to performing calculations. A positive result was considered to be any quantitative result, regardless of the value, or any result of "less than 10 IU/ml." Results which were "target not detected" were considered a negative result. For each of the five different concentration levels, the percent detection rate was calculated (number of detected samples/total number of valid results). PROBIT analysis was used with the preassigned titer values to calculate the concentration giving a 95% detection rate and its 95% confidence interval range.
Linearity and precision. The linearity and assay precision of the TM-ASR assay using the QIAGEN BioRobot 9604 were determined using an eight-member clinical panel provided by Roche Molecular Systems (HCV clinical panel 9905). Panel members were processed, amplified, and detected as described previously. The eight clinical panel members spanned concentrations from approximately 68 to 6,800,000 IU/ml with one negative HCV member. A total of six runs were performed where each positive member of the clinical panel was processed in replicates of 12 in each run and the negative member was processed in replicates of 11 in each run.
Invalid and negative results were excluded prior to performing calculations. The mean titer, standard deviation, and percent CV were determined for each of the seven positive HCV concentration levels. For each of the seven positive levels, nested analysis of variance was performed to obtain estimated intraassay, interassay, and total imprecision. The observed titers were converted to log10 format, and the expected log10 titer versus observed log10 titer was plotted.
Determination of genotype specificity. Genotype inclusively for the TM-ASR assay was determined using clinical samples from eight different genotypes of HCV (1a, 1b, 2b, 3a, 4, 4c/4d, 4h, and 5a) purchased from Millennium Biotech (Ft. Lauderdale, Florida) to determine equivalence of quantitative detection. Samples were extracted, amplified, and detected as previously described. The expected titers for each sample were obtained from Millennium Biotech, and all values were converted to the log10 HCV RNA IU/ml. The differences in the TM-ASR titers versus the expected titers were calculated for each genotype.
Correlation. Assay correlation was determined by processing 100 clinical samples by the Amplicor, Versant, and TM-ASR assays. The selection of samples consisted of 70 samples with titers between 600 IU/ml and 500,000 IU/ml. Fifteen HCV-positive clinical samples had titers less than 600 IU/ml, and 15 samples had titers greater than 500,000 IU/ml. Samples were extracted, amplified, and detected as previously described for each assay. All results were converted to log10 HCV RNA IU/ml, and results were compared through the use of Deming regression.
Contamination control. A total of 576 positive and 576 negative samples were assayed in a "checkerboard" setup across 12 separate TM-ASR runs of 96 samples each. Positive and negative samples used were pools of HCV RNA-positive and NHS samples, respectively. NHS was tested for HCV RNA by COBAS Amplicor HCV assay and was found to be nonreactive, and the positive sample pool was found to have an average HCV RNA titer of 2,595,000 IU/ml (Amplicor measurement with a 1:50 dilution). These experiments used the calibration coefficients generated using Acrometrix calibrators.
Distribution analysis. Data from 22,399 clinical samples tested using TM-ASR and Armored RNA calibration coefficients were gathered through a Microsoft Excel database. Duplicate entries were removed, samples with diluted results were excluded, and only undiluted results were included. Data were stratified in 0.1-log10 increments from 100 to 100,000,000 HCV RNA IU/ml and plotted as percentage of positive samples versus log10 HCV RNA IU/ml. Data from 4,037 Versant HCV bDNA 3.0 samples were collected from the ARUP Network System Repository database, and results were stratified into less than, within, and greater than the reportable range for comparison against the predicted percentage of results expected to fall within these ranges based on the distribution from the TM-ASR distribution.
RESULTS
The TM-ASR calibration coefficients generated from Acrometrix standards were determined over three separate calibration runs by averaging the calibration coefficients calculated on these three runs. The calibration coefficients were as follows: a = –0.0060, b = 0.4532, and c = 0.5929, with calibration coefficient standard deviations of a = 0.00015, b = 0.002635, and c = 0.05397 and percent CVs of a = 2.6%, b = 0.58%, and c = 9.10%.
Inter- and intraassay precision. For the TM-ASR assay, the within-run standard deviation ranged from 0.078 to 0.273 log10 HCV RNA IU/ml and percent CV ranged from 1.56% to 16.5% at 68,000 IU/ml and 68 IU/ml, respectively. Between-run standard deviation log10 HCV RNA IU/ml ranged from 0.054 to 0.174 at 6,800 HCV RNA IU/ml and 6,800,000 IU/ml, respectively. Between-run percent CV ranged from 1.48% to 16.5% at 6,800 HCV RNA IU/ml and 68 HCV RNA IU/ml, respectively. Data are presented in Table 1.
Linearity. The observed titers of the linearity experiments for TM-ASR are presented in Fig. 1a. The regression was as follows: observed = 1.086 x expected – 0.245; n = 800; R2 = 0.999. The linear range of the TM-ASR assay using the Acrometrix calibrators was 680 to 3,400,000 IU/ml (Table 1). Note that as HCV RNA levels increased, the observed HCV RNA level was higher than expected, suggesting that as HCV RNA levels increase above the highest calibrator overestimation occurs.
Limit of detection. The limit of detection was determined for HCV RNA from a clinical specimen low-end dilution panel extracted by the QIAGEN BioRobot 9604 instrument and amplified using TM-ASR reagents on the COBAS TaqMan analyzer. The rate of positive results for a given level was calculated by tabulating the total number of nonnegative results and comparing this value to the total number of valid (QS reactive) results (Table 2). PROBIT analysis performed on these data using the previously determined concentration indicated that 71.3 IU/ml was detected with a 95% confidence range of 55.0 to 105.7 IU/ml.
It was noted that the observed quantitative values for these samples were less than what was expected based upon the previously assigned HCV RNA level. Observed versus assigned HCV RNA levels were compared, and the following were calculated for the observed values: average, standard deviation, percent CV, the actual number of valid results, and the percentage observed versus expected. These data indicate that expected HCV RNA concentrations previously assigned were not consistent with the values obtained empirically, and they are presented in Table 3. In addition, not all replicates produced quantitative values and, in the best case, only 81.3% of the samples assayed had a quantitative result in the TM-ASR assay. Note that the number of valid results is defined as any result where the QS was valid, regardless of the result of the unknown. These data indicate that although the combination of the QIAGEN 96 virus/QIAGEN 9604/TM-ASR assay is capable of detecting 71 IU/ml consistently, the values do not appear to be quantitatively accurate.
Genotype specificity. Samples with genotype 1a seemed to deviate the most from 100% recovery, with the average recovery being 112.9%; genotype 2b samples seemed to perform best, with an average recovery of 102.7%. Note that the expected recovery in these experiments was generated by Millennium Biotech, from which these samples were purchased. No independent quantification was performed on these samples other than HCV TM-ASR. Results are presented in Table 4.
Correlation. Assay correlation was determined by processing 100 clinical samples by the Amplicor, Versant, and TM-ASR assays. The correlation between Amplicor and Versant (Fig. 2a) indicated that HCV RNA levels obtained by the Amplicor assay were slightly higher than those with the Versant assay and displayed compression near the upper limit of the Amplicor assay, with the Deming regression equation Versant = 1.065 x (Amplicor) – 0.446 (R2 = 0.939; n = 80; standard error of the estimate [SEE] = 0.239), and this was similar to previous comparisons (6, 10). The Deming regression between the TM-ASR and Versant assays (Fig. 2b) was TM-ASR = 1.188 x (Versant) – 0.663 (R2 = 0.829; n = 80; SEE = 0.473). The TM-ASR and Amplicor assays (Fig. 2c) Deming regression was TM-ASR = 1.207 x (Amplicor) – 0.919 (R2 = 0.818; n = 86; SEE = 0.544). Also, the TM-ASR and Amplicor assays detected six more samples than the Versant HCV bDNA 3.0 assay, which suggests that the TM-ASR and Amplicor assays are more sensitive than the Versant assay, even though the Amplicor and Versant assays have similar limits of detection claimed by the manufacturers (600 and 615 IU/ml, respectively). Of six samples not detected in the Versant assay, two samples had results of less than 600 IU/ml in both the Amplicor and TM-ASR assays. One sample result was 551 IU/ml in the Amplicor assay and 540 IU/ml in the TM-ASR assay, and the second sample result was 214 IU/ml in the Amplicor assay and 553 IU/ml in the TM-ASR assay. The remaining four samples were greater than 600 IU/ml in both the TM-ASR and Amplicor assays.
Contamination control. Of the 576 positive samples assayed in a "checkerboard" setup across 12 separate runs of 96 samples each, the average HCV RNA titer by TM-ASR was 3,720,000 (6.54 log10) IU/ml versus 2,600,000 (6.41 log10) IU/ml by 1:50 dilution in the COBAS HCV Monitor V2.0 assay. Results of the 576 negative samples included a single NHS sample with an HCV RNA level greater than 100 IU/ml in the TM-ASR assay (3,940 IU/ml). In addition, a single sample was quantified with an HCV RNA level of 68 IU/ml, and six samples returned results of "less than 10 IU/ml." These results indicate that the QIAGEN 9604 virus kit as used on the QIAGEN 9604 instrument paired with the TM-ASR reagents performs well with respect to contamination control using a high-titer specimen interspersed with an HCV RNA-negative specimen.
Armored RNA The TM-ASR calibration coefficients generated using high-titer Armored RNA material were a = –0.0093, b = 0.4657, and c = 0.8129. The average calculated HCV RNA in the 1:4-diluted WHO HCV RNA second international standard 2003 (96/798) was 26,800 (log10 4.42) IU/ml, with a standard deviation of 3,490 (log10 0.06) IU/ml, which agrees well with the expected value of 25,000 (log10 4.40) HCV RNA IU/ml. Linear regression comparing observed versus expected HCV RNA log10 IU/ml (Fig. 1b) showed observed = 1.02 x (expected) – 0.108 (R2 = 0.999; n = 56).
Distribution. Distribution data for the TM-ASR assay are presented in Fig. 3a, with the predicted ranges of the Amplicor and Versant assays shown. These data indicate a skewed normal distribution for samples tested with HCV RNA detectable from 100 to 100,000,000 IU/ml. Distribution data indicated that 43.0% of samples tested had results of less than 100 IU/ml. Analysis of TM-ASR results within the range of the Versant assay (615 to 8,000,000 HCV RNA IU/ml) found average, standard deviation, and median log10 HCV RNA IU/ml values of 5.57, 0.90, and 5.72, respectively. The distribution for the Versant assay is presented in Fig. 3b and indicates that the observed percentage of samples with HCV RNA results less than and greater than the analytical measurement range are less than those predicted by the observed HCV RNA distribution by TM-ASR. The TM-ASR distribution predicts that 44.2% of all samples tested in the Versant assay will fall below the detection limit, while the observed value was 37.6% (n = 1,518). Similarly, the TM-ASR distribution predicts that 7.5% of all samples tested in the Versant assay will be above the analytical measurement range, while the observed value was 1.2% (n = 50). The average, standard deviation, and median log10 HCV RNA IU/ml values for samples quantitated in the Versant assay were 5.62, 0.85, and 5.80, respectively
DISCUSSION
Measurement of HCV RNA has a clearly established role for confirming the diagnosis of infection, monitoring the course of therapy, and establishing the outcome and durability of therapy. These different clinical indications place separate demands on molecular assays measuring HCV RNA. Diagnostic and end-of-treatment testing must be sensitive enough to detect low levels of HCV RNA but does not require quantitative accuracy across a broad range of HCV RNA concentrations. In contrast, current therapeutic "stopping" algorithms require accurate quantification across the broad range of HCV RNA levels seen in infected patients. An assay that is both sensitive and has a broad dynamic range would be ideal.
The TM-ASR assay as described offers substantial improvements in throughput and contamination control over current quantitative HCV RNA assays, while approaching the sensitivity of qualitative HCV RNA assays. These studies document a detection limit of 71 IU/ml for the TM-ASR assay using PROBIT analysis and samples containing a low concentration of HCV RNA. The TM-ASR assay is observed to be only slightly less sensitive than the Food and Drug Administration-approved COBAS Amplicor HCV V2.0 qualitative test with its detection limit of 50 IU/ml (12). This is a substantial improvement over previous quantitative assays, including both the COBAS Amplicor HCV Monitor V2.0 (600 IU/ml) and Versant HCV bDNA V3.0 (615 IU/ml) assays. Although the argument has been made that more sensitive testing may have a value in the analysis of end-therapeutic response (15), definitive evidence is lacking in the literature to justify use of ultrasensitive assays. At this time, algorithms for patient management based on such testing have not emerged as consensus recommendations. In the absence of such studies, limits of detection afforded by the TM-ASR assay may allow for the use of a single assay at all stages of patient analysis, obviating the need for separate qualitative and quantitative testing.
The upper limit of detection for the TM-ASR assay potentially extends beyond 100,000,000 IU/ml when very high concentration standards are available for calibration. In these studies, we have used a version of the TM-ASR assay, calibrated to an upper limit of 143,000,000 IU/ml, allowing for an evaluation of the distribution of HCV RNA levels encountered in a large reference laboratory setting. We have analyzed the results from 22,399 patient tests in order to establish the natural distribution of HCV RNA levels in random samples submitted for reference testing.
The mean viral load was found to be 5.57 log10 HCV RNA IU/ml, with a median log10 HCV RNA IU/ml level of 5.72. Based on our observed distribution of HCV RNA in actual clinical samples, we estimate that an upper limit of 15,000,000 IU/ml will result in 2% of all samples requiring dilution, while a limit of 20,000,000 will result in 1% of all samples requiring dilution to obtain quantitative results. The practical implication of this observation is that calibration far below the potential upper limit of the TM-ASR assay will allow for quantitation of almost all HCV samples with minimal additional cost and inconvenience incurred by the need for sample dilution. The distribution of HCV viral loads was also compared to results obtained from 4,037 samples tested in the Versant assay. The mean viral load of samples tested in the Versant assay was found to be 5.62 log10 HCV RNA IU/ml with a median log10 HCV RNA IU/ml level of 5.80. The small difference in mean and median viral loads between the two assays derived from very large numbers of samples underscores the issue of calibration and standardization of HCV assays in an era of adopted international standards. The small but real differences between these two assays are also seen in an analysis of the predicted versus required percentage of dilutions required to interrogate the full distribution of patient samples. From the TM-ASR distribution we predict that 44.2% of all samples tested in the Versant assay will be less than 615 HCV RNA IU/ml, but the observed value was 37.6%. In addition, the TM-ASR distribution predicts a dilution and repeat rate of 7.5% of all samples tested in the Versant assay, but the observed value was 1.2%.
These results also underscore the challenge of accurately calibrating quantitative HCV RNA assays. The elaborate process of ASR kit calibration described in the study provides one approach to assay calibration based on reference to the WHO HCV RNA international standard 2003 (96/798). In the absence of abundant quantities of high-titer HCV material, standards were created using a dilution series of an HCV Armored construct (Armored RNA). It should be noted that relatively small differences in WHO HCV RNA international standard 2003 (96/798) or manufacture of calibration material have the potential to introduce substantial inaccuracy into subsequent test results. These facts highlight the importance of replicate testing for these calibration studies and verification of the calibrator concentrations against a known standard on the same studies. With limited availability of WHO HCV RNA international standard 2003 (96/798), the accuracy and stability of any secondary standard used for calibration of ASR, research-use-only, and Food and Drug Administration-cleared assays is critical.
The TM-ASR assay calibrated with Acrometrix material appears to perform equally well with respect to HCV genotypes, with the possible exception of a slight overestimation of HCV RNA level genotype 1 compared to the expected value. The TM-ASR assay appears to perform well with regard to within-run and between-run precision, with the highest percent CV found at lower concentrations of HCV RNA. It should be emphasized that these studies and those performed by other investigators (1, 4) have drawn from a single reagent lot. It will be critical to evaluate the performance of the TM-ASR assay using different lots of reagents. The system evaluated, using the QIAGEN BioRobot 9604 and the QIAamp virus kit in combination with the TM-ASR reagents and COBAS TaqMan analyzer, performs well with respect to contamination control, with only a single known negative sample out of 576 returning a quantitative HCV RNA result. The correlation observed between the Amplicor and Versant assays was relatively good, but it was noted that the Amplicor assay returned slightly higher HCV RNA values than the Versant assay, which is consistent with an overestimation of HCV RNA levels in the Amplicor assay noted previously (13), although it could also be explained by compression of results in the Versant assay.
Overall, the TM-ASR assay agrees poorly with the Amplicor and the Versant assays. The comparison of TM-ASR to both the Versant and Amplicor assays demonstrated an SEE nearly double that observed with the Amplicor and Versant correlation, indicating that a sample tested in both assays can have results that differ by 1.5 log10 5% of the time These observations indicate that results for an individual sample from the different assays cannot be used interchangeably, despite their calibration to the WHO HCV RNA international standard 2003 (96/798). This issue is not unique to the TM-ASR assay but has been described in comparisons of the Versant and Amplicor assays to other assays, including the quantitative HCV core antigen assay (Trak-C; Ortho Clinical Diagnostics, Raritan, N.J.) (16) and the HCV Superquant assay (National Genetics Institute, Los Angeles, CA) (8, 9, 14). The practical implication for patients monitored during combination pegylated interferon and ribavirin treatment is that treatment could be mistakenly altered if the quantitative HCV RNA test used by the laboratory were changed during monitoring. Currently, not all quantitative HCV RNA tests produce equivalent results, and physicians must be educated about these differences to ensure appropriate patient care.
The utility of quantitative HCV RNA measurement is an integral tool for the management of HCV-infected patients. With the advent of real-time PCR assays such as the TM-ASR, the ability to measure the full range of HCV RNA levels encountered in patient samples is possible. In addition, the sensitivity of the TM-ASR assay is approaching the limits of detection of the current qualitative tests, allowing for the possibility of using a single test for diagnosis, monitoring, and end-of-therapy assessment. The 96-well format adopted in the TM-ASR assay allows for increased automation and throughput. However, the performance of the TM-ASR assay hinges on the ability of laboratories to properly calibrate these reagents and opens the possibility of incorrect standardization to the WHO HCV RNA international standard 2003 (96/798).
ACKNOWLEDGMENTS
This study was in compliance with human subjects research regulations and was approved by the University of Utah Institutional Review Board.
Special thanks to the Molecular Hepatitis/Retrovirus Laboratory at Associated Regional and University Pathologists (ARUP) for performing the COBAS Amplicor HCV Monitor V2.0 and Versant HCV RNA bDNA 3.0 assays. QIAGEN virus kits and pipette tips were supplied by QIAGEN Inc., Valencia, CA. COBAS HCV TaqMan ASR reagents and K-tubes were supplied by Roche Molecular Systems (RMS), Pleasanton, CA. The costs of reagents for Versant and Amplicor testing were paid for by RMS. PROBIT analysis was performed by Jenny Birkett at RMS, Pleasanton, CA.
REFERENCES
Barbeau, J. M., J. Goforth, A. M. Caliendo, and F. S. Nolte. 2004. Performance characteristics of a quantitative TaqMan hepatitis C virus RNA analyte-specific reagent. J. Clin. Microbiol. 42:3739-3746.
Dufour, D. R., J. A. Lott, F. S. Nolte, D. R. Gretch, R. S. Koff, and L. B. Seeff. 2000. Diagnosis and monitoring of hepatic injury. I. Performance characteristics of laboratory tests. Clin. Chem. 46:2027-2049.
Elbeik, T., J. Surtihadi, M. Destree, J. Gorlin, M. Holodniy, S. A. Jortani, K. Kuramoto, V. Ng, R. Valdes, Jr., A. Valsamakis, and N. A. Terrault. 2004. Multicenter evaluation of the performance characteristics of the bayer VERSANT HCV RNA 3.0 assay (bDNA). J. Clin. Microbiol. 42:563-569.
Forman, M. S., and A. Valsamakis. 2004. Verification of an assay for quantification of hepatitis C virus RNA by use of an analyte-specific reagent and two different extraction methods. J. Clin. Microbiol. 42:3581-3588.
Fried, M. W., M. L. Shiffman, K. R. Reddy, C. Smith, G. Marinos, F. L. Goncales, Jr., D. Haussinger, M. Diago, G. Carosi, D. Dhumeaux, A. Craxi, A. Lin, J. Hoffman, and J. Yu. 2002. Peginterferon alfa-2a plus ribavirin for chronic hepatitis C virus infection. N. Engl. J. Med. 347:975-982.
Germer, J. J., P. J. Heimgartner, D. M. Ilstrup, W. S. Harmsen, G. D. Jenkins, and R. Patel. 2002. Comparative evaluation of the VERSANT HCV RNA 3.0, QUANTIPLEX HCV RNA 2.0, and COBAS AMPLICOR HCV MONITOR version 2.0 assays for quantification of hepatitis C virus RNA in serum. J. Clin. Microbiol. 40:495-500.
Gleaves, C. A., J. Welle, M. Campbell, T. Elbeik, V. Ng, P. E. Taylor, K. Kuramoto, S. Aceituno, E. Lewalski, B. Joppa, L. Sawyer, C. Schaper, D. McNairn, and T. Quinn. 2002. Multicenter evaluation of the Bayer VERSANT HIV-1 RNA 3.0 assay: analytical and clinical performance. J. Clin. Virol. 25:205-216.
Hadziyannis, E., A. Hadziyannis, B. Yen-Lieberman, M. L. Kiwi, S. Hodnick, F. Spanou, C. Starkey, and Z. M. Younossi. 2001. Hepatitis C virus RNA assays: a comparison of SuperQuant and Monitor. J. Clin. Gastroenterol. 33:45-48.
Konnick, E. Q., M. Erali, E. R. Ashwood, and D. R. Hillyard. 2002. Performance characteristics of the COBAS Amplicor hepatitis C virus (HCV) Monitor, version 2.0, international unit assay and the National Genetics Institute HCV Superquant assay. J. Clin. Microbiol. 40:768-773.
Martinot-Peignoux, M., N. Boyer, V. Le Breton, G. Le Guludec, C. Castelnau, R. Akremi, and P. Marcellin. 2000. A new step toward standardization of serum hepatitis C virus-RNA quantification in patients with chronic hepatitis C. Hepatology 31:726-729.
Murphy, D. G., L. Cote, M. Fauvel, P. Rene, and J. Vincelette. 2000. Multicenter comparison of Roche COBAS AMPLICOR MONITOR version 1.5, Organon Teknika NucliSens QT with Extractor, and Bayer Quantiplex version 3.0 for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 38:4034-4041.
Pawlotsky, J. M. 2002. Use and interpretation of virological tests for hepatitis C. Hepatology 36:S65-S73.
Pisani, G., K. Cristiano, M. Wirz, G. M. Bisso, and G. Gentili. 2002. Overestimation of the hepatitis C virus RNA content of reference preparations by the AMPLICOR HCV Monitor test, version 2.0. J. Clin. Microbiol. 40:4765-4767.
Pradat, P., P. Chossegros, F. Bailly, P. Pontisso, G. Saracco, S. Sauleda, M. Thursz, H. Tillmann, H. Vlassopoulou, A. Alberti, J. H. Braconier, J. I. Esteban, S. Hadziyannis, M. Manns, M. Rizzetto, H. C. Thomas, and C. Trepo. 2000. Comparison between three quantitative assays in patients with chronic hepatitis C and their relevance in the prediction of response to therapy. J. Viral Hepat. 7:203-210.
Sarrazin, C., D. A. Hendricks, F. Sedarati, and S. Zeuzem. 2001. Assessment, by transcription-mediated amplification, of virologic response in patients with chronic hepatitis C virus treated with peginterferon alpha-2a. J. Clin. Microbiol. 39:2850-2855.
Veillon, P., C. Payan, G. Picchio, M. Maniez-Montreuil, P. Guntz, and F. Lunel. 2003. Comparative evaluation of the total hepatitis C virus core antigen, branched-DNA, and Amplicor Monitor assays in determining viremia for patients with chronic hepatitis C during interferon plus ribavirin combination therapy. J. Clin. Microbiol. 41:3212-3220.(Eric Q. Konnick, Sheri M.)
University of Utah Health Sciences Center, Salt Lake City, Utah
ABSTRACT
Performance characteristics of the COBAS hepatitis C virus (HCV) TaqMan analyte-specific reagent (TM-ASR) assay using the QIAGEN BioRobot 9604 for RNA extraction were evaluated and compared to the COBAS Amplicor HCV Monitor V2.0 (Amplicor) and Versant HCV bDNA 3.0 (Versant) assays using clinical samples. Calibration of TM-ASR using Armored RNA allowed determination of the distribution of HCV RNA in clinical samples, using 22,399 clinical samples. Limit of detection, linearity, and inter- and intraassay assay precision were determined for the TM-ASR assay using multiple clinical specimen panels across multiple determinations. Genotype specificity for the TM-ASR assay was determined using samples with different HCV RNA genotypes evaluated and compared against predetermined results. Contamination control of the TM-ASR assay was evaluated using pools of HCV RNA-positive and -negative samples tested in a checkerboard pattern over 12 runs of 96 samples. Correlation of the TM-ASR, Amplicor, and Versant assays was determined using 100 paired clinical samples and Deming regression analysis. The TM-ASR performed well with respect to linearity, precision, and contamination control. The correlation between TM-ASR and the Amplicor and Versant assays was poor, with large differences between assay results for individual samples. Calibration of the TM-ASR assay with Armored RNA allowed for a wide dynamic range and description of the distribution of HCV RNA in clinical samples.
INTRODUCTION
The role of hepatitis C virus (HCV) RNA testing has evolved from early research assays with uncertain utility to well-established production tests routinely used for patient care. Recent recommendations emphasize the use of anti-HCV testing by immunoassay or chemiluminescent immunoassay signal-to-cutoff ratios in combination with HCV RNA or anti-HCV by recombinant immunoblot analysis for the identification of infected patients. Molecular testing for HCV RNA remains an important part of the diagnostic algorithm (2). In addition, quantitative HCV RNA testing has become increasingly used for the management of chronically infected patients. Studies have demonstrated a high likelihood of nonresponsiveness to pegylated interferon-ribavirin combination therapy when a 2-log drop in HCV RNA levels is not observed during the first 12 weeks of therapy (5, 12). This finding allows the physician and patient to reassess treatment. Those that stop treatment are spared significant cost and morbidity associated with long-term HCV drug therapy.
The use of HCV kinetics to guide therapy has highlighted the issue of the dynamic range of HCV quantitative tests. The upper limit of detection, especially for PCR assays based on end-point analysis (COBAS Amplicor, Amplicor, and Superquant) is not sufficient to accurately determine the baseline viral load for a significant percentage of patients at the onset of therapy. For these samples, repeat dilution testing is required to establish an initial viral load (9). Quantitative HCV testing with Versant HCV bDNA V3.0 (Versant; Bayer Diagnostics, Tarrytown, NY) (3) offers a significantly higher upper limit of detection but at the cost of lower sensitivity and potentially specificity at its limit of detection (7, 11).
The development of real-time reverse transcription-PCR technology affords the potential for sensitive, broad-dynamic-range testing in a format that is amenable to automation and improved contamination control. However, commercially available assays that utilize real-time PCR technology have been slow to come to market, and developed "home-brew" assays have been limited. The first commercial manufacturer to market with a real-time PCR platform was Roche Diagnostics (Indianapolis, IN) with the introduction of the COBAS HCV TaqMan analyte-specific reagent (ASR) assay (1, 4). While the COBAS HCV TaqMan ASR (TM-ASR) assay potentially offers many advantages over current commercially available technologies, including expanded dynamic range, high throughput, decreased time to result, increased automation, and improved sensitivity, the user is required to calibrate this assay without manufacturer support.
We evaluated the TM-ASR assay using the QIAGEN BioRobot 9604 instrument (QIAGEN Inc., Valencia, CA) and QIAamp virus kit for RNA isolation. The TM-ASR linearity, precision, and genotype inclusion were evaluated, and the TM-ASR assay was compared to the COBAS Amplicor HCV Monitor V2.0 (Amplicor) and Versant assays. In addition, several methods of calibrating the TM-ASR assay were evaluated using both commercially available material and Armored RNA (Ambion Diagnostics, Austin, TX) referenced directly to the WHO HCV RNA second international standard 2003 (96/798).
MATERIALS AND METHODS
Sample selection. EDTA plasma and serum samples submitted to ARUP Laboratories for the determination of HCV RNA levels by Amplicor were used in this study. Samples tested or results compiled from databases were deidentified prior to use or analysis in this study.
COBAS HCV TaqMan ASR: nucleic acid extraction. The extraction of viral RNA and quantitation standard (QS) RNA was achieved by using the QIAGEN 96 virus protocol performed on the QIAGEN BioRobot 9604 (B configuration, build 12). In brief, the extraction consists of sample (220 μl) lysis by protease in a guanidium, carrier RNA, and QS containing lysis buffer (buffer AL) at an elevated temperature (60°C). Sample lysate was then added to 240 μl ethanol, the solution was transferred to the silica-containing QIAGEN column, and the lysate was removed by applying vacuum. The sample was then washed one time with the first wash buffer (AW1 buffer) and two times with the second wash buffer (AW2 buffer), with wash buffer removed by application of vacuum in all cases. Nuclease-free water (buffer AVE) and centrifugation were used to elute extracted RNA from the column. The procedure was similar to what is described in the package insert, but included the following modifications to introduce the recommended concentration of QS to each sample: 900 μl of HCV ASR QS (Roche Diagnostics Corporation, Indianapolis, IN) was added to one vial of QIAGEN carrier RNA (1,350 μg/vial), and 500 μl of the QS/carrier solution was added to 27 ml of buffer AL, which was split into four aliquots of 6.5 ml each and placed on the deck of the QIAGEN BioRobot 9604. TM-ASR master mix was reconstituted by adding 789 μl of manganese acetate (Roche Diagnostics Corporation) to 1 vial of TM-ASR master mix (lot SL1; Roche Diagnostics Corporation). TM-ASR master mix was split into four vials of 1.5 ml each, and 50 μl was aliquoted into individual K-tubes (Roche Diagnostics Corporation) using the QIAGEN BioRobot 9604. After master mix transfer, 50 μl of eluted sample RNA was distributed to the correct K-tube location on the K-tube plate using the QIAGEN BioRobot 9604.
Amplification and detection. Loaded K-tubes were placed on the COBAS TaqMan 96 analyzer (Roche Diagnostics Corporation), where the K-tubes were robotically capped, placed in K-carriers, and loaded into the correct thermocyclers. Cycling conditions were as described in the TM-ASR technical bulletin.
The Versant HCV bDNA 3.0 and COBAS Amplicor HCV Monitor assays were performed as per the manufacturers' instructions.
Samples greater than 600,000 IU/ml in the Amplicor assay were diluted 1:50 in normal human serum (NHS) and retested. Final results in the Amplicor assay were determined by multiplying the diluted HCV RNA IU/ml result by the dilution factor.
Calibration. The TM-ASR assay requires user calibration of the assay with each new lot of reagents. Calibrators are not supplied with the TM-ASR reagents and are not available directly from the manufacturer. Calibrators are processed in the same manner as patient samples, including nucleic acid extraction and amplification. After extraction and amplification, ASR external calibration software, V1.0 and V2.0 (Roche Molecular Systems, Inc., Pleasanton, CA), was used to establish a relationship between the values of the calibrators assigned by the operator and the elbow values of the QS and target probes. The result of this calibration is a set of three calibration coefficients (a, b, and c), which are then entered into the COBAS TaqMan Analyzer software (TaqLink V1.11) at the beginning of subsequent runs and are used to quantitate the unknown samples.
For this study, calibration of the TM-ASR assay was performed using standards obtained from Acrometrix Inc. (Benicia, CA) and consisted of calibrators containing the following nominal HCV RNA concentrations: 500, 5,000, 50,000, 200,000, 500,000, and 2,000,000 IU/ml. Calibration consisted of assaying each calibrator five times on three separate TM-ASR runs, calculating the calibration coefficients for each run, and then averaging a given calibration coefficient to obtain working calibration coefficients. The latest version (V2.0) of the software allows for data from multiple runs to be incorporated into a single set of calibration coefficients.
Calibration using HCV Armored RNA 1b. In order to determine if the linear range of the TM-ASR assay could be expanded by using higher concentrations of standards, Armored RNA was selected because of the large volume of high-titer material which could be readily obtained. A high-titer sample of HCV Armored RNA genotype 1b was obtained and serially diluted using normal human plasma negative for anti-HCV and HCV RNA. Armored RNA standards contained the following calculated HCV RNA IU/ml concentrations: 143,000,000, 14,300,000, 1,430,000, 143,000, 14,300, 1,430, and 717. Calibration of the TM-ASR using Armored RNA calibrators consisted of RNA extraction with the QIAGEN BioRobot 9604 and QIAamp virus kit and RNA amplification and detection using TM-ASR reagents assayed on the COBAS TaqMan analyzer, with each of the calibrators tested in replicates of eight. Four replicates of the WHO HCV RNA second international standard 2003 (96/798) reconstituted in 2 ml of molecular-grade water (1:4 dilution) were processed concurrently to ensure that the calibration coefficients obtained were directly linked to the current WHO international standard. The calibration run was performed using calibration coefficients determined using Acrometrix calibration material, and the results were normalized to the calculated concentration of HCV RNA 2nd International Standard 2003 (96/798). Actual HCV RNA concentrations of the calibrators were calculated by normalizing the average result of the calibrator with the lowest percent coefficient of variation (CV) to the average HCV RNA IU/ml value of the hepatitis C RNA 2nd International Standard 2003 (96/798) and then calculating the concentrations of the remaining calibrators. The calculated HCV RNA IU/ml concentrations of the calibrators were input into ASR external calibration software, V2.0 (Roche Molecular Systems) with the appropriate .RSF file exported from the TaqLink software V1.11 (Roche Molecular Systems), for the calibration run to generate calibration coefficients. Observed results were back-calculated by using ASR external calibration software, V2.0, and compared against the expected HCV RNA values by linear regression analysis.
Limit of detection. To determine the lowest level of HCV RNA in IU/ml that can be detected by the COBAS TaqMan HCV ASR assay, a six-level clinical specimen low-end dilution panel was obtained from Roche Molecular Systems, Inc., extracted, and amplified in replicate over five runs of 96 samples/run. The panel consisted of an HCV specimen of genotype 1a at concentrations of 0, 10, 25, 35, 50, and 100 IU/ml. Each run consisted of five different HCV panel members (n = 16/level/run), as well as a negative panel member (n = 15/run). Invalid results, defined as any result flagged by the COBAS TaqMan Analyzer as "invalid," were excluded prior to performing calculations. A positive result was considered to be any quantitative result, regardless of the value, or any result of "less than 10 IU/ml." Results which were "target not detected" were considered a negative result. For each of the five different concentration levels, the percent detection rate was calculated (number of detected samples/total number of valid results). PROBIT analysis was used with the preassigned titer values to calculate the concentration giving a 95% detection rate and its 95% confidence interval range.
Linearity and precision. The linearity and assay precision of the TM-ASR assay using the QIAGEN BioRobot 9604 were determined using an eight-member clinical panel provided by Roche Molecular Systems (HCV clinical panel 9905). Panel members were processed, amplified, and detected as described previously. The eight clinical panel members spanned concentrations from approximately 68 to 6,800,000 IU/ml with one negative HCV member. A total of six runs were performed where each positive member of the clinical panel was processed in replicates of 12 in each run and the negative member was processed in replicates of 11 in each run.
Invalid and negative results were excluded prior to performing calculations. The mean titer, standard deviation, and percent CV were determined for each of the seven positive HCV concentration levels. For each of the seven positive levels, nested analysis of variance was performed to obtain estimated intraassay, interassay, and total imprecision. The observed titers were converted to log10 format, and the expected log10 titer versus observed log10 titer was plotted.
Determination of genotype specificity. Genotype inclusively for the TM-ASR assay was determined using clinical samples from eight different genotypes of HCV (1a, 1b, 2b, 3a, 4, 4c/4d, 4h, and 5a) purchased from Millennium Biotech (Ft. Lauderdale, Florida) to determine equivalence of quantitative detection. Samples were extracted, amplified, and detected as previously described. The expected titers for each sample were obtained from Millennium Biotech, and all values were converted to the log10 HCV RNA IU/ml. The differences in the TM-ASR titers versus the expected titers were calculated for each genotype.
Correlation. Assay correlation was determined by processing 100 clinical samples by the Amplicor, Versant, and TM-ASR assays. The selection of samples consisted of 70 samples with titers between 600 IU/ml and 500,000 IU/ml. Fifteen HCV-positive clinical samples had titers less than 600 IU/ml, and 15 samples had titers greater than 500,000 IU/ml. Samples were extracted, amplified, and detected as previously described for each assay. All results were converted to log10 HCV RNA IU/ml, and results were compared through the use of Deming regression.
Contamination control. A total of 576 positive and 576 negative samples were assayed in a "checkerboard" setup across 12 separate TM-ASR runs of 96 samples each. Positive and negative samples used were pools of HCV RNA-positive and NHS samples, respectively. NHS was tested for HCV RNA by COBAS Amplicor HCV assay and was found to be nonreactive, and the positive sample pool was found to have an average HCV RNA titer of 2,595,000 IU/ml (Amplicor measurement with a 1:50 dilution). These experiments used the calibration coefficients generated using Acrometrix calibrators.
Distribution analysis. Data from 22,399 clinical samples tested using TM-ASR and Armored RNA calibration coefficients were gathered through a Microsoft Excel database. Duplicate entries were removed, samples with diluted results were excluded, and only undiluted results were included. Data were stratified in 0.1-log10 increments from 100 to 100,000,000 HCV RNA IU/ml and plotted as percentage of positive samples versus log10 HCV RNA IU/ml. Data from 4,037 Versant HCV bDNA 3.0 samples were collected from the ARUP Network System Repository database, and results were stratified into less than, within, and greater than the reportable range for comparison against the predicted percentage of results expected to fall within these ranges based on the distribution from the TM-ASR distribution.
RESULTS
The TM-ASR calibration coefficients generated from Acrometrix standards were determined over three separate calibration runs by averaging the calibration coefficients calculated on these three runs. The calibration coefficients were as follows: a = –0.0060, b = 0.4532, and c = 0.5929, with calibration coefficient standard deviations of a = 0.00015, b = 0.002635, and c = 0.05397 and percent CVs of a = 2.6%, b = 0.58%, and c = 9.10%.
Inter- and intraassay precision. For the TM-ASR assay, the within-run standard deviation ranged from 0.078 to 0.273 log10 HCV RNA IU/ml and percent CV ranged from 1.56% to 16.5% at 68,000 IU/ml and 68 IU/ml, respectively. Between-run standard deviation log10 HCV RNA IU/ml ranged from 0.054 to 0.174 at 6,800 HCV RNA IU/ml and 6,800,000 IU/ml, respectively. Between-run percent CV ranged from 1.48% to 16.5% at 6,800 HCV RNA IU/ml and 68 HCV RNA IU/ml, respectively. Data are presented in Table 1.
Linearity. The observed titers of the linearity experiments for TM-ASR are presented in Fig. 1a. The regression was as follows: observed = 1.086 x expected – 0.245; n = 800; R2 = 0.999. The linear range of the TM-ASR assay using the Acrometrix calibrators was 680 to 3,400,000 IU/ml (Table 1). Note that as HCV RNA levels increased, the observed HCV RNA level was higher than expected, suggesting that as HCV RNA levels increase above the highest calibrator overestimation occurs.
Limit of detection. The limit of detection was determined for HCV RNA from a clinical specimen low-end dilution panel extracted by the QIAGEN BioRobot 9604 instrument and amplified using TM-ASR reagents on the COBAS TaqMan analyzer. The rate of positive results for a given level was calculated by tabulating the total number of nonnegative results and comparing this value to the total number of valid (QS reactive) results (Table 2). PROBIT analysis performed on these data using the previously determined concentration indicated that 71.3 IU/ml was detected with a 95% confidence range of 55.0 to 105.7 IU/ml.
It was noted that the observed quantitative values for these samples were less than what was expected based upon the previously assigned HCV RNA level. Observed versus assigned HCV RNA levels were compared, and the following were calculated for the observed values: average, standard deviation, percent CV, the actual number of valid results, and the percentage observed versus expected. These data indicate that expected HCV RNA concentrations previously assigned were not consistent with the values obtained empirically, and they are presented in Table 3. In addition, not all replicates produced quantitative values and, in the best case, only 81.3% of the samples assayed had a quantitative result in the TM-ASR assay. Note that the number of valid results is defined as any result where the QS was valid, regardless of the result of the unknown. These data indicate that although the combination of the QIAGEN 96 virus/QIAGEN 9604/TM-ASR assay is capable of detecting 71 IU/ml consistently, the values do not appear to be quantitatively accurate.
Genotype specificity. Samples with genotype 1a seemed to deviate the most from 100% recovery, with the average recovery being 112.9%; genotype 2b samples seemed to perform best, with an average recovery of 102.7%. Note that the expected recovery in these experiments was generated by Millennium Biotech, from which these samples were purchased. No independent quantification was performed on these samples other than HCV TM-ASR. Results are presented in Table 4.
Correlation. Assay correlation was determined by processing 100 clinical samples by the Amplicor, Versant, and TM-ASR assays. The correlation between Amplicor and Versant (Fig. 2a) indicated that HCV RNA levels obtained by the Amplicor assay were slightly higher than those with the Versant assay and displayed compression near the upper limit of the Amplicor assay, with the Deming regression equation Versant = 1.065 x (Amplicor) – 0.446 (R2 = 0.939; n = 80; standard error of the estimate [SEE] = 0.239), and this was similar to previous comparisons (6, 10). The Deming regression between the TM-ASR and Versant assays (Fig. 2b) was TM-ASR = 1.188 x (Versant) – 0.663 (R2 = 0.829; n = 80; SEE = 0.473). The TM-ASR and Amplicor assays (Fig. 2c) Deming regression was TM-ASR = 1.207 x (Amplicor) – 0.919 (R2 = 0.818; n = 86; SEE = 0.544). Also, the TM-ASR and Amplicor assays detected six more samples than the Versant HCV bDNA 3.0 assay, which suggests that the TM-ASR and Amplicor assays are more sensitive than the Versant assay, even though the Amplicor and Versant assays have similar limits of detection claimed by the manufacturers (600 and 615 IU/ml, respectively). Of six samples not detected in the Versant assay, two samples had results of less than 600 IU/ml in both the Amplicor and TM-ASR assays. One sample result was 551 IU/ml in the Amplicor assay and 540 IU/ml in the TM-ASR assay, and the second sample result was 214 IU/ml in the Amplicor assay and 553 IU/ml in the TM-ASR assay. The remaining four samples were greater than 600 IU/ml in both the TM-ASR and Amplicor assays.
Contamination control. Of the 576 positive samples assayed in a "checkerboard" setup across 12 separate runs of 96 samples each, the average HCV RNA titer by TM-ASR was 3,720,000 (6.54 log10) IU/ml versus 2,600,000 (6.41 log10) IU/ml by 1:50 dilution in the COBAS HCV Monitor V2.0 assay. Results of the 576 negative samples included a single NHS sample with an HCV RNA level greater than 100 IU/ml in the TM-ASR assay (3,940 IU/ml). In addition, a single sample was quantified with an HCV RNA level of 68 IU/ml, and six samples returned results of "less than 10 IU/ml." These results indicate that the QIAGEN 9604 virus kit as used on the QIAGEN 9604 instrument paired with the TM-ASR reagents performs well with respect to contamination control using a high-titer specimen interspersed with an HCV RNA-negative specimen.
Armored RNA The TM-ASR calibration coefficients generated using high-titer Armored RNA material were a = –0.0093, b = 0.4657, and c = 0.8129. The average calculated HCV RNA in the 1:4-diluted WHO HCV RNA second international standard 2003 (96/798) was 26,800 (log10 4.42) IU/ml, with a standard deviation of 3,490 (log10 0.06) IU/ml, which agrees well with the expected value of 25,000 (log10 4.40) HCV RNA IU/ml. Linear regression comparing observed versus expected HCV RNA log10 IU/ml (Fig. 1b) showed observed = 1.02 x (expected) – 0.108 (R2 = 0.999; n = 56).
Distribution. Distribution data for the TM-ASR assay are presented in Fig. 3a, with the predicted ranges of the Amplicor and Versant assays shown. These data indicate a skewed normal distribution for samples tested with HCV RNA detectable from 100 to 100,000,000 IU/ml. Distribution data indicated that 43.0% of samples tested had results of less than 100 IU/ml. Analysis of TM-ASR results within the range of the Versant assay (615 to 8,000,000 HCV RNA IU/ml) found average, standard deviation, and median log10 HCV RNA IU/ml values of 5.57, 0.90, and 5.72, respectively. The distribution for the Versant assay is presented in Fig. 3b and indicates that the observed percentage of samples with HCV RNA results less than and greater than the analytical measurement range are less than those predicted by the observed HCV RNA distribution by TM-ASR. The TM-ASR distribution predicts that 44.2% of all samples tested in the Versant assay will fall below the detection limit, while the observed value was 37.6% (n = 1,518). Similarly, the TM-ASR distribution predicts that 7.5% of all samples tested in the Versant assay will be above the analytical measurement range, while the observed value was 1.2% (n = 50). The average, standard deviation, and median log10 HCV RNA IU/ml values for samples quantitated in the Versant assay were 5.62, 0.85, and 5.80, respectively
DISCUSSION
Measurement of HCV RNA has a clearly established role for confirming the diagnosis of infection, monitoring the course of therapy, and establishing the outcome and durability of therapy. These different clinical indications place separate demands on molecular assays measuring HCV RNA. Diagnostic and end-of-treatment testing must be sensitive enough to detect low levels of HCV RNA but does not require quantitative accuracy across a broad range of HCV RNA concentrations. In contrast, current therapeutic "stopping" algorithms require accurate quantification across the broad range of HCV RNA levels seen in infected patients. An assay that is both sensitive and has a broad dynamic range would be ideal.
The TM-ASR assay as described offers substantial improvements in throughput and contamination control over current quantitative HCV RNA assays, while approaching the sensitivity of qualitative HCV RNA assays. These studies document a detection limit of 71 IU/ml for the TM-ASR assay using PROBIT analysis and samples containing a low concentration of HCV RNA. The TM-ASR assay is observed to be only slightly less sensitive than the Food and Drug Administration-approved COBAS Amplicor HCV V2.0 qualitative test with its detection limit of 50 IU/ml (12). This is a substantial improvement over previous quantitative assays, including both the COBAS Amplicor HCV Monitor V2.0 (600 IU/ml) and Versant HCV bDNA V3.0 (615 IU/ml) assays. Although the argument has been made that more sensitive testing may have a value in the analysis of end-therapeutic response (15), definitive evidence is lacking in the literature to justify use of ultrasensitive assays. At this time, algorithms for patient management based on such testing have not emerged as consensus recommendations. In the absence of such studies, limits of detection afforded by the TM-ASR assay may allow for the use of a single assay at all stages of patient analysis, obviating the need for separate qualitative and quantitative testing.
The upper limit of detection for the TM-ASR assay potentially extends beyond 100,000,000 IU/ml when very high concentration standards are available for calibration. In these studies, we have used a version of the TM-ASR assay, calibrated to an upper limit of 143,000,000 IU/ml, allowing for an evaluation of the distribution of HCV RNA levels encountered in a large reference laboratory setting. We have analyzed the results from 22,399 patient tests in order to establish the natural distribution of HCV RNA levels in random samples submitted for reference testing.
The mean viral load was found to be 5.57 log10 HCV RNA IU/ml, with a median log10 HCV RNA IU/ml level of 5.72. Based on our observed distribution of HCV RNA in actual clinical samples, we estimate that an upper limit of 15,000,000 IU/ml will result in 2% of all samples requiring dilution, while a limit of 20,000,000 will result in 1% of all samples requiring dilution to obtain quantitative results. The practical implication of this observation is that calibration far below the potential upper limit of the TM-ASR assay will allow for quantitation of almost all HCV samples with minimal additional cost and inconvenience incurred by the need for sample dilution. The distribution of HCV viral loads was also compared to results obtained from 4,037 samples tested in the Versant assay. The mean viral load of samples tested in the Versant assay was found to be 5.62 log10 HCV RNA IU/ml with a median log10 HCV RNA IU/ml level of 5.80. The small difference in mean and median viral loads between the two assays derived from very large numbers of samples underscores the issue of calibration and standardization of HCV assays in an era of adopted international standards. The small but real differences between these two assays are also seen in an analysis of the predicted versus required percentage of dilutions required to interrogate the full distribution of patient samples. From the TM-ASR distribution we predict that 44.2% of all samples tested in the Versant assay will be less than 615 HCV RNA IU/ml, but the observed value was 37.6%. In addition, the TM-ASR distribution predicts a dilution and repeat rate of 7.5% of all samples tested in the Versant assay, but the observed value was 1.2%.
These results also underscore the challenge of accurately calibrating quantitative HCV RNA assays. The elaborate process of ASR kit calibration described in the study provides one approach to assay calibration based on reference to the WHO HCV RNA international standard 2003 (96/798). In the absence of abundant quantities of high-titer HCV material, standards were created using a dilution series of an HCV Armored construct (Armored RNA). It should be noted that relatively small differences in WHO HCV RNA international standard 2003 (96/798) or manufacture of calibration material have the potential to introduce substantial inaccuracy into subsequent test results. These facts highlight the importance of replicate testing for these calibration studies and verification of the calibrator concentrations against a known standard on the same studies. With limited availability of WHO HCV RNA international standard 2003 (96/798), the accuracy and stability of any secondary standard used for calibration of ASR, research-use-only, and Food and Drug Administration-cleared assays is critical.
The TM-ASR assay calibrated with Acrometrix material appears to perform equally well with respect to HCV genotypes, with the possible exception of a slight overestimation of HCV RNA level genotype 1 compared to the expected value. The TM-ASR assay appears to perform well with regard to within-run and between-run precision, with the highest percent CV found at lower concentrations of HCV RNA. It should be emphasized that these studies and those performed by other investigators (1, 4) have drawn from a single reagent lot. It will be critical to evaluate the performance of the TM-ASR assay using different lots of reagents. The system evaluated, using the QIAGEN BioRobot 9604 and the QIAamp virus kit in combination with the TM-ASR reagents and COBAS TaqMan analyzer, performs well with respect to contamination control, with only a single known negative sample out of 576 returning a quantitative HCV RNA result. The correlation observed between the Amplicor and Versant assays was relatively good, but it was noted that the Amplicor assay returned slightly higher HCV RNA values than the Versant assay, which is consistent with an overestimation of HCV RNA levels in the Amplicor assay noted previously (13), although it could also be explained by compression of results in the Versant assay.
Overall, the TM-ASR assay agrees poorly with the Amplicor and the Versant assays. The comparison of TM-ASR to both the Versant and Amplicor assays demonstrated an SEE nearly double that observed with the Amplicor and Versant correlation, indicating that a sample tested in both assays can have results that differ by 1.5 log10 5% of the time These observations indicate that results for an individual sample from the different assays cannot be used interchangeably, despite their calibration to the WHO HCV RNA international standard 2003 (96/798). This issue is not unique to the TM-ASR assay but has been described in comparisons of the Versant and Amplicor assays to other assays, including the quantitative HCV core antigen assay (Trak-C; Ortho Clinical Diagnostics, Raritan, N.J.) (16) and the HCV Superquant assay (National Genetics Institute, Los Angeles, CA) (8, 9, 14). The practical implication for patients monitored during combination pegylated interferon and ribavirin treatment is that treatment could be mistakenly altered if the quantitative HCV RNA test used by the laboratory were changed during monitoring. Currently, not all quantitative HCV RNA tests produce equivalent results, and physicians must be educated about these differences to ensure appropriate patient care.
The utility of quantitative HCV RNA measurement is an integral tool for the management of HCV-infected patients. With the advent of real-time PCR assays such as the TM-ASR, the ability to measure the full range of HCV RNA levels encountered in patient samples is possible. In addition, the sensitivity of the TM-ASR assay is approaching the limits of detection of the current qualitative tests, allowing for the possibility of using a single test for diagnosis, monitoring, and end-of-therapy assessment. The 96-well format adopted in the TM-ASR assay allows for increased automation and throughput. However, the performance of the TM-ASR assay hinges on the ability of laboratories to properly calibrate these reagents and opens the possibility of incorrect standardization to the WHO HCV RNA international standard 2003 (96/798).
ACKNOWLEDGMENTS
This study was in compliance with human subjects research regulations and was approved by the University of Utah Institutional Review Board.
Special thanks to the Molecular Hepatitis/Retrovirus Laboratory at Associated Regional and University Pathologists (ARUP) for performing the COBAS Amplicor HCV Monitor V2.0 and Versant HCV RNA bDNA 3.0 assays. QIAGEN virus kits and pipette tips were supplied by QIAGEN Inc., Valencia, CA. COBAS HCV TaqMan ASR reagents and K-tubes were supplied by Roche Molecular Systems (RMS), Pleasanton, CA. The costs of reagents for Versant and Amplicor testing were paid for by RMS. PROBIT analysis was performed by Jenny Birkett at RMS, Pleasanton, CA.
REFERENCES
Barbeau, J. M., J. Goforth, A. M. Caliendo, and F. S. Nolte. 2004. Performance characteristics of a quantitative TaqMan hepatitis C virus RNA analyte-specific reagent. J. Clin. Microbiol. 42:3739-3746.
Dufour, D. R., J. A. Lott, F. S. Nolte, D. R. Gretch, R. S. Koff, and L. B. Seeff. 2000. Diagnosis and monitoring of hepatic injury. I. Performance characteristics of laboratory tests. Clin. Chem. 46:2027-2049.
Elbeik, T., J. Surtihadi, M. Destree, J. Gorlin, M. Holodniy, S. A. Jortani, K. Kuramoto, V. Ng, R. Valdes, Jr., A. Valsamakis, and N. A. Terrault. 2004. Multicenter evaluation of the performance characteristics of the bayer VERSANT HCV RNA 3.0 assay (bDNA). J. Clin. Microbiol. 42:563-569.
Forman, M. S., and A. Valsamakis. 2004. Verification of an assay for quantification of hepatitis C virus RNA by use of an analyte-specific reagent and two different extraction methods. J. Clin. Microbiol. 42:3581-3588.
Fried, M. W., M. L. Shiffman, K. R. Reddy, C. Smith, G. Marinos, F. L. Goncales, Jr., D. Haussinger, M. Diago, G. Carosi, D. Dhumeaux, A. Craxi, A. Lin, J. Hoffman, and J. Yu. 2002. Peginterferon alfa-2a plus ribavirin for chronic hepatitis C virus infection. N. Engl. J. Med. 347:975-982.
Germer, J. J., P. J. Heimgartner, D. M. Ilstrup, W. S. Harmsen, G. D. Jenkins, and R. Patel. 2002. Comparative evaluation of the VERSANT HCV RNA 3.0, QUANTIPLEX HCV RNA 2.0, and COBAS AMPLICOR HCV MONITOR version 2.0 assays for quantification of hepatitis C virus RNA in serum. J. Clin. Microbiol. 40:495-500.
Gleaves, C. A., J. Welle, M. Campbell, T. Elbeik, V. Ng, P. E. Taylor, K. Kuramoto, S. Aceituno, E. Lewalski, B. Joppa, L. Sawyer, C. Schaper, D. McNairn, and T. Quinn. 2002. Multicenter evaluation of the Bayer VERSANT HIV-1 RNA 3.0 assay: analytical and clinical performance. J. Clin. Virol. 25:205-216.
Hadziyannis, E., A. Hadziyannis, B. Yen-Lieberman, M. L. Kiwi, S. Hodnick, F. Spanou, C. Starkey, and Z. M. Younossi. 2001. Hepatitis C virus RNA assays: a comparison of SuperQuant and Monitor. J. Clin. Gastroenterol. 33:45-48.
Konnick, E. Q., M. Erali, E. R. Ashwood, and D. R. Hillyard. 2002. Performance characteristics of the COBAS Amplicor hepatitis C virus (HCV) Monitor, version 2.0, international unit assay and the National Genetics Institute HCV Superquant assay. J. Clin. Microbiol. 40:768-773.
Martinot-Peignoux, M., N. Boyer, V. Le Breton, G. Le Guludec, C. Castelnau, R. Akremi, and P. Marcellin. 2000. A new step toward standardization of serum hepatitis C virus-RNA quantification in patients with chronic hepatitis C. Hepatology 31:726-729.
Murphy, D. G., L. Cote, M. Fauvel, P. Rene, and J. Vincelette. 2000. Multicenter comparison of Roche COBAS AMPLICOR MONITOR version 1.5, Organon Teknika NucliSens QT with Extractor, and Bayer Quantiplex version 3.0 for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 38:4034-4041.
Pawlotsky, J. M. 2002. Use and interpretation of virological tests for hepatitis C. Hepatology 36:S65-S73.
Pisani, G., K. Cristiano, M. Wirz, G. M. Bisso, and G. Gentili. 2002. Overestimation of the hepatitis C virus RNA content of reference preparations by the AMPLICOR HCV Monitor test, version 2.0. J. Clin. Microbiol. 40:4765-4767.
Pradat, P., P. Chossegros, F. Bailly, P. Pontisso, G. Saracco, S. Sauleda, M. Thursz, H. Tillmann, H. Vlassopoulou, A. Alberti, J. H. Braconier, J. I. Esteban, S. Hadziyannis, M. Manns, M. Rizzetto, H. C. Thomas, and C. Trepo. 2000. Comparison between three quantitative assays in patients with chronic hepatitis C and their relevance in the prediction of response to therapy. J. Viral Hepat. 7:203-210.
Sarrazin, C., D. A. Hendricks, F. Sedarati, and S. Zeuzem. 2001. Assessment, by transcription-mediated amplification, of virologic response in patients with chronic hepatitis C virus treated with peginterferon alpha-2a. J. Clin. Microbiol. 39:2850-2855.
Veillon, P., C. Payan, G. Picchio, M. Maniez-Montreuil, P. Guntz, and F. Lunel. 2003. Comparative evaluation of the total hepatitis C virus core antigen, branched-DNA, and Amplicor Monitor assays in determining viremia for patients with chronic hepatitis C during interferon plus ribavirin combination therapy. J. Clin. Microbiol. 41:3212-3220.(Eric Q. Konnick, Sheri M.)