Comparison of Two Human Immunodeficiency Virus (HIV) RNA Surrogate Assays to the Standard HIV RNA Assay
http://www.100md.com
微生物临床杂志 2005年第12期
Rush Medical College, Chicago, Illinois
University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
The Macfarlane Burnet Institute for Medical Research and Public Health, Melbourne, Australia
New England Research Institute, Watertown, Massachusetts
University of Zurich, Zurich, Switzerland
University of Witwatersrand and National Health Laboratory Services, Parktown, South Africa
Center for Disease Control and Prevention, Atlanta, Georgia
Institute of Microbiology, Medical School, University of Genoa, Genoa, Italy
MTC/SMI, Karolinska Institute, Stockholm, Sweden
Uppsala University, Uppsala, Sweden
National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland
ABSTRACT
Human immunodeficiency virus (HIV) RNA testing is the gold standard for monitoring antiretroviral therapy in HIV-infected patients. However, equipment and reagent costs preclude widespread use of the assay in resource-limited settings. The Perkin-Elmer Ultrasensitive p24 assay and the Cavidi Exavir Load assay both offer potentially simpler, less costly technologies for monitoring viral load. These assays were compared to the Roche Amplicor HIV-1 Monitor Test, v1.5, using panels of clinical samples (subtype B) from HIV-positive subjects and HIV-spiked samples (subtypes A, C, D, CRF_01AE, CRF_02AG, and F). The Ultrasensitive p24 assay detected 100% of the spiked samples with virus loads of >250,000copies/ml and 61% of the clinical samples with virus loads of 219 to 288,850 copies/ml. Detection rates were improved substantially if an external lysis buffer was added to the procedure. The Cavidi assay detected 54 to 100% of spiked samples with virus loads >10,000 copies/ml and 68% of the clinical samples. These detection rates were also greatly improved with a newly implemented version of this kit. Coefficients of variation demonstrate good reproducibility for each of these kits. The results from the Cavidi v1.0, Cavidi v2.0, and Perkin-Elmer, and the Perkin-Elmer Plus external buffers all correlated well with the results from the Roche Monitor Test (r = 0.83 to 0.96, r = 0.84 to 0.99, r = 0.58 to 0.67, and r = 0.59 to 0.95, respectively). Thus, the use of these two assays for monitoring patients, together with less-frequent confirmation testing, offers a feasible alternative to frequent HIV RNA testing in resource-limited settings.
INTRODUCTION
The human immunodeficiency virus (HIV) pandemic has affected countries worldwide, but the impact on resource-limited countries has been especially devastating. Pressure to lower the cost of antiretroviral therapies (ART) has been critical in fighting this battle. The current challenge is to identify simplified assays for monitoring patients on ART that are less expensive and less technically demanding with respect to facilities and instrumentation (6, 12). In the past, the compromise for using simplified methods has often been reduced sensitivity or poor correlation with the gold standards used in industrialized settings (4, 10). We compare here two commercially available kits that measure HIV-specific proteins, p24 and reverse transcriptase (RT), respectively, and utilize simpler technologies to perform the tests. The first method is the Ultrasensitive HIV p24 enzyme-linked immunosorbent assay (ELISA; HIV-1 p24 ELISA plus the ELAST ELISA Amplification System; Perkin-Elmer Life Sciences, Inc.), and the second is the RT assay (Exavir Load Assay; Cavidi Tech AB, Sweden). Both kits offer less expensive alternatives for detecting HIV.
For the Ultrasensitive p24 assay, a standard ELISA format is used for the capture and detection of HIV p24 coupled with a specific signal amplification to increase the assay sensitivity. Heat denaturation of the plasma prior to binding in the ELISA step helps to dissociate immune complexes and denature the antibodies (24) so that they no longer compete for binding to the p24 antigen, a phenomenon that has plagued prior versions of this assay (10, 19, 25). The assay also adds a kinetic readout using the Quanti-Kin Detection System Software that was developed and validated in O. E. Varnier's laboratory (8) to increase the linear range of the assay. HIV p24 concentrations are reported as femtograms of HIV-1 p24/ml of plasma (R.I.L.A.B., S.r.l., Genoa, Italy).
For the RT assay (Cavidi), the RT enzyme is separated from the virus particle using a solid-phase extraction manifold, and the amount of RT enzyme is quantified using a functional assay whereby the RT incorporates bromodeoxyuridine (BrdU) monophosphate into DNA using a poly(A) template bound to a 96-well plate (3, 16). BrdU is then quantified spectrophotometrically using anti-BrdU conjugated to alkaline phosphatase, followed by the addition of its substrate. The RT activity in the unknown sample is compared to that of a recombinant RT enzyme standard with a known concentration. The extrapolated result is reported as fg of RT/ml of plasma or as HIV-1 RNA equivalents/ml based on a conversion factor supplied by the manufacturer. The assay has undergone revisions (17) to improve sensitivity and results for both versions of the kit are reported here.
We sought here to demonstrate the potential utility of measuring HIV p24 or RT activity in plasma by comparing detection rates and sensitivity and correlating the surrogate marker with HIV RNA detection.
(These data were presented in part as a poster on 14 to 17 September 2003 at the 43rd Annual Interscience Conference on Antimicrobial Agents and Chemotherapy Meeting, Chicago, Ill. [control #3803].)
MATERIALS AND METHODS
Specimens. The Virology Quality Assessment (VQA) Program created three panels to evaluate the RT (Cavidi) and the Ultrasensitive p24 (Perkin-Elmer) assays (Table 1). The panels were tested in multiple laboratories (Table 2). The first panel consisted of replicates that were prepared by seeding a well-characterized VQA HIV-1 subtype B stock into sodium citrate-treated HIV-seronegative plasma at defined concentrations (39). The panel was tested by the VQA and University of North Carolina at Chapel Hill (UNC) laboratories using the RT assays (Cavidi v1.0 and v2.0), the Ultrasensitive p24 kit (Perkin-Elmer, using kit lysis buffer), a "modified" version of the p24 kit utilizing an external lysis buffer (EB, kindly provided by Jorg Schupbach), and the HIV RT-PCR assay (Roche). Viral loads for each member of this panel were determined by taking the median of the data obtained from the HIV RT-PCR assays (Roche).
The second panel consisted of plasma from 18 HIV-infected donors enrolled in the VQA donor program (subtype B HIV) and stocks of the NED panel (11) seeded into HIV-seronegative plasma (HIV subtypes A, C, D, CRF_01AE, CRF_02AG, and F). Clinical specimens were included in panel 2 to provide testing with samples that could contribute both virion-associated and non-virion-associated p24 antigen and help to clarify how these two components affect the detection rate of each assay. Spiked samples were used for non-B subtypes due to a lack of available clinical specimens. Eight laboratories tested the second panel one to five times per kit (Table 2). The VQA and UNC laboratories tested the panel using both buffers with the Ultrasensitive p24 (Perkin-Elmer) and the v1.0 RT assay (Cavidi). The remaining laboratories tested the panel once using only one of the assays. Viral loads for this panel ranged from 2.34 to 5.46 log10 HIV RNA copies/ml as defined by triplicate testing performed by the VQA laboratories using the HIV RT-PCR assay (Roche).
The third panel consisted of spiked samples (subtype B HIV). This panel was designed to evaluate the improved sensitivity in the newly enhanced RT assay (Cavidi, v2.0). The range in viral load for this panel was 0 to 10,000 HIV RNA copies/ml with an emphasis on replicates at the low end (100 to 6,000 HIV RNA copies/ml). Testing was performed by the VQA only using the v2.0 RT assay (Cavidi).
Roche Amplicor HIV-1 Monitor Test, v1.5. HIV RNA testing was performed as defined in the package insert. For the standard extraction method, virus from 0.2 ml of plasma was lysed in the kit lysis buffer and the HIV RNA was precipitated using isopropanol and pelleted by centrifugation. After being washed with ethanol, the RNA was resuspended in the kit dilution buffer. Extracted RNA was amplified and detected according to the manufacturer's instructions, and results were reported as HIV RNA copies/ml (or log10 converted results). The equipment that is necessary to perform the Roche Amplicor Monitor Test is listed in Table 3. The estimated cost for this equipment is $13,200 (U.S. dollars).
Cavidi ExaVirLoad assay. Samples tested with the RT assay (Cavidi) were handled according to the manufacturer's directions. For v1.0, the RT from 1 ml of plasma was treated with ion-exchange gel (16) to help facilitate capture using a vacuum driven solid-phase extraction manifold provided by the manufacturer for use with their kit. After the virus particles were washed with kit wash buffer solution, the viral particles were lysed with the kit lysis buffer and the RT-containing eluate was collected. The extracted RT enzyme was loaded into a poly(A)-coated 96-well plate in 2 volumes (15 and 75 μl). A recombinant RT enzyme, which was provided in the kit for generating the standard curve, was also diluted and added to the plate in 2 volumes. Buffers containing reaction components were added to the plate, followed by incubation at 33°C overnight. After a washing step, the plate was developed colorimetrically and read on a spectrophotometer. The results were reported as fg of HIV RT/ml of plasma and as HIV RNA equivalents/ml of plasma (only HIV RNA equivalent results are presented in this study).
Version 2.0 of the RT assay (Cavidi) was modified to improve assay sensitivity (17). Briefly, the virus particles from 1 ml of plasma were coated, captured, and lysed in 330 μl of lysis buffer instead of 600 μl of buffer to concentrate the RT in the eluate; additionally, more RT (from the samples and standards) was added to the 96-well plate (30 and 150 μl of eluate instead of 15 and 75 μl, respectively). Finally, in an attempt to streamline and improve linearity of the assay, day 2 was a day of extended incubation rather than washing and labeling, and day 3 was used for the detection of the assay. Even though the assay still took 3 days, day 2 did not require any hands-on activity. The equipment that is necessary to perform the Cavidi Exavir Load Assay is listed in Table 3. The estimated cost for this equipment is $8,300 (U.S. dollars).
Perkin-Elmer HIV-1 p24 ELISA plus the ELAST ELISA Amplification System. The Ultrasensitive p24 assay (Perkin-Elmer) was performed according to the manufacturer's instructions using the kit lysis buffer. Briefly, the sample (50 μl of plasma) was lysed with 250 μl of prediluted kit lysis buffer, heated at 100°C for 5 min, and cooled to room temperature. A volume of 250 μl of specimen, control, or standard was then added to the respective wells of a 96-well plate. The plate was incubated at 4°C overnight and then washed in an automated plate washer. The captured p24 antigen was labeled with a biotinylated anti-p24 antibody, followed by a streptavidin-horseradish peroxidase treatment step. The biotinyl-tyramide reagent (provided in the ELAST kit) was used to "amplify" the streptavidin-coated bound antigen and was followed by a second streptavidin-horseradish peroxidase step that was then developed colorimetrically and read kinetically for 10 min, followed by a final reading after 30 min when the reaction was stopped with concentrated acid. Quantitative results were reported based on the algorithms incorporated into the p24 assay Quanti-Kin software.
The modified version (Perkin-Elmer plus EB) of this assay was performed according to the manufacturer's instructions, with one deviation. The specimen (50 μl) was first incubated with 25 μl of the external buffer (30 mM Tris-HCl [pH 7.2], 450 mM NaCl, 1.5% Triton X-100, 1.5% deoxycholic acid [sodium salt], 0.3% sodium dodecyl sulfate, 10 mM EDTA) at room temperature for 10 min (26). This sample was then treated with 225 μl of prediluted kit lysis buffer and heat denatured as described above. Quantitative results were also reported based on the algorithms incorporated into the p24 assay Quanti-Kin software. The equipment that is necessary to perform the Perkin-Elmer Ultrasensitive p24 test is listed in Table 3. The estimated cost for this equipment is $6,800 (U.S. dollars).
Statistics. Detection rates were based on the kit-defined limits. For the RT assay (Cavidi) and the Ultrasensitive p24 assay (Perkin-Elmer), no limit of detection is set, but instead each assay's limit of detection is defined based on the combined performance of negative or background specimens and the standard curve. The RT-PCR assay (Roche) limit of detection is 400 copies HIV RNA/ml.
Comparisons of reproducibility were based on coefficients of variation (CVs) because the assays measure different analytes. The mean and standard deviation was estimated from a model that allowed for left censoring where some of the results were below the limit of detection of an assay. A CV was calculated only if at least two positive results were obtained at a given concentration because unbiased estimates of the mean and standard deviation could not be obtained with fewer than two. The CVs for panel 1 were based on the intra-assay standard deviation.
Rank correlations were calculated for each panel in an attempt to provide an estimate of the relationship between the median log10 results obtained with the RT (Cavidi v1.0 and v2.0) assays or the Ultrasensitive p24 (Perkin-Elmer and Perkin-Elmer + EB) assays and the Roche assay or nominal concentrations. A separate correlation was calculated for each run of each assay in each laboratory. Negative samples were excluded from all analyses. The results that were below the limit of detection of an assay were assigned a value of 1.0. On panels 1 and 3, results from the various assays were correlated with nominal concentrations, but for panel 2 the results were correlated with the median RT-PCR (Roche) result because it included clinical samples that did not have a corresponding nominal concentration. On panel 1, correlations with nominal concentrations were calculated for the Ultrasensitive p24 (Perkin-Elmer plus EB) for nominal concentrations greater than 10,000 copies/ml and for the RT assay (Cavidi v1.0 and v2.0) and the RT-PCR assay (Roche) at all positive concentrations. The results from the Ultrasensitive p24 (Perkin-Elmer) were excluded because positive results were only obtained at the two highest concentrations. On panel 2, correlations were calculated for the Ultrasensitive p24 (Perkin-Elmer and Perkin-Elmer plus EB) and the RT assays (Cavidi v1.0 and v2.0). The table includes minimum, median, and maximum correlations for each assay, as well as the number of correlations calculated (i.e., the number of runs available).
RESULTS
Detection rates for panels 1 to 3 are presented in Table 4. These panels were designed to be detectable by the standard HIV RT-PCR (Roche) assay (>400 copies/ml). One clinical sample in panel 2 had a median virus load of 219 copies/ml, and one replicate of this sample was undetectable by the HIV RT-PCR assay (the negative result was excluded from the median calculation).
The Cavidi v1.0 RT assay detected 98% of the specimens in panel 1 with virus loads of 10,000 copies/ml or greater, while the improved RT assay (Cavidi v2.0) detected 100% of the specimens in panel 1 with virus loads of 500 copies/ml or more. For panel 2, the Cavidi v2.0 RT assay also outperformed the v1.0 assay. Detection rates improved from 9 to 100% for the v1.0 assay to 27 to 100% for the v2.0 assay. For panel 3, the Cavidi v2.0 RT assay detected 100% of the specimens with virus loads of 400 copies/ml or greater.
For panel 1, the Ultrasensitive p24 assay (Perkin-Elmer) detected 100% of the specimens with virus loads of 250,000 copies/ml or greater; the addition of the external buffer increased the detection rate to 100% for samples with virus loads of 50,000 copies/ml or greater. Detection rates for the Ultrasensitive p24 (Perkin-Elmer) assay were also improved in panel 2, when the external buffer was included. Detection of samples from panel 2 ranged from 0 to 86% for the Ultrasensitive p24 assay, but improved to 66 to 99% when the external buffer was used in the procedure. Better sensitivity was observed in clinical specimens than in spiked samples, regardless of the buffer used.
For panel 2, the combination of intra- and interassay variation was used because the absence of replication within the panel prevented estimation of the intra-assay standard deviation. For panel 1, median CVs were 31, 31, 27, 27, and 13% for the RT-PCR (Roche), RT (Cavidi v1.0), RT (Cavidi v2.0), Ultrasensitive p24 (Perkin-Elmer), and modified p24 (Perkin-Elmer plus EB) assays, respectively. For panel 2, the median CVs were 26, 26, 17, 18, and 57%.
Correlation values between RT-PCR (Roche) and nominal concentrations were very good, indicating that the expected virus load of the panel members was very close to that which was obtained by RNA testing (Table 5). The Ultrasensitive p24 assay (Perkin-Elmer) gave reasonable correlation values with HIV RNA concentration, but this correlation was improved when an external buffer was used (Perkin-Elmer plus EB). Furthermore, better correlations between RT-PCR and Ultrasensitive p24 values were observed when panel members included only spiked samples versus spiked and clinical samples. The RT assays (Cavidi v1.0 and v2.0) correlated very well with virus load and did not seem to vary if the panel included spiked or clinical samples.
DISCUSSION
Monitoring HIV disease in resource-rich countries involves extensive clinical testing (6, 12). HIV RNA testing by RT-PCR and CD4 testing by flow cytometry are both used to track the status of the viral infection and the progression of immune destruction. Safety testing to monitor drug toxicities and general health status and genotypic and phenotypic testing to monitor individuals with failing regimens in order to detect the emergence of drug resistance add to the burden of monitoring patients. Although tight control of HIV infection has proven beneficial in reducing mortality and morbidity in HIV-infected individuals, cost restraints limit the testing that can be performed in many of the countries where the burden of HIV infection is greatest. Strategies need to be developed for less intensive monitoring, while not compromising the outcome for the patient.
The search for a simplified assay to measure HIV has identified two potential candidates. The first, the Ultrasensitive p24 (Perkin-Elmer) assay, measures the HIV-specific core protein, which may be present as virion-associated or free p24 antigen, although free p24 is mostly bound in immune complexes (25, 27). The correlation between p24 detection and HIV RNA or CD4 cell counts has been evaluated in a number of studies. Schupbach and others have demonstrated that the p24 is predictive of CD4 depletion (14, 28) and disease progression (14, 35), that it decreases when ART is initiated (2, 23, 29), and that it correlates inversely with CD4 changes even when HIV RNA is so low that it cannot be quantitated (30, 31). However, disappointing results have also been reported using this kit (1, 21). Bonard et al. tested plasma samples from 14 patients treated with highly active antiretroviral therapy, found weaker changes in Ultrasensitive p24 antigen testing compared to HIV RNA results, and questioned the reliability of this assay as an RNA surrogate for clinical management of HIV-infected patients. Prado et al. demonstrated a lack of correlation between HIV RNA and the Ultrasensitive p24 assay (Perkin-Elmer) for monitoring patients undergoing a structured treatment interruption. Prado et al. found that discordance between HIV RNA detection and p24 detection after viral rebound was 76% of the treated patients and 49% of the naive patients enrolled in the study. The discordance was attributed to the presence of high-affinity immunoglobulins that interfered with the p24 assay detection. More recently, Schupbach et al. reported that p24 antigen was detectable and remained relatively unchanged during short STI (31). These contradictory findings demonstrate the importance of using standardize methods in attempts to compare assays and demonstrate clinical utility. The two earlier studies utilized the Ultrasensitive p24 kits in absence of the modified external buffer, while the third study included it. Our data demonstrate that the modified buffer does impact the sensitivity of the assay; fully optimized assays should be used for future studies trying to demonstrate utility of this assay for monitoring HIV disease progression.
The p24 data obtained in the current study need to be interpreted carefully. The panels utilized contained two distinct types of samples, those that were spiked with well-characterized virus and those that were obtained from HIV-infected individuals. The reality is that clinical samples will typically contain p24 that is both not virion associated and virion associated. However, in attempting to evaluate the performance of a particular assay there are benefits to using a well-characterized specimen, including the ability to decipher the contribution of immune-complexed versus virion-associated p24 antigen in the detection rates of this assay.
Limits of detection were quite high in samples where only virion-associated p24 was present (i.e., HIV spiked samples). It is estimated that 2,000 to 4,000 p24 Gag proteins are present in each virion, so it would take 6,150 to 12,500 virus particles or 12,500 to 25,000 copies of viral RNA to detect 1 pg of p24 protein (5). Since the limit of detection of the modified p24 assay is approximately 6 pg/ml, it is not surprising that the observed detection rate in spiked samples is approximately 100,000 copies/ml. When clinical samples were tested, the detection rate appeared to be greatly improved, and this is presumed be due to the presence of non-virion-associated p24. Since the contribution of this component may vary between patients (4, 25) and change with disease progression (13, 14, 35), the impact on HIV monitoring is not fully understood and explains the lower correlation with HIV RNA. HIV p24 detection rates were improved with the use of the external buffer (kindly provided by Jorg Schupbach), especially in HIV-spiked samples, where only virion-associated proteins were present. The improved sensitivity with spiked samples suggests that the buffer may help to facilitate dissociation of the protein from the virus particle. Although the added external buffer also appeared to improve the detection of non-B subtypes, a more likely explanation would be that improved detection of the non-B subtype samples was due to improved assay sensitivity.
CV data demonstrate that comparable results were obtained between laboratories with the p24 assay. Correlations between the p24 assay and HIV RNA were lower when the kit buffer was used (r = 0.58 to 0.67) than when the external buffer was used (r = 0.59 to 0.95). Furthermore, correlations were higher when only spiked samples were included. The correlations with HIV RNA reported in this study were not different than those reported (r = 0.48 to 0.69) in other studies which did not utilize the external buffer (20, 22). Respess et al. did document a difference in correlation between untreated (r = 0.69) and treated (r = 0.48) patients, similar to the difference noted in this study between spiked and clinical specimens. This may be attributed to the fact that treatment alters the presence of virion-associated and non-virion-associated p24 antigen that is available for detection. Although more data are need to demonstrate the utility of the p24 assay for monitoring patients on ART, preliminary data suggest that it may prove useful for pediatric HIV diagnosis (15, 22, 37, 33) or diagnosis of acute infection (20). The fact that a small sample volume is required is especially attractive for pediatric diagnosis, and ongoing studies are under way to evaluate the potential for using this assay for detection of p24 in blood spots.
The second candidate to be considered as a surrogate for HIV RNA testing is the RT (Cavidi) assay. A number of studies in Africa, Australia, and France have provided excellent correlations between the RT and HIV RNA assays (3, 9, 34, 36). The fact that the assay only monitors virion-associated RT contributes to this excellent correlation. In addition, the assay could be useful in the detection of other lentiviruses (18), including HIV-2, and the extracted RT may be used in the Cavidi HIV Phenotype RT kit for evaluating resistance in patients on anti-RT regimens (32, 38). The simplified equipment, which includes a standard spectrophotometer, a vacuum driven extraction manifold, and wash buckets for plate washing, make this assay especially attractive for resource-limited urban and rural settings. The assay result may be reported as fg of RT/ml of plasma, but the use of a conversion factor may be used to convert the result into a more familiar readout of HIV RNA equivalents/ml. While the utility of this assay for monitoring patients on ART has yet to be fully evaluated, the roll-out of the v2.0 assay, with its improved sensitivity, should have a major impact. A limiting factor of this assay is sample volume. While a 1-ml plasma input is not problematic for monitoring adult patients, it certainly may be problematic for monitoring pediatric patients. Smaller sample volumes (down to 0.25 ml) may be utilized; however, this reduces assay sensitivity by the dilution factor involved (data from Cavidi).
The RT assay (Cavidi v1.0) showed good detection rates for samples with viral loads of >10,000 copies/ml. The v2.0 assay improved this detection limit to 400 copies/ml, which is comparable to that observed with the HIV RT-PCR assay (Roche) routinely used in many industrialized countries. CV analyses demonstrated comparable reproducibility between the RT and RNA assays. Comparable results were also obtained between the laboratories performing the testing, suggesting the assay is quite robust. Higher CVs were noted in the v2.0 assay than in the v1.0 assay. This was mainly observed in samples with lower viral loads, a phenomenon that has also been noted in HIV RNA assays for samples with viral loads near the limit of detection (7). Correlation between HIV RNA (Roche) and the RT (Cavidi) results were excellent for both versions of the kit (r = 0.83 to 0.99) and did not vary per panel. These results are quite similar to those reported by other investigators (9, 34, 36) using both versions of the RT assay (Cavidi). Preliminary studies involving non-B subtypes (34, 36), suggest that clades CRF02_AG, C, and B are detected well with this assay, but more studies are needed to substantiate these observations and better define detection rates of other clades. Factors that could affect detection rates include the presence of RT inhibitors or resistance mutations in the RT gene. The data presented by Greengrass et al. (9) suggest that the presence of non-nucleoside RT inhibitor-associated resistance mutations may reduce the RT activity by 0.20 log10, but the presence or absence of nucleoside RT inhibitor resistance mutations was not associated with RT activity. More data are needed to confirm this observed effect on RT activity and its impact on monitoring HIV-infected patients.
The benefits of using HIV RNA surrogates are assay simplicity and cost reduction. While kit and reagent costs will vary by country (RT-PCR = $17 to $80, RT assay = $20, Ultrasensitive p24 = $5 to $10), equipment requirements for the HIV RNA surrogate assays are simpler and ca. 40 to 50% less expensive. In addition, although HIV RNA assays require well-developed infrastructures to ensure sterility and containment of nucleic acids and amplified products, the HIV RNA surrogate assays have more simplistic requirements with respect to facilities since nucleic acid extractions and amplification are not part of the assay. Therefore, the use of an HIV RNA surrogate assay to screen patients for drug efficacy and compliance, followed by less-frequent confirmation testing with molecular assays, may provide a useful strategy in resource-poor settings. The two assays evaluated in this study offer feasible RNA surrogates and should be considered for use in this type of strategy.
ACKNOWLEDGMENTS
This study was supported by NIAID contract NO-AI-85354.
We thank Joseph Fitzgibbon of the National Institutes of Health for helpful comments and careful review of the manuscript.
REFERENCES
Bonard, D., F. Rouet, T. A. Toni, A. Minga, C. Huet, D. K. Ekouevi, F. Dabis, R. Salamon, C. Rouzioux, et al. 2003. Field evaluation of an improved assay using heat-dissociated p24 antigen for adults mainly infected with HIV-1 CRF02_AG strains in Cote d'Ivoire, West Africa. J. Acquir. Immune. Defic. Syndr. 34:267-273.
Boni, J., M. Opravil, Z. Tomasik, M. Rothen, L. Bisset, P. J. Grob, R. Luthy, and J. Schupbach. 1997. Simple monitoring of antiretroviral therapy with a signal-amplification-boosted HIV-1 p24 antigen assay with heat-denatured plasma. AIDS 11:F47-F52.
Braun, J., J.-C. Plantier, M.-F. Hellot, E. Tuaillon, M. Gueudin, F. Damond, A. Malmsten, G. E. Corrigan, and F. Simon. 2003. A new quantitative HIV load assay based on plasma virion reverse transcriptase activity for different types, groups and subtypes. AIDS 17:331-336.
Brown, C., R. Kline, L. Atibu, H. Francis, R. Ryder, and T. C. Quinn. 1991. Prevalence of HIV-1 p24 antigenaemia in African and North American populations and correlation with clinical status. AIDS 5:89-92.
Coffin, J. M. 1996. Retroviridae: the viruses and their replication, p. 1767-1847. In B. N. Fields, D. M. Knipe, and P. M. Howley (ed.), Fields virology, 3rd ed. Lippincott-Raven, Philadelphia, Pa.
Crowe, S., S. Turnbull, R. Oelrichs, and A. Dunne. 2003. Monitoring of human immunodeficiency virus infection in resource-constrained countries. CID 37(Suppl. 1):S25-S35.
Erice, A., D. Brambilla, B. Jackson, R. Kokka, B. Yen-Lieberman, and R. Coombs. 2000. Performance characteristics of the Quantiplex HIV-1 RNA 3.0 assay for the detection and quantitation of HIV-1 RNA in plasma. J. Clin. Microbiol. 38:2837-2845.
Giacomini, M., J. L. McDermott, A. A. Giri, I. Martini, F. B. Lillo, and O. E. Varnier. 1998. A novel and innovative kinetic software for virologic colorimetric assays. J. Virol. Methods 73:201-209.
Greengrass, V. L., S. P. Turnbull, J. Hocking, A. L. Dunne, G. Tachedjian, G. E. Corrigan, and S. M. Crowe. 2005. Evaluation of a low cost reverse transcriptase assay for plasma HIV-1 viral load monitoring. Curr. HIV Res. 3:183-190.
Hammer, S., C. Crumpacker, R. D'Aquila, B. Jackson, J. Lathey, D. Livnat, and P. I. Reichelderfer. 1993. Use of virologic assays for detection of human immunodeficiency virus in clinical trials: recommendations of the AIDS Clinical Trials Group Virology Committee. J. Clin. Microbiol. 31:2557-2564.
Huang, D. D., T. A. Giesler, and J. W. Bremer. Sequence characterization of the protease and partial reverse transcriptase proteins of the NED panel, an international HIV type 1 subtype reference and standards panel. AIDS Res. Hum. Retrovir. 19:321-328.
Kent, D. M., D. McGrath, J. P. A. Ioannidis, and M. L. Bennish. 2003. Suitable monitoring approaches to antiretroviral therapy in resource-poor settings: setting the research agenda. CID 37(Suppl. 1):S13-S24.
Lange, J. M., D. A. Paul, H. G. Huisman, F. de Wolf, H. van den Berg, R. A. Coutinho, S. A. Danner, J. van der Noordaa, and J. Goudsmit. 1986. Persistent HIV antigenaemia and decline of HIV core antibodies associated with transition to AIDS. BMJ 293:1459-1462.
Ledergerber, B., M. Flepp, J. Boni, Z. Tomasik, R. W. Cone, R. Luthy, and J. Schupbach. 2000. Human immunodeficiency virus type 1 p24 concentration measured by boosted ELISA of heat-denatured plasma correlates with decline in CD4 cells, progression to AIDS and survival: comparison with viral RNA measurement. J. Infect. Dis. 181:1280-1288.
Lyamuya, E., U. Bredberg-Raden, A. Massawe, E. Urassa, G. Kawo, G. Msemo, T. Kazimoto, A. Ostborn, K. Karlsson, F. Mhalu, and G. Biberfeld. 1996. Performance of a modified HIV-1 p24 antigen assay for early diagnosis of HIV-1 infection in infants and prediction of mother-to-infant transmission of HIV-1 in Dar es Salaam, Tanzania. J. Acquir. Immune. Defic. Syndr. Hum. Retrovir. 12:421-426.
Malmsten, A., X. W. Shao, K. Aperia, G. E. Corrigan, E. Sandstrom, C. F. Kallander, T. Leitner, and J. S. Gronowitz. 2003. HIV-1 viral load determination based on reverse transcriptase activity recovered from human plasma. J. Med. Virol. 71:347-359.
Malmsten, A., X. W. Shao, S. Sjodahl, E. L. Fredriksson, I. Pettersson, T. Leitner, C. F. R. Kallander, E. Sandstrom, and J. S. Gronowitz. 2005. Improved HIV-1 viral load determination based on reverse transcriptase activity recovered from human plasma. J. Med. Virol. 76:291-296.
Malmsten, A., D. H. Ekstrand, L. Akerblom, J. S. Gronowitz, C. F. Kallander, M. Bendinelli, and D. Matteucci. 1998. A colorimetric reverse transcriptase assay optimized for Moloney murine leukemia virus, and its use for characterization of reverse transcriptases of unknown identity. J. Virol. Methods 75:9-20.
Nishanian, P., K. R. Huskins, S. Stehn, R. Detels, and J. L. Fahey. 1990. A simple method for improved assay demonstrates that p24 antigen is present as immune complexes in most sera from infected individuals. J. Infect. Dis. 162:21-28.
Pascual, A., A. Cachafeiro, M. L. Funk, and S. A. Fiscus. 2002. Comparison of an assay using signal amplification of the heat-dissociated p24 antigen with the Roche Monitor human immunodeficiency virus RNA assay. J. Clin. Microbiol. 40:2472-2475.
Prado, J. G., A. Shintani, M. Bofill, B. Clotet, L. Ruiz, and J. Martinez-Picado. 2004. Lack of longitudinal intrapatient correlation between p24 antigenemia and levels of human immunodeficiency virus (HIV) type 1 RNA in patients with chronic HIV infection during structured treatment interruptions. J. Clin. Microbiol. 42:1620-1625.
Respess, R. A., A. Cachafeiro, D. Withum, S. A. Fiscus, D. Newman, B. Branson, O. E. Varnier, K. Lewis, and T. J. Dondero. 2005. Evaluation of an ultrasensitive p24 antigen assay as a potential alternative to human immunodeficiency virus type 1 RNA viral load assay in resource-limited settings. J. Clin. Microbiol. 43:506-508.
Ribas, S. G., P. Ondoa, J. Schupbach, G. van der Groen, and K. Fransen. 2003. Performance of a quantitative human immunodeficiency virus type 1 p24 antigen assay on various HIV-1 subtypes for the follow-up of human immunodeficiency type 1-seropositive individuals. J. Virol. Methods 113:29-34.
Schupbach, J., and J. Boni. 1993. Quantitative and sensitive detection of immune-complexed and free HIV antigen after boiling of serum. J. Virol. Methods 43:247-256. (Erratum, 45:245.)
Schupbach, J. 2003. Viral RNA and p24 antigen as markers of HIV disease and antiretroviral treatment success. Int. Arch. Allergy Immunol. 132:196-209.
Schupbach, J., J. Boni, L. R. Bisset, Z. Tomasik, M. Fischer, H. F. Gunthard, B. Ledergerber, M. Opravil, et al. 2003. HIV-1 p24 antigen is a significant inverse correlate of CD4 T-cell change in patients with suppressed viremia under long-term antiretroviral therapy. J. Acquir. Immune Defic. Syndr. 33:292-299.
Schupbach, J. 2002. Measurement of HIV-1 p24 antigen by signal-amplification-boosted ELISA of heat-denatured plasma is a simple and in expensive alternative to tests for viral RNA. AIDS Rev. 4:83-92.
Schupbach, J., Z. Tomasik, D. Nadal, B. Ledergerber, M. Flepp, M. Opravil, and J. Boni. 2000. Use of HIV-1 p24 as a sensitive, precise and inexpensive marker for infection, disease progression and treatment failure. Int. J. Antimicrob. Agents 16:441-445.
Schupbach, J., J. Boni, M. Flepp, Z. Tomasik, H. Joller, and M. Opravil. 2001. Antiretroviral treatment monitoring with an improved HIV-1 p24 antigen test: an in expensive alternative to tests for viral RNA. J. Med. Virol. 65:225-232.
Schupbach, J., J. Boni, L. R. Bisset, Z. Tomasik, M. Fischer, H. F. Gunthard, B. Ledergerber, M. Opravil, et al. 2003. HIV-1 p24 antigen is a significant inverse correlate of CD4 T-cell change in patients with suppressed viremia under long-term antiretroviral therapy. J. Acquir. Immune Defic. Syndr. 33:292-299.
Schupbach, J., H. Gunthard, B. Joos, M. Fischer, J. Boni, Z. Tomasik, S. Yerly, L. Perrin, M. Battegay, H. Furrer, P. Vernazza, E. Bernasconi, B. Hirschel, et al. 2005. HIV-1 p24 may persist during long-term HAART, increases little during short treatment breaks, and its rebound after treatment stop correlates with the CD4+ loss. J. Acquir. Immune. Defic. Syndr. 40:250-256.
Shao, X. W., A. Malmsten, J. Lennerstrand, A. Sonnerborg, T. Unge, J. S. Gronowitz, and C. F. R. Kallander. 2003. Use of HIV-1 reverse transcriptase recovered from human plasma for phenotypic drug susceptibility testing. AIDS. 17:1463-1471.
Sherman, G. G., G. Stevens, and W. S. Stevens. 2004. Affordable diagnosis of human immunodeficiency virus infection in infants by p24 antigen detection. Pediatr. Infect. Dis. J. 23:173-176.
Sivapalasingam, S. S. Essajee, P. N. Nyambi, V. Itri, B. Hanna, R. Holzman, and F. Valentine. 2005. Human immunodeficiency virus (HIV) reverse transcriptase activity correlates with HIV RNA load: implications for resource-limited settings. J. Clin. Microbiol. 43:3793-3796.
Sterling, T. R., D. R. Hoover, J. Astemborski, D. Vlahov, J. G. Bartlett, and J. Schupbach. 2002. Heat-denatured human immunodeficiency virus type 1 protein 24 antigen: prognostic value in adults with early-stage disease. J. Infect. Dis. 186:1181-1185.
Stevens, G., N. Rekhviashvili, L. E. Scott, R. Gonin, and W. Stevens. 2005. Evaluation of two commercially available, inexpensive alternative HIV-1 viral load assays in HIV-infected subtype C patients from South Africa: the HiSen HIV-1 p24 antigen Ultra kit and the Cavidi ExaVir load HIV-RT assay. J. Clin. Microbiol. 43:857-861.
Sutthent, R., N. Gaudart, K. Chokpaibulkit, N. Tanliang, C. Kanoksinsombath, and P. Chaisilwatana. 2003. P24 antigen detection assay modified with a booster step for diagnosis and monitoring of human immunodeficiency virus type 1 infection. J. Clin. Microbiol. 41:1016-1022.
Tuaillion, E., M. Gueudin, V. Lemee, I. Gueit, P. Roques, G. E. Corrigan, J.-C. Plantier, F. Simon, and J. Braun. 2004. Phenotypic susceptibility to nonnucleoside inhibitors of virion-associated reverse transcriptase from different HIV types and groups. J. Acquir. Immune Defic. Syndr. 37:1543-1549.
Yen-Lieberman, B., D. Brambilla, B. Jackson, J. Bremer, R. Coombs, M. Cronin, S. Herman, D. Katzenstein, S. Leung, H. J. Lin, P. Palumbo, S. Rasheed, J. Todd, M. Vahey, and P. Reichelderfer. 1996. Evaluation of a quality assurance program for quantitation of human immunodeficiency virus type 1 RNA in plasma by the AIDS Clinical Trials Group virology laboratories. J. Clin. Microbiol. 34:2695-2701.(Cheryl Jennings, Susan A.)
University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
The Macfarlane Burnet Institute for Medical Research and Public Health, Melbourne, Australia
New England Research Institute, Watertown, Massachusetts
University of Zurich, Zurich, Switzerland
University of Witwatersrand and National Health Laboratory Services, Parktown, South Africa
Center for Disease Control and Prevention, Atlanta, Georgia
Institute of Microbiology, Medical School, University of Genoa, Genoa, Italy
MTC/SMI, Karolinska Institute, Stockholm, Sweden
Uppsala University, Uppsala, Sweden
National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland
ABSTRACT
Human immunodeficiency virus (HIV) RNA testing is the gold standard for monitoring antiretroviral therapy in HIV-infected patients. However, equipment and reagent costs preclude widespread use of the assay in resource-limited settings. The Perkin-Elmer Ultrasensitive p24 assay and the Cavidi Exavir Load assay both offer potentially simpler, less costly technologies for monitoring viral load. These assays were compared to the Roche Amplicor HIV-1 Monitor Test, v1.5, using panels of clinical samples (subtype B) from HIV-positive subjects and HIV-spiked samples (subtypes A, C, D, CRF_01AE, CRF_02AG, and F). The Ultrasensitive p24 assay detected 100% of the spiked samples with virus loads of >250,000copies/ml and 61% of the clinical samples with virus loads of 219 to 288,850 copies/ml. Detection rates were improved substantially if an external lysis buffer was added to the procedure. The Cavidi assay detected 54 to 100% of spiked samples with virus loads >10,000 copies/ml and 68% of the clinical samples. These detection rates were also greatly improved with a newly implemented version of this kit. Coefficients of variation demonstrate good reproducibility for each of these kits. The results from the Cavidi v1.0, Cavidi v2.0, and Perkin-Elmer, and the Perkin-Elmer Plus external buffers all correlated well with the results from the Roche Monitor Test (r = 0.83 to 0.96, r = 0.84 to 0.99, r = 0.58 to 0.67, and r = 0.59 to 0.95, respectively). Thus, the use of these two assays for monitoring patients, together with less-frequent confirmation testing, offers a feasible alternative to frequent HIV RNA testing in resource-limited settings.
INTRODUCTION
The human immunodeficiency virus (HIV) pandemic has affected countries worldwide, but the impact on resource-limited countries has been especially devastating. Pressure to lower the cost of antiretroviral therapies (ART) has been critical in fighting this battle. The current challenge is to identify simplified assays for monitoring patients on ART that are less expensive and less technically demanding with respect to facilities and instrumentation (6, 12). In the past, the compromise for using simplified methods has often been reduced sensitivity or poor correlation with the gold standards used in industrialized settings (4, 10). We compare here two commercially available kits that measure HIV-specific proteins, p24 and reverse transcriptase (RT), respectively, and utilize simpler technologies to perform the tests. The first method is the Ultrasensitive HIV p24 enzyme-linked immunosorbent assay (ELISA; HIV-1 p24 ELISA plus the ELAST ELISA Amplification System; Perkin-Elmer Life Sciences, Inc.), and the second is the RT assay (Exavir Load Assay; Cavidi Tech AB, Sweden). Both kits offer less expensive alternatives for detecting HIV.
For the Ultrasensitive p24 assay, a standard ELISA format is used for the capture and detection of HIV p24 coupled with a specific signal amplification to increase the assay sensitivity. Heat denaturation of the plasma prior to binding in the ELISA step helps to dissociate immune complexes and denature the antibodies (24) so that they no longer compete for binding to the p24 antigen, a phenomenon that has plagued prior versions of this assay (10, 19, 25). The assay also adds a kinetic readout using the Quanti-Kin Detection System Software that was developed and validated in O. E. Varnier's laboratory (8) to increase the linear range of the assay. HIV p24 concentrations are reported as femtograms of HIV-1 p24/ml of plasma (R.I.L.A.B., S.r.l., Genoa, Italy).
For the RT assay (Cavidi), the RT enzyme is separated from the virus particle using a solid-phase extraction manifold, and the amount of RT enzyme is quantified using a functional assay whereby the RT incorporates bromodeoxyuridine (BrdU) monophosphate into DNA using a poly(A) template bound to a 96-well plate (3, 16). BrdU is then quantified spectrophotometrically using anti-BrdU conjugated to alkaline phosphatase, followed by the addition of its substrate. The RT activity in the unknown sample is compared to that of a recombinant RT enzyme standard with a known concentration. The extrapolated result is reported as fg of RT/ml of plasma or as HIV-1 RNA equivalents/ml based on a conversion factor supplied by the manufacturer. The assay has undergone revisions (17) to improve sensitivity and results for both versions of the kit are reported here.
We sought here to demonstrate the potential utility of measuring HIV p24 or RT activity in plasma by comparing detection rates and sensitivity and correlating the surrogate marker with HIV RNA detection.
(These data were presented in part as a poster on 14 to 17 September 2003 at the 43rd Annual Interscience Conference on Antimicrobial Agents and Chemotherapy Meeting, Chicago, Ill. [control #3803].)
MATERIALS AND METHODS
Specimens. The Virology Quality Assessment (VQA) Program created three panels to evaluate the RT (Cavidi) and the Ultrasensitive p24 (Perkin-Elmer) assays (Table 1). The panels were tested in multiple laboratories (Table 2). The first panel consisted of replicates that were prepared by seeding a well-characterized VQA HIV-1 subtype B stock into sodium citrate-treated HIV-seronegative plasma at defined concentrations (39). The panel was tested by the VQA and University of North Carolina at Chapel Hill (UNC) laboratories using the RT assays (Cavidi v1.0 and v2.0), the Ultrasensitive p24 kit (Perkin-Elmer, using kit lysis buffer), a "modified" version of the p24 kit utilizing an external lysis buffer (EB, kindly provided by Jorg Schupbach), and the HIV RT-PCR assay (Roche). Viral loads for each member of this panel were determined by taking the median of the data obtained from the HIV RT-PCR assays (Roche).
The second panel consisted of plasma from 18 HIV-infected donors enrolled in the VQA donor program (subtype B HIV) and stocks of the NED panel (11) seeded into HIV-seronegative plasma (HIV subtypes A, C, D, CRF_01AE, CRF_02AG, and F). Clinical specimens were included in panel 2 to provide testing with samples that could contribute both virion-associated and non-virion-associated p24 antigen and help to clarify how these two components affect the detection rate of each assay. Spiked samples were used for non-B subtypes due to a lack of available clinical specimens. Eight laboratories tested the second panel one to five times per kit (Table 2). The VQA and UNC laboratories tested the panel using both buffers with the Ultrasensitive p24 (Perkin-Elmer) and the v1.0 RT assay (Cavidi). The remaining laboratories tested the panel once using only one of the assays. Viral loads for this panel ranged from 2.34 to 5.46 log10 HIV RNA copies/ml as defined by triplicate testing performed by the VQA laboratories using the HIV RT-PCR assay (Roche).
The third panel consisted of spiked samples (subtype B HIV). This panel was designed to evaluate the improved sensitivity in the newly enhanced RT assay (Cavidi, v2.0). The range in viral load for this panel was 0 to 10,000 HIV RNA copies/ml with an emphasis on replicates at the low end (100 to 6,000 HIV RNA copies/ml). Testing was performed by the VQA only using the v2.0 RT assay (Cavidi).
Roche Amplicor HIV-1 Monitor Test, v1.5. HIV RNA testing was performed as defined in the package insert. For the standard extraction method, virus from 0.2 ml of plasma was lysed in the kit lysis buffer and the HIV RNA was precipitated using isopropanol and pelleted by centrifugation. After being washed with ethanol, the RNA was resuspended in the kit dilution buffer. Extracted RNA was amplified and detected according to the manufacturer's instructions, and results were reported as HIV RNA copies/ml (or log10 converted results). The equipment that is necessary to perform the Roche Amplicor Monitor Test is listed in Table 3. The estimated cost for this equipment is $13,200 (U.S. dollars).
Cavidi ExaVirLoad assay. Samples tested with the RT assay (Cavidi) were handled according to the manufacturer's directions. For v1.0, the RT from 1 ml of plasma was treated with ion-exchange gel (16) to help facilitate capture using a vacuum driven solid-phase extraction manifold provided by the manufacturer for use with their kit. After the virus particles were washed with kit wash buffer solution, the viral particles were lysed with the kit lysis buffer and the RT-containing eluate was collected. The extracted RT enzyme was loaded into a poly(A)-coated 96-well plate in 2 volumes (15 and 75 μl). A recombinant RT enzyme, which was provided in the kit for generating the standard curve, was also diluted and added to the plate in 2 volumes. Buffers containing reaction components were added to the plate, followed by incubation at 33°C overnight. After a washing step, the plate was developed colorimetrically and read on a spectrophotometer. The results were reported as fg of HIV RT/ml of plasma and as HIV RNA equivalents/ml of plasma (only HIV RNA equivalent results are presented in this study).
Version 2.0 of the RT assay (Cavidi) was modified to improve assay sensitivity (17). Briefly, the virus particles from 1 ml of plasma were coated, captured, and lysed in 330 μl of lysis buffer instead of 600 μl of buffer to concentrate the RT in the eluate; additionally, more RT (from the samples and standards) was added to the 96-well plate (30 and 150 μl of eluate instead of 15 and 75 μl, respectively). Finally, in an attempt to streamline and improve linearity of the assay, day 2 was a day of extended incubation rather than washing and labeling, and day 3 was used for the detection of the assay. Even though the assay still took 3 days, day 2 did not require any hands-on activity. The equipment that is necessary to perform the Cavidi Exavir Load Assay is listed in Table 3. The estimated cost for this equipment is $8,300 (U.S. dollars).
Perkin-Elmer HIV-1 p24 ELISA plus the ELAST ELISA Amplification System. The Ultrasensitive p24 assay (Perkin-Elmer) was performed according to the manufacturer's instructions using the kit lysis buffer. Briefly, the sample (50 μl of plasma) was lysed with 250 μl of prediluted kit lysis buffer, heated at 100°C for 5 min, and cooled to room temperature. A volume of 250 μl of specimen, control, or standard was then added to the respective wells of a 96-well plate. The plate was incubated at 4°C overnight and then washed in an automated plate washer. The captured p24 antigen was labeled with a biotinylated anti-p24 antibody, followed by a streptavidin-horseradish peroxidase treatment step. The biotinyl-tyramide reagent (provided in the ELAST kit) was used to "amplify" the streptavidin-coated bound antigen and was followed by a second streptavidin-horseradish peroxidase step that was then developed colorimetrically and read kinetically for 10 min, followed by a final reading after 30 min when the reaction was stopped with concentrated acid. Quantitative results were reported based on the algorithms incorporated into the p24 assay Quanti-Kin software.
The modified version (Perkin-Elmer plus EB) of this assay was performed according to the manufacturer's instructions, with one deviation. The specimen (50 μl) was first incubated with 25 μl of the external buffer (30 mM Tris-HCl [pH 7.2], 450 mM NaCl, 1.5% Triton X-100, 1.5% deoxycholic acid [sodium salt], 0.3% sodium dodecyl sulfate, 10 mM EDTA) at room temperature for 10 min (26). This sample was then treated with 225 μl of prediluted kit lysis buffer and heat denatured as described above. Quantitative results were also reported based on the algorithms incorporated into the p24 assay Quanti-Kin software. The equipment that is necessary to perform the Perkin-Elmer Ultrasensitive p24 test is listed in Table 3. The estimated cost for this equipment is $6,800 (U.S. dollars).
Statistics. Detection rates were based on the kit-defined limits. For the RT assay (Cavidi) and the Ultrasensitive p24 assay (Perkin-Elmer), no limit of detection is set, but instead each assay's limit of detection is defined based on the combined performance of negative or background specimens and the standard curve. The RT-PCR assay (Roche) limit of detection is 400 copies HIV RNA/ml.
Comparisons of reproducibility were based on coefficients of variation (CVs) because the assays measure different analytes. The mean and standard deviation was estimated from a model that allowed for left censoring where some of the results were below the limit of detection of an assay. A CV was calculated only if at least two positive results were obtained at a given concentration because unbiased estimates of the mean and standard deviation could not be obtained with fewer than two. The CVs for panel 1 were based on the intra-assay standard deviation.
Rank correlations were calculated for each panel in an attempt to provide an estimate of the relationship between the median log10 results obtained with the RT (Cavidi v1.0 and v2.0) assays or the Ultrasensitive p24 (Perkin-Elmer and Perkin-Elmer + EB) assays and the Roche assay or nominal concentrations. A separate correlation was calculated for each run of each assay in each laboratory. Negative samples were excluded from all analyses. The results that were below the limit of detection of an assay were assigned a value of 1.0. On panels 1 and 3, results from the various assays were correlated with nominal concentrations, but for panel 2 the results were correlated with the median RT-PCR (Roche) result because it included clinical samples that did not have a corresponding nominal concentration. On panel 1, correlations with nominal concentrations were calculated for the Ultrasensitive p24 (Perkin-Elmer plus EB) for nominal concentrations greater than 10,000 copies/ml and for the RT assay (Cavidi v1.0 and v2.0) and the RT-PCR assay (Roche) at all positive concentrations. The results from the Ultrasensitive p24 (Perkin-Elmer) were excluded because positive results were only obtained at the two highest concentrations. On panel 2, correlations were calculated for the Ultrasensitive p24 (Perkin-Elmer and Perkin-Elmer plus EB) and the RT assays (Cavidi v1.0 and v2.0). The table includes minimum, median, and maximum correlations for each assay, as well as the number of correlations calculated (i.e., the number of runs available).
RESULTS
Detection rates for panels 1 to 3 are presented in Table 4. These panels were designed to be detectable by the standard HIV RT-PCR (Roche) assay (>400 copies/ml). One clinical sample in panel 2 had a median virus load of 219 copies/ml, and one replicate of this sample was undetectable by the HIV RT-PCR assay (the negative result was excluded from the median calculation).
The Cavidi v1.0 RT assay detected 98% of the specimens in panel 1 with virus loads of 10,000 copies/ml or greater, while the improved RT assay (Cavidi v2.0) detected 100% of the specimens in panel 1 with virus loads of 500 copies/ml or more. For panel 2, the Cavidi v2.0 RT assay also outperformed the v1.0 assay. Detection rates improved from 9 to 100% for the v1.0 assay to 27 to 100% for the v2.0 assay. For panel 3, the Cavidi v2.0 RT assay detected 100% of the specimens with virus loads of 400 copies/ml or greater.
For panel 1, the Ultrasensitive p24 assay (Perkin-Elmer) detected 100% of the specimens with virus loads of 250,000 copies/ml or greater; the addition of the external buffer increased the detection rate to 100% for samples with virus loads of 50,000 copies/ml or greater. Detection rates for the Ultrasensitive p24 (Perkin-Elmer) assay were also improved in panel 2, when the external buffer was included. Detection of samples from panel 2 ranged from 0 to 86% for the Ultrasensitive p24 assay, but improved to 66 to 99% when the external buffer was used in the procedure. Better sensitivity was observed in clinical specimens than in spiked samples, regardless of the buffer used.
For panel 2, the combination of intra- and interassay variation was used because the absence of replication within the panel prevented estimation of the intra-assay standard deviation. For panel 1, median CVs were 31, 31, 27, 27, and 13% for the RT-PCR (Roche), RT (Cavidi v1.0), RT (Cavidi v2.0), Ultrasensitive p24 (Perkin-Elmer), and modified p24 (Perkin-Elmer plus EB) assays, respectively. For panel 2, the median CVs were 26, 26, 17, 18, and 57%.
Correlation values between RT-PCR (Roche) and nominal concentrations were very good, indicating that the expected virus load of the panel members was very close to that which was obtained by RNA testing (Table 5). The Ultrasensitive p24 assay (Perkin-Elmer) gave reasonable correlation values with HIV RNA concentration, but this correlation was improved when an external buffer was used (Perkin-Elmer plus EB). Furthermore, better correlations between RT-PCR and Ultrasensitive p24 values were observed when panel members included only spiked samples versus spiked and clinical samples. The RT assays (Cavidi v1.0 and v2.0) correlated very well with virus load and did not seem to vary if the panel included spiked or clinical samples.
DISCUSSION
Monitoring HIV disease in resource-rich countries involves extensive clinical testing (6, 12). HIV RNA testing by RT-PCR and CD4 testing by flow cytometry are both used to track the status of the viral infection and the progression of immune destruction. Safety testing to monitor drug toxicities and general health status and genotypic and phenotypic testing to monitor individuals with failing regimens in order to detect the emergence of drug resistance add to the burden of monitoring patients. Although tight control of HIV infection has proven beneficial in reducing mortality and morbidity in HIV-infected individuals, cost restraints limit the testing that can be performed in many of the countries where the burden of HIV infection is greatest. Strategies need to be developed for less intensive monitoring, while not compromising the outcome for the patient.
The search for a simplified assay to measure HIV has identified two potential candidates. The first, the Ultrasensitive p24 (Perkin-Elmer) assay, measures the HIV-specific core protein, which may be present as virion-associated or free p24 antigen, although free p24 is mostly bound in immune complexes (25, 27). The correlation between p24 detection and HIV RNA or CD4 cell counts has been evaluated in a number of studies. Schupbach and others have demonstrated that the p24 is predictive of CD4 depletion (14, 28) and disease progression (14, 35), that it decreases when ART is initiated (2, 23, 29), and that it correlates inversely with CD4 changes even when HIV RNA is so low that it cannot be quantitated (30, 31). However, disappointing results have also been reported using this kit (1, 21). Bonard et al. tested plasma samples from 14 patients treated with highly active antiretroviral therapy, found weaker changes in Ultrasensitive p24 antigen testing compared to HIV RNA results, and questioned the reliability of this assay as an RNA surrogate for clinical management of HIV-infected patients. Prado et al. demonstrated a lack of correlation between HIV RNA and the Ultrasensitive p24 assay (Perkin-Elmer) for monitoring patients undergoing a structured treatment interruption. Prado et al. found that discordance between HIV RNA detection and p24 detection after viral rebound was 76% of the treated patients and 49% of the naive patients enrolled in the study. The discordance was attributed to the presence of high-affinity immunoglobulins that interfered with the p24 assay detection. More recently, Schupbach et al. reported that p24 antigen was detectable and remained relatively unchanged during short STI (31). These contradictory findings demonstrate the importance of using standardize methods in attempts to compare assays and demonstrate clinical utility. The two earlier studies utilized the Ultrasensitive p24 kits in absence of the modified external buffer, while the third study included it. Our data demonstrate that the modified buffer does impact the sensitivity of the assay; fully optimized assays should be used for future studies trying to demonstrate utility of this assay for monitoring HIV disease progression.
The p24 data obtained in the current study need to be interpreted carefully. The panels utilized contained two distinct types of samples, those that were spiked with well-characterized virus and those that were obtained from HIV-infected individuals. The reality is that clinical samples will typically contain p24 that is both not virion associated and virion associated. However, in attempting to evaluate the performance of a particular assay there are benefits to using a well-characterized specimen, including the ability to decipher the contribution of immune-complexed versus virion-associated p24 antigen in the detection rates of this assay.
Limits of detection were quite high in samples where only virion-associated p24 was present (i.e., HIV spiked samples). It is estimated that 2,000 to 4,000 p24 Gag proteins are present in each virion, so it would take 6,150 to 12,500 virus particles or 12,500 to 25,000 copies of viral RNA to detect 1 pg of p24 protein (5). Since the limit of detection of the modified p24 assay is approximately 6 pg/ml, it is not surprising that the observed detection rate in spiked samples is approximately 100,000 copies/ml. When clinical samples were tested, the detection rate appeared to be greatly improved, and this is presumed be due to the presence of non-virion-associated p24. Since the contribution of this component may vary between patients (4, 25) and change with disease progression (13, 14, 35), the impact on HIV monitoring is not fully understood and explains the lower correlation with HIV RNA. HIV p24 detection rates were improved with the use of the external buffer (kindly provided by Jorg Schupbach), especially in HIV-spiked samples, where only virion-associated proteins were present. The improved sensitivity with spiked samples suggests that the buffer may help to facilitate dissociation of the protein from the virus particle. Although the added external buffer also appeared to improve the detection of non-B subtypes, a more likely explanation would be that improved detection of the non-B subtype samples was due to improved assay sensitivity.
CV data demonstrate that comparable results were obtained between laboratories with the p24 assay. Correlations between the p24 assay and HIV RNA were lower when the kit buffer was used (r = 0.58 to 0.67) than when the external buffer was used (r = 0.59 to 0.95). Furthermore, correlations were higher when only spiked samples were included. The correlations with HIV RNA reported in this study were not different than those reported (r = 0.48 to 0.69) in other studies which did not utilize the external buffer (20, 22). Respess et al. did document a difference in correlation between untreated (r = 0.69) and treated (r = 0.48) patients, similar to the difference noted in this study between spiked and clinical specimens. This may be attributed to the fact that treatment alters the presence of virion-associated and non-virion-associated p24 antigen that is available for detection. Although more data are need to demonstrate the utility of the p24 assay for monitoring patients on ART, preliminary data suggest that it may prove useful for pediatric HIV diagnosis (15, 22, 37, 33) or diagnosis of acute infection (20). The fact that a small sample volume is required is especially attractive for pediatric diagnosis, and ongoing studies are under way to evaluate the potential for using this assay for detection of p24 in blood spots.
The second candidate to be considered as a surrogate for HIV RNA testing is the RT (Cavidi) assay. A number of studies in Africa, Australia, and France have provided excellent correlations between the RT and HIV RNA assays (3, 9, 34, 36). The fact that the assay only monitors virion-associated RT contributes to this excellent correlation. In addition, the assay could be useful in the detection of other lentiviruses (18), including HIV-2, and the extracted RT may be used in the Cavidi HIV Phenotype RT kit for evaluating resistance in patients on anti-RT regimens (32, 38). The simplified equipment, which includes a standard spectrophotometer, a vacuum driven extraction manifold, and wash buckets for plate washing, make this assay especially attractive for resource-limited urban and rural settings. The assay result may be reported as fg of RT/ml of plasma, but the use of a conversion factor may be used to convert the result into a more familiar readout of HIV RNA equivalents/ml. While the utility of this assay for monitoring patients on ART has yet to be fully evaluated, the roll-out of the v2.0 assay, with its improved sensitivity, should have a major impact. A limiting factor of this assay is sample volume. While a 1-ml plasma input is not problematic for monitoring adult patients, it certainly may be problematic for monitoring pediatric patients. Smaller sample volumes (down to 0.25 ml) may be utilized; however, this reduces assay sensitivity by the dilution factor involved (data from Cavidi).
The RT assay (Cavidi v1.0) showed good detection rates for samples with viral loads of >10,000 copies/ml. The v2.0 assay improved this detection limit to 400 copies/ml, which is comparable to that observed with the HIV RT-PCR assay (Roche) routinely used in many industrialized countries. CV analyses demonstrated comparable reproducibility between the RT and RNA assays. Comparable results were also obtained between the laboratories performing the testing, suggesting the assay is quite robust. Higher CVs were noted in the v2.0 assay than in the v1.0 assay. This was mainly observed in samples with lower viral loads, a phenomenon that has also been noted in HIV RNA assays for samples with viral loads near the limit of detection (7). Correlation between HIV RNA (Roche) and the RT (Cavidi) results were excellent for both versions of the kit (r = 0.83 to 0.99) and did not vary per panel. These results are quite similar to those reported by other investigators (9, 34, 36) using both versions of the RT assay (Cavidi). Preliminary studies involving non-B subtypes (34, 36), suggest that clades CRF02_AG, C, and B are detected well with this assay, but more studies are needed to substantiate these observations and better define detection rates of other clades. Factors that could affect detection rates include the presence of RT inhibitors or resistance mutations in the RT gene. The data presented by Greengrass et al. (9) suggest that the presence of non-nucleoside RT inhibitor-associated resistance mutations may reduce the RT activity by 0.20 log10, but the presence or absence of nucleoside RT inhibitor resistance mutations was not associated with RT activity. More data are needed to confirm this observed effect on RT activity and its impact on monitoring HIV-infected patients.
The benefits of using HIV RNA surrogates are assay simplicity and cost reduction. While kit and reagent costs will vary by country (RT-PCR = $17 to $80, RT assay = $20, Ultrasensitive p24 = $5 to $10), equipment requirements for the HIV RNA surrogate assays are simpler and ca. 40 to 50% less expensive. In addition, although HIV RNA assays require well-developed infrastructures to ensure sterility and containment of nucleic acids and amplified products, the HIV RNA surrogate assays have more simplistic requirements with respect to facilities since nucleic acid extractions and amplification are not part of the assay. Therefore, the use of an HIV RNA surrogate assay to screen patients for drug efficacy and compliance, followed by less-frequent confirmation testing with molecular assays, may provide a useful strategy in resource-poor settings. The two assays evaluated in this study offer feasible RNA surrogates and should be considered for use in this type of strategy.
ACKNOWLEDGMENTS
This study was supported by NIAID contract NO-AI-85354.
We thank Joseph Fitzgibbon of the National Institutes of Health for helpful comments and careful review of the manuscript.
REFERENCES
Bonard, D., F. Rouet, T. A. Toni, A. Minga, C. Huet, D. K. Ekouevi, F. Dabis, R. Salamon, C. Rouzioux, et al. 2003. Field evaluation of an improved assay using heat-dissociated p24 antigen for adults mainly infected with HIV-1 CRF02_AG strains in Cote d'Ivoire, West Africa. J. Acquir. Immune. Defic. Syndr. 34:267-273.
Boni, J., M. Opravil, Z. Tomasik, M. Rothen, L. Bisset, P. J. Grob, R. Luthy, and J. Schupbach. 1997. Simple monitoring of antiretroviral therapy with a signal-amplification-boosted HIV-1 p24 antigen assay with heat-denatured plasma. AIDS 11:F47-F52.
Braun, J., J.-C. Plantier, M.-F. Hellot, E. Tuaillon, M. Gueudin, F. Damond, A. Malmsten, G. E. Corrigan, and F. Simon. 2003. A new quantitative HIV load assay based on plasma virion reverse transcriptase activity for different types, groups and subtypes. AIDS 17:331-336.
Brown, C., R. Kline, L. Atibu, H. Francis, R. Ryder, and T. C. Quinn. 1991. Prevalence of HIV-1 p24 antigenaemia in African and North American populations and correlation with clinical status. AIDS 5:89-92.
Coffin, J. M. 1996. Retroviridae: the viruses and their replication, p. 1767-1847. In B. N. Fields, D. M. Knipe, and P. M. Howley (ed.), Fields virology, 3rd ed. Lippincott-Raven, Philadelphia, Pa.
Crowe, S., S. Turnbull, R. Oelrichs, and A. Dunne. 2003. Monitoring of human immunodeficiency virus infection in resource-constrained countries. CID 37(Suppl. 1):S25-S35.
Erice, A., D. Brambilla, B. Jackson, R. Kokka, B. Yen-Lieberman, and R. Coombs. 2000. Performance characteristics of the Quantiplex HIV-1 RNA 3.0 assay for the detection and quantitation of HIV-1 RNA in plasma. J. Clin. Microbiol. 38:2837-2845.
Giacomini, M., J. L. McDermott, A. A. Giri, I. Martini, F. B. Lillo, and O. E. Varnier. 1998. A novel and innovative kinetic software for virologic colorimetric assays. J. Virol. Methods 73:201-209.
Greengrass, V. L., S. P. Turnbull, J. Hocking, A. L. Dunne, G. Tachedjian, G. E. Corrigan, and S. M. Crowe. 2005. Evaluation of a low cost reverse transcriptase assay for plasma HIV-1 viral load monitoring. Curr. HIV Res. 3:183-190.
Hammer, S., C. Crumpacker, R. D'Aquila, B. Jackson, J. Lathey, D. Livnat, and P. I. Reichelderfer. 1993. Use of virologic assays for detection of human immunodeficiency virus in clinical trials: recommendations of the AIDS Clinical Trials Group Virology Committee. J. Clin. Microbiol. 31:2557-2564.
Huang, D. D., T. A. Giesler, and J. W. Bremer. Sequence characterization of the protease and partial reverse transcriptase proteins of the NED panel, an international HIV type 1 subtype reference and standards panel. AIDS Res. Hum. Retrovir. 19:321-328.
Kent, D. M., D. McGrath, J. P. A. Ioannidis, and M. L. Bennish. 2003. Suitable monitoring approaches to antiretroviral therapy in resource-poor settings: setting the research agenda. CID 37(Suppl. 1):S13-S24.
Lange, J. M., D. A. Paul, H. G. Huisman, F. de Wolf, H. van den Berg, R. A. Coutinho, S. A. Danner, J. van der Noordaa, and J. Goudsmit. 1986. Persistent HIV antigenaemia and decline of HIV core antibodies associated with transition to AIDS. BMJ 293:1459-1462.
Ledergerber, B., M. Flepp, J. Boni, Z. Tomasik, R. W. Cone, R. Luthy, and J. Schupbach. 2000. Human immunodeficiency virus type 1 p24 concentration measured by boosted ELISA of heat-denatured plasma correlates with decline in CD4 cells, progression to AIDS and survival: comparison with viral RNA measurement. J. Infect. Dis. 181:1280-1288.
Lyamuya, E., U. Bredberg-Raden, A. Massawe, E. Urassa, G. Kawo, G. Msemo, T. Kazimoto, A. Ostborn, K. Karlsson, F. Mhalu, and G. Biberfeld. 1996. Performance of a modified HIV-1 p24 antigen assay for early diagnosis of HIV-1 infection in infants and prediction of mother-to-infant transmission of HIV-1 in Dar es Salaam, Tanzania. J. Acquir. Immune. Defic. Syndr. Hum. Retrovir. 12:421-426.
Malmsten, A., X. W. Shao, K. Aperia, G. E. Corrigan, E. Sandstrom, C. F. Kallander, T. Leitner, and J. S. Gronowitz. 2003. HIV-1 viral load determination based on reverse transcriptase activity recovered from human plasma. J. Med. Virol. 71:347-359.
Malmsten, A., X. W. Shao, S. Sjodahl, E. L. Fredriksson, I. Pettersson, T. Leitner, C. F. R. Kallander, E. Sandstrom, and J. S. Gronowitz. 2005. Improved HIV-1 viral load determination based on reverse transcriptase activity recovered from human plasma. J. Med. Virol. 76:291-296.
Malmsten, A., D. H. Ekstrand, L. Akerblom, J. S. Gronowitz, C. F. Kallander, M. Bendinelli, and D. Matteucci. 1998. A colorimetric reverse transcriptase assay optimized for Moloney murine leukemia virus, and its use for characterization of reverse transcriptases of unknown identity. J. Virol. Methods 75:9-20.
Nishanian, P., K. R. Huskins, S. Stehn, R. Detels, and J. L. Fahey. 1990. A simple method for improved assay demonstrates that p24 antigen is present as immune complexes in most sera from infected individuals. J. Infect. Dis. 162:21-28.
Pascual, A., A. Cachafeiro, M. L. Funk, and S. A. Fiscus. 2002. Comparison of an assay using signal amplification of the heat-dissociated p24 antigen with the Roche Monitor human immunodeficiency virus RNA assay. J. Clin. Microbiol. 40:2472-2475.
Prado, J. G., A. Shintani, M. Bofill, B. Clotet, L. Ruiz, and J. Martinez-Picado. 2004. Lack of longitudinal intrapatient correlation between p24 antigenemia and levels of human immunodeficiency virus (HIV) type 1 RNA in patients with chronic HIV infection during structured treatment interruptions. J. Clin. Microbiol. 42:1620-1625.
Respess, R. A., A. Cachafeiro, D. Withum, S. A. Fiscus, D. Newman, B. Branson, O. E. Varnier, K. Lewis, and T. J. Dondero. 2005. Evaluation of an ultrasensitive p24 antigen assay as a potential alternative to human immunodeficiency virus type 1 RNA viral load assay in resource-limited settings. J. Clin. Microbiol. 43:506-508.
Ribas, S. G., P. Ondoa, J. Schupbach, G. van der Groen, and K. Fransen. 2003. Performance of a quantitative human immunodeficiency virus type 1 p24 antigen assay on various HIV-1 subtypes for the follow-up of human immunodeficiency type 1-seropositive individuals. J. Virol. Methods 113:29-34.
Schupbach, J., and J. Boni. 1993. Quantitative and sensitive detection of immune-complexed and free HIV antigen after boiling of serum. J. Virol. Methods 43:247-256. (Erratum, 45:245.)
Schupbach, J. 2003. Viral RNA and p24 antigen as markers of HIV disease and antiretroviral treatment success. Int. Arch. Allergy Immunol. 132:196-209.
Schupbach, J., J. Boni, L. R. Bisset, Z. Tomasik, M. Fischer, H. F. Gunthard, B. Ledergerber, M. Opravil, et al. 2003. HIV-1 p24 antigen is a significant inverse correlate of CD4 T-cell change in patients with suppressed viremia under long-term antiretroviral therapy. J. Acquir. Immune Defic. Syndr. 33:292-299.
Schupbach, J. 2002. Measurement of HIV-1 p24 antigen by signal-amplification-boosted ELISA of heat-denatured plasma is a simple and in expensive alternative to tests for viral RNA. AIDS Rev. 4:83-92.
Schupbach, J., Z. Tomasik, D. Nadal, B. Ledergerber, M. Flepp, M. Opravil, and J. Boni. 2000. Use of HIV-1 p24 as a sensitive, precise and inexpensive marker for infection, disease progression and treatment failure. Int. J. Antimicrob. Agents 16:441-445.
Schupbach, J., J. Boni, M. Flepp, Z. Tomasik, H. Joller, and M. Opravil. 2001. Antiretroviral treatment monitoring with an improved HIV-1 p24 antigen test: an in expensive alternative to tests for viral RNA. J. Med. Virol. 65:225-232.
Schupbach, J., J. Boni, L. R. Bisset, Z. Tomasik, M. Fischer, H. F. Gunthard, B. Ledergerber, M. Opravil, et al. 2003. HIV-1 p24 antigen is a significant inverse correlate of CD4 T-cell change in patients with suppressed viremia under long-term antiretroviral therapy. J. Acquir. Immune Defic. Syndr. 33:292-299.
Schupbach, J., H. Gunthard, B. Joos, M. Fischer, J. Boni, Z. Tomasik, S. Yerly, L. Perrin, M. Battegay, H. Furrer, P. Vernazza, E. Bernasconi, B. Hirschel, et al. 2005. HIV-1 p24 may persist during long-term HAART, increases little during short treatment breaks, and its rebound after treatment stop correlates with the CD4+ loss. J. Acquir. Immune. Defic. Syndr. 40:250-256.
Shao, X. W., A. Malmsten, J. Lennerstrand, A. Sonnerborg, T. Unge, J. S. Gronowitz, and C. F. R. Kallander. 2003. Use of HIV-1 reverse transcriptase recovered from human plasma for phenotypic drug susceptibility testing. AIDS. 17:1463-1471.
Sherman, G. G., G. Stevens, and W. S. Stevens. 2004. Affordable diagnosis of human immunodeficiency virus infection in infants by p24 antigen detection. Pediatr. Infect. Dis. J. 23:173-176.
Sivapalasingam, S. S. Essajee, P. N. Nyambi, V. Itri, B. Hanna, R. Holzman, and F. Valentine. 2005. Human immunodeficiency virus (HIV) reverse transcriptase activity correlates with HIV RNA load: implications for resource-limited settings. J. Clin. Microbiol. 43:3793-3796.
Sterling, T. R., D. R. Hoover, J. Astemborski, D. Vlahov, J. G. Bartlett, and J. Schupbach. 2002. Heat-denatured human immunodeficiency virus type 1 protein 24 antigen: prognostic value in adults with early-stage disease. J. Infect. Dis. 186:1181-1185.
Stevens, G., N. Rekhviashvili, L. E. Scott, R. Gonin, and W. Stevens. 2005. Evaluation of two commercially available, inexpensive alternative HIV-1 viral load assays in HIV-infected subtype C patients from South Africa: the HiSen HIV-1 p24 antigen Ultra kit and the Cavidi ExaVir load HIV-RT assay. J. Clin. Microbiol. 43:857-861.
Sutthent, R., N. Gaudart, K. Chokpaibulkit, N. Tanliang, C. Kanoksinsombath, and P. Chaisilwatana. 2003. P24 antigen detection assay modified with a booster step for diagnosis and monitoring of human immunodeficiency virus type 1 infection. J. Clin. Microbiol. 41:1016-1022.
Tuaillion, E., M. Gueudin, V. Lemee, I. Gueit, P. Roques, G. E. Corrigan, J.-C. Plantier, F. Simon, and J. Braun. 2004. Phenotypic susceptibility to nonnucleoside inhibitors of virion-associated reverse transcriptase from different HIV types and groups. J. Acquir. Immune Defic. Syndr. 37:1543-1549.
Yen-Lieberman, B., D. Brambilla, B. Jackson, J. Bremer, R. Coombs, M. Cronin, S. Herman, D. Katzenstein, S. Leung, H. J. Lin, P. Palumbo, S. Rasheed, J. Todd, M. Vahey, and P. Reichelderfer. 1996. Evaluation of a quality assurance program for quantitation of human immunodeficiency virus type 1 RNA in plasma by the AIDS Clinical Trials Group virology laboratories. J. Clin. Microbiol. 34:2695-2701.(Cheryl Jennings, Susan A.)