Received Date: July 26, 2013; Accepted Date: September 25, 2013; Published Date: September 30, 2013
Citation: Gumbo P, Chideme M, Mangwanya D, Handireketi N, Kambarami D, et al. (2013) Analysis of Bias and ART Enrollment for a Point-of-Care CD4/CD4% Analyzer. J AIDS Clin Res 4: 247. doi:10.4172/2155-6113.1000247
Copyright: © 2013 Gumbo P, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of AIDS & Clinical Research
Point-of-care technologies are a cornerstone for effective HIV care of patients in resource poor settings. They greatly reduce loss to follow up as the patients receive their results immediately and obtain appropriate treatment and other care. There are 593,000 people in urgent need of antiretroviral therapy in Zimbabwe for whom point-of-care technology may be a benefit. The PointCare NOW device was evaluated in order to assess systematic errors between it and conventional flow cytometry and haematology devices, such as the Becton Dickinson FACS Calibur Single Platform and the Sysmex XT-1800i. There being no calibration method for CD4 count on any platform, large systematic errors could affect therapy enrollment decisions. Specimens were collected from 104 patients from the Harare Central Hospital Opportunistic Infection clinic by venipuncture using EDTA tubes. Three samples were automatically excluded as unsuitable for analysis by the PointCare NOW internal control software flag system; for a rejection rate of 2.9%. No other samples were excluded. Each device performed within manufacturer’s replicate precision specifications for all parameters. There was a small but statistically insignificant difference in mean absolute CD4 cell count (+8.6 cells/ μl), mean CD4% (+0.76%) and mean total lymphocyte count (+0.054×103 cells/μl) between the PointCare NOW and the central laboratory systems. There was a small but statistically significant difference in mean total white cell count (+0.28×103 cells/μl) and mean total haemoglobin (-0.74g/dl) measurements and Sysmex XT1800i. We compare two methods for qualifying patients for therapy and show that there was no statistically significant difference between PointCare NOW and FACSCalibur qualifications for either method. One method uses a conventional sharp cut-off and the other uses a newer idea based on patient variability. Our results indicate that the systematic errors determined in our study would not produce disparity in therapy enrollment action taken for a patient receiving results from a PointCare NOW and a FACSCalibur. Our study was completed in 2011 and when compared to other studies carried out in 2009, it suggests that there have been improvements in the PointCare NOW systematic error performance.
Point-of-care; ART; HIV; CD4; CD4%; WBC; LYM; PointCare
POC: Point-of-care; ART: Antiretroviral Therapy; WBC: Total White Blood Cell Count; LYM: Total Lymphocyte Count; ARV: Anti-retroviral; TLC: Total Lymphocyte Counts; Hgb: Total Hemoglobin
The 2009 antenatal clinic survey done in Zimbabwe indicated the prevalence of HIV infection amongst pregnant women to be 16.1%, whilst it was estimated that the prevalence of HIV in Zimbabwe was 14.3% in 2010 amongst the general population . This is a substantial drop from 27.2% in 1998. Use of condoms and high death rates due to lack of anti-retroviral (ARV) drugs up to 2006 were thought to be contributing factors to the drop in prevalence .
CD4 cell counts play a major role in determining HIV patient monitoring as they are used as a criterion for treatment initiation. In Zimbabwe ART is initiated when an adult patient’s CD4 cell count is =350 cells/µl, after which monitoring is done every 6 months. For children less than 5 years of age, the threshold is CD4 % <25 %. Monitoring after ART initiation is used to determine the need for second line drugs. In Zimbabwe, there are 326,000 adults and 27,000 children requiring monitoring.
In some rural settings, HIV positive patients have to walk long distances or pay for transport in order to get to a health facility. If they need a CD4 cell count, then blood is withdrawn and transported to a District lab. They are informed to come back 2 weeks later for the result which might or might not be available. The non availability of the result could be due to machine malfunction or reagent stock outs. Another blood specimen might need to be taken again and this time the patient might be too sick to come back on the fourth week or is unable to afford the transport. All these lead to loss to follow up . An onsite POC machine that could provide CD4 count for adults, CD4% for children and routine hematology would greatly alleviate this suffering, which prompted our evaluation of the PointCare NOW machine.
The primary aim of this study was to evaluate the mean absolute difference (systematic errors) between CD4 parameter results (total CD4 count and CD4%) reported by a PointCare NOW POC device and a central laboratory FACSCalibur flow cytometer and the mean absolute difference between total white cell counts (WBC), total lymphocyte counts (TLC), and total hemoglobin (Hgb) reported by the PointCare NOW POC device and a Sysmex XT-1800i central laboratory hematology analyzer. This was to determine whether adjustments for “bias” need to be made between the POC device and the central laboratory devices. In view of the fact that there is no calibration method for CD4 parameters on any analyzer, it is particularly important to delineate any systematic disagreement between CD4 results that a patient may receive at a central facility and at another time at the pointof- care.
The study also combined systematic error and random errors affecting single comparisons of CD4 count results obtained from the POC device and the central laboratory flow cytometer in determining whether patients would be classified similarly into three categories of ART initiation. These categories are: (1) Begin ART now based on a firm count of <350 cell/µl CD4 count, (2) Wait for the results of the next regular CD4 test, or (3) Repeat CD4 testing in the near future to eliminate suspicion of a physiologically rooted statistical fluctuation in CD4 count. The third aim of the study was to determine whether the automated “flagging” system of internal controls on the PointCare NOW provided a high enough yield of reported results vs. blocked results to be practical.
Our study, where data was obtained in 2011, yielded systematic error, replicate precision, and enrollment classification results that were substantially better than those reported from a 2009 study  of the PointCare NOW machine. This suggested to us that improvements had occurred. We have expanded our study to discuss the observed differences and the implications for patient enrollment in ART.
All samples were collected by venipuncture using Vacutainer needles and holders that are part of the PointCare NOW reagent kit. Blood was drawn to within at least ¾ of the fill mark on the tube (full draw) into 4ml EDTA tubes then mixed by 8 manual-inversions of the tube immediately after collection. These were transported to the laboratory at ambient temperature (between 15 and 30°C) protected from direct sunlight and temperatures above 30°C. These samples were run within 8 hours of collection as recommended by the PointCare NOW manual. The entire tube, with bio-safety cap in place, was placed in the PointCare NOW. After pressing ‘run sample’, the patient’s name and identification number (ID), sample ID, age, sex, blood draw day and time were then entered on the touch screen. The ‘begin sample’ option was then pressed. The patient sample was then run by following the on-screen instructions step by step. When the machine had completed analysis after 8 minutes, a panel of results were printed. The panel includes a total white cell count, white cell differential count, total haemoglobin, CD4 count and CD4%. For this evaluation, we recorded total haemoglobin, CD4 count and CD4%.
External quality control samples are provided in the PointCare NOW kit and were run according to manufacturer’s instructions. Results must be within the target range before proceeding to run patient samples. Daily control results are automatically stored in Levey- Jennings plots. The device performs a background level check after pressing the start up option on the main menu and at the end of the day a daily shut down cycle is automatically performed.
At the outset of the evaluation a precision verification was performed on three donor samples. Subsequently, one hundred and four specimens were collected from adults that attended Harare Central Hospital Opportunistic Infection clinic from the 28th of March 2011 to the 1st of April 2011.
Three samples were automatically excluded as unsuitable for analysis by the PointCare NOW internal control software flag system from the 104 that were collected, for a rejection rate of 2.9% and a reduction of sample size to 101. There was no other sample exclusion in the study.
Data was entered into a Microsoft Excel spreadsheet and transferred into STATA version 10 for analysis. The absolute CD4 count, CD4 percent, absolute lymphocyte, absolute WBC and absolute HGB were the variables of interest. For the absolute CD4 and CD4 percent the comparison was between the Point-Care NOW and the Becton Dickinson FACSCalibur Single Platform flow cytometer . The LYM, WBC and HGB comparisons were between the Point-Care NOW and the Sysmex XT-1800i . Intra- and inter-run variability was assessed using the coefficient of variation. The degree of agreement between measures from the two methods was assessed using the Bland-Altman method of analysis . Absolute CD4 counts were divided into three groups for ART classification: (1-Start ART, 2-Delay ART, 3-Repeat test) and a McNemar test was performed to determine any significant differences in the classification between the FACSCalibur and Point- Care NOW .
The systems were required to perform within manufacturer’s specifications for coefficient of variance (CV) on CD4 count and Hgb. Ten replicate samples from three donors (not among the 104 used for methods comparisons) were used. Table 1 and 2 are a summary of the absolute CD4 cell count and HGB precision from the PointCare NOW device, FACS Calibur Single Platform and Sysmex XT-1800i.
|Mean (cells/µl)||Standard deviation (cells/µl)||Coefficient of Variation (%)|
|Average = 966||Average = 49.5||Average = 5.3|
|FACSCalibur Single Platform|
|Average = 1088||Average = 89.5||Average = 8.2|
Table 1: Summary of precision testing statistics for the absolute CD4 cell count obtained from three normal donors using the PointCare NOW and the FACSCalibur Single Platform.
|Point Care Now|
|Donor||Mean Hgb (g/dl)||Standard deviation (g/dl)||Coefficient of Variation (%)|
|Average = 13.9||Average = 0.1||Average = 0.7|
|Average = 13.1||Average = 0.1||Average = 1.0|
Table 2: Summary of precision testing statistics for haemoglobin obtained from three normal donors using the Point Care Now and the Sysmex XT-1800i.
Bland-altman method comparison
Absolute CD4 cell count: The mean absolute CD4 count using the Point Care NOW instrument was 389.16 cells/µl whilst that with the reference method, FACSCalibur Single Platform was 397.83 cells/ µl. The mean difference in absolute CD4 count values between the two methods was 8.6 cells/µl (95% CI -12.2-29.5) which was not statistically significant (p=0.41). For the two methods the lower limit of agreement was -202.6 (95% CI:-238.7, -166.5) and the upper limit of agreement was 219.96 (95% CI 183.8, 256.1) (Figure 1).
CD4 percentage: The mean percent CD4 count using the Point Care NOW instrument was 20.49 whilst that with the reference method (FACSCalibur Single Platform) was 19.73. The mean difference in percent CD4 count values between the two methods was -.76% cells/ µl which was not significant (p=0.2). The lower and upper limits of agreement for the two methods were -12.60% (95 % CI (-14.63, -10.58)) and the upper limit of agreement was +11.09% (95 % CI (9.06, 13.11)) respectively (Figure 2).
Haemoglobin: The mean absolute Hgb count using the PointCare NOW instrument was 13.25 g/dL whilst that with the reference method (Sysmex XT-1800i) was 12.56 g/dL. The mean difference in absolute Hgb count values between the two methods was -0.74 g/dL (95% CI -1.01 to -0.47) which was significant. The lower and upper limits of agreement for the two methods were -3.5 (95% CI (-3.97, -3.03)) and 2.01 (95% CI (1.54, 2.49) respectively (Figure 3).
Lymphocyte cell count: The mean absolute lymphocyte count using the PointCare NOW instrument was 1.92x103 cells/µl whilst that with the reference method (Sysmex) was 1.98x103 cells/µl. The mean difference in absolute lymphocyte count values between the two methods was 0.054x103 cells/µl (95 % CI -0.019 - 0.089) which was not significant. The lower and upper limits of agreement for the two methods were -.31(95% CI (-0.37, -0.24)) and the upper limit of agreement were .41 (95 % CI (0.35, 0.47) respectively (Figure 4).
Absolute WBC: The mean absolute lymphocyte count using the PointCare NOW instrument was 4.53x103 cells/µl whilst that with the reference method (Sysmex XT1800i) was 4.80x103 cells/µl. The mean difference in absolute WBC count values between the two methods was 0.28x103 cells/µl (95 % CI 0.19 - 0.36) which was significant. The lower and upper limits of agreement for the two methods were -0.56 (95 % CI (-0.70 -0.42) and 1.1 (95 % CI (0.97, 1.25) respectively (Figure 5).
A new classification system was used in this study in which 350 cells/µl was retained as the CD4 count threshold for therapy, but a band of 93 counts above 350 cells/µl was introduced and termed a band calling for repeat measurements before initiating therapy. The size of the band is plus one standard deviation of the known random, dayto- day (not within-day) variability of the CD4 count near 350 cells/µl. Table 3 shows the number of instances of agreement and disagreement between the PointCare NOW and FACSCalibur in classifying results as being highly likely to indicate initiating ART, highly likely to indicate not initiating ART, and requiring a near-term repeat measurement to rule out normal day-to-day fluctuations in patient CD4 count [9-11].
|CD4 Count Range||Classification||FACSCalibur N||PointCare NOW N||Significance of difference (McNemar Test)|
|CD4<350||Start ART||48||43||None p=0.359|
|CD4>443||Delay ART||34||32||None p=0.789|
|350<CD4<443||Repeat Test||19||26||None p=0.265|
Table 3: Classification of ART based on Absolute CD4 count from the FacsCalibur compared to the PointCare NOW.
The introduction of low cost antiretroviral treatment in resource poor settings has led to the need for HIV therapy monitoring and this medical service should be dispersed to locations near the patients . Central laboratory CD4 cell count devices are costly to purchase and operate and cannot be dispersed in large numbers. There is a need for alternative point-of-care devices that less expensive to purchase and operate, yield rapid results, and are easy to use and interpret. These devices need to perform in hot, humid, and frequently dusty conditions where power is unreliable and water shortages are encountered. The PointCare NOW device meets most of these requirements. The reagents and device are temperature resistant, the device can be operated from a battery, there is no need for external water, results are ready in 8 minutes, the sample remains biosafe, and a system of internal flags is intended to assist in results interpretation.
Recent biomathematical studies by Noubary and Hughes indicate that in developing new technologies, method comparison studies should focus on ruling out modest measurement bias, particularly in the vicinity of important thresholds for treatment management . The current study using the PointCare NOW and a FACSCalibur showed that the average absolute difference for CD4 counts to be +8.6 cells/µl. and that the difference was not statistically significant. This suggests that no bias adjustment should be applied to CD4 counts obtained with the PointCare NOW and the central laboratory instrument when a given patient has results from both devices. The absolute difference for lymphocyte counts using the PointCare NOW and the Sysmex analyzer was +0.054x103 cells/µl and was not statistically significant, also indicating that no bias adjustment is needed. On the other hand, the absolute differences between the PointCare NOW and the Sysmex analyzer for haemoglobin (-0.74g/dL) and absolute white cell count (+0.28x103 cells/µl) were statistically significant and a correction for bias would be indicated.
Calling for repeat measurements when CD4 counts approach an ART management threshold from above has also been suggested, provided the measurements are done within days to weeks so as to not unduly delay treatment . This is consistent with the fact that normal CD4 count fluctuations are short term and span a period of days. In this study we introduced a new classification schema where CD4 results in a band of approximately 90 cells/µl above 350 cells/µl would call for a repeat measurement in a few days or weeks to rule out the possibility of physiologically driven random fluctuations in CD4 count causing a patient to be misclassified as ineligible for therapy. Maintaining a 350 cells/µl threshold for starting ART and using the 90 cells/µl band above 350 cells/µl as the repeat measurement zone, our results show that there was no statistical difference between the central laboratory reference system and the PointCare NOW under this classification schema.
In contrast with our 2011 study, an older 2009 study showed substantial bias for the PointCare NOW with central laboratory machines . Our results suggest that improvements had been made in the PointCare NOW in the intervening time and should be noted. The 2009 study reported a CD4 count bias of +153 cells/µl over the entire data range and + 180 cells/µl for CD4<350 cells/µl. In sharp contrast our results were +8.6 cells/µl over the entire data range and +12 cells/µl for CD4< 350 cells/µl. Only part of the 2009 study used replicate precision from more than 7 samples and could be compared with our results. The 2009 results were reported as <11% coefficient of variation. Our study used 10 replicates and we determined an average precision of 5.3% coefficient of variation which is substantially below 11%.
This improved bias and precision performance of the PointCare NOW was also reflected in putative enrolment rates when the ART enrolment algorithm from the 2009 study was applied to our data. The 2009 study reported that the PointCare NOW would have incorrectly deferred ART in approximately 50% of cases using a cut-off at 350 cells/ µl and a plus and minus 5% band in which counts between 332 and 367 cells/µl were taken as similar. In our study we found that percentage to be reduced to less than 6% using the same algorithm.
These comparisons indicate that, in our study, there was no significant difference between PointCare NOW ART enrolment rate and FACSCalibur ART enrolment rates in the hard cut-off and repeat measurement algorithms. Should the repeat measurement algorithm be adopted, there is no degradation expected for PointCare NOW performance.
Recent reports have shown that random fluctuations in CD4 count correlate with fluctuations in total lymphocyte count . Abnormally high or low lymphocyte counts caused by factors other than HIV can produce elevated or depressed CD4 counts that are not representative of the patient’s HIV condition. Clinicians using the Point-Care NOW can examine the reported total lymphocyte count for normalcy whenever a CD4 count is repeated and determine whether the repeat CD4 count has been affected by a short term fluctuation. Having total lymphocyte counts available in a point-of-care system may prove to be significantly advantageous.
The PointCare NOW flag system was effective in calling for a repeat sample run without operator analysis. The device ultimately failed to analyze only 2.9% of samples for CD4 count. In most instances the messages displayed to accompany a flag were clear, but in other instances only a trained individual would be expected to interpret the message. Of the 3 samples with a blocked CD4 count, two gave a screen message of ‘Auto-check shows NEU interference with CD4’. The potential cause and remedial action was as follows:
• Sample age exceeded: re-draw patient - run new sample within 8 hours.
•Damaged patient sample: platelet clumps, or heat damage to cells - Check tube for clots, re-draw patient, run new sample.
• Patient sample with fragile neutrophils - Use alternative method
One of the samples showed a screen message of ‘Auto-check shows LYM interference with CD4’. The potential cause and remedial action was as follows:
• Damaged patient sample: platelet clumps, or heat damage to cells: Check tube for clots, re-draw patient, run new sample.
• Nucleated red blood cells (NRBC’s) (erythroblasts): check blood smear for NRBC’s, notify clinician.
These messages require that trained individuals be available for interpretation which may not be the case in certain areas.
Our study indicates that the PointCare NOW is acceptable and suitable for outreach programs. Patients receiving CD4 counts and CD4% results from a central laboratory and a PointCare NOW will need no bias adjustment whereas small bias adjustments will be in order for total white cell count and haemoglobin. Under the therapy enrolment strategy outlined in this report and others there is statistically no difference found between the PointCare NOW and FACSCalibur results. The flagging system for the PointCare NOW blocked 2.9% of CD4 count results. There is a cost to the medical system inasmuch as these patients would need to return on another day for testing, but this percentage is low enough to be an acceptable cost for quality. Comparisons with earlier studies of the PointCare NOW suggest that improvements have been made. Whether using algorithms for ART enrollment cited in earlier studies or an algorithm that we have presented, the rate of disagreement on ART enrollment between the PointCare NOW (a “No” result) and the central laboratory (a “Yes” result) was a only 6%. Future studies should include methods of deploying the PointCare NOW haematology menu for outreach programs in HIV. Simultaneous reporting of the total lymphocyte count and CD4 count may be clinically advantageous to clinicians making therapy decisions. In addition, owing to the many variables encountered in POC deployment, future studies would be helpful if they delineate features of operator training, supply chain management, repair in the field, and device stability.