Interlaboratory Agreement and Handling Variability for Consistency of Results in Paired Serum Bile Acid Assays
Paired serum bile acid (SBA) samples from 30 dogs with presumptive hepatobiliary disease and 30 apparently healthy dogs were submitted to three diagnostic laboratories to validate agreement between these facilities. Sensitivities and specificities ranged from 50% to 70% and from 74% to 85%, respectively, for all laboratories. The κ index confirmed almost perfect agreement between laboratories. Variation in sample handling and processing by different laboratories should not be considered a primary factor when aberrant SBA values are encountered. Documenting agreement between laboratories is essential as inconsistencies could direct unnecessary medical or surgical intervention when patients are evaluated at different hospitals during the course of their disease.
Introduction
Fasted and postprandial serum bile acid (SBA) assays are commonly used in veterinary medicine for the diagnosis of hepatobiliary dysfunction and portosystemic shunts (PSS) in dogs. The diagnostic value of SBA measurement in the identification of liver dysfunction has previously been compared with more conventional indicators of hepatobiliary disease, such as total bilirubin (TB), albumin, alkaline phosphatase (ALP), alanine aminotransferase (ALT), the ammonia tolerance test, and fasting ammonia concentrations.1–4 Measurements of pre- and postprandial SBA have repeatedly been found to be a valuable and reliable test for the diagnosis of PSS in dogs, with fasting values ≥30 μmol/L exceeding 90% specificity for liver disease and reaching 100% specificity with concentrations ≥50 μmol/L. 1,3,5 However, aberrant SBA values are often encountered in clinical settings. Measurement of inappropriately elevated bile acid concentrations in a peripheral blood sample can be influenced by several variables, including patient factors (e.g., interdigestive gallbladder contraction, delayed gastric emptying, altered gastrointestinal transit time, hemolysis, lipemia, hypertriglyceridemia) as well as errors in sampling protocols or utilization of assays that are unvalidated.6,7
The development of commercially available tests, such as the direct enzymatic photospectrometric method and solid-phase radioimmunoassay (RIA) test kits have allowed greater clinical application for the quantification of SBA concentrations as a diagnostic test for hepatobiliary dysfunction. 5,7–10 To date, variability of results of SBA assays performed by commonly used veterinary laboratories has not been evaluated. As many veterinary patients are examined and treated by a variety of doctors and clinics and test results are shared between care providers, uniformity of results between diagnostic laboratories is essential. It is not an uncommon scenario for veterinary specialists to recommend periodic evaluation of paired SBA concentrations following either intrahepatic or extrahepatic PSS attenuation. Although clients may travel a great distance for the surgical procedure, follow-up evaluations are often performed locally. Differences in sampling protocols and laboratories used are at the discretion of the local veterinarian, and results are typically relayed to the veterinary specialist for interpretation and clinical application. Although SBA concentrations may not dramatically differ postoperatively, trends in these values are closely monitored, and correlations are made with clinical signs to measure successful surgical outcome.11 Based on this information and the patient's clinical status, recommendations may be made for prolonged medical therapy, additional diagnostics, or further surgical intervention.
To the authors’ knowledge, interlaboratory variance of paired SBA assays has not previously been investigated. The purpose of this clinical investigation was to determine and compare the sensitivity and specificity of SBA assays performed at three commonly used veterinary diagnostic laboratories using samples obtained from apparently healthy dogs and those with presumptive underlying liver dysfunction. The hypothesis was that there would be no statistically significant variability of results obtained from different laboratory settings.
Materials and Methods
Sixty client-owned dogs presenting to the James L. Voss Veterinary Medical Center at Colorado State University (CSU) between May 2008 and May 2009 were included in the study after obtaining informed consent from pet owners. Enrolled patients were divided into two groups based on their degree of health status and presumptive diagnosis: 30 apparently healthy dogs and 30 dogs with suspected hepatobiliary dysfunction or portovascular anomalies. Group determination and presumption of health status were made after considering the history, physical examination findings, clinical signs, and conventional diagnostic test results (including blood tests, abdominal ultrasonography, and nuclear scintigraphy). All dogs were >3 kg to allow for adequate blood sampling without increased risk to the patient. The protocol for this study was approved by The Animal Care and Use Committee of CSU.
Patients were fasted for ≥12 hr prior to the initial blood draw (fasted serum sample). The dogs were then fed 2 tablespoons of a high calorie canned dieta, and the paired venous blood sample was drawn 2 hr later (postprandial serum sample). Samples were stored at 4°C (if not submitted for analysis within 30 min). All blood samples were centrifuged for 5 minutes at 3,500–5,000 rpm and analyzed by the CSU Clinical Pathology laboratory within 24 hr. Remaining serum samples were then stored at −80°C for no <3 mo before batch submission to Antech Diagnosticsb and IDEXX Laboratories Incc. Samples were thawed, aliquoted, transferred into polypropylene tubes, and packaged on ice for direct pick-up by Antech Diagnostics or overnight shipment to IDEXX Laboratories. The degree of lipemia and hemolysis in each sample was recorded by the CSU Clinical Pathology Laboratory. Samples were analyzed using the Olympus AU5400 Clinical Chemistry Systemd at both Antech Diagnostics and IDEXX Laboratories and with the Hitachi 971 Automated Chemistry Analyzere at the CSU Clinical Pathology Laboratory. All three laboratories used direct enzymatic methodologye to measure serum bile acids and participated in the Veterinary Laboratory Association Quality Assurance Program to ensure laboratory proficiency.
Statistical Analysis
Sensitivity and specificity were calculated based on the presumptive clinical diagnosis of hepatobiliary dysfunction and reference ranges from each respective laboratory. The values were summarized using standard descriptive statistics. The 95% confidence intervals (95% CI) for sensitivity and specificity were calculated using the Wilson score method. The comparison of sensitivities and specificities between laboratories was performed using a paired McNemar test. The κ index was used as the primary measure to evaluate the level of agreement of diagnostic assessments between laboratories. The 95% CI of the κ indices were computed using the normal approximation method.12 The comparison of the κ indices between fasting and postprandial bile acid samples was performed using the nonparametric bootstrap method.13 To better assess the agreement between two methods of measurement, Bland-Altman plots were used to explore individual differences of serum bile acid assays between the three laboratories.14 A Bland-Altman plot is a scatterplot used to analyze the amount of agreement between two measures by visually depicting the difference (y-axis) between variable averages of these measurements (x-axis). Points plotted near the central line indicate that the two methods of measurement have a high correlation. All statistical tests were two sided and P<0.05 was used to indicate statistical significance. Statistical analyses were performed using SASg.
Results
Breeds enrolled in the “apparently healthy” group included: six mixed-breed dogs, four Labrador retrievers, four golden retrievers, three shih tzu, two Chihuahuas, two Doberman pinchers, and one each of the following: Pekingese, English springer spaniel, border collie, toy poodle, American pit bull terrier, Louisiana Catahoula leopard dog, bassett hound, German shepherd dog, Plott hound, Silky terrier, daschund, and Shiba Inu. Ages ranged from 1.5 yr to 18 yr. Breeds represented in the “presumptive hepatobiliary disease or portosystemic shunts” group included: 6 mixed-breed dogs, four Labrador retrievers, two shih tzu, two Chihuahuas, and one each of the following: Australian shepherd, Pekingese, fox terrier, Maltese, golden retriever, Yorkshire terrier, miniature schnauzer, Scottish deerhound, Weimaraner, Jack Russell terrier, papillon, poodle, Siberian husky, Shetland sheepdog, and border collie. Of these dogs, seven were diagnosed with portosystemic shunts based on abdominal ultrasound, nuclear scintigraphy, and/or surgical confirmation. Ages ranged from 1.6 yr to 15 yr.
Fasted bile acid values for the apparently healthy dogs ranged from 0 μmol/L to 36.00 μmol/L and postprandial values ranged from 0 μmol/L to 53.00 μmol/L. Mean bile acid values were 8.83 μmol/L and 16.12 μmol/L for fasted and postprandial samples, respectively. Fasted bile acid values for all dogs in the hepatobiliary dysfunction and portosystemic shunt group ranged from 0 μmol/L to 288.90 μmol/L. Postprandial bile acid values ranged from 1 μmol/L to 207.00 μmol/L. Mean bile acid values were 37.33 μmol/L and 41.73 μmol/L for fasted and postprandial samples, respectively. Fasted and postprandial bile acids ranged from 27.00 μmol/L to 288.90 μmol/L and from 52.00 μmol/L to 207.00 μmol/L, respectively, for the seven dogs diagnosed with a portovascular anomaly.
The sensitivities and specificities for the three laboratories have been illustrated in Figure 1. For the CSU Clinical Pathology Laboratory, the sensitivities for fasting and postprandial samples were 64% (95% CI, 46–82%) and 57% (95% CI, 39–75%), respectively. The specificities were 82% (95% CI, 70–95%) for fasting samples and 79% (95% CI, 66–93%) for postprandial samples. For Antech Diagnostics, the sensitivities for the fasting and postprandial samples were 50% (95% CI, 29–71%) and 50% (95% CI, 28–72%), respectively. The specificity was 84% (95% CI, 72–97%) for fasting samples and 81% (95% CI, 67–95%) for postprandial samples. For IDEXX, the sensitivities for fasting and postprandial samples were 61% (95% CI, 43–79%) and 70% (95% CI, 53–88%), respectively. The specificity was 85% (95% CI, 73–97%) for fasting samples and 74% (95% CI, 59–90%) for postprandial samples.
Citation: Journal of the American Animal Hospital Association 48, 1; 10.5326/JAAHA-MS-5684
There were no statistically significant differences detected in sensitivity or specificity for fasting or postprandial samples between laboratories. Given this data, there is moderate sensitivity and good specificity for each laboratory when evaluating fasting and postprandial bile acid assays for apparently healthy dogs and those with presumptive hepatic dysfunction or PSS.
The κ index varied between 0.76 and 0.96, indicating a substantial and almost perfect level of agreement between the three diagnostic laboratories (Table 1). The κ index between the CSU Clinical Pathology Laboratory and Antech Diagnostics for fasting samples was significantly smaller than the κ index for the corresponding postprandial samples (0.76 versus 0.96, P<0.001). There were no other statistically significant differences in κ indices between fasting and postprandial samples. The Bland-Altman plots (Figures 2, 3) show that there is a relatively high level of agreement in serum bile acid results between the three laboratories, especially for the postprandial samples. A subgroup analysis indicated that outlying values or highly varied interlaboratory results were associated with dogs with confirmed hepatobiliary disease or portovascular anomalies; however, these findings were statistically insignificant.
*Interpretation of κ index: <0: no agreement; 0–0.2: slight agreement; 0.21–0.40: fair agreement; 0.41–0.60: moderate agreement; 0.61–0.80: substantial agreement; >0.8 almost perfect agreement12 CI, confidence interval; Antech, Antech Diagnostics; Clin Path, Colorado State University Clinical Pathology Laboratory; IDEXX, IDEXX Laboratories, Inc.; SE, standard error.
Citation: Journal of the American Animal Hospital Association 48, 1; 10.5326/JAAHA-MS-5684
Citation: Journal of the American Animal Hospital Association 48, 1; 10.5326/JAAHA-MS-5684
Discussion
Serum bile acids are commonly used for the diagnosis of liver disease and portovascular anomalies in veterinary patients. Verifying and documenting the uniformity of results is essential as inconsistency between laboratories could direct unnecessary or inappropriate diagnostics, medical therapy, or surgical intervention. Because the three laboratories in this study use the same analytic method of serum bile acid measurement, the authors hypothesized that there would be no significant difference between the results. The impact of laboratory variation is unknown for this analyte, and results could be influenced by site differences in such factors as processing time, storage temperature, instrument calibration, and reagents.
This study confirms that there are no significant differences in serum bile acid results between these laboratories. Moderate sensitivity and good specificity were noted for each laboratory when evaluating fasting and postprandial serum bile acid samples in apparently healthy dogs and those with liver disorders. Sensitivity and specificity did not differ between laboratories. The κ index ranged from 0.76 to 0.96 for all samples at all three laboratories, indicating a very high level of agreement of results.12 Bland-Altman plots also depicted little variation in the average discrepancy between laboratories, indicating a relatively high level of agreement in bile acid results. However, this agreement was less consistent with very high bile acid concentrations as evidenced by a broad scatter of points away from the central bias line. These outlying values did correspond to the clinically confirmed cases of liver dysfunction in this study; however, additional correlation and significance could not be found.
Consideration should be given to the limitations of this study. Calculations of sensitivity and specificity were based on a presumptive diagnosis of either hepatobiliary disease or portovascular anomaly. In some patients where definitive diagnosis was unavailable due to financial limitations, euthanasia without necropsy, or other compounding factors, abnormalities in standard blood chemistry parameters of liver function (e.g., ALT, ALP), abdominal ultrasonography, and clinical signs were used to separate patients into the two study groups. The inability to obtain a more definitive evaluation of the patient's disease status could be considered a shortcoming of this study; however, treatment decisions are often made in veterinary medicine based solely on blood chemistry abnormalities and clinical judgment. The consequences of serum hemolysis and lipemia were not directly evaluated in the study, although these factors have been shown to artifactually alter bile acid yield when measured by this enzymatic method.8 Specifically, increasing levels of sample hemolysis have been reported to erroneously decrease the total bile acid measurement, whereas elevated lipemic indices can inaccurately overestimate bile acid results.7,8 Any potential effect of these interfering compounds on laboratory results was minimized by using paired samples. Furthermore, the influence of hemolysis or lipemia would be consistent across samples from each individual dog, and based on the results of this study, did not appear to affect variability.
Additionally, frozen samples evaluated by IDEXX Laboratories and Antech Diagnostics were compared with freshly acquired serum processed by the CSU Clinical Pathology Laboratory. However, these differences in sample handling did not appear to cause significant difference in interlaboratory variability and were in agreement with a previous study in which there was no significant difference in bile acid concentrations measured from samples that had been stored at 4°C, 25°C, and −20°C.8
Conclusion
Although there are numerous variables that may affect the reported concentration of bile acids in peripheral blood, interlaboratory variability is unlikely to be a significant cause and should not be considered a primary factor when aberrant values are obtained. Analysis by any of the three laboratories included in this study can be used for assessment of bile acids in a canine patient and interpreted in the context of their reported reference intervals to aid diagnosis. With this confidence, paired serum bile acid concentrations from differing laboratories could be clinically applied as an accurate and reliable indicator of successful surgical treatment of portosystemic shunts in dogs, or could signal the need for additional medical or surgical intervention.
Sensitivity (light gray bars) and specificity (black bars) for fasting and postprandial bile acid assay samples performed at three reference laboratories (bars indicate 95% confidence intervals).
Bland-Altman plots of serum bile acid assays for fasting samples and the standard deviation of the differences between the two assays.
Bland-Altman plots of serum bile acid assays for postprandial samples and the standard deviation of the differences between the two assays.
Contributor Notes
M. Nanfelt's current affiliation is Carolina Veterinary Specialists, Huntersville, NC.
K. Kennedy's current affiliation is Department of Veterinary Clinical Sciences, College of Veterinary Medicine, Washington State University, Pullman, WA.
