Intralaboratory Reliability and Variability for Allergen-Specific Immunoglobulin Type E Serology Testing
ABSTRACT
Atopic dermatitis is a very common condition affecting dogs and often managed with allergen-specific immunotherapy, which requires accurate identification of causative allergens. Serology testing is used commonly. Serum was collected from 35 atopic dogs and separated into three samples each (1, 2, and 3). Samples 1 and 2 were sent to IDEXX Laboratories the same day; sample 3 was stored at –80°C and submitted ∼30 days later. Specific immunoglobulin type E reactivity to various allergens were determined using monoclonal anti-canine enzyme-linked immunosorbent assay (ELISA) and expressed as ELISA absorbance units. Percent difference ranged from 14.30 to 127.34% for samples 1 and 2. These values increased when comparing samples a month apart (21.78 to 129.65%). Between samples 1 and 2, for each allergen there were differences in interpretation 15.18% of the time; 32 of 35 dogs (91.4%) had at least one allergen with a different interpretation. Comparing sample 3 and the average of samples 1 and 2, differences in interpretation increased to 22.32%; all dogs had at least one allergen that was interpreted differently. These differences in interpretation can alter immunotherapy. Overall, results show the need for better reliability for allergen-specific immunoglobulin type E serology testing using monoclonal anti-canine ELISA.
Introduction
Atopic dermatitis (AD) is a very common dermatosis of dogs and has been defined as a genetically predisposed inflammatory and pruritic allergic skin disease with characteristic clinical features and is associated most commonly with immunoglobulin type E (IgE) antibodies to environmental allergens.1,2 Allergen-specific immunotherapy is frequently prescribed to aid in the management of AD. These are formulated based on allergen reactivity results by intradermal testing (IDT) and/or allergen-specific IgE serology testing (ASIS) in conjunction with history and aerobiology (environment).3 Therefore, reliable and accurate identification of the causative allergens based on testing is necessary for an effective immunotherapy protocol.
In veterinary medicine, ASIS is offered by at least six commercial laboratories in the United States. There is currently little independent oversight on quality control. Previous studies on reliability of ASIS in veterinary medicine have shown significant intra- and interlaboratory variability.4,5 Studies in human medicine have also shown similar findings, as results from ASIS are not interchangeable between different laboratories and assays.6,7
Recently, a commonly used company, Greer Laboratories, was purchased by IDEXX Laboratories. This acquisition could lead to changes in assay reliability. Greer/IDEXX established a proficiency monitoring program for its laboratories that routinely run the monoclonal anti-canine enzyme-linked immunosorbent assay (macELISA)a for ASIS. Results were published in 2012 and 2015, before and after the change in ownership, which revealed that, in contrast to previous studies, variability between and among all laboratories was low. However, it was concluded that until an external and independent quality assurance program is in place, independent, peer-reviewed studies are needed to verify and document reliability.8,9 Therefore, this study sought to independently evaluate intralaboratory reliability and variability of the macELISA test for Greer Laboratories, specifically the Memphis, Tennessee, location. We hypothesized that the assay used should produce reproducible results between all samples.
Materials and Methods
Study Population and Inclusion
The study was approved by the University of Illinois Institutional Animal Care and Use Committee. Written consent was obtained from all clients that participated. Dogs who presented to the University of Illinois Dermatology service between October 2015 and April 2016 were included in the study if they had clinical signs and history consistent with AD based on Favrot’s criteria.10 Current treatment with systemic and/or topical antibiotics, antihistamines, and/or fatty acid supplements was allowed. Dogs under 12 kg were excluded.
Blood Collection and Sample Handling
A minimum of 18 mL of whole blood was collected into glass serum tubes from a jugular or cephalic vein, allowed to clot, and then centrifuged at 1000g for 10 min. The serum from each dog was then divided into three equal aliquots and placed in plastic vials containing at least 2 mL per vial to create samples 1, 2, and 3. The samples were then given separate, randomized first and last names. Samples 1 and 2 were sent at room temperature by overnight mail to IDEXX Laboratoriesa the same day of blood collection as directed by the lab. Sample 3 was sent in the same way about 30 days later, after being stored at –80°C. Each sample was tested with the Midwest United States environmental allergen panel. General categories included fungi, grasses, insects, mites, trees, and weeds, for a total of 49 allergens. The samples were submitted according to the laboratory’s guidelines. The laboratory was unaware of the purpose of the study and identity of the samples.
Allergen-Specific IgE Serology Testing
The specific IgE reactivity to the various allergens in the serum samples was determined using the direct-bind macELISA as described previously.5 All results were expressed as ELISA absorbance units (EAU), which is background corrected. EAU values 0–79 are considered negative. EAU values 80–299 are considered positive (+). EAU values ≥300 are considered significantly positive (+++).
Statistics
A sample size of 35 dogs was determined based on the following a priori conditions: an alpha = 0.05, a power = 0.8, and an expected difference in means of no more than 5 and a standard deviation of 5. For the EAU values, Mauchly’s Test of Sphericity was evaluated. If sphericity was violated, Greenhouse-Geisser correction was used for interpretation. A Cochran’s Q Test was performed to evaluate the interpretation of the EAU values as positive or negative. Bonferroni adjustment was used for multiple comparisons. A commercially available statistical software package was used for the statistical analysisb. A P value of <.05 was considered significant.
Descriptive data was calculated by the average percent difference between EAU values of samples 1 and 2 as well as that of sample 3 compared with the average of samples 1 and 2 for each allergen.
Differences in interpretation (negative versus + versus +++) were recorded between samples 1 and 2 as well as that of sample 3 compared with average of samples 1 and 2. The number of different interpretations as well as percentage of the study population (out of 35) was recorded for each allergen.
Results
A total of 35 dogs were included in the study. There were 17 neutered males, 1 intact male, and 17 spayed females. Median age was 4 yr (range 1–10 yr). Mean weight was 28.6 kg (range 12.3–47.5 kg). Breeds included mixed breed (n = 11), American pit bull terrier (n = 5), Labrador retriever (n = 4), English bulldog (n = 3), boxer (n = 2), golden retriever (n = 2), Old English bulldog (n = 1), Chinese shar pei (n = 1), bull terrier (n = 1), basset hound (n = 1), American bulldog (n = 1), Treeing Walker coonhound (n = 1), cocker spaniel (n = 1), and German shorthair pointer (n = 1).
Analysis of differences between EAU values of samples 1, 2, and 3 of individual allergens yielded a statistically significant difference for Lamb’s quarters (P = .037). Cochran’s Q analysis of positive versus negative results found a statistically significant difference for Sage mix (P = .025). No other significant differences were observed, and all values are displayed in Table 1. For each individual allergen, the average percent difference between samples 1 and 2 as well as that of sample 3 compared with the average of samples 1 and 2 are displayed in Figure 1. The lowest percent difference between samples 1 and 2 was 14.30%, the highest was 127.34%, and the mean for all allergens combined was 27.06%. For the comparison between sample 3 and the average of samples 1 and 2, the lowest percent difference was 21.78%, the highest was 129.65%, and the mean for all allergens combined was 43.94%. Comparing interpretations for each allergen yielded an average of 5.31 different interpretations out of a population of 35 (15.18%). Comparing sample 3 and the average of sample 1 and 2 showed a higher average number of differences in interpretation of 7.81 (22.32%). The data for individual allergens is shown in Figure 2.
Citation: Journal of the American Animal Hospital Association 55, 3; 10.5326/JAAHA-MS-6761
Citation: Journal of the American Animal Hospital Association 55, 3; 10.5326/JAAHA-MS-6761
Discussion
ASIS has several advantages over IDT, including convenience, minimal risk, and shorter withdrawal times for antipruritic medications.3,11 However, the test must be accurate and reproducible to formulate the best immunotherapy protocol for the patient in order for these advantages to be beneficial. Older studies performed in the 1990s showed ASIS assays were not very sensitive or specific when compared with the gold standard of IDT. There are no recognized allergen standards for veterinary laboratories to calibrate their assays and no standardized source of allergen extracts. There are also many different laboratories that use different methods to measure allergen-specific IgE. Examples include use of preassay serum treatment, solid- versus liquid-phase IgE capturing techniques, anti-canine IgE versus high-affinity IgE receptor detecting reagents, and colorimetric versus radiometric methods. There is also no external regulatory agency overseeing quality assurance procedures.12 Therefore, the reproducibility of ASIS has been continually under question over the past 2 decades as ASIS has become more commonly performed because of the advantages above.
The results of our study showed two allergens that yielded statistically significant differences between all samples (Table 1). Lamb’s quarters based on EAU values and Sage mix based on positive versus negative interpretation. However, descriptive statistics looking at percentage differences between samples showed much more variability (Figure 1). Even in samples sent on the same day, there was significant variation as percent difference ranged from 14.30 to 127.34%. Further consideration was given to the fact that variance in negative samples potentially skews the results because a reading of 5 EAU compared with a reading of 20 EAU is a 400% increase. However, when only positive results (EAU > 80) were evaluated, similar values were noted. We also found highly variable differences in the interpretation of results as positive or negative. This variance may be because of inaccuracies of the macELISA testing method by IDEXX. The methodology itself may not be sensitive or specific in detecting allergen-specific IgE. Therefore, it would bind to other IgEs, causing variable and inaccurate results. Inaccuracies of the macELISA may also be a result of analytical variations in the laboratory including sample mix-up or mishandling, incorrectly calibrated or malfunctioning instruments, out-of-date reagents, inappropriate timing or temperature for the assay procedure, failure to follow the assay protocol strictly, and a lack of quality assurance procedures.5 The high variance may even indicate variability testing for IgE as a whole, suggesting that better methods are needed to measure IgE more sensitively and specifically.
Moreover, this variance increased when samples were sent a month apart. The overall range of percentage differences was higher (21.78–129.65%). When compared, the percent difference between sample 3 and the average of samples 1 and 2 was higher than that between samples 1 and 2 by a mean of 16.89%. This suggests that there is more variability when same samples are analyzed at different times. The increase in variability over time indicates that there may be issues with storage. However, based on laboratory claims and previous studies, IgE levels should be stable at –80°C. IDEXX claims on their website that serum samples should be stable for 2 wk at 2–8°C and up to 2 mo frozen.13 Previous studies in both human and veterinary medicine have shown stability of IgE for ASIS stored at –80°C to –14°C for varying times ranging from 4 wk up to 1 yr.6,14,15 There was also potential for freeze/thaw cycles to cause variability as sample 3 of multiple dogs were taken out of the freezer on multiple occasions for a short period (<1 min) of time. However, IgE has been shown to be stable after 25 freeze/thaw cycles, so this is unlikely.16 Another possibility for variation may be changes in IgE concentrations in the shipping process. The laboratory instructions do not call for any specific temperature requirements for shipping. Therefore, temperature during shipping may fluctuate depending on time of day, season, or differences in shipping truck environment. However, this is less likely as all samples were shipped with the same process.
Of particular note, the percent differences of flea and elm were notably higher than other allergens; 127 and 129.65% for flea and 68.24 and 123.72% for elm (percentage difference between samples 1 and 2 and that between sample 3 and the average of samples 1 and 2, respectively), whereas others were in the 20–40% range. This may indicate that IgE for flea and elm may be difficult to accurately measure with the macELISA methodology or that they are particularly unstable compared with other allergen-specific IgEs. IgE for Lamb’s quarters and Sage mix may have similar issues evidenced by statistically significant differences between samples. When looking at allergen groups, mites and trees had particularly higher percentage differences between samples.
To get a better idea of the clinical implications of these differences of EAU values between samples, differences in interpretation of the values (negative versus + versus +++) were analyzed. Based on this analysis, for samples sent on the same day, allergens were interpreted differently 15.18% of the time. When formulating immunotherapy, this would mean on average, around 7 allergens (out of 49) could potentially be different. Of 35 dogs, 32 (91.4%) had at least one allergen with a different interpretation. Comparing samples a month apart, the percentage increased to 22.32%. Similarly, this would mean on average, ∼11 allergens could potentially be different for immunotherapy. All dogs had at least one allergen that was interpreted differently. The chances for a different interpretation are shown for each allergen in Figure 2. Of particular note, Alternaria, flea, red mulberry, and oak mix had percentages ≥40%.
Therefore, although there was not much statistical difference, we feel clinically there were significant differences as seen by the variation and the differences in interpretation.
Various previous studies have looked at the reliability of ASIS. Patterson et al. evaluated intralaboratory reproducibility for Greer, which used the same macELISA methodology that IDEXX now uses.5 The same serum sample was sent 2 days apart, with the later sample stored at –70°C. The results showed an overall lack of reproducibility with 67% of allergens having a significant difference in modified absorbance units between samples 1 and 2. More recently, reports looking at intralaboratory reproducibility and variability were done by Lee et al. (2012 and 2015) and Thom et al.8,9,14. The studies by Lee et al. were funded by Greer/IDEXX to monitor proficiency of their laboratories. In the most recent study, variability was relatively low. The average intra-assay variance among positive calibrators was 6.0% (range 2.7–16.1%) and 11.8% (range 4.9–34.3%) for the negative calibrator. Thom et al. looked at intralaboratory reproducibility of repeated samples sent to the same laboratory in the same day.14 Three different laboratories were evaluated and showed good reproducibility with an intralaboratory disparity of 3.14% (40/1275 measurements).
These more recent studies by Lee et al. suggest that reproducibility is relatively good, which is in contrast to the report by Patterson et al.5,8 The assay tested in Thom et al. used a different methodology, the high-affinity IgE receptor.14 Lee et al. concluded that independent, peer-reviewed studies were needed to confirm this, which was the purpose of our study.8 It should also be noted that the studies by Lee et al. and Thom et al. evaluated reproducibility within the same day.8,9,14 We wanted to evaluate if reproducibility would be different if samples were sent a month apart versus on the same day. Therefore, our study evaluated samples sent on the same day (samples 1 and 2) and those sent a month apart (sample 3). Variances reported in our study (Figure 1) are much higher than those reported by Lee et al. potentially as a result of the reasons mentioned above. It is also possible that pooled serum, as used in the study by Lee et al., may yield more reliable results as they are more similar to the calibrators used. Our results are more similar to those reported in the study by Patterson et al., which was also done independently.5
Limitations of this study include variability in the number of days between samples sent. This was done so that samples would not be sent on Fridays to avoid samples sitting at a shipping facility unrefrigerated for several days. However, it may have been better to design sample collection and submissions so that time between submissions was the same. Some samples were also sent the day after sample collection because of timing. It would have been better to set up a morning collection for all subjects so that all samples would be sent the same day. When looking at differences in interpretation, this study analyzed strictly based on IDEXX interpretations. However, clinicians can interpret and include allergens in immunotherapy based on their own protocol.
Conclusion
In conclusion, our results show that there is need for better reliability for ASIS done by IDEXX Laboratories at the Memphis, Tennessee, location. This contrasts with the results of proficiency monitoring done by Greer/IDEXX. Based on the variability observed in this study, care should be taken when evaluating results of ASIS as the machines had a significant percentage difference when evaluating the same exact sample on the same day. In particular, the values for flea and elm are most variable. Results also suggest that storing samples to send at a later date could result in even more variability. The variability noted could impact immunotherapy recommendations because of differences in interpretation based on the EAU values. Additional studies looking at reproducibility from other laboratories are needed to evaluate reliability of ASIS as a whole.
Average difference between samples for each allergen as a percentage. This is shown separately for sample 1 versus 2 and the average of sample 1 and 2 versus sample 3.
Difference of interpretations between samples for each allergen expressed as a percentage of the population. This is shown separately for sample 1 versus 2 and the average of sample 1 and 2 versus sample 3.
Contributor Notes
Z. Zhou’s present affiliation is the Department of Small Animal Clinical Sciences, College of Veterinary Medicine, Michigan State University, East Lansing, Michigan.
