Sigma metrics and quality goal index: A new road map in clinical chemistry

Abstract

AimSigma measurements are a standard tool for quality assessment of test performance in a laboratory. In this study, we aimed to calculate sigma metrics and quality goal index (QGI) for 28 biochemical parameters. Sigma values of each assay were calculated, based on the bias and coefficient of variation from internal quality control (IQC) and external quality assurance scheme (EQAS).MethodsExternal quality assessment (EQA) and internal quality control data for 28 parameters in a biochemical laboratory were collected from July 2019 to February 2020. The sigma values of each assay were calculated, based on the bias, total error allowable, and coefficient of variation, according to the quality goal index, the main causes of poor performance were determined to guide quality improvement. This study was conducted in the Haseki Training and Research Hospital Biochemistry Laboratory. Sigma metrics calculation was performed as (TEA − Bias)/CV for 28 biochemistry parameters analyzed with AU5800 [Beckman Coulter (BC), USA]. Total allowable errors were followed as per Clinical Laboratory Improvement Amendments (CLIA) guidelines.ResultsAt IQC 1, eight of the 28 parameters (AST, ALT, LDH, CK, ALP, HDL, T. Bil and D. Bil) showed world-class performance. At IQC 2, three of the 28 parameters (ALP, T.bil, and Crea) achieved 6 sigma (world-class performance), and three parameters (Amilase, K, Cl, and Lipase) showed world-class performance for EQC. The quality goal index (QGI) was calculated for items with analysis performance < 3 sigma, and the main causes of poor performance were determined to guide quality improvement.ConclusionSigma metric analysis provides a benchmark for the laboratory to design a protocol for IQC, address poor assay performance, and assess the efficiency of the existing laboratory process. Six Sigma methodology is an effective tool for evaluating the performance of biochemical analytes and conducive to quality assurance and improvement. The quality goal index is a calculation that complements this method.

Keywords

Introduction

Medical Laboratories must produce accurate, precise, and comparable results for correct diagnosis and treatment practices. For this purpose, each of the preanalytical, analytical, and post-analytical processes must be continuously checked and improved. The purpose of clinical laboratory tests is to support the diagnosis, monitor treatment, and assess the risk of disease progression. To have value for creating clinical conclusion, an individual laboratory test result must have a total error small enough to reflect the biological condition being assessed.¹ Also, clinical laboratory testing results are important for ensuring patient safety. Approximately two-thirds of important clinical decisions on patient management are based on laboratory test results.²
In clinical laboratories, medical technologists are trained to focus on achieving Quality Control (QC) results within the defined acceptable limits.³ The “Six Sigma Methodology” is a quality management tool based on statistical calculations, focused on process variables, and providing information about the procedure. The sigma value of a test is a well-defined and quantitative measurement of the quality of this test. Six Sigma is a quality management method to improve assay quality. The sigma methodology has mainly been applied to pre-analytical and analytical processes in clinical laboratories, focusing on the evaluation of biochemical and immunoassay tests. The higher the sigma values, the lower the chance of false test results by the laboratory. It can easily quantify the exact number of errors by combining bias, precision, and total allowable error (TEa). A sigma level < 3 is an indication of a poor performance procedure, whilst good performance is indicated by a sigma level > 3. A Sigma level of 6 or greater indicates world-class performance.⁴ In this study, we aimed to evaluate our laboratory analytical performance with the Clinical Laboratory Improvement Amendments (CLIA) criteria performance (available at: https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA).
The quality goal index (QGI) is a newer parameter to represent the relative extent to which both bias and precision meet their respective quality goals.⁵ In this study, the performance of 28 analytes was evaluated by calculating sigma values from coefficient of variation (CV), bias, and TEa. In addition, QGI analyses were further performed to identify problems related to the measurement procedures for analytes with a sigma value <3.⁶

Materials and Methods

This is a retrospective study, and the data required for the study were extracted between July 2019 to February 2020 in the Haseki Training and Research Hospital Biochemistry Laboratory. Sigma metrics calculation was performed as (TEA − Bias)/CV for 28 biochemistry parameters analyzed with AU5800.[Beckman Coulter (BC), USA] The CV was calculated based on IQC data (BC control serum two-level/one run per day).
This study included the following 28 clinical biochemistry parameters: Albumin(Alb), Alanine Aminotransferase (ALT), Alkaline phosphatase (ALP), Aspartate Aminotransferase (AST), Amylase (Amy), Anti-streptolysin-O (ASO), Iron (Fe), Unsaturated Iron Binding Capacity (UIBC), Direct Bilirubin (D.Bil), Phosphorus (P), Calcium (Ca), Chloride (Cl), Total Cholesterol (T.Chol), Creatinine(Crea), Creatine Kinase (CK), Glucose (Glu), Gamma-Glutamyl Transferase (GGT), High-Density Lipoprotein Cholesterol (HDL), Lactate Dehydrogenase (LDH), Lipase (Lps) , Magnesium (Mg), Potassium (K), Total protein (T.prot), Total bilirubin (T.bil), Sodium (Na), Triglycerides (Trig), Blood urea nitrogen (BUN), Uric acid (UA). The sigma metrics were calculated by the following formula:.[Sigma = (TEa − Bias)/CV].⁷ Whereas TEa is a total allowable error, and bias and CV are the indicators of systematic and random errors, respectively.
QGI represents the relative extent to which both bias and precision meet their respective quality goals. It was calculated using the following formula:.[QGI = Bias/1.5 CV] QGI represents the reason behind the lower sigma value i.e., imprecision, inaccuracy, or both. For analytes that fall short of Six Sigma quality, a QGI score of < 0.8 indicates imprecision, QGI > 1.2 indicates inaccuracy, and QGI score of 0.8-1.2 indicates both imprecision and inaccuracy.⁷
CV is the standard deviation (SD) expressed as a percentage and is a measure of the variability of an assay.[CV = (SD/Mean) × (100)]
Bias is the systematic difference between the expected results obtained by the laboratory test method and the results that would be obtained from an accepted reference method.
TEa was followed as per the CLIA guidelines (Clinical Laboratory Improvement Amendments (CLIA) | CMS). The total error (TE) of parameters was also calculated by the following formula:.[TE = Bias + 1.65 CV].⁶

Results

The bias %, CV%, TEa, sigma, and QGI values of clinical chemistry tests are shown in Table 1. Sigma values calculated with TEa, bias, and CV are shown in Table 2. The QGI according to sigma values are provided in Table 3.
While IQC eight analytes (AST, ALT, LDH, CK, ALP, HDL, T.Bil and D.Bil ) showed an ideal performance of ≥6 sigma for level 1; five analytes (T.prot, UIBC, Alb, Chol, and Lipase) showed an average performance of <3 sigma for IQC 1. Also, IQC three analytes (ALP, T.bil, and Crea) showed a performance of ≥6 sigma for level 2; five analytes (T.chol, Alb, P, Fe, UIBC, and Lipase) showed an average performance of < 3 sigma for level 2. Sigma values of BUN, Glu, AST, Alb, Fe, and HDL were below 3 for EQA. Sigma values Amilase, K, Cl, and Lipase were ≥ 6 for EQA. Sigma values of Glu, BUN, UA, Crea, Trig, AST, Amilase, Ca, P, Mg, Na, K, Cl and ASO were in the range of 3 to 6 For IQC 1. Sigma values of Glu, BUN, UA, Crea, Trig, AST, Fe, Na, D.bil, Amilase, Ca, Na, K, Cl, HDL, ASO and Lps were in the range of 3 to 6 for IQC 2. Also, EQC fourteen analytes (Crea, Trig, ALT, GGT, CK, LDH, CK, ASO, T.Bil ,D.Bil, Aml, Ca, P, Mg, UIBC) showed a performance in the range of 3 to 6 sigma .Sigma value > 6 was found, ALP T-Bil and LDH for both levels of IQC. Sigma values of UIBC and T.chol for both levels of IQC were lower than 3. But EQAs of these tests were in the range of 3 to 6.
Table 3 summarizes the QGI ratio of analytes with lower sigma values (< 3). The quality goal index (QGI) was calculated for items with analysis performance <3 sigma and the main causes of poor performance were determined to guide quality improvement. QGI ratio indicated that out of three and four parameters of IQC1 and IQC2 failed to meet Six Sigma quality performances. The main problem was an inaccuracy in the case of UIBC, lipase, and total cholesterol (QGI > 1.2), imprecision in the case of Alb, P, iron level 2 (QGI < 0.8), and both imprecision and inaccuracy for UIBC (IQC2).

Discussion

Good laboratory practice (GLP) requires every individual laboratory to design a customized Individualized Quality Control Plan (IQCP), a protocol based on Sigma values obtained from Sigma metric analysis (ISO – ISO 15189:2012 – Medical laboratories -Requirements for quality and competence. Available at: https://www.iso.org/standard/56115.html). The incorporation of sigma metrics results in the reduction of laboratory errors by maintaining six standard deviations between the parameter average and its upper and lower limits.⁸
Nanda et all. determined that six sigma values were greater than 6 for some routine biochemistry tests (AST, ALT, ALP, total bilirubin, and uric acid) on Cobas Integra analyzer. Sigma values less than 3 were calculated for total protein, albumin, total cholesterol, and chloride tests in their study.⁹ When these data are compared with our study, it is seen that a sigma value less than 3 is calculated for Alb and UIBC. T.prot and Cl are greater than 3 in our study. Sigma value was found to be less than 3 for albumin in both studies. But EQAS results were good for both of them.
Parameters with low sigma values should be improved with a strict QC strategy. The six-sigma concept is important in controlling the quality of laboratory tests.⁹ Total analytical errors may differ according to accepted error classifications such as Richos, Rilibak, CLIA.¹⁰ However, sigma metric evaluation can be more objective because it is obtained from systemic and randomized error values, bias, and standard deviation.¹¹ In our study, unlike other sigma’s metric clinical chemistry analyzes, we also calculated the quality target index. Our aim here was to identify the problem. Unlike previous studies, we calculated the QGI in tests with sigma values. We have considered their total analytical error value. UIBC, lipase, Fe, P, and total cholesterol were short of sigma metrics with a value < 3. QGI ratio for parameters with sigma < 3 depicted inaccuracies in the case of total cholesterol, UIBC, and lipase (QGI > 1.2), imprecision in the case of tests Albumin, Iron and Phosphor (QGI < 0.8), and imprecision and inaccuracy in the case of UIBC level 2. There are certain limitations in the sigma metrics system because we have observed no problems in the CV % and bias % of glucose (level 2), urea (level 2), and total cholesterol (level 2 and level 3), but sigma shows a lesser value. In the case of AST and ALT, the calculated TEa is higher compared to the allowable error as per CLIA, which is reflected in the QGI and sigma metrics. The tests we analyzed in the biochemistry laboratory were below total analytical error values according to CLIA. This showed us that each laboratory should also perform sigma analysis besides TEA values. We also tried to identify problems by making a QGI evaluation. We have reviewed our daily quality control frequency. A simple guideline for choosing Westgard rules and levels of IQC processed are as follows: for biochemical parameters with Sigma Scale 6 or above (excellent performance), evaluate with one level of QC per day (alternating levels between days) and follow 1-3 s Westgard rule alone. With Sigma Scale 4–6 (good/acceptable performance), evaluate with two levels of control once daily and follow 1-3 s, 2-2 s, R4 s Westgard multi rules. With Sigma Scale 3–4 (poor performance), use two levels of controls twice daily and follow 1-3 s, 2-2 s, R4s, and 4-1 s Westgard’s multi rules. With the Sigma Scale of <3 (problem analyte), root cause analysis should be performed; method performance must be improved before the method can be routinely used..⁸ Finally, sigma metric analysis provides a benchmark for the laboratory to design a protocol for IQC, address poor assay performance, and assess the efficiency of the existing laboratory process.

Declarations

References

1. McPherson RA, Pincus MR, eds. Henry's Clinical Diagnosis and Management by Laboratory Methods. 23rd ed. Elsevier; 2017.
   PubMed · Google Scholar
2. Aslan D, Sert S, Aybek H, Yılmaztürk G. Assessment of total clinical laboratory process performance: normalized OPSpecs charts, Six Sigma, and patient test results. Turk J Biochem. 2005;30(4):296-305.
   PubMed · Google Scholar
3. Singh B, Goswami B, Gupta VK, Chawla R, Mallik V. Application of sigma metrics for the assessment of quality assurance in clinical biochemistry laboratory in India: a pilot study. Indian J Clin Biochem. 2011;26(2):131-135. doi:10.1007/s12291-010-0083-1
   Article · PubMed · Google Scholar
4. Aslan D, Demir S. Six Sigma quality management in laboratory medicine. Turk J Biochem. 2005;30(4):272-278.
   PubMed · Google Scholar
5. Kumar BV, Mohan T. Sigma metrics as a tool for evaluating the performance of internal quality control in a clinical chemistry laboratory. J Lab Physicians. 2018;10(2):194-199. doi:10.4103/jlp.jlp_102_17
   Article · PubMed · Google Scholar
6. Nevalainen D, Berte L, Kraft C, Leigh E, Picaso L, Morgan T. Evaluating laboratory performance on quality indicators with the Six Sigma scale. Arch Pathol Lab Med. 2000;124(4):516-519. doi:10.5858/2000-124-0516-elpoqi
   Article · PubMed · Google Scholar
7. Westgard JO, Westgard SA. The quality of laboratory testing today: an assessment of sigma metrics for analytic quality using performance data from proficiency testing surveys and the CLIA criteria for acceptable performance. Am J Clin Pathol. 2006;125(3):343-354. doi:10.1309/v50h4frvvwx12c79
   Article · PubMed · Google Scholar
8. Westgard JO. Six Sigma Quality Design and Control. 2nd ed. Westgard QC Inc; 2006.
   PubMed · Google Scholar
9. Nanda SK, Ray L. Quantitative application of sigma metrics in medical biochemistry. J Clin Diagn Res. 2013;7(12):2689-2691. doi:10.7860/jcdr/2013/7292.3700
   Article · PubMed · Google Scholar
10. Nar R, Emekli DI. The evaluation of analytical performance of immunoassay tests by using Six Sigma method. J Med Biochem. 2017;36(4):301-308. doi:10.1515/jomb-2017-0026
    Article · PubMed · Google Scholar
11. Hens K, Berth M, Armbruster D, Westgard S. Sigma metrics used to assess analytical quality of clinical chemistry assays: importance of the allowable total error (TEa) target. Clin Chem Lab Med. 2014;52(7):973-980. doi:10.1515/cclm-2013-1090
    Article · PubMed · Google Scholar

Tables

Additional Information

Publisher’s Note
Bayrakol MP remains neutral with regard to jurisdictional and institutional claims.

Rights and Permissions

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). To view a copy of the license, visit https://creativecommons.org/licenses/by-nc/4.0/

View full license details →