Home   Resources   QC Articles   Sigma Metrics, Total Error Budgets & QC

Sigma Metrics, Total Error Budgets & QC

Make sure your test system performance and quality control procedures are aligned with your quality goals

By Curtis A. Parvin, PhD, John Yundt-Pacheco and Max Williams

Good laboratory practice requires that laboratories design their quality control (QC) procedures to assure that reported patient results meet the quality required for their intended use.1 In previous articles in this series, we argued that a laboratory's quality goals should be focused on the risk of reporting unreliable patient results containing measurement error that exceeds the allowable error specified for the analyte.2-5 In this article, we discuss two metrics-sigma values and total error budgets-that can help provide general guidance on how well a laboratory's test system performance and QC procedures align with its specified quality goals.

Sigma values

Laboratory quality specifications are often defined in terms of allowable total error limits (TEa). If the difference between the true concentration of an analyte and the reported concentration in a patient's specimen exceeds TEa the result is considered unreliable. The sigma value expresses the number of analytical standard deviations of the test system process that fit within the specified allowable total error limits. That is,

Sigma = (TEa - Bias) / SD

Bias is the systematic difference between the expected results obtained by the laboratory's test method and the results that would be obtained from an accepted reference. The reference may be another test method, a standard or a consensus reference like a proficiency program or an inter-laboratory peer-comparison program. SD is the total analytical standard deviation of the test method. Equivalently, the quantities can be given as percents:

Sigma = (%TEa - %Bias) / %CV

where %CV is the analytical coefficient of variation of the test method. The Figure gives a graphical example of a test method with 1% bias, 2.5% coefficient of variation and a specified TEa of 10%.

In this case, the sigma value is (10 - 1) / 2.5 = 3.6. That is, 3.6 analytical SDs fit within the 10% quality specification.

Bias can have a significant impact on analytical quality and should usually be removed from the laboratory test system when it is identified. However, eliminating bias below a certain threshold can be difficult and attempts to do so are more likely to increase the overall imprecision of the test method. In general, the value for bias used in sigma computations should be the minimum threshold at which bias is actionable (an attempt to remove it will be made).

Sigma values and QC strategy design

Sigma values are useful for guiding QC strategy design. For a high sigma process it is relatively easy for the laboratory to design a QC procedure to detect any out-of-control condition that could pose a significant risk of producing unreliable results. A relatively large out-of-control condition would have to occur before there would be much chance of producing results that contained errors that exceed the TEa specification and it is easy to design QC procedures that can detect large out-of-control conditions.

Fig. 1 

Graphical Example of a Test Method

On the other hand, for a low sigma process a relatively small out-of-control condition may pose an unacceptably high risk of producing unreliable patient results. It can be challenging to design QC procedures that are good at detecting small out-of-control conditions.

Simple guidelines for choosing appropriate QC rules based on sigma values have been proposed.6 An example of one such guideline is shown in Table 1.

Table 1 

Sample guideline for choosing QC rules based on sigma values

For lower sigma values more QC samples and more powerful QC rules are recommended. Note, a 13S QC rule rejects if any of the QC results differ from their target concentration by more than three SDs. Multirules are combinations of individual QC rules that tend to be more powerful than simple rules such as the 13S QC rule. In general:

  • For large sigma value processes (≥6 sigma), simple QC rules with low false rejection rates are adequate.
  • For intermediate sigma value processes (sigma values between 3.5 and 6) quality goals can be met, but more elaborate QC strategies may be required.
  • For low sigma values (<3.5 sigma) it will be difficult to meet the laboratory's quality goals without finding ways to further reduce the test systems analytical bias, or its analytical imprecision.

Allowable total error, total error and total error budgets

TEa limits specify the measurement error requirements that must be met for the test system to provide patient results that satisfy their purpose. On the other hand, total error (TE) is commonly defined as a summary measure that provides a "reference range" of measurement errors for a test system based on the test system's analytical imprecision and bias. Whereas TEa is a quantity specified by the laboratory that reflects the quality requirement for an analyte, TE is a summary measure reflecting the test system's analytical performance capability. TE is generally defined as:

TE = Bias + Zp*SD

or alternatively as:

%TE = %Bias + Zp*%CV

If Zp is set to 2.33 then about 99% of measurement errors should fall within the TE limits. If Zp is set to 1.645 then about 95% of measurement errors should fall within the TE limits.

The total error budget (TEB) is a quantity that relates the laboratory's test system process capability (TE) to the laboratory's quality requirement (TEa):

TEB = 100*TE / TEa

TEB reflects the percentage of the TEa in patient results that is "consumed" by the laboratory's inherent test system imprecision and bias. How large can TEB be before a lab should be concerned that its test system capability is not well aligned with its quality goals?

We have already shown that sigma values can be used as a guide to the amount of QC effort required to assure that the lab's quality requirements are met. The TEB is a quantity that relates process capability to quality requirements. Table 2 computes both TEB and sigma values assuming TEa = 10% for different amounts of test system bias and imprecision.

Table 2: TEB and Sigma with TE a = 10% as Bias and Imprecision Decrease

Graphical Example of a Test Method

In the first row of Table 2 the bias is 2.5% and the CV is 4.5%. In this case, TE is 9.9% (using the 95% limit definition), TEB is 99%, and the sigma value is 1.67. Thus, a process where nearly 100% of the TEa is consumed by the test system's bias and imprecision is associated with a very poor sigma value.

In the fourth row of Table 2 the bias is 0.5% and the CV is 1.7% (TE = 3.3%, TEB = 33% and sigma = 5.59). A process where only a third of the TEa specification is consumed by the test system's bias and imprecision is associated with a high sigma value. Row 3 of the table suggests that for the test system to be capable of meeting the laboratory's desired quality goals (sigma values ≥3.5), the TEB should not exceed 50%. If more than 50% of your allowable error specification is "consumed" by your test system bias and imprecision, you are going to have a difficult time assuring that your reported patient results are meeting your quality goals. A lab should strive for a TEB of 33% or less.

Dr. Parvin is manager of Advanced Statistical Research; John Yundt-Pacheco is Scientific Fellow; and Max Williams is Division Global Marketing Manager, Bio-Rad

References

  1. International Organization for Standardization (2007) Medical laboratories - particular requirements for quality and competence. ISO 15189. International Organization for Standardization (ISO), Geneva.
  2. Parvin CA, Yundt-Pacheco J, Williams M. The focus of laboratory quality control: Why QC strategies should be designed around the patient, not the instrument. ADVANCE for Administrators of the Laboratory 2011;20(3):48-9.
  3. Parvin CA, Yundt-Pacheco J, Williams M. Designing a quality control strategy: In the modern laboratory three questions must be answered. ADVANCE for Administrators of the Laboratory 2011;20(5):53-4.
  4. Parvin CA, Yundt-Pacheco J, Williams M. The frequency of quality control testing. QC testing by time or number of patient specimens and the implications for patient risk are explored. ADVANCE for Administrators of the Laboratory 2011;20(7):66-9.
  5. Parvin CA, Yundt-Pacheco J, Williams M. Recovering from an out-of-control condition: The laboratory must assess the impact and have a corrective action strategy. ADVANCE for Administrators of the Laboratory 2011;20(11):42-4.
  6. Westgard JO. Six sigma quality design & control, 2nd ed. Madison WI: Westgard QC Inc., 2006.

Copyright 2015

Your Privacy Matters

Before you visit, we want to let you know we use cookies to offer you a better browsing experience. To learn more about how we use cookies, please review our Cookie Policy, accessible from the Manage Preferences link below. We would appreciate your confirmation by either accepting all cookies or by declining and managing your cookie preferences under the Manage Preferences link below.

Back

Cookie Preferences

We use various types of cookies to enhance and personalize your browsing experience on our website. You may review the various types in the descriptions below and decide which cookie preferences you wish to enable. If you wish to decline all non-essential cookies, you may browse our site using strictly-necessary cookies. To learn more about how we use cookies, please visit our Cookie Policy.

Strictly-Necessary Cookies

These cookies are essential for our website to function properly. They either serve as the sole purpose of carrying out network transmissions or they allow you to browse and use features, such as accessing secure areas of the site. These cookies are strictly necessary because services like the shopping cart and invoicing cannot be provided without these cookies. Since these cookies are strictly necessary in order for our website to function, no consent is required to enable them. If you wish to disable these cookies, please update your settings under your browser’s preferences. If these cookies are disabled, please be aware that you will not be able to access certain features of the site like purchasing online.

Functionality Cookies

These cookies improve your browsing experience and provide useful, personalized features. They are used to remember selections that you have made such as your preferred language, region, and username. They also remember changes that you made in text sizes, fonts, and other customizable parts of the Web. Together, this information allows us to personalize features on our website in order to provide you with the best possible browsing experience. The information that these cookies collect is anonymous and cannot track your activity on other websites.

Analytics Cookies

These cookies are used to help ensure that your browsing experience is optimal. They collect anonymous data on how you use our website in order to build better, more useful pages. For instance, we can recognize and count the number of visitors, see how visitors moved around the site, and we can identify which pages returned error messages. This information enables us to enhance your experience and helps us troubleshoot any issues that prevented you from reaching the content that you needed. In order to improve the performance of our site, we use products such as WebTrends OnDemand and Google Analytics to track site usage. You can find the list of products that we use to collect information that is relevant to Analytics Cookies here:

  • Google Analytics
  • Adobe Analytics
  • SessionCam
  • ForeSee
  • WebTrends On Demand

Targeting or Advertising Cookies

These cookies are used to deliver personalized content based on your interests through third-party ad services. This allows us to improve your online experience by helping you find products that are relevant to your interests faster. They remember websites that you have visited and the information is shared with other organizations such as advertisers. These cookies are also used to limit the number of times you see an ad and help measure the effectiveness of a marketing campaign. You can find the list of products that we use to collect information that is relevant to Advertising Cookies here:

  • Marketo
  • Kenshoo
  • Doubleclick
An error has occurred. Error: AdobeAnalyticsModule2 is currently unavailable.