USING TOTAL ERROR To: MONITOR PERFORMANCE, QUALIFY REAGENTS AND CALIBRATORS. AND

  • SELECT Qc PROTOCOLS

REAL WORLD EXAMPLES

Zoe Brooks. George Massarella Cochrane Region Laboratory Program 700 Paris Street, Sudbury, Ont.t Canada P3E 3BS Phone Fax (705)674-8983

David Plaut Baxter Diagnostics, Miami, Florida, 33152 Phone 1-800—441-2790 Ext 6593

American Association for Clinical Chemistry

New York City July 15, 1993


INTRODUCTION

In a poster presented at AACC in 1992, we demonstrated that comparison of Total Error (T E) to a clinically-determined Total Allowable Error (TEa) is a simple and effective way to monitor changes in QC performance and to predict the acceptability of new lots of reagent and calibrators.

Here we apply this theory further in real-world examples to evaluate monthly QC, assess new lots of reagents and select appropriate QC rules.

TE will be greater than T Ea in a method with a large bias and small SD:

BIAS

TktjE

TE will be greater than T Ea in a method with a small bias and large SD:

BIAS

TRUE

• •Tk..;e.

TEa


MONTHLY QC

Routine daily QC was performed in six hospital laboratories using Dade controls in chemistry, blood gas, coagulation, special chemistry and hematology. The transmitted interlab report was imported into Paradox. Data were exported to Quattro Pro to create graphs showing the ratio of TE (Biast + 1.96 x SD) to the clinically acceptable T Ea set by G. Massarel[a for each test/control.

If TE was greater than TEa, tests were flagged and investigations performed using the TE Flowchart.

CHEMISTRY LAB X

JUNE 1993

p -05

SO 0K

so

HIGH

-2

ALB

AST 002 CREAT

1

-t

-1,5

-2

Fas UREA

so 0K

SO

BIAS OK

BIAS

PHOS L’A


The fox Ioni,r.q Tea limits indicate eLirwicaIJy a]. Iowable variation at. observed control nean values r as by G.N. HassareJLa HO r FRCPiCj valueg are modiried time to t;åae and rust: be evaluated for clinical setting.

T EST

cscot’) AST (SGOT) AZ b’,.nnin

ine Phosphatase BJ kam ine Phosphatase Bow lase Amylase el•K

ctn

cetm CEZ efil

CEL

CHI

Ca Lcium Calci.utu Chloride Ch Iorjde Cholesterol

Creatinine Creatinine oarmna-Gtr Camna—GT’ Glueose Past inq Glucose — Fascing LDEI

Phosphate Potassium Potzasgi.uffi godiurn Sodium

cnv

CHI

CH2

CHI CH2

CHI

CHI

cnv

CHL CE12

Total Bilirubin ‘l’oi:al Protein Protein t’rigLycerjdes Tr iglycerides Urate Orate Ot-ea Urea.

CHI CH2 CHI

CHI CH2 CHI

SV OHT’7’S

  1. a

11.0

t-i/L

2.31

JOO, 7

L 17.6

16 .04

L .06

126 . 2

1 .27 2.21. 184. ZI

5.45 1 s. 43

11/1. nmo L/ L

O

0.46 C .37

C.I’1iOI / L mmol ‘L mmö2/L

50.0

6

Li/L

38.2

o.

20

0.30 5.

nmoL/L anol/l.

nmol/L

O

640

umoI/L unxai/L

0.40 40. 0 50.0

rt’(j’10 1 / L mmol iL umol / L

rnB*ol /L

  1. a 141.1

56 . 9

30

g/dL g/dL

103.7

tJ/L 13/1.

:mg/dL

mEq/L

6-00

117.6

191-1

3

rnq/d:, mq/dl.

  1. a 36.0

nq/dL ng/dL

3.2B

6.64

112.46

7.43

0.62 L. 24

o. 5-0 4.0

2

7

o .67 4 0.84 13 7.0

nq/dL

hEq/L mF,q/I.,

mEq/L mg/dL / dL g/dL q/dL mg/dL rnq/d!. mq/dL mq/dL mc.j/dl.’ nq/dL

EXAMPLES

AMYLASE:

PROBLEM: One lab showed mean values

below other labs using the same manufacturer’s reagent. INVESTIGATION: Patient samples confirmed the bias; communication with the manufacturer indicated an incorrect reagent cal factor had been provided, RESOLUTION: Correction of the cal factor

URATE:

PROBLEM: A shift in mean value indicated

high bias. INVESTIGATION: The interlab SDI was high. Comparison of calibrator value to another lab revealed a discrepancy. The manufactureris package insert was re-checked.

RESOLUTION: A revised calibrator value

was entered.


ERROR PROBLEM-SOLVING FLOW CHART

n to C?}

TEST FLAGGED CF PRE’OIOUS 2 nepoars (7)

YES

SD/TE.Z 0.33

NO

0.20

NO

TE/TEa 1.25

YES

INVESTIGATE RANOOM EARO

INVESTIGATE SYSTEMATIC ERROR

TAKE CORRECTIVE ACTION

TAKE CORRECTIVE ACTION

CURRENT PERFORMANCE ACCEPTABLE (7)

YCS

RECORO COMMENTS tN a.c. COG

NO

MONITOR PERFORWNCE CLOSELY

ONGOING PEFCPMÅNCE ACCEPTABLE

YES

AECOAO COMMENTS IN C.C- ACTION LOG

RE-EXAMINE FACTS

IMPROVED (7)

ARRANGE TECHNO’ f ASSISTANCE

TAKE ALTERNATIVE CORRECTIVE ACTON

PATIENTS oun- UNTIL SATISFACTORY PERFORV.ANCE RE äST,4.EUSHO


NEW LOTS

Five different patient samples spanning the analytical range and two sets of control samples were tested using the trial lot of reagent or calibrator for three to five days. Anticipated changes in bias and TE were calculated.

If the new lots TE was < TEa, we accepted the lot and made appropriate changes in QC rule selection if indicated.

If the new lot’s TE was > TEa, we performed further study and rejected the lot if indicated.


ACCEPT LOT

AMYLASE Patients were unchanged; controls showed a marked shift. Pre-assessment of the lot permitted changes in QC means and avoided investigation of potential patient shift. Knowledge of the effect of the lot change on control samples explained variation in replicate proficiency samples.

ALKALINE PHOSPHATASE Means shifted SD. Results were within T Ea. Pre-assessment of the lot allowed re-assignment of QC means to avoid false QC flags.

AMYLASE FEB 1993

250

roo

150

PATIENTS

IOO 150 CURRENT LOT

300

AI-KAI-INE PHOSPHATASE

MAY 1993

goo

250

CONTROLS

PATIENTS

LOT

300 asc


REJECT LOT

PROTHROMBIN TIME Pre-assessment of the new lot showed a clinically unacceptable shift in control and patient values. Other labs confirmed the same shift in patient results. The manufacturer provided a replacement lot.

GENTAMYCIN A change in reagent composition caused a shift in controls and led to clinically-different patient results. Patient samples were referred out for comparison. This method was ultimately discontinued.

PRO-TIME NOV 1992

cow-noug

PATIENTS

15

10

CORRENT

12

GENTAMYCIN NOV 1992

CONTROLS

PATIENTS

4

2

4

8 CURRENT LOT #5TO5BUL

10

12


QC RULES

Appropriate QC rules for 2 control samples per run are selected from Westgardts QC Selection Grids based on the stability of the method and the ASEc in relation to T Ea for each control/test.

WESTGARD QC SELECTION GRID N = 2

C

A

PROCESS STABILITY (error frq•cy)

Poor Mod (2-10X) Ex (<2X)

N/A

1-2.0s

N/A

1-2.5s

1—3./2-2.z

2 to 3

1-2-os

1-2 5s

1-3 os

*SEC

3

1-2 ss

1-3 os

1-3 5s

1-30<4-10

WESTGARD Determine T Ea at medical decision limit

Determine Bias & SD

Calculate ASEc [(TEa-lBiasl)/s1-1.65

Categorize stability

Choose a QCSG; Select the row based on value of ASEc and select a column based on stability

Look up possible control rules and N

MODIFICATION Set T Ea at control means

Enter actual SD and Bias in program

Program calculates ASEc [(TEa-lBiasl)/sl-i.65

Categorize stability

Program specifies QC rules for


SAMPLE REPORT’:

f o llowi.m;j using controls

Selections per run.

are on Selection Grids cowruar. RULE SELECTION

ALB

EX

ALP ALP

BUN

EX

CAL

ceo crtn

MOD

CPR

ax

CRE CGE

Gt„tj CLU

EX

K.

Ex

CTRL YEAN

CEI

CHI

CHI

CHI

54 -GO

5.55 18. as

CH 2

CNJ

s. 06 4.09

93.10

CH2

CH 2

CBI CTt2

CHI

  1. go 525.70

4.37 16. OJ

EX

Ex

•rat

UAC

EX

ear

126.50

CHI. CH2

CHI

  1. JO

.40 .30

.10

.02

-.03

.06

  1. Sa
  2. 00

5.130 9.80

-B .50

7, so

—5 40 -25.20

.07

-.15

1.10 L. OCT

.02 .05

.02

2.30

1.00

  1. 20

-.80 -.40

6.40

SINGLE I HOGTI-RUI„E

1-3

90

1-3.5s

1—3 1-3 , “Gs

-lsw) 1-33/(4-

1st”

69

1-3 . os

  1. 03

1-3 u.

1-2. os

1-3.5s

1.09

1-2.5s

1-3

9 fi

1-3.53

2-05

1-3.03

6.91

1-3 . Ss . 59

2.65

2-3.53

3.55

1-33/(4-1gW)

N/A; trsli

I—Js/a-ag/R-4g/14-isKj

I-3s/2-2s/R—4s/4-1s

v„3G/t4-1sW) I-3s/(4-1$W)

I-3s/a-ag/R-/s/14-1sw)

1-36/ (4-1%}

I-3S/2-2s/R-4s/(4-

I—Js/f4 -mswj

EXAMPLES

For one month’s data from one lab for 24 tests, we reviewed 954 control runs, The number of runs rejected using various QC systems would have been:

Westgard Single Rule QCSG: 4 Westgard MultiRule QCSG: Full Westgard Multirules: 23 79

Some proficiency programs provide goals for internal QC. Setting SD limits in daily QC to match PT goals instead of using the lab’s actual mean and SD fails to detect

significant changes in the test system.


as avnxov a3LL07d L axvun

as nvruov yo

139HV1 -Ld CELLOId auvun

COMPARISON OF PROFICIENCY GOALS TO ACTUAL

xaouvl Ld NO

COMPARISON OF PROFICIENCY GOALS TO ACTUAL


CONCLUSIONS

1 . Comparison of TE to T Ea is a simple and effective way to monitor monthly changes in bias and imprecision.

  1. Comparison of TE to T Ea when changing lots of reagent or calibrator permits appropriate changes in daily QC means, identifies unacceptable lots, and provides an indication of potential changes in proficiency testing material.

  2. Clinical T Ea at control means can be applied to determine appropriate QC rules.