This chapter will discuss why test results are not exactly the same when they are repeated, factors causing this variability, and steps to take to reduce variability and lab errors. As an exercise, you will audit the entire testing process for possible errors.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
What does it mean if a laboratory reports that your patient has a blood glucose level of 120 mg/dL? Does the report really mean that the level is 120 mg/dL or might it really be 115 or less or possibly 125 or more? What does it mean if a laboratory reports that a culture for your patient was negative for Neissera gonorrhorae? Does it mean that no gonococci were present in the patient or that none were present in the sample submitted to the laboratory for culture?Clinical laboratory tests, an extension of the patient's physical assessment, can facilitate, enhance, rule out, or confirm diagnostic or clinical management decisions. Clinicians, using reports generated by clinical laboratories in patient care decisions, need some sense of the reliability of those results, particularly when the testing is being done by an office or hospital ward laboratory operated by staff with little or no technical laboratory training.
In the case of most quantitative analyses, such as the blood glucose determination, processing a single patient sample repetitively in the same analytic system does not produce a single number but rather a range of numbers (Figure 1). The results tend to cluster around an "average" (or "mean") value, and this dispersion of values expresses the reproducibility (or precision) of the determination (Figure 2). The analytic variability of a test can itself vary depending on the analytic system/testing device, the reagents used, and the individual performing the test. It can also vary with time, much as cars assembled on Friday are reputed to be of lower quality than those assembled during the rest of the week.
Therefore, when applying the results of quantitative analyses to diagnostic or patient care decisions, the clinician must consider the specific reported result as only one of a family of points around the "true" level of the analyte in the specimen tested. With very precise tests, repetitive testing produces results that are tightly clustered around the mean value (Figure 2). A wide fuzzy zone (ie, numbers are of low confidence and the range of dispersion is high) around the mean of an imprecise test makes it more difficult for the clinician to be confident that the "true level of the analyte in the sample is close to the reported value. This lack of confidence, in turn, increases the clinician's chance of making an erroneous diagnostic or clinical management decision based on the reported result of an imprecise test. For example, the results of a white blood cell (WBC) count performed by manually diluting the whole blood specimen and visually counting the cells with a microscope and counting chamber are much less precise (as much as six to eight times more variable) than those for an automated WBC count produced with a conventional laboratory cell counter (Figure 3).
Qualitative analyses, such as determination of whether a specimen contains N gonorrhoeae or a urine specimen contains significant levels of chorionic gonadotropin (as determined with a pregnancy test), are subject to similar variability issues. The end point of a qualitative analysis is a dichotomous result ("positive" or "negative") or a semiquantitative result ("+", " ++", " +++", or "++++"). In this kind of testing, predetermined threshold levels are actually used to separate the dichotomous qualitative result (as with the pregnancy test) or define the semiquantitative result classification (as with a urine glucose level determined with a reagent dipstick).
Results of bacteriologic determinations such as the culture identification of a pathogenic organism (eg, N gonorrhoeae) also are subject to analytic variability. Because systems to make these determinations are generally not automated or semiautomated, operator variability represents a much larger issue. That is not to say that variability due to reagents and test systems does not exist, because it does. The ability of selective growth media, for example, to support growth of the designated pathogen and suppress growth of other contaminating organisms can vary from lot to lot and change with time. Discussion of these factors, however is beyond the scope of this module.
Other Sources of Variability in the Testing Process
Patient preparation, specimen collection, identification and transportation, sample preparation, result calculation, and result transcription are a few of the nonanalytic factors that can contribute to variability in the testing process. These factors tend to affect the accuracy of the result (the "true" level of the analyte in the patient) rather than its precision. Factors outside of analytic variability can have a profound impact on a laboratory's ability to produce an accurate result. For example, the blood glucose level in a tube of blood can decrease over time (as much as 25%) because living cells (primarily WBCs) consume glucose to maintain their integrity. The loss of glucose can be slowed by keeping the specimen refrigerated (slowing down the metabolism of the WBCs) or by using metabolic inhibitors, such as fluoride, to stop a major glucose-consuming metabolic pathway.The quality of a laboratory's work can be no better than the quality of the specimens submitted for analysis. All of the factors listed at the beginning of this section, and many more, can adversely affect the reliability of laboratory test results and their ability to reflect the physiologic or pathologic state of the patient (Table 1).
Statistical Definition of Analytic Variability
Testing variation can be defined statistically by analyzing the results of repetitive processing of a single specimen using the same analytic system/device, the same method the same reagents, and the same operator. Because of complexities in the testing process, any automated or manual procedure will produce not one but a group of results. With precise test procedures, all results will fall close to the mean; in other words, the standard deviation (SD), which statistically defines the size of the spread around the mean, will be small (Figure 4).Because the size of the SD also varies with the magnitude of the level at which the specimen is being tested, laboratorians like to use a term that describes the fractional variation around the mean -- the coefficient of variation (CV). The CV is the SD of the data set divided by the mean value of the group of results (multiplied by 100 so it can be expressed as a percent). This is convenient to use because the fractional variation is independent of the specific numeric values of the mean, the SD, and the units that produce it.
If you know the CV for a particular analysis, it can be used to determine the 95% confidence limits (the range within which 95% of a group of values fall -- which is statistically ±2 SD from the mean) of any reported value . This can be done by multiplying the reported value by twice the CV divided by 100 (to change the CV from a percent to a fractional value) Add and subtract this value from the reported result to determine the range within which 95% of probable "true" results (the level that was actually in the sample tested) are likely to fall.
For example, the CV of WBC counts done in a hospital's laboratory was found to be 2%, while the CV of the same procedure on a semiautomated cell counter in a physician's office laboratory was 8%. This means that if a reported WBC count was 12 x 109/L (12,000/µL), the 95% confidence limits of the result produced in the hospital laboratory would fall between 11.52 x 109/L and 12.48 x 109/L (11,520/µL and 12,480/µL), that is, 12.0 x 2 (CV)/100 = 0.48; 12 - 0.48 = 11.52; 12.0 + 0.48 = 12.48. The 95% confidence limits of the same value reported by the office would fall between 10.08 x 109/L and 13.92 x 109/L (10,080/µL and 13,920/µL) (Figure 4). The counts would be even farther from the "true" value 5% of the time but would still be part of the same data set.
It is easy to see that the WBC result from the hospital laboratory is right at the upper limit of the value expected in a healthy population (12 x 109/L [12,000/µL]), and the testing variation is small enough to let the physician interpret it accordingly. The variation implicit in the physician's office laboratory method is such that it would be hard to tell whether the result was clearly normal (10 x 109/L [10,000/µL]), borderline (12 x 109/L [12,000/µL]), or clearly abnormal (14 x 109/L [14,000/µL]). If this example occurred in an office practice, there would obviously be a trade-off between speed and result reliability. The office laboratory's result would be available for immediate patient care decisions, while the return of the hospital laboratory's result might be significantly delayed, particularly if the distance between the office and hospital laboratory is great.
Assuring Quality in Laboratory Testing
Analytic variability is a fact of life in the laboratory. Determining acceptable variability limits is a complex decision involving the clinical demands of the conditions being treated and the ability of laboratory technology to meet the clinical needs of the decision making process. As a consequence, an imprecise test may be acceptable because it provides information that would otherwise not be available to the diagnostic or medical management process. The clinician, however, must factor the reliability of the test result into the clinical decision, particularly at or near decision values or levels, which are those levels at which the diagnosis or medical management decision is likely to change (eg, reduce the insulin dose or leave it unchanged).Quality control protocols alert us to the possibility that problems such as machine wear, light source deterioration, or reagent inactivation may be affecting the reliability of patient care results. These protocols are good at identifying systematic sources of error that can develop in the testing process, but are poor at identifying random errors (eg, an operator drawing a short sample for analysis). Unless a result is so discordant with the patient's condition that the result strains credibility, it is likely that it will be used in the process of making the clinical decision.
A substantial fraction of erroneous laboratory results (about half) are not attributable to analytic variability or test processing problems. They come from other aspects of the cycle, including specimen collection and handling and result reporting (Table 1). Although a large number of essentially unrelated problems affecting the reliability of laboratory test results can develop, a common theme relevant to office laboratory testing is how many of the errors occur because of mistakes in such clerical functions as specimen identification, result calculation, results transcription, and report filing.
When testing is done in an office practice, the laboratory needs to be able to provide the physician-director with documentation that any result was reasonably accurate and precise and that the result was, in fact, relevant to the patient in case any questions are raised about a particular result used in a patient care decision. Such a quality management program must deal with problems that can develop before processing, with appropriate security measures after processing, as well as with variation in the testing process itself. The specifics of quality management of laboratory testing will be the subject of other modules in this series.
Conclusion
The reliability of laboratory results depends on understanding and controlling the sources of error or bias that can enter the testing process. Just as important are preanalytical factors, which can contribute as much variation to the result as analytical factors can. Many clinicians tend to think of test errors as being primarily attributable to problems with the testing process, but this is not necessarily true.To ensure the reliability of test information, the physicians' office laboratory, like any other, must provide the physician-director a firm basis for assuming responsibility for valid test results, for interpreting the results, and for making diagnostic and management decisions. Understanding the office laboratory technology is also basic to providing the quality test information that is essential for medical management decisions.
Exercise
Have one individual who does laboratory testing assay a blood sample (either a patient sample or a quality control sample) for glucose 10 consecutive times, all in one session. If your laboratory does not test for glucose, you may perform any other test that produces a quantitative result. Record the results.Discuss the following questions with your laboratory staff:
Commentary
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Action |
Potential Error |
Test is ordered after history |
Wrong test is ordered No history of conditions affecting test interpretation: Incomplete dietary or drug history |
Physician writes an order in the patient's chart |
Handwriting is illegible Orders are written on the wrong form or the wrong place in the chart Test requirements are not specified: Fasting or random sample |
Nurse or secretary transcribes the order to a requisition slip |
Patient chart is not properly flagged or is put in wrong place Orders are misread, or charts with written orders are overlooked Information is transcribed to the wrong form Lab form is improperly filled out, unclear, or illegible Important details are not transcribed The wrong patient's name is written on the lab requisition The requisition is not verified against the original order |
The requisition is delivered to the laboratory |
The requisition is lost or misplaced on the ward Transportation delays or pickup was beyond the time designated for the test The requisition is delivered to the wrong department or the wrong place in the lab |
The requisition is received and logged into the laboratory |
The requisition is misplaced or lost by the lab clerk: It sticks to another form, is thrown away, or put in the wrong file Information is transcribed incorrectly: Patient name hospital number lab numbers, or names of tests |
Blood pickup slips are prepared and given to a phlebotomist |
The wrong information is written on the slips: Wrong patient's name, incomplete or illegible test list Slip never reaches the phlebotomist |
Phlebotomist goes to the ward to draw blood from the patient |
Phlebotomist draws the wrong patient--does not correctly match the slip with the name on the armband Phlebotomist uses the wrong tube, and an incorrect anticoagulant causes interference with the test Phlebotomist draws an inadequate volume or lets the blood hemolyze Blood is drawn at the wrong time or out of phase with the initial order |
Phlebotomist returns blood to the laboratory |
Substance to be measured deteriorates because of a delay in returning to the lab, improper handling during transport, excessive agitation, adverse environmental conditions (such as failure to keep cool) Serum or plasma is lost, or contamination occurs because the tube is dropped or separated serum or plasma leaks out |
Specimen is logged in, and the work sheet is prepared |
Transcription errors Work lists do not contain all the tests to be processed Patients or tests appear on the wrong lists |
Specimen is processed: Centrifugation and transfer of plasma or serum to other tubes |
Centrifugation causes hemolysis due to heat or excess speed Tube breaks in centrifuge, potentially contaminating other specimens Specimens are transposed during centrifugation or serum separation Transfer tubes are dirty, contaminated, mislabeled, or not matched to work lists |
Specimens and work lists are taken to the processing area |
The technologist is not told which tests should be done The work list or specimen is lost or misplaced The technologist does the test on the wrong tube or the wrong test on the right tube Series of tests are put in incorrect sequence |
Analysis is performed |
Reagent errors: mislabeled, outdated, incorrectly formulated, or contaminated Incorrect pipette is used, or the pipette is incorrectly calibrated or read Read-out instrument errors: improperly calibrated. dirty cuvettes, wrong wavelength selected, weak phototube, incorrectly calibrated to zero Calibration curves are incorrectly formulated or not updated often enough Insufficient or no controls in the run Reagents are added in the wrong sequence |
Results are read, recorded, and calculated |
Technologist misreads the instrument Reading is incorrectly transcribed Calculations are incorrect because of math errors, transposed decimal points, use of the wrong formula, or failure to include physical factors or mathematical constants Results are transcribed to the wrong column or into another patient's file |
Results are transcribed from worksheet to patient's record and placed in lab file for retrieval |
Information is incorrectly transcribed Results are placed in the wrong column or listed against the wrong test Results are placed in the wrong patient's lab report Lab data are misfiled or in a form that makes retrieval impossible Results on the formal lab report do not match telephoned results |
Lab reports are taken to the floor for charting |
Messenger is delayed Report is lost in transit or misplaced on the ward, or chart is not in the rack Report is placed in the wrong patient's chart or the wrong place in the chart |
Physician reads the report, assimilates data into making a diagnosis, or formulates a management decision |
Physician fails to read the test result: Data are not clearly presented, lost in a mass of normal values, misplaced, or out of date sequence Physician does not remember ordering the test and does not anticipate the results Results are incorrectly correlated with previous results or other tests, and the physician interprets them incorrectly |
Reprinted with permission from Medical Laboratory Products (November 1986 20-21), Copyright 1986, Medical Economics Publishing.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
The physician is responsible for the appropriateness of the procedure and for test precision and accuracy in the office setting. To fulfill this responsibility, the practitioner must verify that (1) a quality assurance program is in place to ensure the reliability of test results, (2) testing is processed in an orderly, reproducible fashion in a setting where environmental variation will not affect the result, (3) all extraneous variables are excluded, and (4) records are available to document these facts.
Test Request and Specimen Collection
Tests should be ordered on a request form that includes the patient's name and a unique identifier, the time of collection, and the nature of the specimen. It is good practice, and a requirement of some regulatory and accrediting agencies, to keep a log of all specimens collected, including the time of collection and, in the case of bacteriologic specimens, the source of the specimen. All specimen containers and tubes must also be marked with the patient's name and unique identifier, preferably at the time and place of collection. This is particularly important in the office setting where the tests of various family members (including some with the same first name) may be processed at the same time.Written criteria should be established for unsatisfactory specimens, including what should be done if the name or identifier on the request slip does not match that on the specimen (it is usually best to collect a second specimen, if possible). Protocols should be established for keeping the specimen (when appropriate) for reprocessing should questions develop later. Most serum chemical analyses remain stable for at least 7 days in the refrigerator or 1 month if frozen. A frost-free freezer with periodic warm-ups can adversely affect the stability of many analyses. Anticoagulated whole blood cell counts is stable in the refrigerator for up to 24 hours.
Test Processing
The temperature in the testing environment, as well as in refrigerators, freezers, incubators, water baths, and dry baths (heating blocks) involved in testing or storing reagents or media, should be monitored daily and the results recorded. If storage conditions for reagents or media are disturbed, as in a power outage with a rise in storage temperature, the reagents must be considered unreliable and discarded, even though the expiration date has not yet been reached.The containers for all media and reagents, including urine dipsticks and sticks for blood glucose tests, must be labeled with the date received and the date opened, because their life is relatively short once they have been exposed to air and humidity. Obviously, use the oldest dated reagents first to minimize outdating. As we noted, some materials may have to be discarded before the expiration date if environmental storage problems develop. All working solutions, prepared from more concentrated stock solutions, should be labeled when they are prepared and the expiration date written clearly on the label if appropriate. Remember that stock solutions kept properly in storage will have a different useful life span from derivative working solutions.
For reliable results, all microscopes, centrifuges, chemistry analyzers, blood cell counters and other laboratory equipment must be properly maintained. In taking responsibility for the validity of the test information produced in the office laboratory, the physician must have an equipment log that keeps track of maintenance, problems found, and any corrective action taken. Remember that refrigerators, freezers, incubators, and heating baths or blocks are a class of equipment that can directly affect test reliability and should thus be included in the general maintenance program.
Small pieces of equipment, such as pipettes, should be stored or marked in such a way that there is little chance of inadvertently using the wrong size (eg, a 1-mL pipette instead of a 0.5-mL pipette), and damaged pieces should be discarded. The pipetting and diluting step is one of the most critical in many procedures and requires extreme care on the operator's part. Tests requiring preparation of reagents that involve a pipetting step or dilution of a specimen by pipetting should not be done if the individual doing the test is not proficient in the pipetting process.
Daily quality control tests need to be done to provide early warning of an analytic problem that might adversely affect test results. It also serves as an indication that good laboratory practices are observed, and provides reassurance to the analyst and care giver that he or she can have confidence in the test results. Proficiency testing is done less frequently, but has some of the same objectives. In addition, it serves to compare a laboratory with its peers, assuring that test results are comparable from laboratory to laboratory.
Reporting and Using Test Results
The other important phase of laboratory testing is accurately reporting the results in a timely manner with appropriate interpretation and application of the test results.The results of laboratory tests should be reported and documented in a way that facilitates their interpretation. Unless written directly into the patient's chart, every laboratory result should include the patient's name and some other unique identifier. All laboratory results should include information similar to that required for ordering the test: patient identification, time and date of specimen collection, body site of collection, and name of test. In addition, the test result report needs to contain: test results including the units of measurement, reference intervals (normal values), time and date of analysis, and identity of the analyst. Reports should be written on a form designed for that purpose (never on a paper towel or scrap of paper). All laboratory test reports should be placed in the patient's medical record in chronological order.
Research has shown that a high proportion of written reports have significant numbers of clerical errors, such as transposition of numbers. For this reason, good laboratory practice requires that laboratories routinely check for clerical errors prior to the release of laboratory results. Usually this calls for a second person to read the test result on the report to see if it is reasonable. In a small office and on the hospital ward, the physician should serve this role, always having a high level of skepticism for an unexpected result.
All laboratory reports need to include an appropriate reference interval (normal range) for the test method used. This serves as a point of reference for interpreting and acting on the test result; it also helps to avoid misinterpretation of the result. Additionally, it allows someone who is unfamiliar with the laboratory's reference intervals to make an independent interpretation of the result. Even the laboratory's own staff and physicians may forget the reference intervals in use at the time the test was performed. If the test results are ever brought to court, the reference interval information is essential in establishing credibility of the result. Remember that reference intervals are specific for each test method and frequently cannot be transferred from another test method.
For some tests there are results that indicate a deviation from a healthy condition that is so extreme that there is an immediate threat to the patient's life. These extreme results are called "critical values" or "panic values." Each laboratory should develop a list of those critical values that require immediate notification of the patient's physician so that urgent assessment of the patient's status can be made and action can be taken without delay. The laboratory or nursing ward should have a procedure for notification of the physician and a backup procedure in case the physician is not immediately available.
Timeliness in reporting is just as much a quality issue as are the accuracy and precision of testing. A report delayed past the time when a decision will have a beneficial effect on the patient's condition has very little value other than adding historical data to the medical record. One of the major reasons for testing in the office or hospital ward its to provide useful information to the care giver in a time frame that allows action to be taken when it is most beneficial. Minimizing laboratory errors requires all individuals involved in the testing cycle--the ordering physician, the person obtaining the specimen, the analyst, and the care giver interpreting the test result--to be alert for factors that can interfere with the reliability of the results.
Exercise
Select one test order and follow the ordering, specimen collection, testing, and reporting process from start to finisb. Answer tbe following questions about this test process:
Commentary
The above exercise reviews many of the steps discussed in this module that are intended to minimize laboratory errors. If you found some of the steps missing, this would be a good time to make changes in policies and procedures. Although quality control and proficiency testing are important steps in minimizing errors, periodic audits of a test, from the initial order to the final receipt and review of the result, can detect weaknesses in the testing cycle.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Richard E. Belsey, M.D., Daniel M. Baer, M.D.
Bias: The degree of inaccuracy; a shift in calibration that makes all values higher or lower than the true value.
Coefficient of variation: An index used to describe the precision of a laboratory method. It is calculated with the formula, CV = (standard deviation x 100)/ mean.
Decision value or level: A threshold value of a test result above or below which a physician will respond with a particular action.
Precision: The reproducibility of a test result as described by its variation. The smaller the variation, the greater the precision.
Quality assurance and quality management: A collection of procedures that increase the opportunity to produce reliable test results. These include procedure manuals, quality control, proficiency testing, staff education, and complete documentation of test results, quality control, and problem solving. Quality assurance and quality management are equivalent terms.
Quality control: A means of sampling the precision of testing that uses periodically repeated tests of a sample of known composition. Quality control provides early warning of errors and an indication when patient tests should not be performed until problems with the test have been resolved.
Quality control sample: A sample, similar to patient specimens, that is repeatedly tested to detect testing error prior to performing patient tests.
Reliability: The method's capacity to maintain both accuracy and precision.
Standard deviation: A statistical term for describing variation in a data set (SD = square root of the variance).
Variance: A means of describing the amount of scatter in a data set.