Should biological quality control be replaced?

I’ve been thinking about quality control and quality improvement lately. Mostly this has been about how to go about determining whether the lab has a quality problem with testing and what statistics should be used for this purpose but I was reminded recently about an issue concerning biological quality control that came up a couple months ago on the AARC diagnostics forum. Specifically, one of the participants noted that some of their technicians had refused to perform biological QC on the basis that it violated their HIPAA rights to the privacy of their medical information. Further discussion noted that this was actually a correct interpretation of the HIPAA regulations and that no PFT lab can “force” its technicians to perform biological QC.

I will be the first to admit that I’d never thought about it this way, and I’ve been mulling it over ever since. I’ve performed PFT testing on myself both for formal biological QC and as a quick way to check the operation of a test system for decades but I never thought of my PFT results as being part of my medical information. That’s probably an indication of my own short-sightedness however, and I also realize that over the years I’ve run across a number of testing issues I’d taken for granted up until somebody pointed out a problem with them.

My attitude towards my PFT results may also be due to the fact that I don’t have any notable lung disease. My lab has had technicians who have been asthmatic however, and this has never been a factor in whether they were hired or not (other than not letting them perform methacholine challenges). They’ve usually performed bio-QC on themselves and at the time they seemed to regard it as a way to check on the status of their asthma. In retrospect however, I have to wonder if they were ever concerned that I would use their health status and test information against them in their annual evaluation, or even that the hospital would re-consider their employment because the costs of their health insurance might be higher. Although I don’t think the hospitals I’ve worked for ever thought along these lines, like it or not there are many businesses where this is a factor.

Yesterday I asked myself what would happen if all PFT labs were required to completely end biological quality control because of HIPAA requirements? It didn’t take a lot of thought to realize that there are a number of mechanical test simulators in the marketplace that could do quite well at replacing the biological part of quality control. As importantly, the more I’ve thought about it the more I’ve come to think that biological QC probably isn’t the right way to go about QC in the first place.

First, no matter how experienced we may be there’s still a lot of inherent variability in PFT results, even when we perform the tests on ourselves. This is the nature of any biological system and it’s why bio-QC results are plotted and analyzed using standard deviations on a Levey-Jennings chart.

The problem with this is that the “normal” range is based on the standard deviation of the test results themselves, and that the greater the variability these test results have, the larger the “normal” range is going to be as well. This means that a given test system may be considered to be out of range for one individual with a naturally low variability in test results but still within range for another individual who has a naturally high level of variability.

Another problem with this approach is that it only considers whether changes over time are within a specified range of the average, not whether the results are actually accurate. Say, for example, the calibration syringe used by a lab was significantly inaccurate for some reason (the collar on the stem was loose and slipped, for example). Hopefully this is unlikely but nevertheless it would mean that the test equipment would be mis-calibrated. Biological QC might pick up on the fact that a calibration syringe had changed but not if the calibration syringe was already inaccurate when an individual began their QC.

Finally, most individuals working in PFT Labs are likely in reasonably good health with reasonably normal PFT results. Most patients referred to a PFT lab probably have a lung disorder of one kind or another and their results will often be significantly outside the normal range. So, does QC performed on normal(-ish) individuals mean that the test equipment accurately tests patients with abnormal lung function? Maybe yes, maybe no, but think about about patients with severe airway obstruction who have markedly reduced expiratory flow rates and require a prolonged exhalation time in order to get anywhere near a normal FVC. Is the test system really measuring those low flow rates accurately and since most systems tend to be flow-based, is the exhaled volume being accurately integrated from their expiratory flow? Again, maybe yes, maybe no, but biological QC isn’t very likely to tell you this.

So, what can take the place of the biological QC?

Just as automated systems are replacing humans across the job marketplace there are a number of mechanical simulators that can replace the biological component in QC. Realistically however, because the current demand for these devices is presently low, there isn’t a great deal of selection available.

There are a couple of flow-volume simulators (the 691227 from Hans-Rudolph and the PWG-33 from Piston Medical). These can produce any of the 50 different ATS spirometry waveforms which cover the entire range of pulmonary diseases and are used by manufacturers to test their spirometers. Additional waveforms can be imported and edited as well.

There is a DLCO simulator (the 691044 from Hans-Rudolph). I’ve discussed this previously and it can, depending on the gases used, simulate low, medium or high DLCO results. My lab has been involved in several research studies where we were required to regularly test the equipment used on the study’s subjects with a DLCO simulator and its results were uploaded along with the patient test results.

Finally there is a metabolic (i.e., VO2, VCO2, Ve) simulator (the 17050 series from Vacumed). This is able to simulate a wide variety of tidal volumes and respiratory rates as well as VO2 and VCO2.

At the present time there are no lung volume simulators. Despite this lung volumes measurements using helium dilution or nitrogen washout can be simulated (with a bit of care) using a calibration syringe. Morgan Scientific used to sell a simulator for plethysmographs however this product has since been discontinued (but may return depending on customer interest).

Because the market for simulators is so small, they also tend to be built only when an order has been placed. This means there are no economies of scale, which in turn means that they tend to be more expensive than they “should” be and this has discouraged their more widespread use, particularly since they are not required by any regulatory agencies.

But since simulator technology is well understood and their design is a straightforward engineering issue there’s no particular reason that an automated simulator couldn’t be developed that performed spirometry, lung volumes and DLCO. There’s also no particular reason that an automated QC system couldn’t simulate subjects with mild, moderate, severe and very severe airway obstruction as well as subjects with mild, moderate and severe restriction.

At the present time however, despite the fact that an automated QC system could test PFT systems across their entire operational range with a high degree of accuracy and repeatability there is no demand for an automated QC system and it is therefore unlikely to be built. But it’s possible for this situation to change, and perhaps to do so unexpectedly and as much as I’d like to see our field decide to police itself and mandate lab certifications, inspections and QC reporting, it’s more likely that this will be imposed from outside.

One possibility is that a technician could be fired from a PFT lab position, in part allegedly for refusing to perform biological QC. Another is that a physiology student could refuse spirometry (or some other medical-ish) test and then received a poor grade, allegedly because of this. If these, or similar, cases were brought to court, it’s possible the court would rule in favor of the plaintiff on HIPAA grounds and this interpretation of the law could quickly mean the end of biological QC everywhere.

For that matter CMS (Medicare) or SSA (Social Security) could suddenly decide (perhaps in order to limit payments or because they’re tired of questionable PFT results) to require that any PFT results they pay for or that are used in a payment decision could only be performed on test equipment independently verified by QC simulators.

This issue could also slowly come to a head as the word gets around (mea culpa), and more and more technicians decide they don’t have to do biological QC any more. As much as this may appear to cause problems for some lab managers I have a hard time blaming anybody that doesn’t want to perform biological QC because they want to keep any and all of their medical information private. I don’t think of my own PFT results as being particularly concerning but where do you draw the line? If you worked in a chemistry or hematology lab would you feel the same if you had to regularly provide blood or urine samples for QC? Probably not.

The fact is that biological QC is at best a limited way to monitor test equipment. In my experience equipment problems are far more often found during routine calibration and patient testing than they are because of biological QC. It’s a lot better than not performing any quality control at all but it doesn’t (and can’t) insure equipment accuracy and at best it can only detect change over time that is statistically significant. Finally, I’ve never met a technician (or a manager for that matter) that really liked performing biological QC and documenting the results.

Maybe it’s time to look past biological QC to other ways of ensuring the proper operation of our test equipment. An automated simulator would make it a lot easier to routinely monitor test system accuracy and performance and does not violate any HIPAA regulations. Current simulators have a limited range of abilities, are relatively expensive and are available from only a small number of manufacturers but if simulator QC became a requirement this would likely change very quickly.