Tag Archives: positive predictive value

“Playing the Odds”

The Science of Microbiology, ,

I spent a lot of my medical student days working part-time in the bookmakers, where managing odds and probability underpin the industry. Mathematics was my favourite subject at school, and within that field, probability was my favourite sub-topic.

Now that I am a clinical microbiologist, I can see the similarities. Clinical microbiologists get involved in a lot of the early antimicrobial stewardship decisions for patients based on preliminary/interim microbiology results. This is an area where you definitely need to weigh up the percentages/odds.

For example if you get a positive blood culture with gram positive cocci on the Gram, what will be the probability that this will turn out to be a Staph aureus or even an MRSA? If you are awaiting susceptibilities on an E.coli bacteraemia, what will be the chances it will be an ESBL? These are decisions which have to be made when advising optimal antibiotic therapy. So many factors need to be considered, almost sub-consciously, when calculating these odds.

Of course, with all decisions that involve probability, you will never win (get it right) all of the time. This is where it is critically important to take into account the potential consequences of getting it wrong, i.e. how sick is the patient? how much “reserves” do they have, what level of care are they currently in? In a “game” which involves probabilities, it is careful and prudent management of these probabilities which is key.

The other field of probability that clinical microbiologists must have understanding of are pre- and post- test probability, and positive and negative predictive value. These are key concepts in determining the validity of any test result the laboratory produces.

Playing the odds is an integral part of being a clinical microbiologist. It may even be part of the reason I was attracted to the specialty in the first place. I can even pretend I am still working at the bookies…

Michael

 

“The uncertainty of certainty”

The Science of Microbiology, , ,

There is one thing certain in the microbiology laboratory, that the results will be uncertain. This has nothing to do of course with laboratory systems or the competency of staff members. Just an acceptance that there is no such thing as a certain result…

The other thing to note is that the degree of certainty of results will vary between different tests, not only for separate tests but even for multiple tests contained in the one assay, e.g. any multiplex PCR.

Take for example a multiplex respiratory PCR, containing 24 or so different targets. (Most labs will “demand manage” such expensive assays, allowing them only for immunocompromised patients or the seriously ill. Nevertheless, such assays are becoming increasingly popular.)

In a multiplex respiratory assay, a positive result for rhinovirus is almost certainly going to have a greater chance of being “the genuine article” than a positive result for bocavirus.

This is because each individual target pathogen has a different positive predictive value (PPV), based on both its specificity and its relative prevalence in the tested population. As a result, positive predictive values for individual pathogens within a multiplex can, and do, vary greatly.

But how do we relate such information to the clinicians? Quoting the calculated PPV for each target in a multiplex would make for a long and complex laboratory report. I would not go there… It is probably best to use an appropriate comment for certain results. I.e. “Bocavirus is uncommonly seen in population x, therefore the positive predictive value of this result may be sub-optimal. Close clinical correlation is required.”

Of course, clinicians can increase the degree of certainty by clarifying the “pre-test probability”. I.e. A positive bocavirus result in a 6 month old during the winter season is much more likely to represent a true result than a positive bocavirus result in an adult during the summer season.

With multiplex PCRs, sometimes you are “forced” to perform a test, when it would be better not to know…

Clinicians, in general,  tend to believe that all laboratory results are certain, until we produce one that is very clearly wrong! After that, they will believe all results are uncertain until that trust is rebuilt over time.

To understand certainty of testing, you first of all need to understand the laws of probability. All a laboratory result ever does is convert pre-test probability of disease X into post-test probability. 

It neither confirms nor excludes…

Michael

 

“Dead Certs and Long Shots”

The Science of Microbiology, , ,

The more uncertain the result will be, the more useful the laboratory test generally is…

Sounds a little paradoxical, but it is absolutely true.

If we are looking to confirm something that is almost certain before the lab test is performed, then we need a “super-sensitive” test to fulfil this task. Otherwise we run the risk of giving false negative results.

For example, if we have a teenager with a sore throat and lymphadenopathy, a lymphocytosis and atypical lymphocytes on blood film, then the probability of this being EBV infection is about 90%. There is little point then in doing a confirmatory Monospot test with a sensitivity of 80-85%. This will only lead to giving negative results on patients who actually have EBV infection.

And if we are looking to diagnose a long shot (aka a very unlikely diagnosis) then we had better be sure our laboratory test is “super-specific”, otherwise we will run the risk of giving false positive results.

For example if we want to diagnose dengue fever in a patient with “flu like” symptoms returning from Mexico (an area of relatively low Dengue endemicity), then we need to think twice about performing Dengue serology testing which has a specificity rate of about 95%. You are just as likely to report a positive test in someone who doesn’t actually have Dengue.

What we are doing in actual practice here is taking our pre-test probability, and using it to give a prevalence rate (by proxy) in our tested population. Once we know this, then we can use our test sensitivity and specificity to calculate positive and negative predictive values, not always with the results we would like…

Laboratory specialists tend to be more aware of testing limitations such as these. Clinicians, in general,  tend to just take the laboratory results as gospel.

But I believe it is ultimately the laboratory’s responsibility to stress the limitations of using laboratory testing for “Dead Certs or Long Shots”, and either prevent such testing taking place, or put big disclaimers on the results.

Michael