Tag Archives: positive predictive value

“Think twice”

What have the following got in common?

  • E.coli resistant to nitrofurantoin
  • E.coli resistant to fosfomycin
  • Haemophilus influenzae resistant to ciprofloxacin
  • Group B streptococcus resistant to penicillin.
  • Coagulase negative staphylococci resistant to vancomycin
  • Candida albicans resistant to fluconazole

In my area of the world anyway (New Zealand), the percentage resistance rates of the above micro-organism/antimicrobial combinations is less than 1%. i.e. the prevalence is very low.

And because the prevalence is very low, unless your susceptibility testing methods are very specific, the positive predictive value of the result will also be very low. Thus , there will generally be a large number of false positives amongst such results.

Such a result should therefore automatically trigger a double check of everything, with a close look at the audit trail leading to the result. In some circumstances, repeating the test or sending the isolate to a reference laboratory may be the best option even if the result looks genuine.

We always need to be very careful when reporting low prevalence results, because even though we would like them to be, our tests are generally not perfect…

Michael

 

 

“The Hit Parade”

It is really useful to know your “hit rate” for any particular laboratory test. I.e. the average number of tests required to produce one positive result in your tested population.

So what is an acceptable hit rate?

Well, this depends on the severity of the disease you are looking for, how easy it is to treat, and also how easy it is to clinically suspect based on the symptomatology.

Low hit rates are generally more acceptable for serious diseases. e.g. our hit rate for gonorrhoea based on our NAAT is roughly one in a hundred. For HIV it is approximately one in several hundred. But the consequences of missing these diseases are very serious, so the low hit rate can be rationalised.

Other low hit rates are not so easy to explain away. Our hit rate for Hepatitis A is approximately 1 in a 1000. This is a disease which is rarely serious and also has some distinctive epidemiological and clinical characteristics.

After rotavirus vaccination commenced in NZ last year the rotavirus hit rate has dropped to less than 1 in a 100. Rotavirus generally does not cause particularly serious illness in the relatively affluent NZ population.

Detection of Trichomonas in genital specimens also has a local hit rate of approximately one in a hundred. Move into the older age groups and this hit rate drops to 1 in a 1000. Trichomonas infection is not a particularly severe disease in the bigger scheme of things.

Hit rates are not just important from an economic/efficiency point of view. There is an important quality issue here as well. Unless you are using a highly specific test, low hit rates often lead to poor positive predictive value, and the dreaded false positives….

So what can we do about it?

If the hit rate is unjustifiably low, one can try and focus the testing on patients more likely to have the disease. For example, Hepatitis A testing could be restricted to patients with a significant increase in liver function tests. Along the same lines routine Trichomonas testing could be restricted to certain age categories most likely to have the disease.

With more sophisticated IT capability these days, one can often determine hit rates for different tests within a few minutes with a simple computer search. Thus looking at hit rates for different tests is becoming an increasingly important aspect of laboratory management.

 Have a think about a few of the tests that you perform in your local laboratory. What are your hit rates like? And is there anything that you can do about them?

Michael

 

“Don’t believe everything you read…”

I know it is difficult to believe but I was doing a little bit of background reading recently on Monospot tests (looking for heterophile antibodies to Epstein Barr Virus). On reading a guideline I came across this statement…

“The presence of heterophile antibodies in a symptomatic adolescent or young adult has a sensitivity of approximately 90%, and specificity of almost 100% for glandular fever.”

Now the question I have is what is wrong with this statement?

The answer is that sensitivity and specificity are functions of the test itself. Different tests for the same disease from different manufacturers may have different sensitivity and specificity.

However once you start applying the test to population cohorts such as symptomatic adolescents, then you need to start talking in terms of positive and negative predictive value.

….and the paradox is when you use a test such a Monospot, with a sensitivity of approximately 80%, in a high prevalence population such as symptomatic adolescents, then your negative predictive value will be relatively lower than in a low prevalence cohort, as there will be a significant amount of people who will test negative who actually have the disease (false negatives).

Sometimes we need to think about the science behind the statements in the guideline and make sure it makes sense in our heads.

Don’t believe everything you read, (especially when it is written by me!)

Michael

For a really nice presentation on sensitivity, specificity, PPV etc click here (5-10 minute read)