Tag Archives: metaanalyses

“Publication Bias: It is believed to be true, so it will be…”

I always worry a little when people rely heavily on literature based evidence to support their beliefs. I am not saying it is not important, but it is not the be all and end all. One of the main enemies of the medical literature is publication bias. Publication bias obviously affects all science, not just microbiology.

One (of a number) of textbook definitions of publication bias is as follows: “Publication bias occurs when authors are more likely to submit, or editors accept, positive than null (negative or inconclusive) results.”

However I think publication bias works at many more levels than the definition above suggests…

There are several reasons why a paper supporting a hypothesis is more likely to be read in the medical journals:

  • From the moment the author decides to test a hypothesis, there is a sub-concious desire to prove that the hypothesis is correct. This may affect the design of the trial, the collating of the results, the writing of the article, the conclusions etc etc.
  • A paper which supports a hypothesis is more likely to be submitted for publication than one which doesn’t.
  • A paper supporting a hypothesis is more likely to be reviewed positively and accepted than papers that don’t.
  •  Even if the article is accepted and published, a paper supporting a hypothesis is more likely to be read and publicised than one that doesn’t.

So “publication bias” can be nasty at many levels. Trial registers attempt to alleviate some of the problems of publication bias, but are only partially successful and don’t address all the factors involved.

Publication bias is bad enough when it involves small personal trials. However when looking at large scale trials that are sponsored by major companies where the stakes are very high (ie millions of dollars hanging on the result of the trial), then publication bias in all its forms (as described above) becomes much worse, and probably dwarves any form of statistical bias.

Meta-analyses are equally prone to publication bias and possibly more so unless the individual trials are very carefully vetted.

Publication bias I think just reflects human nature. It is also an excuse for me to read less journal articles….