How We Evaluate Evidence

Every article begins with the same question: what does the evidence actually show?

The answer is rarely what a headline suggests. This page sets out how we reach ours, so that you can check our reasoning rather than take our word for it.

Two kinds of writing, two standards

Evidence reviews are structured. They follow a written method, appraise each source, grade the certainty of the evidence on a declared scheme, and document their decisions so another person could repeat them. They are narrative reviews, not systematic reviews or peer-reviewed journal articles, and we do not present them as such.

Clinical explainers and commentary are not graded reviews. They draw on the reviews and on established clinical knowledge, and they are labelled so you always know which kind of piece you are reading.

We work from primary sources

We read the original material: peer-reviewed research, clinical guidelines, regulatory documents, legislation, official statistics and published datasets. We do not build claims on news coverage, press releases or other people’s summaries.

Where a source is paywalled and we cannot obtain it, we say so rather than citing it second-hand. An uncheckable citation is not evidence.

How we appraise a study

Study design sets the ceiling on what a finding can support, and execution decides whether it reaches it. A case series cannot answer a question about causation however large it is; a randomised trial with serious attrition may answer less than its design promises.

Our reviews are narrative, not systematic. We do not run a registered protocol, an exhaustive database search, dual screening, or a single risk-of-bias instrument applied uniformly to every study — and we say so on every review, rather than implying a completeness we have not attempted. What we do instead is traceable and stated on the page.

Every load-bearing claim is traced to its source, and each source’s status is recorded in one of four tiers: read in primary (full text obtained and checked), read in abstract only (flagged at every use), held but retrieval-limited, or not obtained (cited only through a secondary source or a pooling meta-analysis, and flagged as an open gate). Every source is listed by tier, so you can see not only what we cite but how well we have actually read it.

Two rules govern the rest. A secondary source is never used as a primary for grading: a review article may frame a question or lead us to the underlying studies, but any graded claim is traced to the primary. And a study-overlap map records which cohorts are independent and which share a lineage or are pooled by the same meta-analysis, so that agreement between figures drawn from the same underlying data is never mistaken for independent corroboration.

What our certainty marks mean

We mark the certainty of a body of evidence, not the value of a treatment. These are different claims and they are constantly confused.

Our certainty marks are editorial judgements, not formal GRADE ratings, and we label them that way wherever they appear. Each mark is made on a small set of declared axes — how far the evidence supports causation rather than association, the size and consistency of the observed effect, how directly the studied population maps to the patients the question is about, and, where it bears on the reading, how clinically important the outcome is. The number of axes and their definitions are not fixed across the site: every review sets out the exact scheme it uses and grades each axis on an ordinal scale, from well-supported down to not established, with the basis shown. We use our own editorial scheme rather than a formal instrument because it makes the judgement legible — you can see what we weighed, and disagree with any part of it.

A mark of low certainty means the research is weak. It does not mean the treatment does not work, and it does not mean it does. It means the evidence cannot yet tell us, and anyone using a low-certainty mark to argue that something is ineffective has misread it — often deliberately.

We separate evidence from interpretation

Evidence does not speak for itself. Every article distinguishes between what the evidence demonstrates, what the researchers believe it may mean, what we think it means, and what follows for policy or practice.

These are four different claims with four different standards of proof, and collapsing them is how most bad argument in this field is made. You should always be able to see which one you are reading.

Where lived experience belongs — and where it does not

Lived experience determines which questions are worth asking and which outcomes are worth measuring. Researchers routinely study what is easy to measure rather than what matters to patients — bone density rather than fractures, hormone levels rather than how people feel and function. Knowing the experience is how you notice that gap.

Lived experience does not change how a study is graded. It is not evidence of effect, it does not raise the certainty of a weak finding, and it cannot make a badly designed trial reliable. A study is appraised on its methods and nothing else, whatever we may wish it had found.

Holding both of those at once is the whole discipline of this publication.

We acknowledge uncertainty

Some questions have strong, consistent evidence. Many in this field do not. Where the answer is unknown, we say it is unknown rather than choosing the answer that suits us.

Confidence should be earned by evidence, not manufactured through certainty of tone.

When the evidence disagrees with us

We have published findings that are unhelpful to positions we hold, and we will go on doing it. A publication that has never reported an inconvenient result is not reporting; it is advocating with citations attached.

If better evidence emerges, we update the work and record what changed and why. Revising a conclusion in the face of stronger evidence is not a failure of the method. It is the method.

We correct in public

Errors are corrected openly, dated, and left attached to the original piece. We do not amend quietly and we do not delete. Our Corrections Policy sets out the process.

We apply the same standard outward: we record errors we find in the published literature and whether they have been corrected, because an uncorrected error goes on being cited long after it is known to be wrong.

What you can expect

Work grounded in sources you can read yourself. Reasoning shown rather than asserted. Uncertainty stated plainly. Complexity respected rather than smoothed away. And conclusions that change when the evidence does.

Knowledge is not static. Neither are we.