Editorial · Deposit Standard
This is how an Eden Openly Evidence Review is made, checked, deposited and corrected.
It sits alongside two companion pages. Editorial Standards govern what we write. Reading the Evidence explains how to read clinical evidence, and why it has the shape it does. This page is about the third thing: how we know our own documents are right — and what we do when they are not.
Every rule below exists because something went wrong. The failures are named. That is the point of publishing it.
Why this is public. Eden Openly is written under a pseudonym. We cannot offer you an institution, a named peer reviewer, or a professional reputation to lean on. What we can offer is the method, in full: the checks, the gates, the provenance of every source, and a correction record that names our own errors alongside anyone else’s.
You should not take our word for anything. You should be able to check. This document exists so that you can.
The three rules everything else follows from
1. The error is almost never in the primary. It is in the transmission.
Medical evidence travels: primary study → systematic review → appraisal document → policy proposition → public argument → clinical decision. There is no verification step at any hop. Each layer trusts the layer above, applies its own transformation, and passes the result on.
Errors are not caught. They are laundered — and they arrive at the far end carrying an institutional badge.
In building one review, we verified 41 errors in the evidence chain underlying a national policy consultation. Roughly eight were in the primary research papers. The rest were introduced afterwards — in an appraisal table, in a systematic review’s one-line summary, in an author manuscript that was never corrected, in a news article, and in our own reading of all four.
Not one of them was detectable by a check that asks only whether a citation resolves. All of them were visible to someone who opened the source.
2. A check that is built and not read is not a check.
One version of an Evidence Review was deposited with thirty-four citations resolving to the wrong papers. Nine references were misaligned in the list. A numerical integrity check passed cleanly — no orphaned references, no dangling citations — because every number still resolved to a reference. It simply resolved to the wrong one.
A claim-to-reference map had been built specifically to catch this. It had recorded the misalignment in plain sight: eleven claims about a testosterone evidence review, filed under a written ministerial statement.
Nobody read it.
The map is now a gate, and the gate is a human being reading it line by line. No script can close it, because the script would be checking numbers again.
3. The criterion is fidelity to the source, not the direction of the result.
Corrections are kept because they are faithful to the evidence — never because of which way they cut.
This is not a posture. In one review, correcting an error restored an adverse safety signal that an earlier draft had wrongly dissolved. Another correction removed a claim of harm that the primary paper did not support. Some corrections cut both ways and cannot be scored at all — and where that is true, we say so and decline to score them.
We publish no tally of which way our corrections fall. An earlier version of one correction record did, and the count was not derived — it was asserted, and it was wrong. That is itself recorded as a correction. A record in which every error points the same way is a record nobody should believe.
Provenance of sources
Printed in full in every deposit:
And it applies from ER-015 onward — not backwards. This convention was adopted on 13 July 2026. Reviews published before it were produced without a per-reference provenance record, and that record cannot now be reconstructed: some sources were read in full, some in published abstract, some through a publisher’s web page. Those reviews therefore make no per-reference provenance claim, they say so in their own deposits, and nothing should be read into the silence of a reference in them. We would rather tell you which of our documents can carry this claim than let a standard adopted today appear to vouch for work done before it.
Silence is an assertion, not an absence. It says: read in full, in the original. That is a strong claim, deliberately. It is the claim the whole corpus is worth nothing without.
Every exception is stated on the reference itself, in fixed wording, so that it can be counted:
| Class | What the reference says |
|---|---|
| Abstract only | Read in published abstract; full text not obtained. |
| Extract or summary | Read in the original summary. / Read in the official translation. |
| Not obtained | Not obtained. |
The counts in the Verification statement are derived from the document at build time, never typed. Nothing is stored twice, so nothing can drift.
And one distinction that took us longer than it should have
“Read in primary” is not the same as “quoted from the primary.” A paper can be opened, read in full, and still quoted from its abstract — because the abstract is nearer to hand. We did exactly this: a set of bone-density figures was taken from a paper’s abstract, in a paper we had read in full, and the abstract disagreed with the paper’s own Results at three of four measurement sites.
This failure passes every provenance check, because the provenance claim is true. The rule now reads: figures come from the Results. Where an abstract and a results section disagree, the results text governs and the disagreement is stated.
How certainty is marked
A single certainty rating cannot carry four different questions, and collapsing them is how most public argument about medical evidence goes wrong. Every graded outcome answers these separately:
| Axis | The question |
|---|---|
| Causal attribution | Did the treatment cause it — or the surrounding care, the passage of time, or who was selected into treatment? |
| Observed effect size | How large was the measured change? |
| Clinical importance | Would a patient notice, or a clinician act? |
| Population directness | Was it measured in the population the claim is about — or borrowed from adults, or from cisgender cohorts? |
These come apart constantly. “The evidence is weak” usually means causal attribution is weak — which says nothing about whether the effect is large, or whether it matters.
Marks are rendered as a coloured disc and a word: 🟢 robust, 🟡 moderate, 🔴 limited. The word is what survives greyscale, printing, and a screen reader. The colour is decorative; the word carries the meaning.
A note on our own inconsistency. Reviews published before ER-015 use a single combined traffic light, described on Reading the Evidence, which folds these four questions into one mark. Folding them is the thing this scheme exists to stop. The four-axis scheme is now the standard, and earlier reviews will be updated to it at their next version. Until then, the corpus carries both, and you should know which you are looking at.
Certainty marks are editorial judgements against a stated scheme. They are not formal GRADE ratings, and they are open to challenge.
What stops a deposit
These are enforced in code. The build halts; it does not warn.
| The gate | The failure it exists for |
|---|---|
| The reference list must be a numbered list | A review’s entire reference list once rendered unnumbered for several drafts. Every citation in the text pointed at nothing. The reference count, the citation sweep and the claim-to-reference map all reported it fine — each was checking the entries, and none was checking the list around them. |
| No orphaned references, no dangling citations | The baseline arithmetic check. Necessary, and nowhere near sufficient — see rule 2. |
| The source manifest must agree with the reference list | A deposit cannot claim four provenance exceptions while its manifest records three. |
| All counts derived, never typed | A deposit once shipped a verification statement whose reference count had drifted from its own reference list. |
And one gate that cannot be automated: the claim-to-reference map must be read, line by line, before deposit. Not merely built. Building it and not reading it is how thirty-four citations pointed at the wrong papers through a check that passed.
The archive gate
Every reference claiming to have been read in full has the document in a source archive, and the build refuses to run without it. The archive is keyed by DOI, and each source is checked weekly against PubMed for corrections, errata and retractions.
Until 14 July 2026 this section said the opposite. It said the gate was off, that the claim a source had been read in full rested on our word rather than on an artefact, and that we would rather tell you so than describe a check we were not running. It is now running.
Switching it on found five things in a review that had already been deposited four times. One reference was superseded — two corrigenda had been issued, and the article was never amended in place. One had been corrected in 2022. One could not be obtained at all: the claim it carried was true, and it was removed anyway, because a claim that cannot be evidenced is a claim that cannot be made. One passage carried two citations, neither of which supported it — a claim with two irrelevant citations is worse than a claim with none. And one archived source had a colon where its identifier needed a slash: a wrong identifier does not fail. It answers.
None of it was found by reading more carefully. All of it was found by a build that refuses.
Corrections and versions
Every Evidence Review carries a correction record, published in full. Three columns: what was wrong · where it came from · what changed. It names our own errors, and it names their causes.
It is not an apology. It is the credential. A pseudonymous review with no correction record is asking to be trusted. One with a correction record is asking to be checked.
| Version | What it means |
|---|---|
| 1.0 | First deposit |
| 1.1, 1.2 … | Corrections that do not change a certainty grading |
| 2.0 | Any change to a certainty grade, a finding, or the bottom line |
Every version receives its own permanent identifier (a DOI) under a single concept DOI for the review. Superseded versions remain accessible.
We do not silently correct. A corpus whose corrections are invisible has no credential at all.
When we find an error in someone else’s work
We find errors in published papers, in systematic reviews, and in the appraisal documents that inform national policy. What we do about them is governed by four rules.
- Verification means the document was opened. An error is not an error until we have read the source and checked the claim against it. Anything unverified is published nowhere and reported to no one.
- Report first. Publish afterwards. Errors go to the body that can correct them, in writing, before they go into anything we publish. A finding that has been reported and left unaddressed is a record. The same finding published without warning is an allegation.
- One notice, once, everything verified. A drip of corrections is a campaign.
- Trivial errors go to the people who can fix them, not to the people who would enjoy them.
Where a review reports errors in a third-party document, the deposit states plainly: these discrepancies were reported to [body] on [date]; as of [date] the published documents are unchanged. That single sentence is what a scientist does and a campaigner does not.
What this standard does not do
- It does not make our reviews peer-reviewed. They are not. They are narrative reviews, written by one author, under adversarial review. They do not follow PRISMA and they do not use a formal risk-of-bias instrument.
- It does not make them right. It makes them checkable, and it makes their errors visible when they are found. Those are different things, and the second is the only one we can promise.
- It does not replace clinical judgement. Nothing we publish is a prescribing protocol or a policy position, and no clinical decision should rest on it alone.
The short version. We publish the method because we cannot publish a name. Every gate here exists because something went wrong, and the thing that went wrong is named. The most important rule in the document is that a check nobody reads is not a check — and we know that because we built one, didn’t read it, and shipped thirty-four citations pointing at the wrong papers.
If you find an error in anything we publish, tell us. It will go in the correction record, with your finding credited, whichever way it cuts.
Version history of this standard
| Version | Date | What changed |
|---|---|---|
| 1.0 | 13 July 2026 | First published. Documented from the ER-014 and ER-015 builds. |