Source Verification and Review-Until-Clean: A Fact-Checking Workflow for AI Content

AI-assisted writing has pushed the cost of producing an article toward zero and the risk of producing a wrong one to unprecedented levels: the model fluently completes details you never provided, search snippets quietly drop qualifiers, and your own research notes are already mixed with that morning's inferences. This article documents the workflow GPTMap's editorial team uses -- three writing rules governing the draft, a five-step review loop governing publication -- demonstrated with a real verification pass (the sentence-by-sentence check of Anthropic's September 1 announcement, run on 2026-09-02). The goal of the whole process is a single property: every sentence in the piece can be pointed to in a source by any reader or reviewer.

1. The Three Writing Rules

Rule one: key facts must be reproducible. Every date, price, feature detail, and quote in the text must map to one reproducible load of an official source -- not a search snippet, not a secondhand report, and not even your own research notes from that morning. The criterion: before publishing, re-fetch the source once and point to each key fact in it; anything unpointable gets deleted or downgraded to clearly-labeled hearsay.

Rule two: no embellishment beyond the source. A verified source does not make every sentence you build around it true. Three high-frequency failures:

• Directional comparisons -- the source says the free tier gets 3 tasks and you write that paid tiers are "more generous";
• Mechanism invention -- the source does not say how results are delivered and you write "push / notification / email";
• Detail inflation -- the source says "new email alerts" and you write "alerts for new email from specified senders".

All three read harmlessly, like connective tissue, but none is asserted by the source. The criterion matches rule one: every description, comparison, and explanation must be pointable in the original.

Rule three: parameters and enums come from the current extraction. For API content, every parameter name and enum/tier value in the text must be pointable in this session's fetched official documentation -- the extraction output is a whitelist, and anything outside it does not get written. APIs evolve fast and memory mixes versions; this rule exists to stop "the older API's parameter, from memory, in the new version's article".

2. A Derived-Number Rule

Any number you computed yourself -- count totals, day gaps, percentage deltas -- is produced by a command, not by mental math:

# Article count (this site's zh posts on the day)
ls apps/web/content/posts/*.zh.md | wc -l

# Days remaining (e.g., to a shutdown date)
python3 -c "from datetime import date; print((date(2026,9,24)-date(2026,9,1)).days)"

Convention checklist before citing a number:
[ ] Task-set / dataset version and date
[ ] Scoring convention (partial/strict, no-tools/with-tools)
[ ] Effort tier and default
[ ] Safeguards on? How are interventions scored?
[ ] Error margins and trial counts
[ ] Harness (evaluation framework) name

The same family of problem: self-referential statistics need an as-of date. "This site now has 71 articles" without a date is a time bomb -- in three months it is guaranteed wrong. Write "as of 2026-09-01, 71 articles", or omit the total.

3. The Five-Step Review Loop

The last stage before publishing is a loop, not a pass:

1. Re-fetch every source. Reload and archive each official URL the piece relies on. Sources that no longer load downgrade their facts. Re-fetching matters because official pages change between writing day and publish day.
2. Compare sentence by sentence. Dates, prices, mechanism claims, quotes -- point to each one's location in the re-fetched original. There is no shortcut here; "it felt fine on a read-through" does not catch "more generous" or "push notification".
3. Run the three-class phrase scan. Comparatives (more/faster/looser/higher), mechanism verbs (push/store/fail/automatically), qualifiers (specified/default/automatic). Check every hit against the source. The scan is triage: hits are a to-check list, not an error list; strip code blocks from the range so indentation false positives do not bury real findings.

The scan patterns for the three risk classes (grep-ready):

grep -nE '(more|faster|looser|higher|stronger|cheaper)' draft.md   # comparatives
grep -nE '(push|store|fail|automatically|sync)' draft.md           # mechanism verbs
grep -nE '(specified|default|automatic)' draft.md                  # qualifiers

4. Fix and re-diff. Fix with scripts that save each file immediately -- a crashed fix script loses its fixes silently; after fixing, re-read the diff to confirm the fix added no new claims and dropped nothing.
5. Re-run until a finding-free round. Fixes can introduce new problems, so the full review re-runs. One consecutive finding-free round is the ship condition.

4. A Worked Example: Verifying a Real Announcement

On 2026-09-02, this site ran the full loop against Anthropic's Claude Fable 5.1 / Mythos 5.1 announcement from the day before. Replay: step one, re-fetched anthropic.com/claude-fable-and-mythos-5-1 and the newsroom page, confirming titles, byte magnitudes, and the Sep 1, 2026 date field. Step two, pointed to the original sentence behind every draft fact -- "two safeguard tiers of the same model", "$10/$50 per MTok", "cache reads at $0.25/MTok, down 75%", "about -25% typical, up to about -45% highly agentic", "EFS in phases starting later this fall", "Mythos currently limited to US organizations", "effort defaults: High in Claude Code, Medium in Claude Cowork and Claude.ai". Step three, the scan surfaced comparative phrasing like "fewer false positives (-60%)", checked against the original's "cybersecurity false positives down 60%" -- convention holds. Step four, fixed two vague draft phrasings (one had turned the official "estimated" into a certainty; the qualifier was restored). Step five, a re-run came back with zero findings, and the piece shipped.

The telling detail: all seven pricing and convention facts pointed back to a single reproducible load, and what actually caught problems was not "a read-through" but the step-three phrase scan and the step-four re-diff.

5. Common Mistakes and Troubleshooting

• Substituting "the source is official" for "I read the original": official domains get paraphrased too; every hop in the citation chain adds a verification obligation.
• Verifying only once: fixes introduce new problems, sources update the same day, scripts fail silently -- a single pass is how things slip. Loop to zero findings.
• Letting scan false positives bury real findings: strip code blocks, URLs, and code comments from the scan range; order hits by risk.
• Mixing notes into evidence: label research notes as "what the source said" vs "what I inferred", and draft only from the first kind.
• Fixes that never land: fix scripts must save per file and verify the write; re-diff afterward.

6. Next Steps

• This loop's published track record on a news piece: OpenAI Ecosystem Week 42 Flash: Assistants API Shut Down, Sol Price Cut, Transcription Deprecations.
• The full pre-publish checklist this workflow feeds into: The GPTMap editorial checklist: 12 steps to a publishable OpenAI article.
• The sibling discipline for benchmark numbers: How to Read the Benchmark Charts in a Model Launch: Effort Curves, Log Cost Axes, and Safeguard Interventions.
• Maintaining articles after publication: GPTMap Article Maintenance Guide: Re-testing, Version Sync, and Sunset.