How do you get accurate answers out of these things?

What actually cuts hallucination, and what just feels like it does? You have seen the tricks list: tell the model it's an expert, ask it to double-check itself, demand citations, force clean JSON. Some of those have real science behind them. Most do not. Today I'm ranking them by evidence.

Start at the top, because the winner is clear: retrieval grounding. Instead of asking the model to answer from memory, you hand it the source documents and make it answer from those. Of every lever we reviewed, this is the single best-documented one for factual accuracy.

Two results carry the claim. A peer-reviewed clinical decision support evaluation, published in 2025, tested retrieval-augmented setups and got hallucination rates down to 5.8 percent. That one is peer-reviewed and carries no vendor flag in our review. The second comes from a ServiceNow team at NAACL's 2024 industry track: adding retrieval cut hallucinations in structured outputs from 68 percent to 10. That is a vendor writing about its own system, so flag it. But the direction matches the independent work, and the survey literature treats retrieval as a leading class of mitigation while also cataloging its limits.

And it has limits. Retrieval only helps if the retrieval is good. Several analyses from 2024 and 2025, mostly preprints, show that noisy or irrelevant retrieved passages drag accuracy down, and one puts it bluntly: incorrect retrieval causes a sharp decline in overall system correctness. In practice the bottleneck is usually the retriever. The model takes the blame.

Second proven lever: voting. Wang and colleagues, peer-reviewed at ICLR 2023 and widely replicated since, sampled many reasoning paths from the same model and took the majority answer. Accuracy rose 17.9 percent on the GSM8K math benchmark, 11 percent on SVAMP, 12.2 on AQuA. No training, no new tools. You ask several times and count the votes. Replication across groups is what earns this one its grade.

Now the levers that feel like they work. Telling the model it's an expert: Zheng and colleagues, published in Findings of EMNLP 2024, an independent team, ran 162 personas against nine models and found no significant accuracy gain. Picking the best persona did no better than picking at random.

Demanding citations: the ALCE benchmark, peer-reviewed at EMNLP 2023, measured whether model citations actually support the claims made. On its hardest dataset, even the best models lacked complete citation support roughly half the time. A citation gives you something to audit. It is not proof.

Few-shot examples earned their reputation honestly, with an expiration date. The original GPT-3 paper at NeurIPS 2020, which is OpenAI measuring its own model, so flag it, showed TriviaQA rising from 64.3 percent zero-shot to 71.2 with examples. Then the era turned. Microsoft reported in 2024 that few-shot examples actively hurt its o1 reasoning model, and DeepSeek's own report says the same about theirs. Both vendors, both reporting against their own interest. On older chat models, examples help. On reasoning models, test before you paste.

And asking the model to check its own work? That one has the most surprising evidence in the episode, and Whitney takes it next.

The verdict: for retrieval grounding, the evidence is strong. Peer-reviewed results, vendor and independent, pointing the same way, with a replicated voting result standing beside it. The persona trick is also strongly evidenced, in the negative direction.

Here is what you should do. If accuracy matters, spend your effort on retrieval quality first: which documents the model sees, and whether they are the right ones. For reasoning tasks, sample more than one answer and take the vote. Skip the expert costume, and treat every citation as a lead to check rather than a receipt.

Sources

• Evaluating Retrieval-Augmented Generation Variants for Clinical Decision Support · Electronics (MDPI), peer-reviewed, 2025 [PR journal][no vendor flag] · https://www.mdpi.com/2079-9292/14/21/4227
• Ayala & Bechard, Reducing Hallucination in Structured Outputs via Retrieval-Augmented Generation · NAACL 2024 Industry Track [PR][Vendor: ServiceNow] · https://aclanthology.org/2024.naacl-industry.19/
• A Survey on Hallucination in Large Language Models · arXiv 2311.05232 · https://arxiv.org/pdf/2311.05232
• Evaluating RAG Under Adversarial Poisoning Attacks · arXiv 2412.16708, 2024 · https://arxiv.org/pdf/2412.16708
• Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models · ICLR 2023 [PR][widely replicated] · https://arxiv.org/abs/2203.11171
• Zheng et al., persona study · Findings of EMNLP 2024 [PR][Independent] (question-pool citation, cluster T4)
• Gao et al., Enabling Large Language Models to Generate Text with Citations (ALCE) · EMNLP 2023 [PR] · https://arxiv.org/abs/2305.14627
• Brown et al., Language Models are Few-Shot Learners · NeurIPS 2020 [PR][Vendor: OpenAI] · https://arxiv.org/pdf/2005.14165
• Microsoft few-shot/o1 finding (2024) and DeepSeek-R1 report [both Vendor, against interest] (question-pool citation, cluster T4)

Can the model catch its own mistakes? You have probably tried this. An answer comes back looking shaky, you type "are you sure?", and the model apologizes and produces something new. It feels like quality control. Michael just ranked the levers with strong evidence behind them; this is the popular lever he set aside for me, because the science on it is the most surprising in the episode.

The paper to know is Huang and colleagues at ICLR 2024. It comes out of Google DeepMind, so a vendor, and notice the direction: this result makes their own technology look worse, which is the kind of vendor result you can weight more comfortably. They tested what the field calls intrinsic self-correction. The model reviews its own answer with no new information coming in. No test results, no documents, no verifier, just the model and its previous output. Reasoning accuracy stayed flat or got worse. And the detail that should stop you cold: when the model did change its answer, it flipped right answers to wrong ones. The review pass you added for safety was actively breaking correct work.

That is one paper, so here is the sweep. Kamoi and colleagues published a meta-survey in TACL 2024, peer-reviewed, independent team. They went through the self-correction literature looking for successful cases, and they found zero successful self-correction studies where the model lacked outside information. Zero. Every genuine win had something external feeding the loop.

The "are you sure" reflex has its own paper. The FlipFlop experiment, an arXiv preprint, showed that simply challenging a model causes performance drops through sycophancy. The model reads your pushback as evidence it was wrong, even when it was right, and folds. Related work documents self-bias: models favor their own outputs when judging them. So the model is a bad editor of itself twice over. It grades its own work too kindly, and it caves the moment you frown at it.

Now the other side, because there is one. SELF-REFINE, peer-reviewed at NeurIPS 2023, reported gains around 20 percent on average from iterative self-feedback. Both results are real, and the reconciliation is in what the feedback contains. Self-refinement helps with fluency and instruction-following, the polish of the answer. For factual reasoning, the levers brief and the Huang result agree: nothing improves unless external feedback enters the loop.

And when external feedback does enter, the picture reverses. Reflexion, peer-reviewed at NeurIPS 2023, independent team, looped a model on real test results and lifted coding accuracy on HumanEval from 80 percent to 91. Same loop shape as self-correction. The difference is that the tests, an outside verifier with ground truth, told the model exactly what was wrong. The loop was never the problem. The mirror was.

Think about what this means for the products you are being sold right now. Any tool that promises reliability because "the model checks its own work" is promising you the intrinsic version, the one with zero successful studies behind it. When you hear that pitch, ask one question: checks it against what? If the answer is "against itself," you now know the literature on that.

The verdict: the evidence here is strong, and it is negative for the popular version. A model reviewing itself with nothing new to look at does not reliably catch its mistakes, and challenging it can make results worse. External verification works, and that finding is well replicated.

So here is what to do. Stop asking the model whether it is sure; you are measuring its sycophancy, and you risk flipping a right answer to a wrong one. Keep your revision loops, and wire something external into every one of them: run the test suite, check the claim against retrieved documents, use a separate verifier, or put a human eye on it. The loop is fine. Feed it facts instead of feelings.

Sources

• Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet · ICLR 2024 [PR][Vendor: DeepMind, against interest] · https://arxiv.org/abs/2310.01798
• Kamoi et al., self-correction meta-survey · TACL 2024 [PR][Independent] (question-pool citation, cluster T2)
• Madaan et al., SELF-REFINE: Iterative Refinement with Self-Feedback · NeurIPS 2023 [PR] · https://openreview.net/pdf?id=S37hOerQLB
• Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment · arXiv 2311.08596 · https://arxiv.org/pdf/2311.08596
• Reflexion · NeurIPS 2023 [PR][Independent] (question-pool citation, cluster T2: HumanEval 80% to 91% via looping on test results)
• Levers brief: docs/research/levers-that-improve-outputs.md (self-critique findings, navigation playbook item 3)

Does forcing JSON output cost you accuracy? If you build with these models, you do this constantly. Your code needs to parse the answer, so you turn on structured output and lock the model to a schema. The question is whether that lock squeezes the thinking. This one is genuinely contested, and the fight itself is the story, so let me walk you through the rounds.

Round one. Tam and colleagues published "Let Me Speak Freely?" at EMNLP 2024's industry track. Industry track, vendor-adjacent authors, so carry that flag with the result. They compared free-form generation against format-restricted generation on math and symbolic reasoning tasks and reported a significant decline in reasoning ability under format restrictions. Stricter constraints produced greater degradation. That paper traveled fast, and "JSON mode makes models dumber" became folk knowledge in about a month.

Round two. The rebuttal came from dottxt, and I want to say this flag as loudly as I can: dottxt is a company that sells structured generation tooling. A finding that structured output hurts accuracy is a finding that their product hurts you. They are grading their own homework. With that fully on the table, here is what they found: when they matched the prompts between the two conditions, the degradation failed to reproduce. Their argument was that the original comparison had given the free-form condition better prompts, so the study was measuring prompt quality, and the format took the blame. A motivated rebuttal can still be a correct one. You just verify harder.

Round three, and the current best reading. A 2026 synthesis from an independent group put the earlier results side by side and found that the penalty tracks model capacity. Strong models absorb the format constraint and lose little. Weak models pay for it, up to 36 points. A capacity-dependent penalty would also explain how one team measured a real drop and another measured nothing. And the loss is recoverable: let the model reason in free text first, then format the answer in a second step, and the penalty largely disappears.

One caution rides alongside all three rounds. A 2025 benchmark of constrained decoding, JSONSchemaBench, measured what schema enforcement actually guarantees: compliance. The JSON will parse. Nothing about the values inside is guaranteed. Valid and wrong is a failure mode you test for separately, and no output format will catch it for you.

The verdict for this video: contested, so here are both sides, and I am declaring no winner between the original paper and the rebuttal. What the independent synthesis adds is a practical resolution: the JSON tax appears real, it is sized by model capacity, and a two-step pattern makes it mostly avoidable.

Now the verdict for the whole episode, because the three videos stack into one playbook. I showed you that retrieval grounding is the best-documented lever against hallucination; that evidence is strong. Whitney showed you that models do not catch their own mistakes without help, and that external verifiers are what actually work; that evidence is strong too. And structured output is contested, with a cheap workaround while the fight settles.

So here are the three moves. Ground the model with retrieval, so it answers from documents instead of memory, and put your engineering effort into getting the right documents in front of it. Verify externally, with tests, retrieval checks, or a separate verifier, and never by asking the model to reassure you. And when you need JSON, especially from a smaller model, split the job: get the reasoning in free text first, then do the formatting as its own cheap second step.

Sources

• Tam et al., Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models · EMNLP 2024 Industry Track [PR][Vendor-adjacent] · https://aclanthology.org/2024.emnlp-industry.91/
• dottxt rebuttal (2024) [Vendor-funded: sells structured generation] · failed to reproduce Tam et al. with matched prompts (question-pool citation, cluster T5)
• 2026 independent synthesis [Independent] · capacity-dependent penalty, up to 36 points on weak models; reason-first-format-second recovers the loss (question-pool citation, cluster T5)
• JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models · arXiv 2501.10868, 2025 (measures compliance, coverage, efficiency; not reasoning accuracy) · https://arxiv.org/abs/2501.10868
• Levers brief: docs/research/levers-that-improve-outputs.md (structured-output findings, navigation playbook items 1, 3, 4)