Learn / Search quality
How do you reduce hallucinations in RAG?
Short answer
Reduce hallucinations by making sure the right evidence is retrieved, instructing the model to answer only from it, requiring citations, verifying that each claim is supported, and refusing when the documents do not cover the question.
With troveGEN
troveGEN reduces hallucinations at both ends: better retrieval going in, and checked answers coming out, with citations that must resolve to a real passage.
Start with retrieval, not the prompt
A model that is given the wrong passages will answer fluently from them. The first defence is therefore search quality: documents read correctly, passages split sensibly, hybrid search, and a reranker. Missing evidence is the most common root cause of a wrong answer.
Ground the answer
- Tell the model to answer only from the supplied passages and to say so when they are not enough.
- Number the passages and require a citation marker for each claim.
- Keep the context focused: a few relevant passages work better than a long pile.
Verify, do not trust
Citations can be fabricated, so check them. A citation should point to a passage that really exists, and the claim next to it should be supported by that passage. Entailment models, which judge whether a passage supports a statement, can score this automatically, and arithmetic can be checked directly.
Know when to refuse
The safest answer to a question the documents do not cover is "I could not find this". Relevance thresholds and a check that the retrieved evidence really relates to the question prevent a confident answer built on an unrelated passage.
Use exact computation for numbers
Totals, averages and counts over tables should be computed, not generated. Let software do the arithmetic over the extracted table and have the model explain the result.
Key takeaways
- Fix retrieval first; a model cannot cite evidence it was never given.
- Require citations and verify them against the passages.
- Refuse, and compute numbers exactly, instead of guessing.
How troveGEN helps with reducing hallucinations
When you turn on answers, troveGEN validates every citation against a real passage, removes any that do not resolve, scores each claim for support with a local model, refuses when the evidence does not cover the question, and computes totals over tables exactly before the model explains them.
What troveGEN provides
- Citations validated against real passages; unsupported ones are removed
- Claim-by-claim support scoring with a local model
- Refusal when the documents do not cover the question
- Exact totals and averages computed over tables, then explained by the model
- Relevance thresholds so unrelated passages are not used as evidence
Frequently asked questions
Can hallucinations be eliminated?
Not entirely. They can be reduced substantially and made visible, through citations, claim checks and refusals, so that a reader can verify what matters.
Do bigger models hallucinate less?
Often somewhat, but a large model with poor retrieval still answers from the wrong evidence. Improving retrieval is usually cheaper and more effective.
What is groundedness?
A measure of how much of an answer is supported by the retrieved passages. It is also called faithfulness.
How does troveGEN help with reducing hallucinations?
troveGEN reduces hallucinations at both ends: better retrieval going in, and checked answers coming out, with citations that must resolve to a real passage. It provides: Citations validated against real passages; unsupported ones are removed; Claim-by-claim support scoring with a local model; Refusal when the documents do not cover the question; Exact totals and averages computed over tables, then explained by the model; Relevance thresholds so unrelated passages are not used as evidence.