The four ways RAG goes quietly wrong

Four failures, none of which raise

Failure	What it looks like	What actually happened
Chunk boundary	a confident half-answer	the sentence completing it is in the next chunk
Stale context	last year's policy, stated firmly	the corpus was never re-indexed
Lost in the middle	the answer was retrieved and ignored	it sat in the middle of a long context
Conflicting sources	one of two contradictory answers	both were retrieved; the model picked one

None of these throw an exception. Every one produces a fluent, plausible answer, which is what makes them expensive.

Lost in the middle is the one with a cheap fix. Models attend most strongly to the start and end of a long context, so the best passage placed fourth of seven is the one least likely to be used. Reorder so the strongest chunks sit at the edges — best first, second-best last, the weakest buried in the middle.

Your turn: write edge_order(chunks) taking chunks best-first and returning them reordered so the strongest are at the outside.

def edge_order(chunks):
    """Reorder best-first chunks so the strongest sit at the start and the end."""
    return list(chunks)

Detecting a contradiction rather than averaging it

When two retrieved passages disagree — the handbook says 30 days, the updated policy says 14 — a model will usually pick one, state it confidently, and give you no hint that the other existed.

You cannot fix that in the prompt alone. What you can do is notice: if the retrieved passages disagree on the fact being asked about, that is a signal worth surfacing, and often worth answering with "your documents disagree" rather than picking a side.

The cheap version, which catches a surprising amount: extract the answer-shaped values from the passages and see whether they agree.

Your turn: write disagreement(passages, pattern) returning the sorted distinct values matching pattern across the passages. Two or more means the sources conflict.

import re


def disagreement(passages, pattern):
    """Return the sorted distinct values matching pattern across the passages."""
    return []