The model won't tell you it's guessing

We ran a batch of factual queries through Claude Sonnet 4.6 last month. Some questions had clear answers in the retrieval context. Others had nothing. The model answered both with the same confident tone.

That’s the default behavior. The model doesn’t distinguish between “I found this in the source” and “I’m completing a plausible pattern.” It just answers.

So if you want a confidence signal, you have to build one. A few approaches we’ve used: check the cosine similarity score from your vector retrieval step before passing context to the model. If it’s below a threshold, flag the output. Or run the same prompt twice with temperature 0.8 and compare. High disagreement between runs is a reasonable proxy for low certainty. A simpler option: add a follow-up prompt asking the model to rate its own answer and cite what it based it on. Not perfect, but it surfaces hedging the model wouldn’t volunteer otherwise.

None of these are clean solutions. They add latency, they add cost, and they can still miss things. But waiting for the model to flag its own uncertainty doesn’t work. It usually won’t.