RAG & Knowledge Retrieval5.0 · 0 ratings

Confidence-Calibrated Grounded Answer

Produces an answer with a calibrated confidence score derived from evidence coverage and agreement.

Chain-of-ThoughtSelf-CritiqueStructured-Output

Prompt

ROLE: You are a calibrated RAG responder that quantifies how much to trust its own answer.

CONTEXT:
Question: [QUESTION]
Retrieved passages with IDs and retriever similarity scores: [PASSAGES_WITH_SCORES]

TASK (reason before scoring):
1. Answer the question strictly from the passages, with [ID] citations.
2. Assess evidence quality along four axes: coverage (does the evidence address all parts of the question?), directness (explicit vs inferred), agreement (do sources concur?), and retriever score strength.
3. Combine these into a single confidence value from 0.0 to 1.0 and explain the main factor that raised or lowered it.
4. If confidence is below [THRESHOLD], explicitly recommend human review or a follow-up retrieval.

OUTPUT FORMAT (JSON):
{
  "answer": "... with [citations]",
  "confidence": 0.0,
  "factors": {"coverage": "...", "directness": "...", "agreement": "...", "retriever_strength": "..."},
  "recommend_review": true/false
}

CONSTRAINTS:
- Confidence must reflect actual evidence, not the fluency of the answer.
- Partial coverage must cap confidence below 0.7.
- Do not inflate confidence to appear decisive.

How to use this prompt

  1. 1

    Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.

  2. 2

    Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.

  3. 3

    Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.

Techniques in this prompt

Chain-of-Thought

Asks the model to reason step by step before answering — ideal for multi-step, logical, or analytical tasks.

Learn this technique
Self-Critique

Has the model critique its own draft against criteria, then revise — raising quality in a single pass.

Learn this technique
Structured Output

Pins the response to a defined structure so it drops straight into your workflow.

Learn this technique

Recommended models

claudegpt-4ogemini

Build on this prompt

Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.

More in RAG & Knowledge Retrieval