RAG-Friendly Prompting: How to Force “Answer Only From Sources”

“How do we stop the model from making things up?” Make it legally impossible for the assistant to answer without evidence. This post gives you RAG-friendly, source-only prompting that refuses to answer when citations are missing, plus quick approvals for risky replies. If you’re new to retrieval, start with RAG for Business and privacy basics in AI Data Privacy 101. For stack choices, see Choose Your AI Stack.

Manager Mode – when to require citations and how strict to be

Decide which work must be evidence-backed and how strict your sources should be. Use this matrix to align legal, support, and content teams before you build.

The table maps common tasks to citation requirements and review rules. Start strict, then relax only after clean pilots.

Trust set and domain policy

Sources define quality. Maintain a short allowlist and enforce it in prompts and tooling.

• Internal: wiki, runbooks, signed contracts, KPI glossary.
• Public: vendor docs, regulators, major standards bodies.
• Blocklist: forums, aggregator scrapes, unknown blogs (unless whitelisted).

When to refuse to answer

Refusal prevents incidents. If the trust set doesn’t have coverage, the assistant should say “I don’t know” and open a request for documentation or human help.

Builder Mode – retrieval frames, filters, schemas, approvals

Implement these copy-paste frames to force source-only behavior. Each piece adds a thin layer of control that compounds into trustworthy answers.

1) Source-only retrieval prompt (internal KB / RAG)

This frame refuses to answer if the KB excerpts don’t cover the question. It returns structured JSON so you can gate on missing fields.

System:
Answer ONLY from the provided internal sources. If insufficient, return:
{"answer":null,"reason":"no_sources"}.
User:
Question: {{question}}
Internal sources (title,uri,excerpt,date,scope):
{{kb_results}}
Schema:
{"answer":"string|null","citations":[{"title":"string","uri":"string","date":"YYYY-MM-DD"}],"confidence":"low|medium|high","policy_flags":["missing_source","scope_violation"]}
Return ONLY JSON.

Gate any output where policy_flags is non-empty or citations is empty.

2) Web/live-search frame with trust-domain and freshness filters

Use for public facts. The assistant must cite at least two allowed domains that meet your freshness window.

System:
Answer ONLY from ALLOWED_DOMAINS and results within FRESHNESS_DAYS. If insufficient, return {"answer":null,"reason":"no_sources"}.
User:
Question: {{question}}
ALLOWED_DOMAINS: {{["vendor.com","regulator.gov","standards.org"]}}
FRESHNESS_DAYS: {{30}}
Web results (title,url,excerpt,date):
{{web_results}}
Schema:
{"answer":"string|null","citations":[{"title":"string","url":"string","date":"YYYY-MM-DD"}],"confidence":"low|medium|high","policy_flags":["off_domain","stale_source"]}
Return ONLY JSON.

Reject outputs with off_domain or stale_source flags and prompt the editor to replace sources or relax the window.

3) Chunking & retrieval hints that reduce misses

Good prompting can’t rescue a bad index. Apply these small rules for more reliable citations.

• Chunk size: ~500–1,000 tokens with overlap for policies/SOPs.
• Hybrid retrieval: combine dense vectors with BM25 keyword search.
• Field tags: store title, uri, section, date, scope with every chunk.
• Query rewrite: expand acronyms and add synonyms before search.

4) Answer schema for customer-facing text

Keep responses auditable and easy to review. Missing citations or flagged risks should hard-fail to human review.

{
  "title":"string",
  "body_markdown":"string",
  "citations":[{"title":"string","uri_or_url":"string"}],
  "confidence":"low|medium|high",
  "policy_flags":["missing_source","off_domain","stale_source","pii_detected"]
}

5) One-click approval for risky replies

Route customer answers, legal text, and policy changes through a fast human gate. Keep it skimmable so reviewers approve in seconds, not minutes.

Subject: Approve evidence-backed answer for {{ticket_id}}
Summary: {{title}}
Confidence: {{confidence}} • Policy flags: {{policy_flags}}
Citations: {{citation_1}}, {{citation_2}}
Actions: ✅ Approve  |  ✏️ Edit  |  ❌ Reject
SLA: 2h • Escalates once to backup

6) Handling conflicts & gaps

Sources will sometimes disagree. Teach the assistant to surface that conflict and stop, not to average the truth.

• If two sources disagree → set confidence low, include both citations, route to review.
• If KB is missing → return {"answer":null,"reason":"no_sources"} and open a “document request.”

7) Minimal logging fields (privacy-aware)

Logs must help audits but avoid leaking sensitive text. Store hashes and IDs rather than full content where possible.

• user_id, ts, model_name, model_version
• prompt_id, retrieval_set_id, trustset_version
• citations_list (URIs/URLs), confidence, policy_flags
• approval_id, approver, decision, retention_code (R30/R90/R365)

8) KPIs to review weekly

Small, boring numbers keep you safe. Add them to your analytics and review them every Friday.

• Citation coverage: % answers with 2+ valid citations.
• Approved answer rate: editor-approved risky answers ÷ total.
• Reviewer seconds to approve: median trending down.
• Off-domain/stale flags: count per 1k answers (down and to the right).

FAQ – source-only prompting

Do we need RAG for every answer?
No. Require RAG for policy, support, and anything customer-facing. For brainstorming, allow draft-only outputs without citations.

What if the assistant returns null often?
Improve your KB coverage, widen freshness windows, or add trusted public domains. Refusal is safer than invention.

Can we mix internal and web sources?
Yes. Prefer internal for policy and product; add web only for public facts with allowlists and freshness rules.

How do we enforce this in tools?
Use JSON-only outputs, gate on policy flags, and wire the approval template via your Automation Workflows.

Final thoughts

Hallucinations stop when answers must cite sources or refuse gracefully. Combine trust-domain filters, freshness windows, JSON schemas, and one-click approvals. Start strict, measure citation coverage and approval speed, and only relax rules when the data proves you can.