Cover diagram: RAG reference and attribution system connects answer assertions, evidence fragments, source metadata and audit verification into a chain of evidence

Date: 2026-06-19
Suitable for readers: engineers, product leaders and technical leaders who are working as an enterprise knowledge base, document quiz, customer service assistant, research assistant, compliance quiz or internal Agent.

Abstract

In the previous article, I talked about how to evaluate the retrieval quality of RAG.

This article continues to go down: How should the reference and attribution of RAG be done?

Many systems will put several "source links" at the end of the answer, but this does not mean that the attribution is reliable. A really useful reference answers three questions:

  1. Who said this?
  2. What evidence is it based on?
  3. Does this evidence really support this conclusion?

If only one document name is displayed, the user cannot know which sentence in the answer comes from; if only a chunk is displayed ID, the user cannot judge whether the source is trusted, expired or out of power; if the model only writes the reference number by itself, but does not do post-check verification, the reference can easily become a decoration.

This article gives a set of engineering design: split the RAG output into claim, citation, evidence span, source metadata and audit record; use structured output constraint model; claim-to-citation checker for posterior verification; in the UI, make the source into a clickable, highlightable, and auditable chain of evidence.

Contents

1. References are not decorations, but product contracts

In a production system, the reference bears at least four responsibilities.

Responsibility Problem failures
can verify Can users click on the evidence and see the original text? The answer cannot be checked
to be blamed Where does the conclusion come from? Unable to replay after error
Can be audited Does the system record the evidence retrieved at that time? Online accident cannot be restored
manageable Is the source expired, more powerful, and low credible? Compliance and Security Risks

So the reference is not the "beautify answer" format, but the RAG system's commitment to the user: The key conclusion in the answer must be able to return to the evidence.

It can be seen in the OpenAI File Search document that when using a File to Search, the response will include file_search_call and with file_citation Annotation message; if you want to analyze the search results, you can also explicitly include file_search_call.results. This means that the reference should not just stay at the natural language layer, but should be part of the system output.

The Web Search document also emphasizes that Inline Citations should be clearly visible and clickable when displaying Web results to users or information in them. This principle also applies to enterprise RAGs: references should be seen, clicked, and verified by users.

2. Three concepts: source, evidence, assertion

The most common problem with reference design is to mix Source, Evidence and Claim together.

2.1 Source: Source

Source is the original data unit, for example:

  • PDF file;
  • Notion page;
  • contract version;
  • work order record;
  • database row;
  • web URL;
  • Internal policy documentation.

It answers "what is this material".

Source metadata should at least contain:

{
  "source_id": "refund_policy_2026_03",
  "title": "退款政策 2026-03 版",
  "source_type": "policy_doc",
  "version": "2026-03",
  "updated_at": "2026-03-18",
  "owner": "finance_ops",
  "access_scope": ["customer_service", "finance"],
  "canonical_url": "kb://policies/refund/2026-03",
  "content_hash": "sha256:..."
}

2.2 Evidence: Evidence

Evidence is the smallest text or structured fragment that can support a conclusion. It can be a chunk, a piece of original text, a table cell, a database record, or a document area.

It answers "Which paragraph material supports this statement".

{
  "evidence_id": "refund_policy_2026_03#chunk_09",
  "source_id": "refund_policy_2026_03",
  "page": 4,
  "section": "跨境退款",
  "span": {
    "start_char": 1832,
    "end_char": 1938
  },
  "quote": "跨境支付订单需进入人工审核,审核完成后按原支付渠道退款。",
  "retrieval_score": 0.82
}

2.3 Claim: Assertion

Claim is a key conclusion in the answer that can be judged true and false.

For example:

跨境支付订单需要进入人工审核。

It is not a whole paragraph of answer, nor is it a topic. It should be small enough to ask: "Can this sentence be supported by a certain piece of evidence?"

FActScore splits long text into Atomic Facts, and then judges whether each fact is supported by reliable knowledge sources. This idea is very inspiring for RAG: Reference checks should also try to make assertion-levels, not paragraph-levels.

3. How to choose the reference granularity

Different risk scenarios require different reference granularity.

Granularity Example When to use it Risk
documentation level "Source: Refund Policy 2026-03" Low-risk summary, internal knowledge browsing Users want to flip the document by themselves
paragraph level lead to a section or chunk General knowledge quiz, customer service assistant Chunk is still difficult to check when it is too long
Sentence/Segment Level A sentence in the original text of highlighting Policies, contracts, medical, financial Higher cost
field level Bring to table cells or database fields Report, Contract Field, Order Status Structured ingestion is required
assertion level Each claim binds the citations High-risk Q&A, audit scenarios Requires post-checking

My advice is:

  • For low-risk internal knowledge bases, at least paragraph-level references.
  • For policies, contracts, and customer service processes, a fragment-level reference to key conclusions.
  • For legal, medical, financial, compliance, and external commitments, make assertion-level references, and keep audit records.

Don't pursue all sentences with references in the first place. A better way is to mark the "key assertions":

  • number;
  • date;
  • conditions;
  • exception;
  • rights and obligations;
  • operating steps;
  • Conclusions that affect users.

These places must have verifiable evidence.

4. A set of recommended data structures

The reference system of RAG can be organized with four types of objects.

4.1 Evidence Pack

Before entering the model, organize the search results into the Evidence Pack. Note that the Evidence ID for the model must be stable, short and referenced.

{
  "query_id": "refund_001",
  "evidence_pack": [
    {
      "evidence_id": "E1",
      "source_id": "refund_policy_2026_03",
      "title": "退款政策 2026-03 版",
      "version": "2026-03",
      "updated_at": "2026-03-18",
      "section": "跨境退款",
      "quote": "跨境支付订单需进入人工审核,审核完成后按原支付渠道退款。",
      "access_scope": ["customer_service", "finance"],
      "source_hash": "sha256:abc..."
    },
    {
      "evidence_id": "E2",
      "source_id": "refund_policy_2026_03",
      "title": "退款政策 2026-03 版",
      "version": "2026-03",
      "updated_at": "2026-03-18",
      "section": "到账时间",
      "quote": "退款通常在 1-5 个工作日内退回原支付渠道。",
      "access_scope": ["customer_service", "finance"],
      "source_hash": "sha256:abc..."
    }
  ]
}

Don't just give the model a large paragraph of text. Short IDs, titles, versions, quotes, and source metadata are all important.

4.2 Answer Contract

Let the model output structured answers, not just Markdown.

{
  "answer": "跨境支付订单需要进入人工审核。审核完成后,退款通常会在 1-5 个工作日内退回原支付渠道。",
  "claims": [
    {
      "claim_id": "C1",
      "text": "跨境支付订单需要进入人工审核。",
      "citations": ["E1"]
    },
    {
      "claim_id": "C2",
      "text": "审核完成后,退款通常会在 1-5 个工作日内退回原支付渠道。",
      "citations": ["E1", "E2"]
    }
  ],
  "insufficient_evidence": false
}

This structure brings three benefits:

  • Claim can be checked one by one;
  • The reference can be rendered as a user-clickable UI;
  • Audit logs can be recorded.

4.3 Citation Object

When it is finally displayed to the user, Citation should not just E1. It should be parsed into a readable source.

{
  "citation_id": "E1",
  "label": "退款政策 2026-03 版,跨境退款",
  "source_id": "refund_policy_2026_03",
  "quote": "跨境支付订单需进入人工审核,审核完成后按原支付渠道退款。",
  "locator": {
    "page": 4,
    "section": "跨境退款",
    "start_char": 1832,
    "end_char": 1938
  },
  "url": "kb://policies/refund/2026-03#page=4",
  "updated_at": "2026-03-18"
}

4.4 Audit Record

When there is a problem online, you can't just look at the answers that the user sees. To be able to restore what the system saw at that time.

{
  "trace_id": "rag_20260619_000431",
  "user_id_hash": "u_...",
  "query": "跨境订单退款多久到账?",
  "retrieval": {
    "queries": ["跨境订单退款多久到账", "跨境支付退款人工审核"],
    "filters": {"department": "customer_service"},
    "top_k": 8,
    "result_ids": ["E1", "E2", "E3"]
  },
  "answer_contract": "...",
  "citation_check": {
    "unsupported_claims": [],
    "missing_citations": [],
    "stale_sources": []
  },
  "model": "production-answer-model",
  "generated_at": "2026-06-19T17:57:23+08:00"
}

Pay attention to privacy and compliance: The log does not necessarily save the full user identity or sensitive text, but you must save enough hash, source id, version, reference and verification results to facilitate review.

5. Let the model output a verifiable answer

Models are good at writing natural language, but natural language is not suitable for direct auditing. An available generation instruction should clarify three things.

5.1 Only allow references to the IDs in the Evidence Pack

你只能引用 evidence_pack 中出现的 evidence_id。
不要编造 citation id。
如果没有足够证据,设置 insufficient_evidence=true,并说明缺什么证据。

This can reduce the source of the model to create out of thin air.

5.2 Each critical assertion must have CITATIONS

关键断言包括数字、日期、条件、例外、步骤、限制、权限、法律/财务/医疗影响。
每个关键断言必须放入 claims 数组,并绑定一个或多个 citations。
不要把多个无关事实塞进同一个 claim。

This reduces the question of "there is only one reference in the entire answer".

5.3 Downgrade when uncertain

如果证据之间冲突,必须指出冲突来源,不要自行选择一个结论。
如果证据不足,必须输出 insufficient_evidence=true。
如果答案需要外部最新信息,而 evidence_pack 没有,必须说明无法基于当前资料回答。

This reduces "referenced illusions of packaging".

6. How to do Claim-to-Citation

After the model output is referenced, the posterior verification is also required. A basic checker can be divided into four steps.

6.1 Parsing CLAIMS

Let's split the answer first into claim. The most stable is to let the model output directly claims array; if the history system can only output Markdown, it can post-process the key assertion.

6.2 Check whether the Citation ID exists

Check if each Citation is from the Evidence Pack. Any ID that does not exist is a hard error.

6.3 Check if the quote supports claim

There are three levels of support relationships:

Method When to use it Limitations
String/field rules Exact numbers, dates, order status Unfriendly to rewriting
structured comparison Forms, databases, contract fields Structured when ingested
LLM judge / NLI Open text, complex induction Calibration and sampling manual review required

Although the OpenAI Graders document is in the platform migration period, the ideas are still valuable: the evaluation can be combined in String Check, Text Similarity, Score Model, Python Grader and other methods. For RAG references, you can also combine rule checks and model checks.

6.4 Failure CLAIM to do policy processing

Don't just use the checker as a log. It should affect product behavior:

check result Recommended processing
Citation ID does not exist Refuse to show this answer, retry or downgrade
claim no citation Delete the claim or ask for a model rewrite
Citation does not support claim Delete, rewrite or mark as insufficient evidence
source expired Retrieve with the new version, or prompt version conflict
source is more than right Block now and log security events

In high-risk scenarios, the Citation Check failure should not be "sickly online".

7. Complex Scenarios: Multi-source, Conflict, Table, Summary, and Permissions

7.1 Multi-source support

Some conclusions require common support for multiple evidence.

{
  "claim_id": "C3",
  "text": "跨境退款需要人工审核,审核后仍按原支付渠道退回。",
  "citations": ["E1", "E2"],
  "support_type": "joint"
}

This CLAIM cannot just check a single Citation. To allow Joint Support: E1 supports "manual review", and E2 supports "original payment channel return".

7.2 Conflict evidence

If two sources conflict, don't let the model be silenced.

{
  "claim_id": "C4",
  "text": "关于跨境退款到账时间,当前资料存在版本冲突。",
  "citations": ["E2", "E7"],
  "conflict": true,
  "resolution": "E2 是 2026-03 版,E7 是 2025-10 版;应优先使用 E2。"
}

Conflict is not a bad thing. The bad thing is that the system pretends not to conflict.

7.3 Table and Database Fields

Table references should not just refer to the entire table. It is best to save cells, row and column names, and filter criteria.

{
  "evidence_id": "E12",
  "source_id": "pricing_table_2026_q2",
  "locator": {
    "row_key": "enterprise",
    "column": "sla_credit",
    "sheet": "SLA"
  },
  "value": "10%",
  "quote": "enterprise / sla_credit = 10%"
}

Otherwise, the user will open a large table and still do not know where the answer is.

7.4 Abstract References

Summary is most likely to have the problem of "every sentence seems to have a basis, but the original text cannot be found".

The recommended practice is:

  • First establish a citation for the original evidence;
  • Abstract sentences refer to the original evidence, not the intermediate products of the abstract;
  • If you have to refer to the intermediate summary, also keep the source ids used when the summary is generated.

7.5 Permission and Tenant Isolation

The reference system cannot disclose the source title, file name, or quote that the user does not have permission to see.

Error example:

根据 VIP 客户折扣表,ACME 的折扣为 27%。

Even if the answer does not expand the document content, the filename itself may leak sensitive information.

The correct way is to retrieve the forced permission filtering before retrieving, and check the Citation again before displaying. access_scope. When permissions fail, the Citation Label should not be sent to the frontend.

8. How to display references in UI

A good reference UI does not stack footnotes at the end of the answer, but allows users to quickly check.

8.1 Inline citation

Reference tags are displayed next to the key sentence:

跨境支付订单需要进入人工审核 [1]。审核完成后,退款通常会在 1-5 个工作日内退回原支付渠道 [1][2]。

Mark to be able to click. After clicking, the source card is displayed.

8.2 Source drawer

Source cards show at least:

  • document title;
  • version;
  • update time;
  • section/page;
  • original quote;
  • Permission range or visibility statement;
  • "Open original text" link;
  • "Which sentence should the reference support".

8.3 The original text is highlighted

If the source is a PDF, web page, knowledge base page, click Citation to jump to the original location and highlight the reference clip. Users should not manually find answers in PDF 80.

8.4 Uncertainty display

If there is insufficient evidence or conflict, do not pretend to be determined with a green tick.

Three states can be used:

Status Explanation
supported Reference fragment directly supports claim
partially support References to support some facts, still missing another part
Insufficient evidence There is not enough source to support

8.5 References do not block reading

The purpose of citing UI is not to show off. The text should be smooth, and the evidence should be able to unfold. In particular, it is necessary to avoid interrupting reading of large footnotes in particular on the mobile terminal.

9. Online logs and audit records

After the RAG error, I am most afraid that there is only one user complaint:

它引用了错误政策。

But you don't know at the time:

  • What is the user problem;
  • What is the retrieval of the query that is rewritten;
  • What is the top-k result;
  • what evidence is in the context;
  • What Claims are output by the model;
  • Whether the Citation Checker is passed;
  • what version of the knowledge base was at that time;
  • The front end shows what sources finally show.

So the online trace records at least:

Field Use
trace_id Associate user feedback, logs and reviews
query original question
rewritten_queries Review search
filters Review permissions and versions
retrieved_evidence_ids See Recall and Sort
context_evidence_ids Look at the evidence of the final model
answer_claims See assertion split
citations See reference relationship
citation_check_result See if you are showing sickness
source_hashes Review Knowledge Base Version
model_and_prompt_version regression test

If the data is sensitive, save the hash and locator, and if necessary, re-fetch the evidence through the controlled background. Don't turn the system into unauditable for logging.

10. How to Evaluate Reference Quality

ALCE evaluates long-form answers with citations on dimensions including fluency, correctness, and citation quality. WebGPT collects references to help users assess factual accuracy. RARR explores adding attribution to existing model outputs and revising unsupported content. Together, these studies make a case for measuring citation quality separately, since an overall answer satisfaction score can conceal problems with the sources.

A set of practical indicators can be designed as follows:

Metric Meaning
Citation Coverage The ratio of citation in the key claim
Citation Support Rate The ratio of CITATION actually supports claim
Unsupported Claim Rate Claim ratios that are exported without evidence
Missing Citation Rate Claim ratio that should be referenced but not referenced
Citation Precision The true proportions in the cited evidence
Citation Recall Proportion of reference in Gold Evidence
Over-citation Rate Proportion of a simple claim binding too many weak related sources
Stale Citation Rate Proportion of citing expired sources
Permission-safe Citation Rate Reference ratios without an overpowered source

The evaluation set covers these samples:

  • Direct support for documents;
  • Joint multi-evidence support;
  • Reference partial support;
  • References are not supported;
  • The answer is correct but the reference is wrong;
  • The answer is wrong but the reference seems relevant;
  • conflict between old and new versions;
  • Insufficient permissions;
  • no answer questions;
  • table field problem;
  • Summary question.

If you don't have a sample of "wrong answer but wrong reference" in your eval, the system can easily learn to use relevant documents as decoration.

11. An example of a minimum validator

The following example only does basic rule verification: whether the Citation ID exists, whether the claim has a reference, and whether the reference quote quotes the key phrase in the claim. In the real system, semantic entailment, field alignment, LLM Judge and manual sampling are also required.

import json
import re
from pathlib import Path


def normalize(text):
    return re.sub(r"\s+", "", text.lower())


def keyword_overlap(claim, quote):
    claim_norm = normalize(claim)
    quote_norm = normalize(quote)

    # 真实项目应使用领域词表或实体抽取,这里只保留一个最小示例。
    tokens = re.findall(r"[\u4e00-\u9fff]{2,}|[a-zA-Z0-9%.-]+", claim)
    important = [t for t in tokens if len(t) >= 2]
    if not important:
        return 0.0

    hits = sum(1 for token in important if normalize(token) in quote_norm)
    return hits / len(important)


def check_answer(answer_contract, evidence_pack, threshold=0.5):
    evidence_by_id = {item["evidence_id"]: item for item in evidence_pack}
    errors = []
    warnings = []

    for claim in answer_contract.get("claims", []):
        claim_id = claim.get("claim_id", "<missing>")
        claim_text = claim.get("text", "")
        citations = claim.get("citations", [])

        if not citations:
            errors.append({
                "claim_id": claim_id,
                "type": "missing_citation",
                "message": "关键断言没有引用"
            })
            continue

        support_scores = []
        for citation_id in citations:
            evidence = evidence_by_id.get(citation_id)
            if evidence is None:
                errors.append({
                    "claim_id": claim_id,
                    "type": "unknown_citation",
                    "citation_id": citation_id
                })
                continue

            score = keyword_overlap(claim_text, evidence.get("quote", ""))
            support_scores.append(score)

        if support_scores and max(support_scores) < threshold:
            warnings.append({
                "claim_id": claim_id,
                "type": "weak_support",
                "best_score": round(max(support_scores), 3),
                "message": "引用可能不足以支持该断言,需要语义校验或重写"
            })

    return {
        "passed": not errors,
        "errors": errors,
        "warnings": warnings
    }


if __name__ == "__main__":
    answer_contract = json.loads(Path("answer_contract.json").read_text(encoding="utf-8"))
    evidence_pack = json.loads(Path("evidence_pack.json").read_text(encoding="utf-8"))
    print(json.dumps(check_answer(answer_contract, evidence_pack), ensure_ascii=False, indent=2))

This checker is rough, but it illustrates an important principle: References must be checked by the program, and cannot fully believe the model itself says "I'm referring".

For high-risk scenarios, you can add another LLM Judge:

你是引用校验器。判断 Claim 是否被 Evidence 直接支持。

Claim:
{{claim_text}}

Evidence:
{{quote}}

只输出 JSON:
{
  "supported": true | false | "partial",
  "reason": "一句话说明",
  "missing_fact": "如果 partial 或 false,说明缺什么"
}

LLM Judge is also evaluated. Don't make Judge a new uncontrollable black box.

12. Common Mistakes

12.1 Reference is equivalent to trust

References may be irrelevant, expired, overpowered, or taken out of context. The real thing to check is Support Relation.

12.2 Only the source is listed at the end of the answer

The list of sources at the end is suitable for further reading and is not suitable to prove each key conclusion. The key assertion requires inline citation.

12.3 Let the model compile the Citation ID by itself

The Citation ID must come from the Evidence Pack provided by the system. The model cannot invent the source.

12.4 Reference the entire PDF

A citation to an entire 80-page PDF is too broad. Identify at least the section, page, or chunk. In high-risk settings, point to the exact quotation or supporting span.

12.5 Ignore tables and structured data

Many business answers come from tables, database fields, or configuration items. Don't throw away the row and column information after converting them into a large piece of text.

12.6 Reference UI cannot be clicked

Non-clickable references cannot be checked. Users can ultimately choose to believe or not to believe the model.

12.7 Do not save the current source version

The knowledge base will be updated. The source that is clicked today is not necessarily the source that the model saw when it answered yesterday. Audit logs must be logged in version or hash.

13. Landing road map

Week 1: Standardize the Evidence ID

  • Assign short IDs to each chunk that enters the context;
  • Keep source ID, title, version, quote, locator;
  • Only these IDs are allowed in Prompt.

Week 2: Change the Output Contract

  • Let the model output answer + claims + citations + insufficient_evidence;
  • granular constraints on key assertions;
  • Insufficient evidence is allowed to fail, rather than forcibly answering.

Week 3: Add Reference Checker

  • Check if the Citation ID exists;
  • Check if each key claim has a reference;
  • Check if the source is expired or overpowered;
  • Rewrite, delete, or downgrade the weakly supported claim.

Week 4: Change the UI and logs

  • Inline Citation is clickable;
  • Source drawer displays quote and locator;
  • original text highlighting;
  • Trace saves Retrieval, Context, Claims, Citations, Checker Result.

Week 5: Do Reference Quality Eval

  • Mark a batch of claims-to-evidence samples;
  • Evaluation of Citation Coverage, Support Rate, Unsupported Claim Rate;
  • Return user feedback and online failure samples.

14. Pre-release self-check list

At least check these questions before going online:

  • Is the evidence of each entry context stable? evidence_id?
  • Can the model only reference the ID in the Evidence Pack?
  • Whether the key assertion is broken into claims?
  • Is there a citation for each key claim?
  • Can Citation resolve to Source, Quote, Locator and version?
  • Is there a checker checking the Citation ID, missing references, weak support, expiration and overpower?
  • Will the Citation Check fail, rewrite, or downgrade instead of showing it as usual?
  • Can the front end click the reference and highlight the original text?
  • Does the audit log hold source hash, model version, prompt version, and checker results?
  • Does eval set contain "what is correct but the reference is wrong", "the answer is wrong but the reference seems to be relevant", "multi-source joint support", "no answer" and "insufficient permission" samples?

If these questions cannot be answered, the reference is still just the format, not the engineering ability.

15. Summary

RAG's reference and attribution is not to add a few footnotes to the answer. It is a complete link from data ingestion, retrieval, context structure, model output, post-check, front-end display to audit logs.

The minimum usable design can be summarized as:

Source -> Evidence Span -> Evidence Pack -> Claim -> Citation -> Support Check -> UI Highlight -> Audit Record

As long as this link is disconnected, the RAG will return to the state of "seemingly sourced, but cannot be verified".

The next article recommends continuing to write: How to do the knowledge base of RAG with Chunking: from the original document to the retrievable unit of evidence.

References