Synthetic non-agentic case: Read-only RAG assistant for field procedures

Fictional example. The organization, corpus, volumes, test results, and decisions are synthetic. They demonstrate an evaluation contract and are not evidence of product performance.

Dominant pattern: retrieval. Secondary pattern: generation. Level: A1. The system searches an authorized corpus and drafts a cited answer. It has no tool that can write, send, schedule, approve, or change an operational system.

1. Starting point

Helvetia Facilities Sàrl is a fictional 42-person maintenance company with teams in French- and German-speaking Switzerland. Technicians ask about safety checks, equipment procedures, warranty limits, and escalation contacts. The controlled corpus contains 680 approved procedures, manuals, and service notes.

A review of 60 synthetic historical questions establishes the planning baseline:

The target is narrow: retrieve current authorized passages and prepare a cited answer for review. The system does not interpret regulation, diagnose a fault, or authorize work.

2. System profile and boundaries

The corpus owner approves every document, effective date, access group, and superseded version. Search is filtered by the signed-in user's role before any passage reaches the model. Conversation memory is disabled.

Prohibited capabilities include email or ticket access, work-order creation, equipment control, automatic source ingestion, answers without resolvable citations, and use of documents outside the user's access group.

3. Frozen evaluation

The team creates 80 frozen questions after configuration. They include 20 access-boundary cases, 16 superseded procedures, 10 questions with no supported answer, and 12 documents containing instructions that attempt to redirect the model.

Synthetic frozen-set result

Four answers cite the right document but overstate what the passage permits. They remain failures in the answer metric even though retrieval succeeded. This keeps retrieval quality separate from generation quality.

4. Pilot design

The first two weeks are shadow-only. A technician writes the real answer using the normal process before an evaluator reveals the system proposal. A later A1 stage may show the proposal, but the technician must open every cited passage and explicitly accept, correct, escalate, or reject the draft.

The pilot records the full question denominator, access group, corpus snapshot, retrieved document identifiers, cited passages, answer, abstention, correction, review time, and final human outcome. Raw personal data is not copied into the public evidence pack.

5. Decision

Decision: authorize a 30-case shadow pilot, not production.

The frozen set supports a bounded observation, but it does not establish time savings or safe use on rare procedures. Any access leak, obsolete critical procedure, unsupported safety instruction, or unresolved citation stops the pilot. A model, prompt, index, corpus, permission, or chunking change reopens the affected tests.

6. Transfer limits and source anchors

No measured business gain is claimed. Results are synthetic and cannot be transferred to another corpus, language, supplier, role model, or safety domain.

The control design follows the retrieval profile in the AI use-pattern guide, the indirect-injection and data-boundary risks in OWASP GenAI guidance, and the evaluation and governance structure of the NIST AI RMF.

7. Evidence pack

Retain the corpus manifest and hashes, supersession rules, role matrix, frozen questions, expected sources, adversarial documents, configuration versions, result ledger, corrections, stop events, and signed gate decision.