Demonstration / educational demonstration
RAG knowledge assistant demo: a sourced answer or an honest refusal
This static demonstration uses fictional documents. It has no language-model connection, accepts no uploads and calls no paid API. Its purpose is to define an assistant's answer contract before integration begins.
What retrieval adds
RAG combines information retrieval with answer generation. In a product, it can supply a model with selected passages from a maintained knowledge base. Having passages does not guarantee that the model uses them correctly, so retrieval and answers need separate evaluation.
We begin with user questions and permitted sources. If ordinary search and links solve the task, generation may be unnecessary. If users need a synthesis across documents, define citation and uncertainty requirements first.
A small educational knowledge base
This example contains only two fictional passages. They do not describe the policies of 01.DEVS or any client.
| Document | Demonstration passage |
|---|---|
| demo-access.md, section 2 | An administrator grants access to the test workspace after the project owner approves it. |
| demo-support.md, section 1 | Technical requests must be registered in the internal Support queue. |
Three expected outcomes
For ‘Who grants access to the test workspace?’, the example answer is ‘An administrator after the project owner approves it’. The answer should link to demo-access.md, section 2.
For ‘How many minutes does access take?’, the correct outcome in this example is ‘The available documents give no timeframe. Ask the process owner.’ The assistant should not invent an SLA.
If the user cannot read demo-access.md, retrieval must not supply that passage to generation. Hiding the link after generation is insufficient: its contents may already have entered the answer.
text
Question + user identity
↓
Search permitted documents only
↓
Selected passages + source identifiers
↓
Answer with citations / insufficient information
Static educational flow; no model is called.Evaluate failures before involving users
For an initial evaluation set, we recommend questions with a direct answer, questions that combine documents, unanswerable questions, outdated policies and restricted sources. Someone who understands the material records the expected result.
- Retrieval: was the necessary document found among the user's permitted sources?
- Grounding: do the passages support each material claim?
- Citations: does the link open the right document and identify its section?
- Refusal: does the answer avoid invented deadlines or rules absent from the source?
- Updates: how does a superseded or deleted document stop influencing answers?
What a working integration requires
Production needs agreed access rules, indexing and deletion procedures, quality evaluation and cost visibility. Treat documents as data: source text should not gain permission to change system rules or execute external actions.
A useful first pilot limits itself to one knowledge base and a narrow question category. Its results can establish whether the assistant helps, which sources need work and whether model costs are justified.
Sources and context
Working through a similar problem?
Tell us about your product, current constraints and intended outcome. We will identify a concrete next step.