Almost everyone building an AI support tool reaches for the same architecture: retrieval-augmented generation. Embed your documents, search them at query time, stuff the results into the prompt, let the model write the answer. The demo works in an afternoon.
Then real customers arrive, and the gaps show up fast. Here's what separates a RAG demo from a RAG product.
Grounding beats fluency
A model will happily write a confident, well-structured answer to a question your knowledge base never addressed. That's the most dangerous failure mode in support — a wrong answer that sounds right.
We treat retrieved content as data, not instructions, and we tell the model so explicitly. When retrieval comes back empty or weak, the agent says it doesn't know and offers a handoff instead of inventing an answer.
Keep the index fresh
Knowledge goes stale. A price changes, a policy updates, a product ships. If your index reflects last quarter, your agent is confidently out of date.
- A nightly re-crawl re-ingests sources and re-embeds what changed.
- Every chunk is watermarked, so re-ingestion is idempotent — no duplicates, no drift.
- When retrieval repeatedly finds nothing for a question, we log a knowledge gap so you can see exactly what content is missing.
Chunking is a product decision
Embedding quality matters less than people think; chunk boundaries matter more. Split too coarsely and retrieval drags in irrelevant text. Split too finely and you lose the context that makes an answer correct.
We chunk on semantic boundaries, keep a little overlap, and store enough metadata to reassemble context at answer time.
The honest answer is a feature
The single biggest trust win in customer support isn't a clever answer — it's a reliable "I'll get a teammate for that." An agent that knows the edge of its knowledge is an agent customers can trust inside it.