All projects

RAG

Enterprise RAG Knowledge Assistant

A grounded question-answering assistant over internal policy, product and operations documentation with citation-backed responses.

7

Pipeline stages instrumented

Golden Q/A

Evaluation set

Streaming

Response mode

Architecture

Hover or tap any stage to see its role in the pipeline.

Data flow

Source documents are parsed and normalised, split with structure-aware chunking, embedded and upserted into the vector index with metadata. At query time the question is embedded, candidates are fetched via hybrid search, reranked by a cross-encoder, and the top passages are composed into a grounded prompt. The LLM streams an answer with citations back through the API.

Problem statement

Internal knowledge was scattered across policy PDFs, wiki pages and ticket history. Teams spent significant time searching, and generic LLM answers could not be trusted because they were ungrounded.

Business objective

Deliver accurate, source-cited answers from internal documentation with response quality that could be measured and defended before rollout.

Solution overview

A retrieval pipeline that ingests and normalises documents, embeds semantically coherent chunks into a vector store, retrieves and reranks candidates, then synthesises an answer constrained to retrieved context with inline citations.

AI / ML approach

Hybrid retrieval (dense embeddings + keyword) with a cross-encoder reranker, structure-aware chunking, and a strict grounding prompt that must abstain when retrieved context is insufficient.

Model / LLM used

Instruction-tuned LLM for synthesis, embedding model for retrieval, cross-encoder for reranking

Technology stack

PythonLangChainFAISSFastAPIRedisDockerAWS

Key capabilities

  • Citation-backed answers
  • Abstains when context is insufficient
  • Metadata filtering by document type and access scope
  • Streaming responses
  • Incremental re-indexing

Evaluation approach

  • Golden question set with human-reviewed reference answers
  • Retrieval hit-rate and mean reciprocal rank at k
  • Groundedness and citation-correctness checks on sampled responses
  • Latency budget tracked per pipeline stage

Challenges

  • Naive fixed-size chunking split tables and procedures mid-context
  • Semantically similar but outdated documents surfaced in retrieval
  • Reranking added latency to every request

Solution decisions

  • Moved to structure-aware chunking with heading and table preservation
  • Added recency and document-status metadata filters before reranking
  • Cached embeddings and reranked only the top candidate window

Business impact

Turned unstructured internal documentation into a trustworthy, auditable answering surface where every response can be traced back to its source passage.

More projectsDiscuss a similar system