02 / GENERATIVE AI & RETRIEVAL / CASE STUDY
RAGAssist
An evidence-first RAG workbench for document question answering—built to show sources, acknowledge insufficient evidence, and measure where retrieval and generation succeed or fail.
The challenge
A fluent answer is not necessarily a trustworthy answer. When documents are the source of truth, the system must retrieve relevant passages, cite its evidence and avoid claiming certainty where the source is insufficient.
The workbench answers questions over a fictional sample policy or a temporary user-uploaded document collection. It exposes the retrieved passages rather than hiding them behind the final response.
How it works
Document content is ingested, chunked, encoded with Sentence Transformers, indexed in FAISS and ranked for a question. LangChain structures the retrieval-and-generation workflow; a Groq-hosted model produces answers with citation references. FastAPI serves the API and a lightweight web frontend presents the evidence.
Document → chunks → embeddings → FAISS retrieval → LLM answer → source citations + evidence inspection.
Evaluation & decisions
The nine-question retrieval baseline achieved mean Recall@3 = 1.00, with mean reciprocal rank MRR@3 = 0.8148. A larger chunk configuration improved MRR@3 to 0.9444 on the same small question set. The answer audit reported 75% overall correctness but found unnecessary abstentions.
This contrast is important: good retrieval scores did not automatically produce reliable final answers. The documentation reports the failure cases rather than hiding them.
Engineering considerations
The application supports PDF/TXT ingestion, temporary document workspaces, source inspection, API validation, automated tests and a public Cloudflare-hosted frontend. The API uses a free backend that may sleep; the frontend can load before the document engine is warm.
Do not upload sensitive documents. Temporary workspaces can be lost when the free backend restarts; cited text should be checked independently.
Metrics come from a small, curated evaluation set; they are not claims about arbitrary documents or general production performance.