OLAMIDE OLAITAN
PROJECT / 02 / GENERATIVE AI & RETRIEVAL
CONTACT ↗
BACK TO SELECTED WORK

02 / GENERATIVE AI & RETRIEVAL / CASE STUDY

RAGAssist

An evidence-first RAG workbench for document question answering—built to show sources, acknowledge insufficient evidence, and measure where retrieval and generation succeed or fail.

RAGFAISSLangChainFastAPIEvaluation
PROJECT IN FOCUSDOCUMENTED APPROACH / VERIFIED REPOSITORY
01 / RECALL@31.00Nine-question retrieval baseline
02 / BEST MRR@30.9444Evaluated chunking variant
03 / OVERALL ANSWER CORRECTNESS75%Curated answer audit

The challenge

A fluent answer is not necessarily a trustworthy answer. When documents are the source of truth, the system must retrieve relevant passages, cite its evidence and avoid claiming certainty where the source is insufficient.

The workbench answers questions over a fictional sample policy or a temporary user-uploaded document collection. It exposes the retrieved passages rather than hiding them behind the final response.

How it works

Document content is ingested, chunked, encoded with Sentence Transformers, indexed in FAISS and ranked for a question. LangChain structures the retrieval-and-generation workflow; a Groq-hosted model produces answers with citation references. FastAPI serves the API and a lightweight web frontend presents the evidence.

Document → chunks → embeddings → FAISS retrieval → LLM answer → source citations + evidence inspection.

Evaluation & decisions

The nine-question retrieval baseline achieved mean Recall@3 = 1.00, with mean reciprocal rank MRR@3 = 0.8148. A larger chunk configuration improved MRR@3 to 0.9444 on the same small question set. The answer audit reported 75% overall correctness but found unnecessary abstentions.

This contrast is important: good retrieval scores did not automatically produce reliable final answers. The documentation reports the failure cases rather than hiding them.

Engineering considerations

The application supports PDF/TXT ingestion, temporary document workspaces, source inspection, API validation, automated tests and a public Cloudflare-hosted frontend. The API uses a free backend that may sleep; the frontend can load before the document engine is warm.

Do not upload sensitive documents. Temporary workspaces can be lost when the free backend restarts; cited text should be checked independently.

Metrics come from a small, curated evaluation set; they are not claims about arbitrary documents or general production performance.

NEXT CASE STUDY / CONTINUE EXPLORING

ModelOps

↗