OLAMIDE OLAITAN
PROJECT / 01 / NLP & MODEL EVALUATION
CONTACT ↗
BACK TO SELECTED WORK

01 / NLP & MODEL EVALUATION / CASE STUDY

SupportSense

An end-to-end banking intent classifier. A measured comparison between classical and neural approaches, with a deployment decision based on evidence and engineering trade-offs.

ClassificationNLPModel selectionGradioSQL
PROJECT IN FOCUSDOCUMENTED APPROACH / VERIFIED REPOSITORY
01 / OFFICIAL TEST ACCURACY89.32%Selected Linear SVM
02 / MACRO F10.8934Official test split
03 / INTENT CLASSES77BANKING77 dataset

The challenge

Support teams receive short, varied descriptions of card, transfer and payment problems. The engineering question is whether an automated intent model can categorise these messages reliably enough to demonstrate a support-routing workflow—not to claim that the system solves a banking issue.

The solution is a text classification workflow with a reproducible evaluation and a Gradio inference interface.

How it works

Training data is split for model development, and the official test partition is held aside for final comparison. The application uses a saved text-processing and classification pipeline, avoiding retraining on every request.

Data preparation → TF-IDF feature extraction → Linear SVM → intent label → Gradio prediction UI.

Evaluation & decisions

Three families were implemented: TF-IDF with Linear SVM, a PyTorch embedding classifier and a TensorFlow/Keras embedding classifier. The selected SVM achieved 89.32% test accuracy and approximately 0.8934 macro F1, outperforming the neural alternatives in this experiment.

PyTorch test accuracy: 83.31%. TensorFlow/Keras test accuracy: 87.86%. Model selection prioritised observed performance, simpler inference and maintainability.

Engineering considerations

The project includes reusable inference code, SQL / PostgreSQL integration, automated pytest tests, and a public Gradio demo. The Linear SVM produces decision scores, not calibrated probabilities, and can misclassify unfamiliar or ambiguous customer requests.

No real banking records are required for the demo. The deployed service runs on a free host and can take time to start.

Results are from the documented BANKING77 evaluation, not live-production accuracy. This is not a banking advice or decision-making system.

NEXT CASE STUDY / CONTINUE EXPLORING

RAGAssist

↗