01 / NLP & MODEL EVALUATION / CASE STUDY
SupportSense
An end-to-end banking intent classifier. A measured comparison between classical and neural approaches, with a deployment decision based on evidence and engineering trade-offs.
The challenge
Support teams receive short, varied descriptions of card, transfer and payment problems. The engineering question is whether an automated intent model can categorise these messages reliably enough to demonstrate a support-routing workflow—not to claim that the system solves a banking issue.
The solution is a text classification workflow with a reproducible evaluation and a Gradio inference interface.
How it works
Training data is split for model development, and the official test partition is held aside for final comparison. The application uses a saved text-processing and classification pipeline, avoiding retraining on every request.
Data preparation → TF-IDF feature extraction → Linear SVM → intent label → Gradio prediction UI.
Evaluation & decisions
Three families were implemented: TF-IDF with Linear SVM, a PyTorch embedding classifier and a TensorFlow/Keras embedding classifier. The selected SVM achieved 89.32% test accuracy and approximately 0.8934 macro F1, outperforming the neural alternatives in this experiment.
PyTorch test accuracy: 83.31%. TensorFlow/Keras test accuracy: 87.86%. Model selection prioritised observed performance, simpler inference and maintainability.
Engineering considerations
The project includes reusable inference code, SQL / PostgreSQL integration, automated pytest tests, and a public Gradio demo. The Linear SVM produces decision scores, not calibrated probabilities, and can misclassify unfamiliar or ambiguous customer requests.
No real banking records are required for the demo. The deployed service runs on a free host and can take time to start.
Results are from the documented BANKING77 evaluation, not live-production accuracy. This is not a banking advice or decision-making system.