Before I had any professional software title, I built the tool I actually needed: something that triages my inbox before I open it. Ai-Email-summariser connects to Gmail over OAuth2 (or any IMAP server as a fallback), classifies each email as important or junk, labels them accordingly, and uses GPT-3.5-turbo to write a short summary of the ones I open.
The design decision I keep getting asked about in interviews is why the classifier is TF-IDF plus Random Forest instead of an LLM. The honest answer has three parts, and the third one is the interesting one.
The LLM is the expensive part, so it only does one job
Summarisation genuinely needs a language model. Classification does not. An API call to decide "is this junk" costs money, adds latency, and fails at a different level than local code fails. The classifier runs locally with scikit-learn, so triage is free, instant, and works offline. GPT-3.5-turbo only gets called when I actually open an email to read it. Splitting the two jobs meant one API call per email I read instead of one per email that exists.
The classifier design, stated plainly
The vectoriser is TF-IDF with a 5000-feature cap, English stop words removed, and unigrams plus bigrams. The classifier is a Random Forest with 100 estimators and class_weight='balanced', which matters because my real inbox is not 50/50: the junk-heavy reality would otherwise let a model score well by predicting "junk" constantly.
The input features are deliberately crude: subject, sender, and body concatenated into one string. No fancy sender-reputation graph, no header analysis. I wanted the first version to be auditable, and with TF-IDF I can print which words pushed a classification either way. That turned out to matter for trust: when the model flags something important that I thought was junk, being able to see why makes me fix my assumptions instead of distrusting the tool.
The cold-start problem is a UX problem
An untrained classifier is useless, and a tool that demands training data before it works is a tool nobody sets up. So the first prediction on any fresh install returns a neutral 0.5/0.5 with the label defaulting to important, and the interface shows that confidence number next to every email. You are never asked to trust the model. You are asked to correct it.
Every correction (i1 marks email one important, j2 marks two junk) goes into a JSON feedback file that deduplicates by a composite id built from the subject, sender, and a hash of the body preview, so marking the same email twice updates the label instead of duplicating the training row. Each retrain happens on demand from that file, milliseconds, on my machine. The model persists to a joblib file and reloads on the next run.
This is the part I would defend hardest in a design review: the feedback loop is the product. The model itself is the boring part, a textbook scikit-learn pipeline, and that is fine. What makes the tool usable by week two is that it becomes mine by week two, trained on what I personally consider junk.
The tests
Eight tests in unittest with mocked Gmail responses: prediction structure, the train-and-persist path, feedback deduplication (adding the same email twice updates one row and flips the label), the Gmail integration with a mocked service, and three edge cases that taught me the most. An empty email, a 10,000-character body, and a body full of special characters including multi-script text all have to return a valid prediction structure rather than an exception, because the alternative is the classifier crashing on the one weird email that matters most.
The duplicate-feedback test is the one I would keep if I could keep only one. It encodes the assumption that feedback should correct, not accumulate, which is exactly the bug that would silently poison the training data if someone refactored the feedback manager later.
What I would change
Two things, both known. First, the neutral fallback on prediction failure is silent; it should log so I can see how often the vectoriser is being fed garbage. Second, there is no evaluation set at all, because the feedback file doubles as training data. The moment the feedback pool gets large enough, I want a held-out split so accuracy is measured instead of felt.
The stack is Python with scikit-learn, the Gmail API with IMAP fallback, OpenAI for summaries only, and unittest with mocks. Nothing exotic, and that is the point: this is what a first production-shaped ML system looks like when the goal is a tool you run every day rather than a benchmark score.
Full repo: https://github.com/Reaver1000/Ai-Email-summariser
