Project case study
ML - NLP · 2026
Toxic Comment Classification with Fairness-Aware Models
Compared engineered-feature, TF-IDF, and DistilBERT models for multilabel toxic comment classification with fairness-aware evaluation.
Highlights
- Built and evaluated transformer-based toxic comment classifiers using DistilBERT.
- Compared TF-IDF, engineered-feature, and transformer approaches under class imbalance.
- Assessed fairness tradeoffs across identity groups using false positive and false negative rates.
Gallery
What
Online platforms need scalable moderation systems that can detect harmful content while avoiding unfair over-censorship of identity-related language.
This project compares toxic comment classification models across predictive performance, fairness, interpretability, and computational cost.
How
Built a shared preprocessing pipeline for the Jigsaw toxic comment dataset, preserving raw, cleaned, and TF-IDF-ready text representations for different modeling approaches.
Compared engineered-feature models, TF-IDF logistic regression and SVM baselines, and fine-tuned DistilBERT models under a multilabel classification setup.
Results
The shared DistilBERT model achieved the strongest overall performance with AP = 0.681, outperforming TF-IDF logistic regression at AP = 0.576.
Fairness analysis showed a tradeoff: transformer models reduced false positives for identity groups, while TF-IDF models sometimes caught more harmful content but over-flagged identity-related language.
Key Takeaways
Overall accuracy is not enough for moderation systems; subgroup error rates can reveal materially different risks across identity groups.
A hybrid deployment strategy is strongest: use transformers for prediction while retaining interpretable models for auditing, monitoring, and fairness review.