---
library_name: sklearn
tags:
  - spam-detection
  - scikit-learn
  - ensemble
  - tfidf
  - lime
  - shap
  - eli5
  - nlp
  - text-classification
  - explainable-ai
license: mit
pipeline_tag: text-classification
---

# Spam Email Classifier — sklearn Voting Ensemble with XAI (v2)

**ENGT 375 — Applied Machine Learning | Spring 2026 | Old Dominion University**

> **Disclaimer:** This model was created as a student project for Applied Machine Learning at ODU. It is intended for **educational and research purposes only** and should not be used as a sole spam/phishing filter in production. Classification accuracy may vary, and the model may produce incorrect or misleading results. Always use established email security tools for real-world spam filtering.

A voting ensemble classifier (Random Forest + Logistic Regression + Linear SVM with calibration) for spam email detection, with LIME, SHAP, and ELI5 explainability support. This is the **v2** model — a beginner-friendly course-facing rewrite of the original `spam-xai-project`, retrained on the full 99,999-sample corpus. The Gradio workflow was previously merged in from the now-retired `spam-classifier-gradio` project.

## Model Details

- **Architecture:** `VotingClassifier` (soft voting)
  - `RandomForestClassifier(n_estimators=200, class_weight='balanced')`
  - `LogisticRegression(max_iter=1000, class_weight='balanced')`
  - `CalibratedClassifierCV(LinearSVC(class_weight='balanced'))`
- **Features:** 3,000 TF-IDF features (unigrams + bigrams + trigrams) + 24 hand-crafted metadata features
- **Framework:** scikit-learn 1.6+
- **Task:** Binary classification (spam / ham)
- **Threshold:** Optimized for 99% ham precision via precision-recall curve

## Evaluation Results

| Model | Accuracy | Precision | Recall | F1 Score |
|-------|----------|-----------|--------|----------|
| **VotingEnsemble (deployed)** | **0.9740** | **0.9661** | **0.9795** | **0.9727** |
| RandomForest | 0.9775 | 0.9757 | 0.9767 | 0.9762 |
| LogisticRegression | 0.9657 | 0.9552 | 0.9732 | 0.9641 |
| SVM | 0.9689 | 0.9625 | 0.9721 | 0.9673 |

## Training Details

| Parameter | Value |
|-----------|-------|
| Training examples | 69,999 |
| Test examples | 30,000 |
| Total samples | 99,999 (full dataset retrain) |
| Random state | 42 |
| Optimal threshold | 0.3714 |
| Total features | 3,024 (3,000 TF-IDF + 24 metadata) |
| Voting strategy | Soft voting |
| TF-IDF ngram_range | (1, 3) |
| TF-IDF max_features | 3,000 |
| TF-IDF min_df | 2 |
| TF-IDF max_df | 0.90 |
| TF-IDF sublinear_tf | True |
| Class weighting | balanced |

## Metadata Features (24)

`exclamation_density`, `dollar_sign_count`, `caps_word_ratio`, `spam_phrase_count`, `ham_phrase_count`, `net_spam_context`, `url_count`, `html_tag_count`, `email_length`, `avg_sentence_length`, `capitalization_ratio`, `has_specific_date`, `has_specific_time`, `date_reference_count`, `has_unsubscribe`, `has_physical_address`, `has_proper_greeting`, `has_contact_info`, `registration_language_score`, `cta_to_info_ratio`, `shortener_url_ratio`, `legitimate_platform_count`, `gov_edu_url_count`, `question_mark_count`

## Dataset

- [VoltageVagabond/spam-email-dataset](https://huggingface.co/datasets/VoltageVagabond/spam-email-dataset)
- Sources: Kaggle 190K spam dataset (stratified-sampled to 100K) + GitHub email-dataset
- v2 trains on the **full** 99,999-sample corpus (no subsampling), split 69,999 train / 30,000 test

## Files

| File | Purpose |
|------|---------|
| `voting_model.joblib` | Trained VotingClassifier ensemble (~152MB) |
| `tfidf_vectorizer.joblib` | Fitted TF-IDF vectorizer (3,000 features) |
| `meta_scaler.joblib` | MinMaxScaler for the 24 metadata features |
| `feature_names.joblib` | Feature name list (used by LIME/SHAP/ELI5 for labels) |
| `optimal_threshold.joblib` | Calibrated decision threshold (0.3714) |
| `training_sample.joblib` | Sample of training data for LIME/SHAP background |
| `training_report.json` | Training metrics and classification report |

## Usage

```python
import joblib
from scipy.sparse import hstack, csr_matrix
from utils import preprocess_text, compute_metadata_features

model = joblib.load("voting_model.joblib")
tfidf = joblib.load("tfidf_vectorizer.joblib")
scaler = joblib.load("meta_scaler.joblib")
threshold = joblib.load("optimal_threshold.joblib")

email = "Congratulations! You've won a free iPhone!"
text_features = tfidf.transform([preprocess_text(email)])
meta_features = scaler.transform([compute_metadata_features([email])])
features = hstack([text_features, csr_matrix(meta_features)])

proba = model.predict_proba(features)[0][1]
label = "SPAM" if proba >= threshold else "HAM"
print(f"{label} ({proba:.1%} confidence)")
```

## Interactive Demo

- **Live Gradio Space (v2):** [VoltageVagabond/spam-xai-classifier-v2](https://huggingface.co/spaces/VoltageVagabond/spam-xai-classifier-v2)

## Baseline Random Forest (preserved for comparison)

The earlier version of this project used a single calibrated Random Forest with GridSearchCV (no ensemble). Those artifacts are preserved in `models/_baseline_rf/` for comparison and reproducibility:

- **Architecture:** `CalibratedClassifierCV(RandomForestClassifier)` with isotonic regression
- **Features:** 11 metadata features (older shorter list) + 3,000 TF-IDF features
- **Training:** GridSearchCV over `n_estimators=[100,200]`, `max_depth=[20,None]`, 3-fold CV
- **Calibration:** 5-fold isotonic
- **Files in `models/_baseline_rf/`:**
  - `random_forest_spam.joblib` — calibrated model (~872 MB)
  - `random_forest_raw.joblib` — uncalibrated model (~168 MB)
  - `tfidf_vectorizer.joblib`, `meta_scaler.joblib`, `feature_names.joblib`, `optimal_threshold.joblib`, `training_sample.joblib`, `training_config.joblib`

The baseline model is **not** the deployed model — the voting ensemble (top of this card) is. The baseline is here so you can compare ensemble vs. single-model performance, or reproduce the older results.

## Intended Use

This model is an **educational demonstration** of sklearn ensemble methods combined with explainable AI (XAI), created as part of a university course project. It is suitable for:

- Learning how voting ensembles combine multiple classifiers (Random Forest + Logistic Regression + SVM)
- Understanding TF-IDF text vectorization combined with hand-crafted metadata feature engineering
- Exploring LIME, SHAP, and ELI5 explanations for model interpretability
- Comparing ensemble vs. single-model performance using the included baseline
- Following the Kuzlu et al. (2020) feature reduction methodology to evaluate which XAI tool gives the most useful feature rankings

It is **not** intended for production spam filtering.

## Limitations

- **Bag-of-words approach** — TF-IDF cannot distinguish legitimate marketing from spam when the vocabulary overlaps significantly (tested with a real Lenovo Rewards email — misclassified as spam at 78% confidence)
- **Binary classification only** (spam/ham) — no multi-class or severity ranking
- Trained on **English emails only** — not suitable for other languages
- Static vocabulary — cannot adapt to new spam patterns without retraining
- Threshold tuning is dataset-specific and may not generalize to all email distributions
- The voting ensemble model is large (~152 MB) which makes it tight for free HF Spaces

## Related Models

| Model | Description | Link |
|-------|-------------|------|
| spam-classifier-mlx | Qwen 3.5 0.8B MLX LoRA fine-tune | [VoltageVagabond/spam-classifier-mlx](https://huggingface.co/VoltageVagabond/spam-classifier-mlx) |
| spam-classifier-liquid | Liquid AI LFM2.5-1.2B LoRA fine-tune | [VoltageVagabond/spam-classifier-liquid](https://huggingface.co/VoltageVagabond/spam-classifier-liquid) |
| spam-xai-classifier-v2 (Space) | Live Gradio web app for this model | [VoltageVagabond/spam-xai-classifier-v2](https://huggingface.co/spaces/VoltageVagabond/spam-xai-classifier-v2) |

## Citation

```bibtex
@misc{voltagevagabond2026spamxai,
  title={Spam Email Classifier — sklearn Voting Ensemble with XAI (v2)},
  author={VoltageVagabond},
  year={2026},
  howpublished={\url{https://huggingface.co/VoltageVagabond/spam-xai-model-v2}},
  note={ENGT 375 — Applied Machine Learning, Old Dominion University, Spring 2026}
}
```