---
library_name: sklearn
tags:
  - spam-detection
  - scikit-learn
  - ensemble
  - tfidf
  - lime
  - shap
  - eli5
  - nlp
  - text-classification
  - explainable-ai
license: mit
pipeline_tag: text-classification
---

## Senior Project Notice

This repository was created for a senior project in ENGT 375 Applied Machine Learning at Old Dominion University. It is provided for educational and research demonstration purposes only. It is not intended for production use, security filtering, or making real-world spam/phishing decisions. Always use established security tools for operational email protection.

# Spam Email Classifier — sklearn Voting Ensemble with XAI (v2)

**ENGT 375 — Applied Machine Learning | Spring 2026 | Old Dominion University**
A voting ensemble classifier (Random Forest + Logistic Regression + Linear SVM with calibration) for spam email detection, with LIME, SHAP, and ELI5 explainability support. This is the **v2** model — a beginner-friendly course-facing rewrite of the original `spam-xai-project`, retrained on the full 99,999-sample corpus. The Gradio workflow was previously merged in from the now-retired `spam-classifier-gradio` project.

## Model Details

- **Architecture:** `VotingClassifier` (soft voting)
  - `RandomForestClassifier(n_estimators=200, class_weight='balanced')`
  - `LogisticRegression(max_iter=1000, class_weight='balanced')`
  - `CalibratedClassifierCV(LinearSVC(class_weight='balanced'))`
- **Features:** 3,000 TF-IDF features (unigrams + bigrams + trigrams) + 24 hand-crafted metadata features
- **Framework:** scikit-learn 1.6+
- **Task:** Binary classification (spam / ham)
- **Threshold:** Optimized for 99% ham precision via precision-recall curve

## Evaluation Results

| Model | Accuracy | Precision | Recall | F1 Score |
|-------|----------|-----------|--------|----------|
| **VotingEnsemble (deployed)** | **0.9740** | **0.9661** | **0.9795** | **0.9727** |
| RandomForest | 0.9775 | 0.9757 | 0.9767 | 0.9762 |
| LogisticRegression | 0.9657 | 0.9552 | 0.9732 | 0.9641 |
| SVM | 0.9689 | 0.9625 | 0.9721 | 0.9673 |

## Training Details

| Parameter | Value |
|-----------|-------|
| Training examples | 69,999 |
| Test examples | 30,000 |
| Total samples | 99,999 (full dataset retrain) |
| Random state | 42 |
| Optimal threshold | 0.3714 |
| Total features | 3,024 (3,000 TF-IDF + 24 metadata) |
| Voting strategy | Soft voting |
| TF-IDF ngram_range | (1, 3) |
| TF-IDF max_features | 3,000 |
| TF-IDF min_df | 2 |
| TF-IDF max_df | 0.90 |
| TF-IDF sublinear_tf | True |
| Class weighting | balanced |

## Metadata Features (24)

`exclamation_density`, `dollar_sign_count`, `caps_word_ratio`, `spam_phrase_count`, `ham_phrase_count`, `net_spam_context`, `url_count`, `html_tag_count`, `email_length`, `avg_sentence_length`, `capitalization_ratio`, `has_specific_date`, `has_specific_time`, `date_reference_count`, `has_unsubscribe`, `has_physical_address`, `has_proper_greeting`, `has_contact_info`, `registration_language_score`, `cta_to_info_ratio`, `shortener_url_ratio`, `legitimate_platform_count`, `gov_edu_url_count`, `question_mark_count`

## Dataset

- [VoltageVagabond/spam-email-dataset](https://huggingface.co/datasets/VoltageVagabond/spam-email-dataset)
- Sources: Kaggle 190K spam dataset (stratified-sampled to 100K) + GitHub email-dataset
- v2 trains on the **full** 99,999-sample corpus (no subsampling), split 69,999 train / 30,000 test

## Files

| File | Purpose |
|------|---------|
| `voting_model.joblib` | Trained VotingClassifier ensemble (~152MB) |
| `tfidf_vectorizer.joblib` | Fitted TF-IDF vectorizer (3,000 features) |
| `meta_scaler.joblib` | MinMaxScaler for the 24 metadata features |
| `feature_names.joblib` | Feature name list (used by LIME/SHAP/ELI5 for labels) |
| `optimal_threshold.joblib` | Calibrated decision threshold (0.3714) |
| `training_sample.joblib` | Sample of training data for LIME/SHAP background |
| `training_report.json` | Training metrics and classification report |

## Usage

```python
import joblib
from scipy.sparse import hstack, csr_matrix
from utils import preprocess_text, compute_metadata_features

model = joblib.load("voting_model.joblib")
tfidf = joblib.load("tfidf_vectorizer.joblib")
scaler = joblib.load("meta_scaler.joblib")
threshold = joblib.load("optimal_threshold.joblib")

email = "Congratulations! You've won a free iPhone!"
text_features = tfidf.transform([preprocess_text(email)])
meta_features = scaler.transform([compute_metadata_features([email])])
features = hstack([text_features, csr_matrix(meta_features)])

proba = model.predict_proba(features)[0][1]
label = "SPAM" if proba >= threshold else "HAM"
print(f"{label} ({proba:.1%} confidence)")
```

## Interactive Demo

- **Live Gradio Space (v2):** [VoltageVagabond/spam-xai-classifier-v2](https://huggingface.co/spaces/VoltageVagabond/spam-xai-classifier-v2)

## Baseline Random Forest (preserved for comparison)

The earlier version of this project used a single calibrated Random Forest with GridSearchCV (no ensemble). Those artifacts are preserved in `models/_baseline_rf/` for comparison and reproducibility:

- **Architecture:** `CalibratedClassifierCV(RandomForestClassifier)` with isotonic regression
- **Features:** 11 metadata features (older shorter list) + 3,000 TF-IDF features
- **Training:** GridSearchCV over `n_estimators=[100,200]`, `max_depth=[20,None]`, 3-fold CV
- **Calibration:** 5-fold isotonic
- **Files in `models/_baseline_rf/`:**
  - `random_forest_spam.joblib` — calibrated model (~872 MB)
  - `random_forest_raw.joblib` — uncalibrated model (~168 MB)
  - `tfidf_vectorizer.joblib`, `meta_scaler.joblib`, `feature_names.joblib`, `optimal_threshold.joblib`, `training_sample.joblib`, `training_config.joblib`

The baseline model is **not** the deployed model — the voting ensemble (top of this card) is. The baseline is here so you can compare ensemble vs. single-model performance, or reproduce the older results.

## Intended Use

This model is an **educational demonstration** of sklearn ensemble methods combined with explainable AI (XAI), created as part of a university course project. It is suitable for:

- Learning how voting ensembles combine multiple classifiers (Random Forest + Logistic Regression + SVM)
- Understanding TF-IDF text vectorization combined with hand-crafted metadata feature engineering
- Exploring LIME, SHAP, and ELI5 explanations for model interpretability
- Comparing ensemble vs. single-model performance using the included baseline
- Following the Kuzlu et al. (2020) feature reduction methodology to evaluate which XAI tool gives the most useful feature rankings

It is **not** intended for production spam filtering.

## Limitations

- **Bag-of-words approach** — TF-IDF cannot distinguish legitimate marketing from spam when the vocabulary overlaps significantly (tested with a real Lenovo Rewards email — misclassified as spam at 78% confidence)
- **Binary classification only** (spam/ham) — no multi-class or severity ranking
- Trained on **English emails only** — not suitable for other languages
- Static vocabulary — cannot adapt to new spam patterns without retraining
- Threshold tuning is dataset-specific and may not generalize to all email distributions
- The voting ensemble model is large (~152 MB) which makes it tight for free HF Spaces

## Related Models

| Model | Description | Link |
|-------|-------------|------|
| spam-classifier-mlx | Qwen 3.5 0.8B MLX LoRA fine-tune | [VoltageVagabond/spam-classifier-mlx](https://huggingface.co/VoltageVagabond/spam-classifier-mlx) |
| spam-classifier-liquid | Liquid AI LFM2.5-1.2B LoRA fine-tune | [VoltageVagabond/spam-classifier-liquid](https://huggingface.co/VoltageVagabond/spam-classifier-liquid) |
| spam-xai-classifier-v2 (Space) | Live Gradio web app for this model | [VoltageVagabond/spam-xai-classifier-v2](https://huggingface.co/spaces/VoltageVagabond/spam-xai-classifier-v2) |

## Citation

```bibtex
@misc{voltagevagabond2026spamxai,
  title={Spam Email Classifier — sklearn Voting Ensemble with XAI (v2)},
  author={VoltageVagabond},
  year={2026},
  howpublished={\url{https://huggingface.co/VoltageVagabond/spam-xai-model-v2}},
  note={ENGT 375 — Applied Machine Learning, Old Dominion University, Spring 2026}
}
```