Update README.md

README.md

@@ -1,332 +1,332 @@
----
-license: mit
-language:
-  - en
-tags:
-  - text-classification
-  - ai-text-detection
-  - deberta-v3
-  - binary-classification
-  - nlp
-datasets:
-  - liamdugan/raid
-  - artem9k/ai-text-detection-pile
-  - gsingh1-py/train
-  - cc_news
-  - blog_authorship_corpus
-  - webis/tldr-17
-  - ChristophSchuhmann/essays-with-instructions
-  - HuggingFaceH4/stack-exchange-preferences
-  - pile-of-law/pile-of-law
-metrics:
-  - accuracy
-  - f1
-  - precision
-  - recall
-  - roc_auc
-pipeline_tag: text-classification
-model-index:
-  - name: GLYPH
-    results:
-      - task:
-          type: text-classification
-          name: AI-Generated Text Detection
-        metrics:
-          - name: Accuracy
-            type: accuracy
-            value: 0.9885
-          - name: F1
-            type: f1
-            value: 0.9901
-          - name: Precision
-            type: precision
-            value: 0.9851
-          - name: Recall
-            type: recall
-            value: 0.9952
-          - name: ROC-AUC
-            type: roc_auc
-            value: 0.9990
-          - name: MCC
-            type: mcc
-            value: 0.9765
----
-
-# GLYPH — High-Accuracy AI Text Detector
-
-GLYPH is a binary text classifier built on [DeBERTa-v3-base](https://huggingface.co/microsoft/deberta-v3-base) that distinguishes human-written text from AI-generated text. It achieves **98.85% accuracy**, **0.999 ROC-AUC**, and **0.990 F1** on a held-out test set spanning 10 human writing domains and 14 AI model families — from GPT-2 (1.5B) through GPT-4 (~1T).
-
-The model was trained on ~50K texts covering academic papers, news articles, blog posts, Reddit discussions, legal filings, Wikipedia, student essays, and technical Q&A on the human side, and outputs from 24 distinct AI model configurations across 10 model families on the AI side. It produces well-separated, high-confidence predictions (mean confidence 0.976) and remains accurate even at the strictest decision thresholds.
-
-## Key Results
-
-| Metric | Value |
-|---|---|
-| **Accuracy** | 98.85% |
-| **F1 Score** | 0.9901 |
-| **Precision** | 98.51% |
-| **Recall** | 99.52% |
-| **ROC-AUC** | 0.9990 |
-| **Average Precision** | 0.9993 |
-| **MCC** | 0.9765 |
-| **Human Accuracy** | 97.94% |
-| **AI Accuracy** | 99.52% |
-| **Mean Confidence** | 0.976 |
-| **F1 @ 0.95 threshold** | 0.987 |
-
-All metrics evaluated on a held-out test set of 5,050 texts (2,136 human / 2,914 AI) with no overlap in source texts, split hashes, or temporal leakage with the training set.
-
-## Per-Source Performance
-
-### Human Text Sources
-
-| Source | Domain | n | Accuracy | Confidence |
-|---|---|---|---|---|
-| PubMed Abstracts | Biomedical research | 300 | **100.0%** | 0.988 |
-| Blog / Opinion | Personal blogs | 200 | **100.0%** | 0.987 |
-| Reddit Writing | Informal / social | 300 | **100.0%** | 0.985 |
-| Wikipedia | Encyclopedic | 500 | **99.8%** | 0.987 |
-| CC-News | Journalism | 392 | **99.5%** | 0.981 |
-| arXiv Abstracts | Academic / scientific | 444 | **90.8%** | 0.948 |
-
-arXiv abstracts are the hardest category — highly formulaic academic prose with structural similarity to AI output. Even so, detection accuracy is 90.8% with 94.8% mean confidence, and the remaining errors are concentrated in a small subset of unusually short or template-heavy abstracts.
-
-### AI Model Families
-
-| Model | Family | Params | n | Accuracy | F1 |
-|---|---|---|---|---|---|
-| GPT-3.5-Turbo | OpenAI | 175B | 223 | **100.0%** | 1.000 |
-| GPT-4 | OpenAI | ~1T | 215 | **100.0%** | 1.000 |
-| Llama-2-70B-Chat | Meta | 70B | 191 | **100.0%** | 1.000 |
-| MPT-30B | MosaicML | 30B | 211 | **100.0%** | 1.000 |
-| MPT-30B-Chat | MosaicML | 30B | 191 | **100.0%** | 1.000 |
-| Mistral-7B-Instruct-v0.1 | Mistral AI | 7B | 194 | **100.0%** | 1.000 |
-| Mistral-7B-v0.1 | Mistral AI | 7B | 203 | **100.0%** | 1.000 |
-| Llama-3.1-8B-Instruct | Meta | 8B | 238 | **99.6%** | 0.998 |
-| Phi-3.5-Mini-Instruct | Microsoft | 3.8B | 238 | **99.6%** | 0.998 |
-| Command-Chat | Cohere | 52B | 198 | **99.5%** | 0.997 |
-| Text-Davinci-002 | OpenAI | 175B | 176 | **99.4%** | 0.997 |
-| Llama-3.2-3B-Instruct | Meta | 3B | 238 | **99.2%** | 0.996 |
-| GPT-2-XL | OpenAI | 1.5B | 198 | **98.5%** | 0.992 |
-| Cohere Command | Cohere | 52B | 200 | **97.5%** | 0.987 |
-
-Detection is robust across four generations of language models (GPT-2 through GPT-4), three access paradigms (open-weight, API-only, and proprietary), and parameter counts spanning three orders of magnitude (1.5B to ~1T).
-
-### Performance by Text Length
-
-| Length Bucket | n | Accuracy | F1 |
-|---|---|---|---|
-| Very Long (>2000 words) | 103 | **100.0%** | 1.000 |
-| Long (500–2000 words) | 862 | **99.9%** | 0.999 |
-| Short (50–150 words) | 1,976 | **98.5%** | 0.989 |
-| Medium (150–500 words) | 1,634 | **98.8%** | 0.989 |
-| Very Short (<50 words) | 475 | **98.1%** | 0.899 |
-
-Performance degrades gracefully with shorter inputs. Even on texts under 50 words — where the model has minimal signal — accuracy remains above 98%.
-
-### Threshold Sensitivity
-
-The model produces well-calibrated, high-confidence outputs. Performance holds across aggressive decision thresholds:
-
-| P(AI) Threshold | F1 | Precision |
-|---|---|---|
-| 0.50 (default) | 0.990 | 0.985 |
-| 0.60 | 0.991 | 0.987 |
-| 0.70 | 0.992 | 0.990 |
-| 0.80 | 0.992 | 0.992 |
-| 0.90 | 0.991 | 0.993 |
-| 0.95 | 0.987 | 0.996 |
-
-At a 0.95 threshold, precision reaches 99.6% with only a 0.3% drop in F1 — suitable for high-stakes applications where false accusations of AI usage carry serious consequences.
-
-## Architecture
-
-| Component | Details |
-|---|---|
-| Base model | `microsoft/deberta-v3-base` (184M parameters) |
-| Architecture | DeBERTa-v3 with disentangled attention and enhanced mask decoder |
-| Task head | Linear classifier (768 → 2) with 0.15 dropout |
-| Tokenizer | SentencePiece (slow tokenizer, `use_fast=False`) |
-| Max sequence length | 512 tokens |
-| Output | `[P(human), P(AI)]` softmax probabilities |
-
-DeBERTa-v3 was chosen over RoBERTa and BERT alternatives due to its disentangled attention mechanism, which separately encodes content and position. This is particularly relevant for AI text detection: language models have characteristic positional dependencies in how they distribute tokens across a sequence, and disentangled attention gives the classifier direct access to these patterns.
-
-## Training
-
-### Configuration
-
-| Parameter | Value |
-|---|---|
-| Trainable parameters | 184,423,682 (100% — all layers unfrozen) |
-| Optimizer | AdamW (weight decay 0.01) |
-| Learning rate | 2e-5 (cosine schedule) |
-| Warmup | 10% of total steps |
-| Effective batch size | 64 (16 × 4 gradient accumulation) |
-| Precision | bf16 mixed precision |
-| Gradient checkpointing | Enabled (non-reentrant) |
-| Label smoothing | 0.05 |
-| Class weights | human=1.182, ai=0.867 |
-| Epochs | 8 (early-stopped at 3.17) |
-| Best checkpoint | Epoch 1.19 (by validation F1) |
-| Training time | ~49 minutes on RTX 4070 Ti 12GB |
-| Final train loss | 0.186 |
-| Final eval loss | 0.150 |
-
-### Why Fully Unfrozen?
-
-Initial experiments with 4 frozen encoder layers (standard practice from PAN-CLEF 2025 literature) yielded only 80% accuracy with severe human-side bias — the model classified 44% of human texts as AI. Freezing 4 of 12 layers in DeBERTa-base locks 33% of the network, far more aggressive than the 21% reported for DeBERTa-large. Unfreezing all layers with cosine LR decay and 10% warmup resolved the bias entirely, lifting human accuracy from 55.6% to 97.9% without sacrificing AI detection (97.4% → 99.5%).
-
-### Dataset Composition
-
-**Total: 50,458 texts** (40,364 train / 5,044 validation / 5,050 test)
-
-Stratified by source with hash-based deduplication to prevent data leakage.
-
-#### Human Sources (10 domains, ~29K target)
-
-| Domain | Source | Target Count | Text Type |
-|---|---|---|---|
-| Academic (STEM) | arXiv API | 5,000 | Abstracts across 8 categories (cs.CL, cs.AI, cs.LG, physics, math, q-bio, econ, stat) |
-| Academic (Medical) | PubMed API | 3,000 | Biomedical research abstracts |
-| Encyclopedic | Wikipedia API | 5,000 | Article sections across 10 topic categories |
-| Journalism | CC-News (HuggingFace) | 4,000 | News articles |
-| Literary / Creative | Project Gutenberg | 2,000 | Public domain book excerpts |
-| Informal / Social | Reddit (webis/tldr-17) | 3,000 | Writing-focused subreddit posts |
-| Student / Educational | PERSUADE corpus | 2,000 | Student essays |
-| Technical / Q&A | StackExchange | 2,000 | Technical answers |
-| Blog / Opinion | Blog Authorship Corpus | 2,000 | Personal blog posts |
-| Legal / Formal | Pile of Law | 1,000 | Legal opinions and case summaries |
-
-#### AI Sources (24 model configurations across 10 families)
-
-**Locally generated via LM Studio (8 models, Q4_K_M quantization):**
-
-| Model | Family | Parameters |
-|---|---|---|
-| Llama-3.1-8B-Instruct | Meta Llama | 8B |
-| Llama-3.2-3B-Instruct | Meta Llama | 3B |
-| Mistral-7B-Instruct-v0.3 | Mistral AI | 7B |
-| Qwen2.5-7B-Instruct | Alibaba Qwen | 7B |
-| Qwen2.5-14B-Instruct | Alibaba Qwen | 14B |
-| Gemma-2-9B-Instruct | Google | 9B |
-| Phi-3.5-Mini-Instruct | Microsoft | 3.8B |
-| DeepSeek-V2-Lite-Chat | DeepSeek | 16B (MoE) |
-
-Local generation used 4 temperature/sampling configurations (default, creative, precise, varied) across 6 prompt strategies (direct, continue, rewrite, expand, style_mimic, question_answer) with a system prompt enforcing natural human-like output — no markdown, no meta-commentary, no self-referential AI language.
-
-**HuggingFace datasets (16 additional model families):**
-
-| Dataset | Models Added | Reference |
-|---|---|---|
-| RAID (ACL 2024) | ChatGPT-3.5, GPT-4, GPT-3-Davinci, Cohere Command, Llama-2-70B-Chat, Mistral-7B-v0.1, Mixtral-8x7B, MPT-30B, GPT-2-XL | [liamdugan/raid](https://huggingface.co/datasets/liamdugan/raid) |
-| AI Text Detection Pile | GPT-2/3/J/ChatGPT (mixed) | [artem9k/ai-text-detection-pile](https://huggingface.co/datasets/artem9k/ai-text-detection-pile) |
-| NYT Multi-Model | GPT-4o, Yi-Large, Qwen-2-72B, Llama-3-8B, Gemma-2-9B, Mistral-7B | [gsingh1-py/train](https://huggingface.co/datasets/gsingh1-py/train) |
-
-This combination ensures coverage of proprietary API models (GPT-3.5, GPT-4, GPT-4o, Cohere), large open models exceeding consumer GPU VRAM (Llama-2-70B, Qwen-2-72B, Mixtral-8x7B, Yi-Large), older architectures (GPT-2, GPT-3, GPT-J), and mixture-of-experts models (Mixtral, DeepSeek-V2-Lite). RAID data was filtered to non-adversarial generations only (`attack=="none"`) for training data quality.
-
-## Usage
-
-### With Transformers
-
-```python
-from transformers import AutoTokenizer, AutoModelForSequenceClassification
-import torch
-
-model_name = "ogmatrixai/glyph"  # Replace with your repo path
-tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)
-model = AutoModelForSequenceClassification.from_pretrained(model_name)
-model.eval()
-
-text = "Your text to classify here..."
-
-inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
-with torch.no_grad():
-    logits = model(**inputs).logits
-    probs = torch.softmax(logits, dim=-1)
-    
-p_human, p_ai = probs[0].tolist()
-label = "AI-generated" if p_ai > 0.5 else "Human-written"
-confidence = max(p_human, p_ai)
-
-print(f"{label} (confidence: {confidence:.1%})")
-```
-
-### With Pipeline
-
-```python
-from transformers import pipeline
-
-detector = pipeline(
-    "text-classification",
-    model="ogmatrixai/glyph",  # Replace with your repo path
-    tokenizer=AutoTokenizer.from_pretrained("ogmatrixai/glyph", use_fast=False),
-)
-
-result = detector("Your text here...")
-print(result)
-# [{'label': 'LABEL_1', 'score': 0.98}]  # LABEL_0 = human, LABEL_1 = AI
-```
-
-### Important Notes
-
-- **Tokenizer**: Always use `use_fast=False`. The fast tokenizer for DeBERTa-v3 has a confirmed regression in `transformers>=4.47` ([#42583](https://github.com/huggingface/transformers/issues/42583)) that crashes on load.
-- **Max length**: The model was trained with `max_length=512`. Longer texts should be truncated or chunked with predictions aggregated.
-- **Labels**: `LABEL_0` = human, `LABEL_1` = AI-generated.
-
-## Limitations and Ethical Considerations
-
-### Known Limitations
-
-1. **English only.** GLYPH was trained exclusively on English text. Performance on other languages is untested and likely degraded.
-
-2. **Training distribution.** The model has seen outputs from 24 specific AI model configurations. Novel architectures, heavily fine-tuned models, or future model families may evade detection. AI text detection is fundamentally adversarial — no static detector provides permanent robustness.
-
-3. **arXiv abstracts remain the hardest domain** at 90.8% accuracy. Highly formulaic academic writing with rigid structural conventions shares surface features with AI-generated text. Users in academic integrity contexts should treat borderline predictions on scientific abstracts with appropriate caution.
-
-4. **Short texts (<50 words)** have reduced F1 (0.899) despite high accuracy (98.1%). With minimal token-level signal, the model occasionally produces confident but incorrect predictions. For short-form content, consider requiring higher confidence thresholds.
-
-5. **Adversarial attacks.** The training data includes only non-adversarial AI outputs. Paraphrasing attacks, homoglyph substitution, targeted prompt engineering, and watermark-removal techniques were not included. Dedicated adversarial robustness (e.g., RAID adversarial subsets) is a planned enhancement.
-
-6. **Mixed authorship.** GLYPH classifies at the document level. It does not detect partial AI usage (e.g., AI-written paragraphs embedded in a human-written essay). Sentence-level or span-level detection requires a different approach.
-
-7. **512-token window.** Texts are truncated at 512 tokens. For long documents, this means classification is based on the opening ~350–400 words only. Sliding-window aggregation is recommended for long-form content.
-
-### Ethical Considerations
-
-AI text detection carries real consequences — academic penalties, professional reputation damage, content moderation decisions. False positives (human text classified as AI) are particularly harmful. While GLYPH's false positive rate is low (2.06% on the test set, 44 out of 2,136 human texts), no detector achieves zero false positives.
-
-**Recommendations for responsible deployment:**
-
-- Never use GLYPH as the sole basis for punitive action. Use it as one signal among many (metadata, behavioral patterns, stylometric analysis).
-- Apply a high confidence threshold (≥0.95) for consequential decisions. At this threshold, precision reaches 99.6%.
-- Provide users with the confidence score, not just a binary label. A text scored at P(AI)=0.52 is fundamentally different from one scored at P(AI)=0.99.
-- Maintain an appeals process. Statistical classifiers will always produce errors.
-- Acknowledge the base rate problem. In populations where AI usage is rare, even a 2% FPR produces many false accusations relative to true detections.
-
-## Training Infrastructure
-
-| Component | Specification |
-|---|---|
-| GPU | NVIDIA GeForce RTX 4070 Ti (12GB VRAM) |
-| CPU | Intel Core i7-14700K (20 cores) |
-| RAM | 48GB DDR5 |
-| Framework | PyTorch 2.6+ / HuggingFace Transformers |
-| Precision | bf16 mixed precision |
-| Total training time | 49 minutes |
-| Experiment tracking | Weights & Biases |
-
-## Citation
-
-```bibtex
-@misc{glyph2026,
-  title={GLYPH: High-Accuracy AI Text Detection with DeBERTa-v3},
-  author={OGMatrixAI},
-  year={2026},
-  url={https://huggingface.co/ogmatrixai/glyph}
-}
-```
-
-## Acknowledgments
-
-Training data incorporates the [RAID benchmark](https://huggingface.co/datasets/liamdugan/raid) (Dugan et al., ACL 2024), the [AI Text Detection Pile](https://huggingface.co/datasets/artem9k/ai-text-detection-pile), and the [NYT Multi-Model dataset](https://huggingface.co/datasets/gsingh1-py/train). Human text sources include arXiv, PubMed, Wikipedia, CC-News, Project Gutenberg, Reddit, StackExchange, Blog Authorship Corpus, PERSUADE, and Pile of Law. The base model is [DeBERTa-v3-base](https://huggingface.co/microsoft/deberta-v3-base) by Microsoft Research.
+---
+license: mit
+language:
+  - en
+tags:
+  - text-classification
+  - ai-text-detection
+  - deberta-v3
+  - binary-classification
+  - nlp
+datasets:
+  - liamdugan/raid
+  - artem9k/ai-text-detection-pile
+  - gsingh1-py/train
+  - cc_news
+  - blog_authorship_corpus
+  - webis/tldr-17
+  - ChristophSchuhmann/essays-with-instructions
+  - HuggingFaceH4/stack-exchange-preferences
+  - pile-of-law/pile-of-law
+metrics:
+  - accuracy
+  - f1
+  - precision
+  - recall
+  - roc_auc
+pipeline_tag: text-classification
+model-index:
+  - name: GLYPH
+    results:
+      - task:
+          type: text-classification
+          name: AI-Generated Text Detection
+        metrics:
+          - name: Accuracy
+            type: accuracy
+            value: 0.9885
+          - name: F1
+            type: f1
+            value: 0.9901
+          - name: Precision
+            type: precision
+            value: 0.9851
+          - name: Recall
+            type: recall
+            value: 0.9952
+          - name: ROC-AUC
+            type: roc_auc
+            value: 0.9990
+          - name: MCC
+            type: mcc
+            value: 0.9765
+---
+
+# GLYPH — High-Accuracy AI Text Detector
+
+GLYPH is a binary text classifier built on [DeBERTa-v3-base](https://huggingface.co/microsoft/deberta-v3-base) that distinguishes human-written text from AI-generated text. It achieves **98.85% accuracy**, **0.999 ROC-AUC**, and **0.990 F1** on a held-out test set spanning 10 human writing domains and 14 AI model families — from GPT-2 (1.5B) through GPT-4 (~1T).
+
+The model was trained on ~50K texts covering academic papers, news articles, blog posts, Reddit discussions, legal filings, Wikipedia, student essays, and technical Q&A on the human side, and outputs from 24 distinct AI model configurations across 10 model families on the AI side. It produces well-separated, high-confidence predictions (mean confidence 0.976) and remains accurate even at the strictest decision thresholds.
+
+## Key Results
+
+| Metric | Value |
+|---|---|
+| **Accuracy** | 98.85% |
+| **F1 Score** | 0.9901 |
+| **Precision** | 98.51% |
+| **Recall** | 99.52% |
+| **ROC-AUC** | 0.9990 |
+| **Average Precision** | 0.9993 |
+| **MCC** | 0.9765 |
+| **Human Accuracy** | 97.94% |
+| **AI Accuracy** | 99.52% |
+| **Mean Confidence** | 0.976 |
+| **F1 @ 0.95 threshold** | 0.987 |
+
+All metrics evaluated on a held-out test set of 5,050 texts (2,136 human / 2,914 AI) with no overlap in source texts, split hashes, or temporal leakage with the training set.
+
+## Per-Source Performance
+
+### Human Text Sources
+
+| Source | Domain | n | Accuracy | Confidence |
+|---|---|---|---|---|
+| PubMed Abstracts | Biomedical research | 300 | **100.0%** | 0.988 |
+| Blog / Opinion | Personal blogs | 200 | **100.0%** | 0.987 |
+| Reddit Writing | Informal / social | 300 | **100.0%** | 0.985 |
+| Wikipedia | Encyclopedic | 500 | **99.8%** | 0.987 |
+| CC-News | Journalism | 392 | **99.5%** | 0.981 |
+| arXiv Abstracts | Academic / scientific | 444 | **90.8%** | 0.948 |
+
+arXiv abstracts are the hardest category — highly formulaic academic prose with structural similarity to AI output. Even so, detection accuracy is 90.8% with 94.8% mean confidence, and the remaining errors are concentrated in a small subset of unusually short or template-heavy abstracts.
+
+### AI Model Families
+
+| Model | Family | Params | n | Accuracy | F1 |
+|---|---|---|---|---|---|
+| GPT-3.5-Turbo | OpenAI | 175B | 223 | **100.0%** | 1.000 |
+| GPT-4 | OpenAI | ~1T | 215 | **100.0%** | 1.000 |
+| Llama-2-70B-Chat | Meta | 70B | 191 | **100.0%** | 1.000 |
+| MPT-30B | MosaicML | 30B | 211 | **100.0%** | 1.000 |
+| MPT-30B-Chat | MosaicML | 30B | 191 | **100.0%** | 1.000 |
+| Mistral-7B-Instruct-v0.1 | Mistral AI | 7B | 194 | **100.0%** | 1.000 |
+| Mistral-7B-v0.1 | Mistral AI | 7B | 203 | **100.0%** | 1.000 |
+| Llama-3.1-8B-Instruct | Meta | 8B | 238 | **99.6%** | 0.998 |
+| Phi-3.5-Mini-Instruct | Microsoft | 3.8B | 238 | **99.6%** | 0.998 |
+| Command-Chat | Cohere | 52B | 198 | **99.5%** | 0.997 |
+| Text-Davinci-002 | OpenAI | 175B | 176 | **99.4%** | 0.997 |
+| Llama-3.2-3B-Instruct | Meta | 3B | 238 | **99.2%** | 0.996 |
+| GPT-2-XL | OpenAI | 1.5B | 198 | **98.5%** | 0.992 |
+| Cohere Command | Cohere | 52B | 200 | **97.5%** | 0.987 |
+
+Detection is robust across four generations of language models (GPT-2 through GPT-4), three access paradigms (open-weight, API-only, and proprietary), and parameter counts spanning three orders of magnitude (1.5B to ~1T).
+
+### Performance by Text Length
+
+| Length Bucket | n | Accuracy | F1 |
+|---|---|---|---|
+| Very Long (>2000 words) | 103 | **100.0%** | 1.000 |
+| Long (500–2000 words) | 862 | **99.9%** | 0.999 |
+| Short (50–150 words) | 1,976 | **98.5%** | 0.989 |
+| Medium (150–500 words) | 1,634 | **98.8%** | 0.989 |
+| Very Short (<50 words) | 475 | **98.1%** | 0.899 |
+
+Performance degrades gracefully with shorter inputs. Even on texts under 50 words — where the model has minimal signal — accuracy remains above 98%.
+
+### Threshold Sensitivity
+
+The model produces well-calibrated, high-confidence outputs. Performance holds across aggressive decision thresholds:
+
+| P(AI) Threshold | F1 | Precision |
+|---|---|---|
+| 0.50 (default) | 0.990 | 0.985 |
+| 0.60 | 0.991 | 0.987 |
+| 0.70 | 0.992 | 0.990 |
+| 0.80 | 0.992 | 0.992 |
+| 0.90 | 0.991 | 0.993 |
+| 0.95 | 0.987 | 0.996 |
+
+At a 0.95 threshold, precision reaches 99.6% with only a 0.3% drop in F1 — suitable for high-stakes applications where false accusations of AI usage carry serious consequences.
+
+## Architecture
+
+| Component | Details |
+|---|---|
+| Base model | `microsoft/deberta-v3-base` (184M parameters) |
+| Architecture | DeBERTa-v3 with disentangled attention and enhanced mask decoder |
+| Task head | Linear classifier (768 → 2) with 0.15 dropout |
+| Tokenizer | SentencePiece (slow tokenizer, `use_fast=False`) |
+| Max sequence length | 512 tokens |
+| Output | `[P(human), P(AI)]` softmax probabilities |
+
+DeBERTa-v3 was chosen over RoBERTa and BERT alternatives due to its disentangled attention mechanism, which separately encodes content and position. This is particularly relevant for AI text detection: language models have characteristic positional dependencies in how they distribute tokens across a sequence, and disentangled attention gives the classifier direct access to these patterns.
+
+## Training
+
+### Configuration
+
+| Parameter | Value |
+|---|---|
+| Trainable parameters | 184,423,682 (100% — all layers unfrozen) |
+| Optimizer | AdamW (weight decay 0.01) |
+| Learning rate | 2e-5 (cosine schedule) |
+| Warmup | 10% of total steps |
+| Effective batch size | 64 (16 × 4 gradient accumulation) |
+| Precision | bf16 mixed precision |
+| Gradient checkpointing | Enabled (non-reentrant) |
+| Label smoothing | 0.05 |
+| Class weights | human=1.182, ai=0.867 |
+| Epochs | 8 (early-stopped at 3.17) |
+| Best checkpoint | Epoch 1.19 (by validation F1) |
+| Training time | ~49 minutes on RTX 4070 Ti 12GB |
+| Final train loss | 0.186 |
+| Final eval loss | 0.150 |
+
+### Why Fully Unfrozen?
+
+Initial experiments with 4 frozen encoder layers (standard practice from PAN-CLEF 2025 literature) yielded only 80% accuracy with severe human-side bias — the model classified 44% of human texts as AI. Freezing 4 of 12 layers in DeBERTa-base locks 33% of the network, far more aggressive than the 21% reported for DeBERTa-large. Unfreezing all layers with cosine LR decay and 10% warmup resolved the bias entirely, lifting human accuracy from 55.6% to 97.9% without sacrificing AI detection (97.4% → 99.5%).
+
+### Dataset Composition
+
+**Total: 50,458 texts** (40,364 train / 5,044 validation / 5,050 test)
+
+Stratified by source with hash-based deduplication to prevent data leakage.
+
+#### Human Sources (10 domains, ~29K target)
+
+| Domain | Source | Target Count | Text Type |
+|---|---|---|---|
+| Academic (STEM) | arXiv API | 5,000 | Abstracts across 8 categories (cs.CL, cs.AI, cs.LG, physics, math, q-bio, econ, stat) |
+| Academic (Medical) | PubMed API | 3,000 | Biomedical research abstracts |
+| Encyclopedic | Wikipedia API | 5,000 | Article sections across 10 topic categories |
+| Journalism | CC-News (HuggingFace) | 4,000 | News articles |
+| Literary / Creative | Project Gutenberg | 2,000 | Public domain book excerpts |
+| Informal / Social | Reddit (webis/tldr-17) | 3,000 | Writing-focused subreddit posts |
+| Student / Educational | PERSUADE corpus | 2,000 | Student essays |
+| Technical / Q&A | StackExchange | 2,000 | Technical answers |
+| Blog / Opinion | Blog Authorship Corpus | 2,000 | Personal blog posts |
+| Legal / Formal | Pile of Law | 1,000 | Legal opinions and case summaries |
+
+#### AI Sources (24 model configurations across 10 families)
+
+**Locally generated via LM Studio (8 models, Q4_K_M quantization):**
+
+| Model | Family | Parameters |
+|---|---|---|
+| Llama-3.1-8B-Instruct | Meta Llama | 8B |
+| Llama-3.2-3B-Instruct | Meta Llama | 3B |
+| Mistral-7B-Instruct-v0.3 | Mistral AI | 7B |
+| Qwen2.5-7B-Instruct | Alibaba Qwen | 7B |
+| Qwen2.5-14B-Instruct | Alibaba Qwen | 14B |
+| Gemma-2-9B-Instruct | Google | 9B |
+| Phi-3.5-Mini-Instruct | Microsoft | 3.8B |
+| DeepSeek-V2-Lite-Chat | DeepSeek | 16B (MoE) |
+
+Local generation used 4 temperature/sampling configurations (default, creative, precise, varied) across 6 prompt strategies (direct, continue, rewrite, expand, style_mimic, question_answer) with a system prompt enforcing natural human-like output — no markdown, no meta-commentary, no self-referential AI language.
+
+**HuggingFace datasets (16 additional model families):**
+
+| Dataset | Models Added | Reference |
+|---|---|---|
+| RAID (ACL 2024) | ChatGPT-3.5, GPT-4, GPT-3-Davinci, Cohere Command, Llama-2-70B-Chat, Mistral-7B-v0.1, Mixtral-8x7B, MPT-30B, GPT-2-XL | [liamdugan/raid](https://huggingface.co/datasets/liamdugan/raid) |
+| AI Text Detection Pile | GPT-2/3/J/ChatGPT (mixed) | [artem9k/ai-text-detection-pile](https://huggingface.co/datasets/artem9k/ai-text-detection-pile) |
+| NYT Multi-Model | GPT-4o, Yi-Large, Qwen-2-72B, Llama-3-8B, Gemma-2-9B, Mistral-7B | [gsingh1-py/train](https://huggingface.co/datasets/gsingh1-py/train) |
+
+This combination ensures coverage of proprietary API models (GPT-3.5, GPT-4, GPT-4o, Cohere), large open models exceeding consumer GPU VRAM (Llama-2-70B, Qwen-2-72B, Mixtral-8x7B, Yi-Large), older architectures (GPT-2, GPT-3, GPT-J), and mixture-of-experts models (Mixtral, DeepSeek-V2-Lite). RAID data was filtered to non-adversarial generations only (`attack=="none"`) for training data quality.
+
+## Usage
+
+### With Transformers
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+model_name = "ogmatrixllm/glyph"  # Replace with your repo path
+tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)
+model = AutoModelForSequenceClassification.from_pretrained(model_name)
+model.eval()
+
+text = "Your text to classify here..."
+
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
+with torch.no_grad():
+    logits = model(**inputs).logits
+    probs = torch.softmax(logits, dim=-1)
+    
+p_human, p_ai = probs[0].tolist()
+label = "AI-generated" if p_ai > 0.5 else "Human-written"
+confidence = max(p_human, p_ai)
+
+print(f"{label} (confidence: {confidence:.1%})")
+```
+
+### With Pipeline
+
+```python
+from transformers import pipeline
+
+detector = pipeline(
+    "text-classification",
+    model="ogmatrixai/glyph",  # Replace with your repo path
+    tokenizer=AutoTokenizer.from_pretrained("ogmatrixai/glyph", use_fast=False),
+)
+
+result = detector("Your text here...")
+print(result)
+# [{'label': 'LABEL_1', 'score': 0.98}]  # LABEL_0 = human, LABEL_1 = AI
+```
+
+### Important Notes
+
+- **Tokenizer**: Always use `use_fast=False`. The fast tokenizer for DeBERTa-v3 has a confirmed regression in `transformers>=4.47` ([#42583](https://github.com/huggingface/transformers/issues/42583)) that crashes on load.
+- **Max length**: The model was trained with `max_length=512`. Longer texts should be truncated or chunked with predictions aggregated.
+- **Labels**: `LABEL_0` = human, `LABEL_1` = AI-generated.
+
+## Limitations and Ethical Considerations
+
+### Known Limitations
+
+1. **English only.** GLYPH was trained exclusively on English text. Performance on other languages is untested and likely degraded.
+
+2. **Training distribution.** The model has seen outputs from 24 specific AI model configurations. Novel architectures, heavily fine-tuned models, or future model families may evade detection. AI text detection is fundamentally adversarial — no static detector provides permanent robustness.
+
+3. **arXiv abstracts remain the hardest domain** at 90.8% accuracy. Highly formulaic academic writing with rigid structural conventions shares surface features with AI-generated text. Users in academic integrity contexts should treat borderline predictions on scientific abstracts with appropriate caution.
+
+4. **Short texts (<50 words)** have reduced F1 (0.899) despite high accuracy (98.1%). With minimal token-level signal, the model occasionally produces confident but incorrect predictions. For short-form content, consider requiring higher confidence thresholds.
+
+5. **Adversarial attacks.** The training data includes only non-adversarial AI outputs. Paraphrasing attacks, homoglyph substitution, targeted prompt engineering, and watermark-removal techniques were not included. Dedicated adversarial robustness (e.g., RAID adversarial subsets) is a planned enhancement.
+
+6. **Mixed authorship.** GLYPH classifies at the document level. It does not detect partial AI usage (e.g., AI-written paragraphs embedded in a human-written essay). Sentence-level or span-level detection requires a different approach.
+
+7. **512-token window.** Texts are truncated at 512 tokens. For long documents, this means classification is based on the opening ~350–400 words only. Sliding-window aggregation is recommended for long-form content.
+
+### Ethical Considerations
+
+AI text detection carries real consequences — academic penalties, professional reputation damage, content moderation decisions. False positives (human text classified as AI) are particularly harmful. While GLYPH's false positive rate is low (2.06% on the test set, 44 out of 2,136 human texts), no detector achieves zero false positives.
+
+**Recommendations for responsible deployment:**
+
+- Never use GLYPH as the sole basis for punitive action. Use it as one signal among many (metadata, behavioral patterns, stylometric analysis).
+- Apply a high confidence threshold (≥0.95) for consequential decisions. At this threshold, precision reaches 99.6%.
+- Provide users with the confidence score, not just a binary label. A text scored at P(AI)=0.52 is fundamentally different from one scored at P(AI)=0.99.
+- Maintain an appeals process. Statistical classifiers will always produce errors.
+- Acknowledge the base rate problem. In populations where AI usage is rare, even a 2% FPR produces many false accusations relative to true detections.
+
+## Training Infrastructure
+
+| Component | Specification |
+|---|---|
+| GPU | NVIDIA GeForce RTX 4070 Ti (12GB VRAM) |
+| CPU | Intel Core i7-14700K (20 cores) |
+| RAM | 48GB DDR5 |
+| Framework | PyTorch 2.6+ / HuggingFace Transformers |
+| Precision | bf16 mixed precision |
+| Total training time | 49 minutes |
+| Experiment tracking | Weights & Biases |
+
+## Citation
+
+```bibtex
+@misc{glyph2026,
+  title={GLYPH: High-Accuracy AI Text Detection with DeBERTa-v3},
+  author={OGMatrix},
+  year={2026},
+  url={https://huggingface.co/ogmatrixllm/glyph}
+}
+```
+
+## Acknowledgments
+
+Training data incorporates the [RAID benchmark](https://huggingface.co/datasets/liamdugan/raid) (Dugan et al., ACL 2024), the [AI Text Detection Pile](https://huggingface.co/datasets/artem9k/ai-text-detection-pile), and the [NYT Multi-Model dataset](https://huggingface.co/datasets/gsingh1-py/train). Human text sources include arXiv, PubMed, Wikipedia, CC-News, Project Gutenberg, Reddit, StackExchange, Blog Authorship Corpus, PERSUADE, and Pile of Law. The base model is [DeBERTa-v3-base](https://huggingface.co/microsoft/deberta-v3-base) by Microsoft Research.