README.md

---
license: other
language:
- en
library_name: transformers
tags:
- sclm
- stateful
- memory
- earcp
- text-generation
- conversational
pipeline_tag: text-generation
base_model: mistralai/Mistral-7B-v0.1
widget:
- text: The wizard Elara lived in Silverwood forest. One day, she discovered
  example_title: Fantasy Story
- text: In the year 2050, humanity had finally achieved
  example_title: Science Fiction
- text: The detective examined the crime scene carefully. The clues pointed to
  example_title: Mystery
inference:
  parameters:
    max_new_tokens: 100
    temperature: 0.7
    top_p: 0.9
    repetition_penalty: 1.1
---

[![PyPI version](https://badge.fury.io/py/saclm.svg)](https://badge.fury.io/py/saclm)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)

# 🧠 SCLM: Stateful Coherent Language Model

**SCLM** adds **persistent latent memory** to transformer language models, enabling better coherence across long conversations and multi-turn generation.

## 🎯 Key Features

- **Persistent State**: Memory that evolves across conversation turns
- **Entity Coherence**: Maintains context about characters, places, and objects
- **Edit Mode**: Make local changes without affecting global memory
- **Lightweight**: Only 91.7M additional parameters (2.44% overhead)

## 📊 Architecture: EARCP

```
EARCP = Encapsulation + Alignment + Revision + Coherence + Propagation
```

| Component | Function |
|-----------|----------|
| **Encapsulation** | GRU-style state update from hidden states |
| **Alignment** | Cross-attention between state and hidden layers |
| **Revision** | Drift detection and correction |
| **Coherence** | Mixture-of-Experts for consistency |
| **Propagation** | State injection into transformer layers |

## 🔧 Model Details

| Parameter | Value |
|-----------|-------|
| Base Model | mistralai/Mistral-7B-v0.1 |
| EARCP Parameters | 91.7M |
| Latent State Dim | 256 |
| Injection Layers | [8, 16] |
| Alpha (injection strength) | 0.02 |
| Experts | 2 |

## 🚀 Quick Start

```python
# Note: Full SCLM requires custom loading (see below)
# The inference widget uses the base model only

from transformers import AutoTokenizer
import torch

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("amewebstudio/ananke-sclm")

# For full SCLM functionality, load weights separately:
# 1. Load base Mistral-7B
# 2. Load EARCP weights from earcp_weights.pt
# 3. Apply SCLM wrapper
```

## 📈 Validation Results

| Test | Result |
|------|--------|
| Forward Pass | ✅ |
| State Evolution | ✅ (norm: 0 → 4.6 → 7.5) |
| Coherent Generation | ✅ |
| Edit Mode | ✅ |
| Entity Memory | ✅ (Elara, Nimbus retained) |

## 💡 Use Cases

- **Interactive Fiction**: Characters and plot points remain consistent
- **Long Conversations**: Context persists without growing prompts
- **Creative Writing**: Maintain story coherence across chapters
- **Role-Playing**: NPCs remember past interactions

## 📝 Citation

```bibtex
@article{amega2025sclm,
  title={SCLM: Stateful Coherent Language Models with EARCP Architecture},
  author={Amega, Mike},
  year={2025},
  note={Ame Web Studio}
}
```

## 👤 Author

**Mike Amega** - [Ame Web Studio](https://github.com/Volgat)

Business Source License 1.1 (BSL-1.1) - See LICENSE for details.

Copyright (c) 2025 (Ame Web Studio). All Rights Reserved.

👤 Author
Mike Amega - Ame Web Studio
📧 amewebstudio35@gmail.com

---

*SCLM is an experimental architecture exploring persistent memory in language models.*