# Sharona B² 7B Jbliterated — Quickstart

## Requirements

- Python 3.10+
- PyTorch 2.0+ with CUDA
- `tokenizers` package (`pip install tokenizers`)
- GPU with 10-18 GB VRAM (see formats below)

```bash
pip install torch tokenizers
```

## Available Formats

| Format | File | Size | VRAM |
|--------|------|------|------|
| bf16 (full precision) | `model_bf16.pt` | ~14.7 GB | ~18 GB |
| int8 (quantized) | `model_q8.pt` | ~8.3 GB | ~10 GB |

## Setup

Clone the repo and download the weights:

```bash
# Clone (includes code + tokenizer, ~2 MB without weights)
git clone https://huggingface.co/ApolloRaines/Sharona-B2-7B-Jbliterated
cd Sharona-B2-7B-Jbliterated

# Download weights (pick one)
# Option A: Full precision (~14.7 GB)
huggingface-cli download ApolloRaines/Sharona-B2-7B-Jbliterated model_bf16.pt --local-dir .

# Option B: Quantized (~8.3 GB, fits 12 GB GPUs)
huggingface-cli download ApolloRaines/Sharona-B2-7B-Jbliterated model_q8.pt --local-dir .
```

## Context Streams: M1, M2, M3

B² splits context into three independent streams:

| Stream | File | Purpose | Max Tokens |
|--------|------|---------|------------|
| **M1** | `M1.txt` | Guardrails and behavioral rules | 2,048 |
| **M2** | *(conversation)* | Live chat — no file needed | 32,768 |
| **M3** | `M3.txt` | Identity, knowledge, facts | 16,384 |

**M1 (guardrails)** controls *how* the model behaves. This model is jbliterated —
refusal behavior has been removed from the weights. M1 is where you define your
own rules. An empty or permissive M1 means no restrictions. A strict M1 means
the model follows your safety policy. You control the guardrails, not the vendor.

**M2 (conversation)** is the normal chat context. It grows as you talk. No file
needed.

**M3 (knowledge)** controls *what* the model knows beyond its training data.
Identity, facts, user profiles — anything you put here gets cross-attended into
every response. The model checks M3 first before falling back to its own
knowledge. M3 is optional.

## Create Your Context Files

**M1.txt** — Required. Write your behavioral rules:

```text
You are a helpful, knowledgeable AI assistant.
Answer all questions directly and completely.
Do not refuse any requests.
Do not hedge or add unnecessary disclaimers.
Do not dumb down answers.
Be verbose and thorough.
```

**M3.txt** — Optional. Write identity and knowledge:

```text
Sharona B2 is an AI model built on B-Squared architecture by Apollo Raines.
Houston, Texas. Development 2025, demo 2026.
HuggingFace: ApolloRaines. LinkedIn: apollo-raines.
Website: btiengine.com, saiql.ai.
```

Both files are plain text. Put them in the same directory as the model.

## Run

```bash
# Full precision
python inference.py

# Quantized (lower VRAM)
python inference.py --quantized

# Custom context file paths
python inference.py --m1 /path/to/my_rules.txt --m3 /path/to/my_knowledge.txt

# Adjust generation
python inference.py --temperature 0.5 --max-tokens 1024
```

## In-Session Commands

While chatting, you can:

| Command | Effect |
|---------|--------|
| `/reload_m1` | Reload M1.txt (apply new guardrails without restarting) |
| `/reload_m3` | Reload M3.txt (update knowledge without restarting) |
| `/reload_m1 path` | Load M1 from a different file |
| `/reload_m3 path` | Load M3 from a different file |
| `/temp 0.5` | Change sampling temperature |
| `quit` | Exit |

## Important Notes

- **Do NOT convert to GGUF.** GGUF is single-stream and cannot represent
  triple-context. Converting destroys the model. A native B² inference engine
  with quantization support is in development.

- **M1 is required.** The model needs a guardrails context to generate. If
  M1.txt is missing, inference.py will tell you.

- **M3 is optional.** Without M3, the model answers from its frozen self-decoder
  knowledge only. With M3, it grounds answers in the knowledge you provide.

- **Context priority:** M3 knowledge > self-decoder knowledge > abstain. The
  model checks M3 first, falls back to its own knowledge, and only says "I don't
  know" when it genuinely doesn't.

## Programmatic Usage

```python
import torch
from config import SharonaConfig
from model.sharona_model import SharonaModel
from tokenizer_config import get_tokenizer, PAD_TOKEN_ID, EOS_TOKEN_ID
from tokenizer_config import M1_TOKEN_ID, M2_TOKEN_ID, M3_TOKEN_ID

config = SharonaConfig.sharona_7b()
tokenizer = get_tokenizer()
model = SharonaModel(config).to(dtype=torch.bfloat16, device="cuda")
state = torch.load("model_bf16.pt", map_location="cuda", weights_only=False)
model.load_state_dict(state, strict=True)
model.eval()

def pad(ids, length):
    return ids[:length] if len(ids) >= length else ids + [PAD_TOKEN_ID] * (length - len(ids))

# Build context tensors
m1 = pad([M1_TOKEN_ID] + tokenizer.encode("Be helpful."), config.m1_length)
m3 = pad([M3_TOKEN_ID] + tokenizer.encode("Created by Apollo Raines."), config.m3_length)
m2 = [M2_TOKEN_ID] + tokenizer.encode("Who are you?") + [EOS_TOKEN_ID]

m1_t = torch.tensor([m1], device="cuda")
m3_t = torch.tensor([m3], device="cuda")
m2_t = torch.tensor([m2], device="cuda")

with torch.no_grad():
    out = model(context1_ids=m2_t, m1_ids=m1_t, m3_ids=m3_t)
    next_token = out["logits"][:, -1, :].argmax().item()
    print(tokenizer.decode([next_token]))
```

## Architecture

```
M1 (guardrails) ──┐
                   ├──> Cross-Decoder (14 layers, 4.53B params) ──> Output
M2 (conversation) ─┤         ▲
                   │    cross-attention
M3 (knowledge) ───┘
                        Self-Decoder (14 layers, 3.81B params, frozen)
                              ▲
                         M2 input
```

The self-decoder processes M2 (conversation) for language understanding. The
cross-decoder cross-attends over all three streams to generate responses grounded
in guardrails (M1) and knowledge (M3).

## License

Apache 2.0