docs/M01-identity.md

M01 — Identity & Manifests

Spec version: v1.0 Depends on: X04 (config), PyNaCl, cryptography (for TLS cert generation) Depended on by: X02 (events), X01 (transport), M02 (discovery), M03 (bus), and every service that signs anything

---

1. Responsibility

• Cryptographic identity (Ed25519 keypair) for one device
• Signing and verification of arbitrary canonical-JSON payloads
• Creation, signing, verification of node manifests
• Creation, signing, verification of community manifests
• Canonical JSON encoding (the one that signatures are computed over)
• TLS certificate generation (self-signed, pinned to the device key on first contact)
• Optional: capability tokens (Phase 2)

This module owns no event log — that is X02. It owns no networking — that is X01. It owns only crypto and document formats.

---

2. File layout

hearthnet/identity/
├── __init__.py         # re-exports
├── keys.py             # KeyPair, canonical_json, sign, verify, TLS cert
├── manifest.py         # NodeManifest, CommunityManifest, builders, verifiers
└── tokens.py           # Phase 2; stubs only in MVP

---

3. Public API

3.1 keys.py

# hearthnet/identity/keys.py
from dataclasses import dataclass
from pathlib import Path
from typing import Any

NodeID    = str  # "ed25519:XXXX-XXXX-XXXX-XXXX" (short) or "ed25519:<base64-url-nopad>" (full)
Signature = str  # "ed25519:<base64-url-nopad>"

@dataclass(frozen=True)
class KeyPair:
    """Wraps Ed25519 signing key + verify key plus their string forms."""
    signing_key:    nacl.signing.SigningKey
    verify_key:     nacl.signing.VerifyKey
    node_id_full:   str          # "ed25519:<base64-url-nopad of 32 bytes>"
    node_id_short:  str          # "ed25519:XXXX-XXXX-XXXX-XXXX" (8 bytes, base32, 4-grouped)

    def sign(self, payload: dict) -> dict:
        """Return a new dict equal to payload with signature field added."""

    def sign_bytes(self, data: bytes) -> Signature:
        """Sign raw bytes; return 'ed25519:<base64-url-nopad>'."""

# --- generation / loading ---

def generate() -> KeyPair:
    """Create a fresh Ed25519 keypair (uses os.urandom)."""

def load(keys_dir: Path) -> KeyPair:
    """Load device.ed25519 + device.pub from keys_dir.
       Raises IdentityError('keys_missing') if not present,
       IdentityError('keys_invalid') if malformed.
       Validates permissions are 0600 on Unix."""

def load_or_generate(keys_dir: Path) -> KeyPair:
    """Load if present; otherwise generate and persist."""

def save(kp: KeyPair, keys_dir: Path) -> None:
    """Persist signing key as device.ed25519 (0600) and verify key as device.pub."""

# --- public-key handling ---

def short_node_id(verify_key_bytes: bytes) -> str:
    """First 8 bytes base32, grouped: 'ed25519:XXXX-XXXX-XXXX-XXXX'."""

def full_node_id(verify_key_bytes: bytes) -> str:
    """All 32 bytes, base64-url no pad, prefixed: 'ed25519:<b64>'."""

def parse_node_id(node_id: str) -> bytes:
    """Decode the full form back to 32 bytes. Short form raises ValueError."""

def verify_key_from_full(node_id_full: str) -> nacl.signing.VerifyKey:
    """Convenience: parse and wrap."""

# --- canonical JSON ---

def canonical_json(obj: Any) -> bytes:
    """Sorted keys, no whitespace, no trailing zeros on numbers, UTF-8.
       Used everywhere a signature is computed."""

# --- signing / verification ---

def sign_payload(payload: dict, kp: KeyPair) -> dict:
    """Add a 'signature' field over canonical_json(payload \\ {signature})."""

def verify_payload(payload: dict, vk: nacl.signing.VerifyKey) -> bool:
    """True iff the 'signature' field validates over canonical_json(payload \\ {signature})."""

def verify_payload_with_node_id(payload: dict, expected_node_id_full: str) -> bool:
    """Convenience: derive verify key from node_id and call verify_payload."""

# --- TLS ---

def generate_self_signed_cert(kp: KeyPair, host: str = "0.0.0.0") -> tuple[bytes, bytes]:
    """Generate an X.509 self-signed cert+key for TLS. The CN includes the short node id.
       Returns (cert_pem, key_pem). The cert is rotated when the device key changes (i.e. never)."""

class IdentityError(Exception):
    """code in {'keys_missing','keys_invalid','keys_permissions','bad_node_id','sign_failed','verify_failed'}"""
    code: str

3.2 manifest.py

# hearthnet/identity/manifest.py
from dataclasses import dataclass, field
from typing import Any
from datetime import datetime, timezone, timedelta

# ---- types ----

@dataclass(frozen=True)
class Endpoint:
    transport: str                  # "https"
    host:      str
    port:      int

@dataclass(frozen=True)
class HardwareSpec:
    gpu:           str | None
    vram_gb:       float
    ram_gb:        float
    cpu_cores:     int
    disk_free_gb:  float

@dataclass(frozen=True)
class CapabilitySpec:
    """As it appears INSIDE a node manifest (subset of the full descriptor in M03)."""
    name:             str
    version:          str           # "1.0"
    stability:        str           # "stable" | "beta" | "experimental"
    schema_hash:      str           # "blake3:..."
    params:           dict[str, Any]
    max_concurrent:   int

@dataclass(frozen=True)
class NodeManifest:
    version:           int          # always 1 for this contract version
    contract_version:  str          # "1.0"
    node_id:           str          # full form
    display_name:      str
    community_id:      str
    profile:           str          # "anchor" | "hearth" | "spark" | "bridge"
    endpoints:         list[Endpoint]
    hardware:          HardwareSpec
    capabilities:      list[CapabilitySpec]
    uptime_seconds:    int
    load:              dict[str, Any]
    issued_at:         str          # RFC 3339
    expires_at:        str          # RFC 3339
    signature:         str

    def as_dict(self) -> dict: ...
    def is_expired(self, now: datetime | None = None) -> bool: ...

# ---- node manifest builder & verifier ----

def build_node_manifest(
    kp: KeyPair,
    community_id: str,
    display_name: str,
    profile: str,
    endpoints: list[Endpoint],
    hardware: HardwareSpec,
    capabilities: list[CapabilitySpec],
    uptime_seconds: int,
    load: dict[str, Any],
) -> NodeManifest:
    """Build and sign a fresh node manifest. issued_at = now,
       expires_at = now + MANIFEST_TTL_SECONDS."""

def parse_node_manifest(blob: bytes | dict) -> NodeManifest:
    """Parse JSON bytes or dict into a typed NodeManifest. Does NOT verify signature."""

def verify_node_manifest(manifest: NodeManifest, *, now: datetime | None = None) -> None:
    """Verify signature + expiry. Raises IdentityError on failure:
       'invalid_signature' | 'expired' | 'bad_manifest' (malformed structure)."""

# ---- community manifest ----

@dataclass(frozen=True)
class CommunityPolicy:
    min_signatures_to_invite:   int
    min_signatures_to_demote:   int
    min_signatures_to_revoke:   int
    capability_token_ttl_seconds: int
    federation_enabled:         bool
    default_member_can_invite:  bool

@dataclass(frozen=True)
class CommunityMember:
    node_id:   str
    level:     str         # "anchor" | "trusted" | "member"
    added_at:  str
    added_by:  str

@dataclass(frozen=True)
class RevokedEntry:
    node_id:    str
    revoked_at: str

@dataclass(frozen=True)
class CommunityManifest:
    version:               int
    community_id:          str
    name:                  str
    root_key:              str
    created_at:            str
    lamport_at_creation:   int
    policy:                CommunityPolicy
    members:               list[CommunityMember]
    revoked:               list[RevokedEntry]
    head_lamport:          int
    signature:             str

    def is_member(self, node_id: str) -> bool: ...
    def level_of(self, node_id: str) -> str | None: ...
    def is_revoked(self, node_id: str) -> bool: ...

def build_community_manifest(
    root_kp: KeyPair,
    name: str,
    policy: CommunityPolicy,
) -> CommunityManifest:
    """Build and sign the genesis community manifest. lamport_at_creation = 0, head_lamport = 0,
       members = [root as 'anchor'], revoked = []."""

def regenerate_community_manifest_from_state(
    materialised_state: dict,
    signing_kp: KeyPair,
) -> CommunityManifest:
    """Given a materialised view from the event log, produce a fresh signed manifest.
       Signing key may be any anchor's key, not just root."""

def parse_community_manifest(blob: bytes | dict) -> CommunityManifest: ...
def verify_community_manifest(cm: CommunityManifest) -> None: ...

3.3 tokens.py (Phase 2 — stubs only in MVP)

# hearthnet/identity/tokens.py
@dataclass(frozen=True)
class CapabilityToken:
    issuer:     str             # NodeID issuing the token
    subject:    str             # NodeID who may use it
    capability: str             # "rag.query"
    issued_at:  str
    expires_at: str
    nonce:      str             # ULID
    signature:  str

def issue_token(...) -> CapabilityToken: ...     # Phase 2
def verify_token(token: CapabilityToken, expected_issuer: str) -> None: ...  # Phase 2

---

4. On-disk format

<DATA>/keys/
├── device.ed25519     # raw 32-byte signing key, no header, 0600
└── device.pub         # raw 32-byte verify key, 0644

Single-key design (no rotation, no chain). Rationale: a community member's key is their identity; rotation creates a new identity. If a key is compromised, the right action is to revoke that NodeID and the user re-onboards with a new key.

---

5. Behaviour

5.1 First-run flow

load_or_generate(keys_dir) →
  if device.ed25519 exists:
    read, validate permissions, parse → KeyPair
  else:
    generate → save → KeyPair

5.2 Manifest TTL and republish

The node orchestrator (node.py) republishes manifest every MANIFEST_REPUBLISH_INTERVAL_SECONDS (20s). Receivers consider any manifest with expires_at < now as expired and drop it.

5.3 Verification budget

Verification is cheap (Ed25519 is fast), but at scale (1000 events/sec) it can add up. Verifying nodes cache (payload_hash, sig, result) for 60 seconds to avoid re-verifying identical bytes.

5.4 Pinned keys

The first time we see a NodeID's verify key, we pin it in memory. A subsequent manifest with the same NodeID but a different verify key is rejected with invalid_signature and logged at warning. This is TOFU (trust-on-first-use) and is acceptable inside a community where invites carry the verify key.

---

6. Errors

All errors raise IdentityError with a code in: keys_missing, keys_invalid, keys_permissions, bad_node_id, sign_failed, verify_failed, bad_manifest, expired, invalid_signature.

These codes map cleanly onto the wire error codes in CONTRACT §9.

---

7. Configuration

From X04 §3:

config.identity.keys_dir       # where keys live
config.identity.auto_generate  # default True

---

8. Tests

Unit

• test_keypair_roundtrip — save / load returns equal keys
• test_short_node_id_format — matches ed25519:[A-Z2-7]{4}(-[A-Z2-7]{4}){3}
• test_canonical_json_is_stable — sort + no-whitespace is deterministic
• test_canonical_json_numbers — 1.0, 1.00, 1 all canonicalise to 1
• test_sign_verify_roundtrip — random payloads
• test_verify_fails_on_modified_field — flip one byte, verify returns False
• test_node_manifest_expiry — exact TTL boundary behaviour
• test_community_manifest_genesis — root key is sole anchor
• test_tls_cert_includes_short_node_id — CN match

Integration

• test_manifest_round_trip_over_http — server emits, client receives, parses, verifies
• test_tofu_key_pinning — second different key for same NodeID is rejected

---

9. Cross-references

What	Where
Wire signing rules	CONTRACT §1.3, §5.1
Canonical JSON	CONTRACT §1.2
Node manifest schema	CONTRACT §6.1
Community manifest schema	CONTRACT §6.2
Used by event signing	X02 §3
Used by transport TLS pinning	X01 §4
Used by bus per-request verification	M03 §5.6

---

10. Open questions

1. Mobile clients store keys in localStorage — not great. Phase 2: WebCrypto + IndexedDB.
2. Key rotation — currently impossible without identity change. Phase 3 may add a "linked-keys" event.
3. HSM / Yubikey support — out of scope; possible Phase 3.