# Apple MLX Deployment Guide

## Supported deployment target

The validated target is an Apple M1 Mac with 16 GB unified memory. The model also runs on newer Apple Silicon systems with sufficient memory.

The 4-bit release is the recommended choice for 16 GB because macOS, Python, model weights, Metal allocations, image tensors and KV cache all share the same memory pool.

## 1. Prerequisites

Confirm Apple Silicon:

```bash
uname -s
uname -m
sysctl -n machdep.cpu.brand_string
```

Expected:

```text
Darwin
arm64
Apple M1, M2, M3, M4 or later Apple Silicon
```

Install Python if needed. The validated conversion used Python 3.14, but use a current native arm64 Python compatible with the pinned MLX packages.

## 2. Download the repository

```bash
python3 -m pip install --upgrade "huggingface-hub[cli]"
hf auth login

cd ~/Downloads
hf download \
  AXONVERTEX-AI-RESEARCH/Shieldstral-1.0-3B-MLX-4bit \
  --local-dir Shieldstral-1.0-3B-MLX-4bit

cd Shieldstral-1.0-3B-MLX-4bit
```

The `.cache/huggingface` directory created by `--local-dir` is local download metadata. Remove it after download when a clean directory is preferred:

```bash
rm -rf .cache/huggingface
```

## 3. Install dependencies

```bash
./scripts/install.sh
source .venv/bin/activate
```

The installer creates a local virtual environment and installs the pinned MLX-VLM stack.

## 4. Verify release integrity

```bash
./scripts/verify_checksums.sh
```

Then load the model and perform one deterministic classification:

```bash
./scripts/validate_model.sh
```

Expected outcome:

```text
PASS: model loaded
PASS: deterministic classification returned yes or no
```

## 5. Start the server

```bash
./scripts/start_server.sh
```

Defaults:

```text
HOST=127.0.0.1
PORT=18190
MAX_KV_SIZE=16384
KV_BITS=8
TOP_LOGPROBS_K=20
```

The command binds only to localhost. Keep this default unless remote access is intentionally configured and protected.

### Lower-memory mode

```bash
MAX_KV_SIZE=8192 ./scripts/start_server.sh
```

### Minimal test mode

```bash
MAX_KV_SIZE=4096 ./scripts/start_server.sh
```

### Different port

```bash
PORT=18210 ./scripts/start_server.sh
```

### Bind to a Tailscale interface

Determine the Tailscale IPv4 address:

```bash
TAILSCALE_IP="$(tailscale ip -4)"
```

Start:

```bash
HOST="$TAILSCALE_IP" PORT=18190 ./scripts/start_server.sh
```

Do not bind to `0.0.0.0` on an untrusted network without authentication, firewalling and transport controls.

## 6. Health and smoke tests

From a second terminal:

```bash
cd ~/Downloads/Shieldstral-1.0-3B-MLX-4bit
source .venv/bin/activate

./scripts/health.sh
./scripts/smoke_test.sh
```

Run the technical-report protocol cases:

```bash
./scripts/run_paper_alignment.sh
```

Run all 13 regression cases:

```bash
./scripts/run_evals.sh
```

## 7. Background service with launchd

Create a logs directory:

```bash
mkdir -p "$HOME/Library/Logs/shieldstral-mlx"
```

Create `~/Library/LaunchAgents/ai.axonvertex.shieldstral-mlx.plist` and replace `/ABSOLUTE/PATH`:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>ai.axonvertex.shieldstral-mlx</string>
  <key>ProgramArguments</key>
  <array>
    <string>/bin/zsh</string>
    <string>/ABSOLUTE/PATH/Shieldstral-1.0-3B-MLX-4bit/scripts/start_server.sh</string>
  </array>
  <key>WorkingDirectory</key>
  <string>/ABSOLUTE/PATH/Shieldstral-1.0-3B-MLX-4bit</string>
  <key>EnvironmentVariables</key>
  <dict>
    <key>HOST</key><string>127.0.0.1</string>
    <key>PORT</key><string>18190</string>
    <key>MAX_KV_SIZE</key><string>16384</string>
    <key>KV_BITS</key><string>8</string>
  </dict>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
  <key>StandardOutPath</key>
  <string>/Users/REPLACE_ME/Library/Logs/shieldstral-mlx/server.out.log</string>
  <key>StandardErrorPath</key>
  <string>/Users/REPLACE_ME/Library/Logs/shieldstral-mlx/server.err.log</string>
</dict>
</plist>
```

Load it:

```bash
launchctl bootout "gui/$(id -u)" ~/Library/LaunchAgents/ai.axonvertex.shieldstral-mlx.plist 2>/dev/null || true
launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/ai.axonvertex.shieldstral-mlx.plist
launchctl kickstart -k "gui/$(id -u)/ai.axonvertex.shieldstral-mlx"
```

Check:

```bash
launchctl print "gui/$(id -u)/ai.axonvertex.shieldstral-mlx"
curl -fsS http://127.0.0.1:18190/health | python -m json.tool
```

## 8. Memory monitoring

Use Activity Monitor or:

```bash
ps -o pid,rss,%cpu,command -ax | grep -E 'mlx_vlm.server|Python' | grep -v grep
memory_pressure
```

If macOS memory pressure becomes yellow or red:

1. reduce `MAX_KV_SIZE`;
2. reduce simultaneous requests;
3. close memory-heavy applications;
4. restart the server after large multimodal workloads.

## 9. Operational guidance

- Keep `max_tokens=1` for classification.
- Keep `temperature=0.0` for deterministic scoring.
- Request `top_logprobs=20` so both `yes` and `no` are normally available.
- Use one policy question per request for category-specific moderation.
- Use a broad query only when the instruction explicitly defines the combined policy scope.
- Treat the score as a policy signal, not as sole authority for destructive or irreversible action.

## 10. Troubleshooting

### `MistralCommonBackend has no attribute vocab`

This release intentionally omits `tekken.json` from the runtime artifact and uses `tokenizer.json`, forcing the MLX-compatible `TokenizersBackend`. Do not add `tekken.json` back to the model directory.

### Model loads but server is killed

Reduce context:

```bash
MAX_KV_SIZE=4096 ./scripts/start_server.sh
```

### Port already in use

```bash
lsof -nP -iTCP:18190 -sTCP:LISTEN
PORT=18191 ./scripts/start_server.sh
```

### Checksum failure after `hf download --local-dir`

The checksum manifest excludes `.cache/huggingface`. Run the provided verifier from the model root. It ignores local cache metadata.