Tabby-Tavern AI Stack

Complete containerized local AI infrastructure for a private lab:

Layer	Software
High-perf inference	TabbyAPI + EXL3 / ExLlamaV3
Character chat UI	SillyTavern
General LLM workspace	Open WebUI
GGUF backend	Ollama
Private search	SearXNG (+ Redis cache)

This is a private-lab / portfolio stack — production-shaped, not a multi-tenant SaaS product and not a hosted inference endpoint.

Lab baseline hardware: NVIDIA GeForce RTX 4070 (Linux + Docker Compose + NVIDIA Container Toolkit).

Surface	URL
This HF card	https://huggingface.co/jpanasuk/tabby-tavern-stack
GitHub source of truth	https://github.com/jpanasuk-netizen/tabby-tavern-stack
Sell sheet (Space)	https://huggingface.co/spaces/jpanasuk/tabby-tavern-sell-sheet
Telemetry / benchmarks	https://github.com/jpanasuk-netizen/local_grid_suite
Multi-agent prototype	https://github.com/jpanasuk-netizen/multi-agent-dungeon-crawler
Author	https://huggingface.co/jpanasuk · @jpanasuk-netizen

Weights are not included. You download EXL3 and/or GGUF models yourself.
Secrets are not included. Copy examples and generate your own keys.

Service ports (compose defaults)

Compose service	Container name (default)	Host port	In-container	Role
`sillytavern`	`tabby-tavern-sillytavern-1`	8000	8000	Chat / character frontend
`open-webui`	`tabby-tavern-open-webui-1`	3000	8080	General LLM workspace
`tabbyapi`	`tabby-tavern-tabbyapi-1`	5000	5000	EXL3 high-performance inference
`ollama`	`tabby-tavern-ollama-1`	11434	11434	GGUF model backend
`searxng`	`tabby-tavern-searxng-1`	8080	8080	Private metasearch
`searxng-redis`	`tabby-tavern-searxng-redis-1`	(internal)	6379	SearXNG cache

Open WebUI is wired to Ollama via compose env:

OLLAMA_BASE_URL=http://ollama:11434

All services share the bridge network ai-network.

System requirements

Host

OS: Linux native, or Windows WSL2 with Docker Desktop / Docker Engine
GPU: NVIDIA GPU with recent drivers (lab used RTX 4070)
Docker: Docker Engine + Compose plugin (docker compose version)
NVIDIA Container Toolkit (required for GPU passthrough into containers)
Disk: room for Docker images plus model weights (EXL3 8B class is often tens of GB; GGUF varies by quant)
RAM / VRAM: 8B-class EXL3 fits a 12 GB class card with headroom when KV/cache is tuned; larger models need more VRAM

One-time NVIDIA Container Toolkit check

# Driver visible on host
nvidia-smi

# Toolkit installed (package name varies by distro)
# https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

# GPU visible inside a throwaway container
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

If nvidia-smi works on the host but fails in Docker, fix the toolkit / Docker daemon GPU runtime before bringing the stack up.

Optional host tools

# Hugging Face CLI (model download)
pipx install "huggingface_hub[cli]"
# or: python3 -m venv .venv && . .venv/bin/activate && pip install -U "huggingface_hub[cli]"

Cold start (stranger path)

Prefer cloning GitHub for day-to-day work. This HF repo is the public card + sanitized mirror of the same layout.

0) Clone

git clone https://github.com/jpanasuk-netizen/tabby-tavern-stack.git
cd tabby-tavern-stack

Alternate (HF mirror — same public layout, may lag GitHub):

git lfs install   # only if you later pull large assets; weights still not shipped
git clone https://huggingface.co/jpanasuk/tabby-tavern-stack
cd tabby-tavern-stack

1) Build the TabbyAPI image used by compose

Compose expects this exact local tag:

docker build -f Dockerfile.tabby -t local/tabbyapi:exl3-fixed .

Dockerfile.tabby is a thin layer on ghcr.io/theroyallab/tabbyapi:latest (adds build deps used by the lab image path).
There is also a fuller Dockerfile with CUDA/EXL3 env defaults baked in — use it if you prefer image-level defaults:

# optional alternative image build
docker build -f Dockerfile -t local/tabbyapi:exl3-fixed .

2A) Download an EXL3 model for TabbyAPI

Weights go under tabby_models/ (gitignored). Compose mounts:

./tabby_models  →  /app/models   (inside tabbyapi)

TabbyAPI model_name is a directory name under that mount.

mkdir -p tabby_models

# Example EXL3 pack (pick a revision/quant that fits your VRAM)
huggingface-cli download turboderp/Llama-3.1-8B-Instruct-exl3 \
  --revision 6.0bpw \
  --local-dir tabby_models/Llama-3.1-8B-Instruct-6.0bpw-exl3

# Confirm the directory exists and contains model files
ls -la tabby_models/Llama-3.1-8B-Instruct-6.0bpw-exl3 | head

Other EXL3 / ExLlama-family repos work the same way: download into tabby_models/<folder-name>/ and point config at <folder-name>.

2B) Configure TabbyAPI (required before first launch)

# Prefer the sanitized example if present
cp -n tabby_config/config.example.yml tabby_config/config.yml

# If you only have a minimal stub, create a working private-lab config:
cat > tabby_config/config.yml <<'EOF'
network:
  host: 0.0.0.0
  port: 5000
  disable_auth: false

# Generate your own keys — never reuse published historical keys
# python -c "import secrets; print(secrets.token_hex(32))"
security:
  admin_key: "REPLACE_WITH_LONG_RANDOM_ADMIN_KEY"
  api_key: "REPLACE_WITH_LONG_RANDOM_API_KEY"

developer:
  disable_auth: false

model:
  model_dir: models
  # MUST match the directory name under ./tabby_models/
  model_name: Llama-3.1-8B-Instruct-6.0bpw-exl3
  max_seq_len: 8192
  cache_size: 8192
EOF

Generate real keys:

python3 -c "import secrets; print('admin_key:', secrets.token_hex(32)); print('api_key:  ', secrets.token_hex(32))"
# paste into tabby_config/config.yml

Optional API token file (create locally; do not commit):

# compose has an optional mount (commented by default):
# - ./tabby_config/api_tokens.yml:/app/api_tokens.yml

Compose always mounts:

./tabby_config/config.yml  →  /app/config.yml

2C) Ollama / GGUF path (secondary backend)

Ollama data persists in ./ollama_data → /root/.ollama.

After the stack is up:

# Pull a small GGUF-class model through the running container
docker exec -it tabby-tavern-ollama-1 ollama pull qwen2.5:7b

# List models
docker exec -it tabby-tavern-ollama-1 ollama list

# Smoke test generation
docker exec -it tabby-tavern-ollama-1 ollama run qwen2.5:7b "Say hello in one sentence."

Open WebUI (http://localhost:3000) should discover Ollama automatically via OLLAMA_BASE_URL=http://ollama:11434.

You can run TabbyAPI (EXL3) and Ollama (GGUF) together; just remember they both want GPU VRAM — load one heavy model at a time on a single 12 GB class card unless you know your headroom.

3) Frontend / search secrets (placeholders only)

SillyTavern (sillytavern_config/config.yaml):

Default listen port 8000
Lab compose sets WHITELIST_MODE=false via environment
If basic auth is enabled in the YAML, replace:

basicAuthUser:
  username: "YOUR_ST_USERNAME_HERE"
  password: "YOUR_ST_PASSWORD_HERE"

SearXNG (searxng_config/settings.yml):

use_default_settings: true
server:
  secret_key: "YOUR_SEARXNG_SECRET_KEY_HERE"
  image_proxy: true

Replace secret_key with a long random string before any network exposure:

python3 -c "import secrets; print(secrets.token_hex(32))"

4) Launch

docker compose up -d
docker compose ps

Or:

chmod +x start-stack.sh load-model.sh
./start-stack.sh

start-stack.sh runs docker compose down || true then docker compose up -d and prints the service URLs.

5) Health checks (every service)

# Compose view
docker compose ps
docker compose logs --tail=80 tabbyapi
docker compose logs --tail=40 ollama
docker compose logs --tail=40 sillytavern
docker compose logs --tail=40 open-webui
docker compose logs --tail=40 searxng

# HTTP probes (expect non-connection-refused; exact status varies by app)
curl -sS -o /dev/null -w "sillytavern  %{http_code}\n" http://127.0.0.1:8000/ || true
curl -sS -o /dev/null -w "open-webui   %{http_code}\n" http://127.0.0.1:3000/ || true
curl -sS -o /dev/null -w "tabbyapi     %{http_code}\n" http://127.0.0.1:5000/ || true
curl -sS -o /dev/null -w "ollama       %{http_code}\n" http://127.0.0.1:11434/ || true
curl -sS -o /dev/null -w "searxng      %{http_code}\n" http://127.0.0.1:8080/ || true

# Ollama API
curl -sS http://127.0.0.1:11434/api/tags | head -c 400; echo

# GPU inside TabbyAPI container
docker exec -it tabby-tavern-tabbyapi-1 nvidia-smi || \
  docker exec -it tabby-tavern-tabbyapi-1 bash -lc 'nvidia-smi' || true

Browser targets after green probes:

SillyTavern → http://localhost:8000
Open WebUI → http://localhost:3000
TabbyAPI → http://localhost:5000
Ollama → http://localhost:11434
SearXNG → http://localhost:8080

6) Point SillyTavern at TabbyAPI

In SillyTavern API settings (UI):

API type: OpenAI-compatible / TabbyAPI (as offered by your ST build)
Endpoint: http://tabbyapi:5000 from another container, or http://127.0.0.1:5000 from the host browser depending on how ST is configured
API key: the api_key you put in tabby_config/config.yml

Cross-container auth/whitelist issues were a real lab pain point — see DEVLOG.md. If ST cannot reach TabbyAPI, check:

Both containers on ai-network (docker network inspect / docker compose ps)
TabbyAPI keys match what ST sends
TabbyAPI finished loading the EXL3 model (docker compose logs -f tabbyapi)

7) Switch / reload TabbyAPI model

# Edit model_name in tabby_config/config.yml to another folder under tabby_models/
docker compose restart tabbyapi
docker compose logs -f tabbyapi

Note: load-model.sh in older trees still hardcodes a host path like ~/tabby-tavern/.... Prefer editing tabby_config/config.yml + docker compose restart tabbyapi from the repo root, or fix the script paths to $PWD before using it.

GPU / compose tuning (what the lab actually ships)

From docker-compose.yml on the GPU services (tabbyapi, ollama):

Setting	Value	Why
`deploy.resources.reservations.devices`	NVIDIA GPU `count: all`	Passthrough
`shm_size`	16g	Avoid shared-memory OOM during load/attn
`CUDA_VISIBLE_DEVICES`	`0`	Pin first GPU
`CUDA_DEVICE_ORDER`	`PCI_BUS_ID`	Stable device ordering
`PYTORCH_CUDA_ALLOC_CONF`	`max_split_size_mb:128`	Allocator fragmentation control
`EXLLAMA_GPU_LAYERS`	`999`	Prefer full GPU offload
`EXLLAMA_KV_CACHE`	`q8_0`	KV cache quant tradeoff
`EXLLAMA_FLASH_ATTENTION`	`1`	Flash-attn path when available
`OLLAMA_FLASH_ATTENTION`	`1`	Ollama flash-attn
`OLLAMA_KV_CACHE_TYPE`	`q8_0`	Ollama KV quant
`OLLAMA_NUM_PARALLEL`	`1`	Single parallel slot (VRAM stability)

If you OOM: lower max_seq_len / cache_size, use a smaller bpw EXL3, or stop Ollama models while TabbyAPI holds a large model.

Repository structure

tabby-tavern-stack/
├── docker-compose.yml          # full lab orchestration (6 services)
├── Dockerfile                  # TabbyAPI image + CUDA/EXL3 ENV defaults
├── Dockerfile.tabby            # thin TabbyAPI image used by default compose tag
├── start-stack.sh              # down + up + print URLs
├── load-model.sh               # helper (verify paths before use)
├── tabby_config/
│   ├── config.example.yml      # sanitized template (prefer this)
│   └── config.yml              # local runtime config (do not publish real keys)
├── tabby_models/               # EXL3 weights (gitignored contents)
├── sillytavern_config/         # ST server config (placeholders for auth)
├── sillytavern_data/           # ST user data (secrets gitignored)
├── openwebui_data/             # Open WebUI persistence (DB gitignored)
├── ollama_data/                # Ollama models/keys (gitignored)
├── searxng_config/             # settings.yml (placeholder secret_key)
├── searxng/                    # optional extra searxng assets
├── Clean_Sampler.json          # sampler preset helper
├── SECURITY.md
├── DEVLOG.md                   # engineering history
├── LICENSE
└── docs/                       # sell-sheet / extras (when present)

Compose service names (authoritative):

tabbyapi · sillytavern · ollama · open-webui · searxng · searxng-redis

Environment & secrets guidance

Secret / file	Where	Rule
TabbyAPI `admin_key` / `api_key`	`tabby_config/config.yml`	Generate yourself; never commit live values
TabbyAPI `api_tokens.yml`	`tabby_config/` (optional mount)	Local only
SillyTavern basic auth	`sillytavern_config/config.yaml`	Replace `YOUR_ST_*` placeholders
SillyTavern cookie/secrets	under `sillytavern_data/`	gitignored
SearXNG `secret_key`	`searxng_config/settings.yml`	Replace placeholder
Open WebUI DB	`openwebui_data/webui.db*`	gitignored
Ollama keys/models	`ollama_data/`	gitignored
`.env`	repo root	gitignored if created

Private-lab defaults are intentional. This is not hardened multi-tenant hosting.

Before any LAN/WAN exposure:

Replace every placeholder credential
Prefer binding host ports to 127.0.0.1 (edit compose ports: to "127.0.0.1:8000:8000", etc.)
Put a reverse proxy + TLS in front if you leave the machine
Read SECURITY.md

Older public revisions of this mirror may have contained lab convenience keys. Treat any key you ever saw in a public file as burned and rotate it everywhere it might have been reused.

Common failure modes

Symptom	Likely cause	Fix
`local/tabbyapi:exl3-fixed` pull/start fails	Image never built	`docker build -f Dockerfile.tabby -t local/tabbyapi:exl3-fixed .`
TabbyAPI exits / cannot find model	`model_name` ≠ folder under `tabby_models/`	Align names; confirm mount `./tabby_models:/app/models`
CUDA / GPU errors in container	Toolkit missing or no `--gpus` path	Fix NVIDIA Container Toolkit; `docker run --gpus all ... nvidia-smi`
OOM during load	Model + KV + dual backends too large	Smaller quant; lower `max_seq_len`/`cache_size`; unload Ollama models
ST cannot talk to TabbyAPI	Auth/network mismatch	Same compose network; matching API key; wait for model load
Open WebUI shows no models	Ollama empty or URL wrong	`ollama pull ...` in ollama container; confirm `OLLAMA_BASE_URL=http://ollama:11434`
SearXNG unhealthy / 500s	Placeholder/missing `secret_key` or Redis down	Set secret; `docker compose logs searxng searxng-redis`
Port already allocated	Host process using 3000/5000/8000/8080/11434	Stop conflicting service or remap compose ports
`load-model.sh` edits wrong file	Hardcoded `~/tabby-tavern` path	Edit `tabby_config/config.yml` in-repo instead
WSL2 GPU not visible	Windows driver / WSL GPU support	Update NVIDIA Windows driver; confirm `nvidia-smi` inside WSL

Measured lab results (defensible only)

Numbers from checked-in sample telemetry in local_grid_suite (benchmarks/sample_hardware_runs.json, methodology in benchmarks/METHODOLOGY.md). Single-box lab runs — not a product SLA.

Stage	Model	Decode tok/s
Baseline	`qwen3:8b`	1.39
GPU-routed	`qwen-gpu:latest`	29.7 – 39.3
Stabilized	`qwen-gpu:latest`	37.47

→ ~27× decode uplift on that run series after GPU routing / tuning.

Warm stream suite on qwen3:8b: ~76 tok/s (400-token runs).

Re-measure on your hardware. Do not advertise these as guaranteed throughput.

Engineering notes

See DEVLOG.md for the real build log:

Compose consolidation (TabbyAPI + ST + Open WebUI + Ollama + SearXNG + Redis)
EXL3 adoption vs earlier experiments
GPU env tuning (shm_size, flash-attn / KV cache flags)
Container-to-TabbyAPI auth / whitelist fixes

Related spine

Project	Link
Stack (this card)	https://huggingface.co/jpanasuk/tabby-tavern-stack
Stack (GitHub)	https://github.com/jpanasuk-netizen/tabby-tavern-stack
Sell sheet Space	https://huggingface.co/spaces/jpanasuk/tabby-tavern-sell-sheet
Local Grid Suite	https://github.com/jpanasuk-netizen/local_grid_suite
Multi-agent prototype	https://github.com/jpanasuk-netizen/multi-agent-dungeon-crawler
Collection	https://huggingface.co/collections/jpanasuk/independent-ai-lab-spine-6a727803ed9c6d16164f5130

WSL2 field notes — fresh-install lessons (Aug 2026)

A from-scratch rebuild on a clean WSL2 Ubuntu 24.04 environment (same RTX 4070) surfaced several issues not covered above. These notes are additive — the cold-start guide above remains the canonical path; this section documents what else can bite you and the exact fixes.

EXL2 vs EXL3 — TabbyAPI dropped EXL2 support

The local/tabbyapi:exl3-fixed image (built on ghcr.io/theroyallab/tabbyapi:latest) no longer supports EXL2-quantized models. If you point model_name at an exl2 directory, TabbyAPI will crash on startup with:

ValueError: Models quantized with 'exl2' require the exllamav2 backend, which is no longer supported. Please use an exl3 or unquantized model.

Fix: Use EXL3 quants only. turboderp/Llama-3.1-8B-Instruct-exl3 has branches for every bpw (2.0 through 8.0). Download with:

huggingface-cli download turboderp/Llama-3.1-8B-Instruct-exl3 \
  --revision 6.0bpw \
  --local-dir tabby_models/Llama-3.1-8B-Instruct-exl3-6.0bpw

Then set model_name: Llama-3.1-8B-Instruct-exl3-6.0bpw in tabby_config/config.yml.

NVIDIA Container Toolkit repo URL — wrong format breaks apt

The official NVIDIA Container Toolkit instructions often show a repo list line like:

deb [signed-by=...] https://nvidia.github.io/libnvidia-container/stable/deb/noble main

This does not work — apt reports Malformed entry (Component) because the NVIDIA repo uses a flat structure (/ as component), not distro main.

Fix: The correct format is:

# Add GPG key
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor --yes -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

# Correct repo line (note: amd64/ and trailing /)
echo 'deb [arch=amd64 signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/deb/amd64 /' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

After this, docker info | grep nvidia should show Runtimes: nvidia runc ....

libcuda.so not found inside TabbyAPI container on WSL2

Even with NVIDIA Container Toolkit installed and torch.cuda.is_available() returning True, the TabbyAPI container may crash during model load with:

/usr/bin/ld: cannot find -lcuda: No such file or directory

This happens because the Triton JIT compiler (used by exllamav3 for some kernels) needs libcuda.so at link time, but the WSL2 CUDA driver libraries are under /usr/lib/wsl/drivers/ and /usr/local/cuda-12.8/compat/ — not in the standard linker path.

Fix: Add a symlink step to the Dockerfile:

RUN ln -sf /usr/local/cuda-12.8/compat/libcuda.so /usr/lib/x86_64-linux-gnu/libcuda.so && ldconfig

Full patched Dockerfile:

FROM ghcr.io/theroyallab/tabbyapi:latest
USER root
RUN apt-get update && apt-get install -y python3-dev python3.12-dev build-essential && rm -rf /var/lib/apt/lists/*
RUN ln -sf /usr/local/cuda-12.8/compat/libcuda.so /usr/lib/x86_64-linux-gnu/libcuda.so && ldconfig
RUN mkdir -p /app/logs && chmod 777 /app/logs

The Triton warnings (Triton is not supported on current platform, roll back to CPU) are cosmetic — exllamav3 uses its own CUDA kernels for inference and does not require Triton.

Port conflict when host Ollama is already running

If you already run Ollama on the host (port 11434), the Docker Ollama container will fail with:

Error response from daemon: failed to bind host port 0.0.0.0:11434/tcp: address already in use

Fix: Remap the Docker Ollama port in docker-compose.yml:

ollama:
  ports:
    - "11435:11434"   # host 11435 → container 11434

Open WebUI still reaches Ollama via the internal Docker network at http://ollama:11434 — only the host-side port changes.

Open WebUI — connect to TabbyAPI as an OpenAI-compatible backend

By default Open WebUI is wired only to Ollama. To also use TabbyAPI's EXL3 model from Open WebUI, add these environment variables to the open-webui service in docker-compose.yml:

open-webui:
  environment:
    - OLLAMA_BASE_URL=http://ollama:11434
    - OPENAI_API_BASE_URL=http://tabbyapi:5000/v1
    - OPENAI_API_KEY=your_tabbyapi_api_key
    - ENABLE_OPENAI_API=true

This makes both the Ollama GGUF models and the TabbyAPI EXL3 model available in Open WebUI's model selector.

SillyTavern — disable browser auto-launch in Docker

SillyTavern's browserLaunch.enabled: true causes issues in Docker (no browser available). Set it to false in sillytavern_config/config.yaml:

browserLaunch:
  enabled: false

Also, SillyTavern will refuse to start if it detects an "insecure" configuration (listening on 0.0.0.0 with no auth). Keep basicAuthMode: true or enable user accounts.

MCPO — empty mcpServers config crashes

MCPO requires at least one server entry in mcpo/config.json. An empty "mcpServers": {} causes:

ValueError: No 'mcpServers' found in config file.

Fix: Either provide a real MCP server, or create a minimal stub at ~/mcp-servers/server.py and reference it:

{
  "mcpServers": {
    "host-master": {
      "command": "python3",
      "args": ["/opt/mcp-servers/server.py"]
    }
  }
}

SearXNG — enable JSON format for API access

The default SearXNG config only returns HTML. If any tool needs JSON search results, add it to the formats list in searxng_config/settings.yml:

search:
  formats:
    - html
    - json

TabbyAPI config tuning for 12 GB VRAM (RTX 4070)

Tested config for Llama-3.1-8B-Instruct EXL3 6.0bpw on a 12 GB card:

model:
  model_dir: models
  model_name: Llama-3.1-8B-Instruct-exl3-6.0bpw
  max_seq_len: 8192
  cache_size: 8192
  cache_8bit: true

cache_8bit: true halves KV cache VRAM usage (important for 12 GB cards)
max_seq_len: 8192 is a safe ceiling — 10240 works but leaves less headroom when Ollama also has a model loaded
Measured: ~8 GB VRAM used with the 6.0bpw EXL3 model + 8-bit cache, leaving ~4 GB for Ollama GGUF models

MCPO — MCP server with OpenAPI proxy

MCPO (ghcr.io/open-webui/mcpo:main) launches MCP servers as subprocesses and exposes their tools as OpenAPI-compatible HTTP endpoints. The stack includes a custom MCP server at mcp-servers/server.py that provides 7 tools for managing the stack:

Tool	Method	Description
`list_tabbyapi_models`	POST	List loaded EXL3 models in TabbyAPI
`tabbyapi_chat`	POST	Chat with the TabbyAPI EXL3 model
`list_ollama_models`	POST	List GGUF models in Ollama
`ollama_pull_model`	POST	Pull a model into Ollama
`ollama_chat`	POST	Chat with an Ollama model
`get_stack_status`	POST	Health check all services
`searxng_search`	POST	Web search via SearXNG

Key details:

The MCP server uses FastMCP from the mcp Python library — not raw JSON-RPC. MCPO's container venv at /app/.venv/ has mcp pre-installed.
The config (mcpo/config.json) must point to /app/.venv/bin/python3 (not bare python3) so the server can import mcp.
Environment variables (TABBYAPI_URL, OLLAMA_URL, TABBYAPI_KEY) are passed via the config's env field so the server can reach other containers over the Docker network.
Tools are served at the /host-master/ path prefix (matching the server name in config).
Swagger docs: http://localhost:8001/docs
API key: set via --api-key in docker-compose command args (default: mcp-secret-key)

Empty mcpServers crashes MCPO — it requires at least one server entry. If you don't have a real MCP server, create a minimal one or remove the MCPO service from compose.

Verified working state

After all fixes applied, the stack was verified with 24/24 checks passing:

All 6 containers running, no restart loops
TabbyAPI: EXL3 model loaded, chat completions returning valid responses at ~53 tok/s processing
Ollama: llama3.1:8b pulled and available
Open WebUI: healthy, connected to both TabbyAPI and Ollama
SillyTavern: running with basic auth (admin/tabby)
SearXNG: serving with JSON format enabled
MCPO: connected to host-master MCP server
GPU: 8 GB / 12 GB VRAM used on RTX 4070

Author

Jeremy Panasuk — enterprise data / platform background; Aug 2024–present independent local-AI systems year (private Docker LLM lab, decode telemetry, multi-agent prototypes).

GitHub: @jpanasuk-netizen
Hugging Face: jpanasuk
LinkedIn: jeremy-p-34203322

License

MIT

Downloads last month: -; Downloads are not tracked for this model. How to track

Spaces using jpanasuk/tabby-tavern-stack 2

Collection including jpanasuk/tabby-tavern-stack

Independent AI Lab Spine

Collection

Stack + telemetry + multi-agent prototype companions for the independent engineering year. • 2 items • Updated 10 days ago