---
title: Compact Re-trace Generator
emoji: "🧠"
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
suggested_hardware: l4x4
---

# Compact re-trace generator

Batch-only Docker Space. It serves four independent vLLM endpoints (one per L4),
streams the source dataset once, and continuously commits validated Parquet shards
and a checkpoint to the output dataset repository.

## Required Space secrets

* `HF_TOKEN` — fine-grained token with **read** access to the gated Llama model and
  **write** access to `OUTPUT_DATASET`.
* `OUTPUT_DATASET` — e.g. `your-name/compact-retraced-reasoning-v1`.

## Optional Space variables

* `MODEL_ID` (default: `meta-llama/Llama-3.1-8B-Instruct`)
* `SOURCE_DATASET` (default: `SupraLabs/reasoning-corpus-4K-5M-v1`)
* `MAX_ROWS` (default: `0`, meaning no limit)
* `SHARD_ROWS` (default: `1000`)
* `MAX_SOURCE_TOKENS` (default: `5000`; skips longer source rows)
* `MAX_THINKING_TOKENS` (default: `220`)
* `MAX_OUTPUT_TOKENS` (default: `900`)
* `TEMPERATURE` (default: `0.7`)
* `SEED` (default: `20260730`)

Run a small pilot first: set `MAX_ROWS=100`. Inspect the output dataset, then create
a fresh dataset repo (or intentionally delete only those pilot shards) before the
full paid run. The process resumes from `state/checkpoint.json`.