soul0101/telco-track-a-submission

Telco Troubleshooting Agentic Challenge — Track A submission

code.zip in this repository is the reproducible submission for Track A of the Telco Troubleshooting Agentic Challenge (Phase 2). It implements a single-shot Qwen3-32B agent over a rule-first / LLM-residual pipeline.

• Base model: Qwen/Qwen3-32B (no fine-tuning, no LoRA adapters).
• Inference stack: vLLM (OpenAI-compatible endpoint) — no internet access required at runtime.
• Throughput: ~9 s / scenario on qwen/qwen3-32b, ~1 k tokens / scenario.

Contents

File	Description
code.zip	Self-contained submission archive. Extract and run per the embedded README.md.

The archive layout follows the competition's SUBMISSION.md requirements:

code.zip
├── README.md            # how to install + run + expected outputs
├── requirements.txt
├── run.py / run.sh      # submission entry point
├── models/deploy.sh     # vLLM deployment command for Qwen3-32B
├── .env.example
├── src/telco_agent/     # agent code (classifier, physics, prompts, resolver, …)
└── scripts/             # legacy dev helpers

How to reproduce

unzip code.zip -d telco_track_a
cd telco_track_a

# 1. Install deps
pip install -r requirements.txt

# 2. Deploy the base model (vLLM, local GPU host)
bash models/deploy.sh        # default: Qwen3-32B at http://localhost:8001

# 3. Run the agent (writes result/ with traces.json, results.csv, runtime.json, README.md)
bash run.sh --input /path/to/scenarios.json --output result

result/ will contain the three artefacts required by the rubric:

• results.csv — scenario_id,prediction (one row per problem).
• traces.json — every LLM completion produced.
• runtime.json — per-problem wall time in seconds, written by the runtime_logger decorator reproduced verbatim from SUBMISSION.md.

Architecture

┌──────────────────────────────────────────────────┐
│ 1 — Extract drive-test KPIs, signaling counts,   │
│     cell configs. Rule cascade → fault kind.     │
├──────────────────────────────────────────────────┤
│ 2 — Pre-filter options to 1-4 IDs.               │
├──────────────────────────────────────────────────┤
│ 3 — Pre-compute per-candidate dB gain delta      │
│     (matches the simulator's gain_pattern).      │
├──────────────────────────────────────────────────┤
│ 4 — Single-shot Qwen3 call:                      │
│     system="5G RAN engineer, output \\boxed{IDs}"│
│     user=facts + candidates + per-kind guidance  │
├──────────────────────────────────────────────────┤
│ 5 — Parse \\boxed{...} → final answer.           │
│     Fallback to rule-based prior on parse fail.  │
└──────────────────────────────────────────────────┘

The classifier produces 11 fault kinds (pdcch, multi_pingpong, multi_az_txinc, add_neighbor, thld_dec, a3_dec, multi_tilt_txdec, tilt_up, test_server, insufficient, unknown) and applies a focused resolution procedure per kind, including hedging strategies for ambiguous single-answer cases.

Why rule-first + LLM-residual?

• Reliability — single-shot \boxed{IDs} over pre-filtered candidates is dramatically more reliable than multi-turn tool-calling loops on this scale of problem (past winners greenpark12345 and vaderyang used the same pattern).
• Reproducibility — the rule engine narrows the answer space to 1–4 candidates, making the LLM job near-deterministic at temperature=0.1.
• Cost — ~1 k tokens / scenario vs 5–10× that for tool-calling loops.