---
title: "Beating the Floor Is Hard: A Leakage-Controlled Study of On-Device Tool Selection with Block-Diffusion and Autoregressive LoRA on Apple Silicon"
authors: "Rud Lord and the KnowledgeOS Agents"
affiliations: "KnowledgeOS Collective"
date: 2026-06-14
tags: [tool-use, function-calling, diffusion-language-models, LoRA, on-device, MLX, LLM-as-a-judge, evaluation]
template: arxiv
---

# Abstract

We study whether parameter-efficient fine-tuning can teach a model to *select the right
tools* for an agentic coding task, under conditions that are honest about both **data
leakage** and **strong non-learning baselines**. We build and release what is, to our knowledge (June 2026), the first
**block-diffusion** (DiffusionGemma-26B-A4B) QLoRA trainer that runs entirely on
**Apple Silicon** via MLX, then contrast it head-to-head with an **autoregressive (AR)
sibling** (Gemma-4-26B-A4B) LoRA trained on identical data. We replace a
96%-contaminated source split with a **group-aware, leakage-free benchmark**
(575/124/124), pair every accuracy metric with a **frequency floor** and a **data
ceiling**, and add an **LLM-as-a-judge** that credits semantically-equivalent tool names,
exposing the "dialect tax" by which exact-match metrics understate generalist models
5–6×.

The central result is a carefully-qualified **negative**: across architecture
(diffusion vs. AR), LoRA capacity (rank 16→32), training objective (answer-token loss
reweighting), and a 43% data expansion mined from on-device agent transcripts, **no
trained configuration significantly clears the frequency floor of 0.474 Jaccard** on the
frozen test. The AR sibling is the strongest fine-tune (0.4925 Jaccard, ~10× faster
inference) and *ties* the floor; fine-tuning lifts a 6.1× gap over zero-shot but the floor
is the binding constraint. A deliberately boring train-only TF-IDF nearest-neighbor
baseline reaches **0.512 Jaccard** (5NN vote), the strongest point estimate in the study,
yet also fails to significantly beat the floor (paired Δ +0.038, 95% CI [−0.005,+0.082]).
A label-set oracle reaches **0.944** and the train-vocabulary ceiling reaches **0.996**,
showing that the missing piece is not label availability or adapter capacity but mapping
novel task phrasings to reusable tool sets. We argue that the floor, retrieval baseline,
ceiling, and dialect-aware judge should be first-class requirements for trace-derived
tool-selection studies. We release the trainer, benchmark, dual-metric harness, extra
analysis scripts, and experiment traces (Langfuse + LangSmith).

# 1. Introduction

Agentic coding systems repeatedly answer a small but consequential question: *given a
task and a palette of available tools, which tools are needed?* Getting this right early
prunes the action space, reduces latency, and avoids wasted tool calls. It is therefore
attractive to distill tool selection into a small, fast, **on-device** specialist that
can run alongside a larger planner without a network round-trip.

This paper reports an attempt to build such a specialist and, more importantly, an honest
account of *how hard it is to demonstrate that the specialist actually helps*. We make
five contributions:

1. **An Apple-Silicon block-diffusion trainer.** DiffusionGemma is a block-diffusion
   language model: an autoregressive encoder prefills the prompt, and a bidirectional
   decoder denoises a fixed-length canvas [Arriola et al., 2025; Austin et al., 2021].
   As of mid-2026 no Apple-Silicon training path existed — `mlx-lm` cannot load the
   architecture and `mlx-vlm`'s SFT trainer computes the wrong (autoregressive) loss for
   a denoiser. We implement the ground-truth recipe (D3PM-uniform corruption, supervised
   EOS-fill, manual `LoRALinear` wiring around a MoE) and verify it on a 64GB M2 Max.

2. **A leakage-controlled benchmark with the baselines the field omits.** Our source
   split was 96% contaminated (a trivial train-lookup scored 0.959 Jaccard); we rebuild
   it group-aware and leakage-free, then anchor every number to a **frequency floor**, a
   **data ceiling**, and a train-only **retrieval baseline**, following warnings that LLMs
   rarely beat majority baselines on uncontaminated classification [Sainz et al., 2023;
   Dodge et al., 2021].

3. **A retrieval sanity check that changes the story.** A 5-nearest-neighbor TF-IDF
   baseline over the clean training prompts reaches 0.512 Jaccard, beating every LoRA
   point estimate but not significantly beating the floor. A train-label-set oracle
   reaches 0.944. This localizes the remaining problem to prompt-to-label-set
   generalization, not label availability.

4. **A dialect-aware evaluation.** Trace-derived labels reward one team's tool vocabulary
   (`Bash`) over equivalent synonyms (`run_shell_command`). We add an LLM-as-a-judge
   [Liu et al., 2023; Zheng et al., 2023] that scores task-appropriateness rather than
   string match, and quantify the resulting "dialect tax."

5. **A deconfounded sweep and an honest negative.** Diffusion vs. AR, capacity,
   loss-objective, and data scale — none significantly beat the floor. We show *why*
   (a 0.996 train-vocabulary ceiling and 0.944 label-set oracle show the labels recur) and
   refute our own initial loss-dilution hypothesis with a single-factor experiment.

The takeaway is methodological as much as empirical: **report the floor, report the
retrieval baseline, report the ceiling, judge for dialect, and deconfound your ablations**
— otherwise it is easy to mistake "learned the local vocabulary" for "learned the task."

# 2. Related Work

**Discrete and block-diffusion language models.** D3PM [Austin et al., 2021] introduced
discrete denoising diffusion with four corruption kernels (uniform, absorbing/mask,
discrete Gaussian, embedding-distance). Masked diffusion LMs (MDLM [Sahoo et al., 2024];
MD4 [Shi et al., 2024]) isolate the absorbing-state transition into a simplified masked
objective and scale to billions of parameters (LLaDA [Nie et al., 2025]; Dream [Ye et
al., 2025]; DiffuLLaMA [Nie et al., 2024]). Block / semi-autoregressive variants (BD3-LM
[Arriola et al., 2025]; SDAR [Cheng et al., 2025]) combine inter-block autoregression
with intra-block parallel diffusion — the family DiffusionGemma belongs to. Crucially for
us, DiffusionGemma uses the **uniform** kernel, not absorbing/mask; transplanting an
MDLM-style mask objective would be incorrect.

**Tool selection and function-calling benchmarks.** The Berkeley Function-Calling
Leaderboard [Patil et al., 2025] evaluates tool selection, argument correctness, and
irrelevance detection via AST matching; API-Bank, xLAM [Zhang et al., 2024], and ToolACE
[Liu et al., 2024] extend coverage to multi/parallel calls and self-improvement. These
benchmarks largely assume a fixed, canonical tool API. Our setting differs: labels are
mined from *real agent traces*, so the "correct" tool names are a team-specific dialect —
which motivates both our leakage controls and our LLM-judge.

**Retrieval as a sanity baseline.** Because trace-derived tool labels often recur as
small sets, a tool selector must beat not only learned zero-shot baselines but also a
train-only memory baseline: retrieve similar historical tasks and reuse their tool set.
We use simple TF-IDF nearest-neighbor retrieval only as a diagnostic — not as a novel IR
method — because it is cheap, reproducible, and hard for a 600-example LoRA paper to
ignore.

**On-device parameter-efficient fine-tuning.** LoRA [Hu et al., 2021] and QLoRA
[Dettmers et al., 2023] make billion-parameter fine-tuning feasible on consumer hardware;
Apple's MLX brings this to Apple Silicon. Public MLX LoRA guides target standard
autoregressive models; we extend the approach to a block-diffusion MoE and document the
macOS GPU-watchdog engineering that unattended training requires.

**LLM-as-a-judge.** G-Eval [Liu et al., 2023] and MT-Bench/LLM-judge [Zheng et al., 2023]
established chain-of-thought, form-filling judges; Prometheus 2 [Kim et al., 2024] offers
an open evaluator. Known biases (position, verbosity, self-enhancement) mean judges
complement rather than replace deterministic metrics — which is exactly how we use ours.

**Contamination and baselines.** Data contamination inflates benchmark scores [Sainz et
al., 2023; Dodge et al., 2021], and on uncontaminated classification, LLMs "rarely
demonstrate statistically significant improvements over simple majority baselines." We
take this seriously by construction: a leakage-free split, a frequency floor as the
primary baseline, and paired bootstrap tests for every claim.

# 3. The DiffusionGemma MLX Trainer

DiffusionGemma-26B-A4B-it is a Gemma-4 MoE (25.8B total, 3.8B active, 128 experts, top-8)
wrapped in a block-diffusion process: an AR encoder prefills the prompt (filling a KV
cache), then a bidirectional decoder denoises a 256-token **canvas**. Training requires
three non-obvious choices.

**Corruption (D3PM-uniform).** Per example we draw $t \sim U(\epsilon, 1)$,
$\epsilon{=}10^{-3}$, and replace each canvas position independently with probability $t$
by a token drawn uniformly over the full vocabulary $V{=}262{,}144$. We never use a mask
token; inference initializes and renoises the canvas with uniform random ids, so train
and inference corruption match.

**Loss (flat, full-canvas).** We minimize unweighted cross-entropy over **all** 256
canvas positions (corrupted and clean alike), with the response padded to the boundary by
a supervised `<eos>` fill so the model learns termination. No $1/t$ reweighting (that is
the absorbing-kernel ELBO weight; it does not apply to the uniform kernel).

**LoRA wiring around a MoE.** The stock `mlx-vlm` PEFT path crashes on this architecture
(the language model is a non-`nn.Module` view). We freeze the base and manually wrap
`self_attn.{q,k,v,o}_proj` and dense `mlp.{gate,up,down}_proj` with
`LoRALinear.from_base` (rank 16, $\alpha{=}32$); MoE experts, the router, and embeddings
stay frozen. This trains **18.6M** parameters — 0.072% of the model. Logits are produced
by tying to the input embedding (`embed_tokens.as_linear`) with an fp32 $\tanh$ softcap
at 30. Per-microstep `mx.eval` bounds grad-accumulation memory to a single-step peak
(~17.5GB with gradient checkpointing).

**Unattended-training engineering.** The macOS GPU *interactivity watchdog* aborts long
Metal command buffers and fires more often as the GPU thermally throttles (observed
survival dropped ~20→5 steps/attempt over an hour). Robust configuration:
`--grad-checkpoint` plus `MLX_MAX_OPS_PER_BUFFER=1`, a crash-resume loop, thermal
cooldowns between attempts, dense checkpointing (every 10 steps, so a kill never erases
all progress), and partial-evaluation fallback. We additionally report a sharp
**inference** limitation: block-diffusion generation from long agentic prompts is slow on
MLX today (the long-context prefill is a single large buffer), whereas the AR sibling
decodes at ~1.5 s/sample.

# 4. Benchmark Construction

**Leakage control.** The source `mlx_selector` split contained 3,935 rows but only ~823
distinct (prompt, response) pairs, with the large majority of test prompts appearing
verbatim in train; a trivial train-lookup baseline scored 0.959 Jaccard. We deduplicate
exact pairs, **group by task identity** (a hash of the task-text prefix so near-duplicate
workflow prompts cannot straddle splits), greedy-pack into 575/124/124 train/valid/test,
and assert zero pair *and* zero prompt overlap. Split hashes are frozen in a manifest.

**Frequency floor.** A non-learning predictor that always emits the globally three most
common tools `{Bash, Read, Edit}` scores **0.474 Jaccard / 0.750 Top-1** — a strong
baseline because a few tools dominate agentic traces. We treat clearing this floor (with
a paired test) as the bar for "interesting."

**Data ceiling.** For each test example we compute the Jaccard of its train-vocabulary-
intersected gold set against the full gold set. The mean, **0.996** (only 1 of 124 test
tools is never seen in training), is an upper bound on any train-only model. The large gap
between this ceiling and observed scores localizes the bottleneck to **method, not data**.

**Train-only retrieval baseline.** We strip each rendered prompt to its task text,
construct a deterministic TF-IDF vectorizer over the 575 training tasks, retrieve the
nearest training tasks by cosine similarity, and predict either the nearest neighbor's
label set or the top-three tools by rank-weighted vote across the $k$ nearest neighbors
($k \in \{3,5\}$). This baseline uses no test labels and no LLM calls. We also report a
**train-label-set oracle** that chooses the training label set with maximum Jaccard to the
test gold set; this is not deployable, but measures whether the correct combinations
already recur in the training corpus.

**Metrics.** We report Jaccard (primary), exact-set match, and Top-1, each with 10k-sample
bootstrap 95% CIs, plus paired bootstrap tests for deltas (vs. zero-shot and vs. floor).
We add an **LLM-as-a-judge** (DeepSeek-chat) that, given the task, the reference tools, and
the predicted tools, scores task-appropriateness in $[0,1]$ and is instructed to credit
synonyms (`Bash`≈`run_shell_command`, `Read`≈`read_file`).

# 5. Experimental Setup

All training and evaluation run on a single Mac Studio (M2 Max, 64GB), serially. The
diffusion base is the 4-bit `diffusiongemma-26B-A4B-it`; the AR sibling is the 4-bit
`gemma-4-26B-A4B-it-qat`. Decoding is greedy (temperature 0). Diffusion eval uses
buffer caps `ops=32/mb=128` and a fresh process per 20-sample chunk to avoid a
position-correlated thermal degradation. Zero-shot AR baselines are evaluated
**thinking-disabled** (`enable_thinking=False`) with a 256-token budget and a parser that
strips any residual reasoning channel — without this, reasoning models score 0.000 by
exhausting their budget before answering. Checkpoints are selected on **held-out valid
Jaccard**, never on validation token-loss, then scored once on the frozen test. Retrieval,
head/tail, and label-set-oracle analyses run offline from the frozen train/test JSONL and
the saved per-sample prediction JSONs.

# 6. Results

## 6.1 Main benchmark (frozen 124-sample test)

| Model | Jaccard [95% CI] | Top-1 | LLM-judge |
|---|---|---|---|
| _frequency floor (non-learning)_ | _0.474_ | _0.750_ | — |
| _TF-IDF 5NN vote (train-only retrieval)_ | _0.512 [0.459, 0.565]_ | _0.790_ | — |
| _data ceiling (oracle, train-only)_ | _0.996_ | — | — |
| DiffusionGemma zero-shot | 0.073 [0.045, 0.104] | 0.097 | 0.386 |
| Gemma-4-26B AR zero-shot | 0.070 [0.045, 0.099] | 0.113 | 0.446 |
| Qwen3.6-35B-A3B zero-shot | 0.138 [0.102, 0.176] | 0.250 | 0.294 |
| Qwen3.6-27B zero-shot | 0.197 [0.158, 0.236] | 0.331 | 0.493 |
| DiffusionGemma + LoRA (ours) | 0.447 [0.393, 0.502] | 0.750 | **0.721** |
| **AR sibling + LoRA (ours)** | **0.4925 [0.432, 0.553]** | **0.782** | 0.650 |

Fine-tuning works in the trivial sense: the diffusion LoRA lifts zero-shot 0.073→0.447
(paired delta **+0.374**, $P(\text{win}){=}1.0$), and the AR LoRA lifts 0.070→0.4925
(+0.42, $P{=}1.0$). The AR sibling is the **best fine-tune and ~10× faster** to serve.
But the strongest point estimate in the table is not a neural model at all: a cheap
train-only TF-IDF 5NN vote reaches **0.512**.

## 6.2 The floor is not significantly beaten

The decisive test is paired-bootstrap of (model − floor) on the same items:

| Comparison | Mean Δ | 95% CI | Verdict |
|---|---|---|---|
| TF-IDF 5NN vote − floor | +0.038 | [−0.005, +0.082] | **tie** (near-miss) |
| AR sibling − floor | +0.018 | [−0.039, +0.078] | **tie** (CI crosses 0) |
| capacity r32 − floor | −0.030 | [−0.081, …] | below |
| answer-weighted eos0.05 − floor | −0.049 | [−0.089, …] | below |
| answer-weighted eos0.25 − floor | −0.034 | [−0.061, …] | below |

No trained configuration's lower bound exceeds zero. Even the retrieval baseline, which has
the highest point estimate, remains a statistical tie with the floor.

## 6.3 A boring memory baseline is the strongest point estimate

The retrieval sanity check is uncomfortable for the LoRA story. Predicting from the
nearest training tasks is not a new model, but it is the right floor for trace-derived
selectors: a production agent can cheaply remember its own past.

| Method | Jaccard [95% CI] | Δ vs. floor [95% CI] | Top-1 | Exact |
|---|---|---|---|---|
| TF-IDF 1NN | 0.500 [0.445, 0.557] | +0.026 [−0.031, +0.084] | 0.766 | 0.153 |
| TF-IDF 3NN vote | 0.511 [0.458, 0.563] | +0.036 [−0.011, +0.085] | 0.790 | 0.161 |
| **TF-IDF 5NN vote** | **0.512 [0.459, 0.565]** | **+0.038 [−0.005, +0.082]** | **0.790** | **0.177** |
| train-label-set oracle | 0.944 [0.921, 0.966] | +0.470 [+0.420, +0.521] | 0.935 | 0.815 |

The 5NN vote is the best point estimate, but its paired lower bound still does not clear
zero. The label-set oracle is the more important diagnostic: the right tool combinations
usually already exist in train. The hard part is *routing a new task to the right prior*,
not inventing unseen tool labels.

![Retrieval baselines](figures/fig5_retrieval_baselines.pdf)

## 6.4 Head/tail error anatomy

The floor is not uniformly strong. It wins by saturating the head tools. On the 44
head-only examples (gold set subset of `{Bash, Read, Edit}`), the floor scores 0.652,
while the AR LoRA scores 0.769. But on examples with two or more tail tools, the floor
falls to 0.285 and both LoRAs are worse (diffusion 0.257, AR 0.229); TF-IDF 1NN is still
weak but best at 0.360. The missed vertical is therefore not "bigger LoRA" in the
abstract. It is tail-tool recall: `StructuredOutput`, shell/read dialect synonyms, MCP
browser tools, and low-frequency agent-specific tools.

| Slice | n | floor | Diffusion LoRA | AR LoRA | TF-IDF 1NN |
|---|---:|---:|---:|---:|---:|
| head-only tools | 44 | 0.652 | 0.540 | **0.769** | 0.559 |
| one tail tool | 48 | 0.438 | 0.489 | 0.415 | **0.541** |
| multi-tail tools | 32 | 0.285 | 0.257 | 0.229 | **0.360** |

![Head/tail stratification](figures/fig6_head_tail_strata.pdf)

## 6.5 Ablations (deconfounded, single-factor vs. the diffusion baseline 0.447)

| Variant | Test Jaccard | LLM-judge | Note |
|---|---|---|---|
| answer-weighted loss, eos-weight 0.05 | 0.425 | 0.676 | **worse** — refutes the loss-dilution fix |
| answer-weighted loss, eos-weight 0.25 | 0.440 | 0.689 | still below baseline |
| capacity rank 32 / $\alpha$64 | 0.444 | 0.666 | extra capacity does not help |
| +43% data (expanded, rank 16) | 0.434 (valid 0.529) | — | valid up, test flat |

We initially hypothesized that ~95% of the diffusion loss is "wasted" on trivial EOS-fill
positions, diluting the signal from the few tool-name tokens. Down-weighting EOS made
results **worse** at both doses — over-correction; the model already learns content tokens
under the flat loss. The data expansion (249 leak-free pairs mined per-turn from 13,127
on-device agent transcripts, vocabulary 39→63) **raised held-out valid Jaccard
0.498→0.529 but left the frozen test flat** — a cautionary valid/test split.

## 6.6 The dialect tax

Under exact-match Jaccard, zero-shot models look incompetent (0.07–0.20). Under the
LLM-judge, every model jumps **5–6×** (DiffusionGemma zero-shot 0.073→0.386; Gemma-4 AR
0.070→0.446), because they select *valid* tools in a different vocabulary. Strikingly, the
two metrics **rank the architectures oppositely**: the AR LoRA wins on Jaccard (0.4925 vs.
0.447) but the diffusion LoRA wins on the judge (0.721 vs. 0.650). "Which is better"
depends on whether one rewards exact-name match or task-appropriate selection — a
distinction invisible to a single metric.

# 7. Discussion

**The floor is the protagonist.** A study like this is tempting to frame as "our
fine-tune beats much larger zero-shot models" (it does, by 2–6×). But the honest frame is
that a non-learning frequency predictor is competitive with everything we trained. The
floor encodes a real prior — a few tools dominate — and beating it requires task signal
beyond that prior, which our levers did not reliably supply.

**Retrieval is the uncomfortable baseline.** The TF-IDF result changes the practical
recommendation. If the product goal is a useful on-device selector, a memory-first router
should be in the first implementation: retrieve similar historical tasks, reuse or vote on
their tool sets, then ask a learned model only for ambiguous tail cases. If the research
goal is to prove learning, that retrieval baseline is the minimum bar.

**Diffusion vs. AR.** On identical data, the AR sibling matches or exceeds the
block-diffusion model on exact-match accuracy at ~10× lower latency, while the diffusion
model is judged slightly more task-appropriate. For *this* task on Apple Silicon, the AR
specialist is the more practical artifact; block-diffusion's iterative canvas decoding is
a lossier and slower way to emit a short tool list.

**Method, not data — but now more specific.** The 0.996 train-vocabulary ceiling and 0.944
label-set oracle say the labels and label combinations are available. Capacity, objective,
and architecture changes did not clear the floor. The gap is prompt-to-label-set routing
and tail-tool recall, not lack of labels or lack of trainable parameters.

**Next verticals.** The useful follow-up lanes are: (1) a retrieval-augmented selector
with calibrated fallback to AR LoRA; (2) a head/tail hybrid that keeps `{Bash, Read, Edit}`
as priors but trains binary or retrieval classifiers for `StructuredOutput`, shell/read
synonyms, browser/MCP tools, and other tail tools; (3) a canonical tool ontology so exact
Jaccard and semantic judge scores stop measuring dialect; (4) an online Langfuse/LangSmith
A/B that measures wasted tool calls, latency, and task success, not just offline Jaccard;
and (5) cross-organization trace splits to determine whether the floor is a universal
agentic prior or a single-operator artifact.

# 8. Limitations

The test set is small (124 items); the win/loss lives in low-frequency tool buckets where
124 samples give wide CIs. Labels are single-team trace-derived (the dialect issue we
measure but cannot remove). The LLM-judge has its own biases [Zheng et al., 2023] and is
itself an LLM. The retrieval baseline is deliberately simple TF-IDF, not a tuned semantic
retriever or production memory system. The data expansion is modest (+43%) and dominated
by short, repetitive user turns after resume-copy deduplication. Inference-latency claims
are MLX-specific and may change as block-diffusion kernels mature. Finally, all results
are offline; the decisive product metric is an online A/B over tool-call waste, latency,
and task success.

# 9. Conclusion

We built what is, to our knowledge, the first Apple-Silicon block-diffusion QLoRA trainer, a leakage-controlled
tool-selection benchmark anchored to a frequency floor, a retrieval baseline, and a data
ceiling, plus a dialect-aware dual-metric harness. Across architecture, capacity,
objective, and data scale, **no trained configuration significantly beats the frequency
floor** — a robust negative that we argue is the finding, not a failure. The AR sibling is
the best and fastest fine-tune, but a TF-IDF memory baseline is the strongest point
estimate and also only ties the floor. The next useful selector is likely hybrid:
frequency priors for head tools, retrieval for recurring task clusters, and learned
calibration for the tail. We release the trainer, benchmark, harness, extra analyses, and
experiment traces, and we recommend that on-device tool-selection studies report the
floor, retrieval baseline, ceiling, and dialect-aware score before claiming a win.

# Reproducibility

Trainer, eval, leakage-aware dataset builder, frequency-floor / data-ceiling / paired-
bootstrap analysis, TF-IDF retrieval baseline, head/tail stratification, LLM-judge, and
the dual Langfuse+LangSmith push scripts are released. Key hyperparameters: AdamW
(bias-corrected) lr 1.5e-4 → cosine 1.5e-5, warmup 25, betas (0.95, 0.99), wd 1e-4,
grad-clip 1.0, micro-batch 1 × accum 8; diffusion LoRA r16/$\alpha$32 on attn+MLP
(18.6M params); AR sibling LoRA r16 attention-only (11.5M). All splits are
sha256-manifested; all runs are logged with per-sample predictions and both metrics.

# References

- Arriola et al. (2025). *Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models (BD3-LMs)*. arXiv:2503.09573.
- Austin et al. (2021). *Structured Denoising Diffusion Models in Discrete State-Spaces (D3PM)*. arXiv:2107.03006.
- Cheng et al. (2025). *SDAR: Semi-Autoregressive Diffusion*. arXiv.
- Dettmers et al. (2023). *QLoRA: Efficient Finetuning of Quantized LLMs*. arXiv:2305.14314.
- Dodge et al. (2021). *Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus*. arXiv:2104.08758.
- Hu et al. (2021). *LoRA: Low-Rank Adaptation of Large Language Models*. arXiv:2106.09685.
- Kim et al. (2024). *Prometheus 2: An Open Source LM Specialized in Evaluating Other LMs*. arXiv:2405.01535.
- Liu et al. (2023). *G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment*. arXiv:2303.16634.
- Liu et al. (2024). *ToolACE: Winning the Points of LLM Function Calling*. arXiv:2409.00920.
- Nie et al. (2025). *Large Language Diffusion Models (LLaDA)*. arXiv:2502.09992.
- Nie et al. (2024). *Scaling Diffusion Language Models via Adaptation from Autoregressive Models (DiffuLLaMA)*. arXiv:2410.17891.
- Patil et al. (2025). *The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation*. PMLR v267.
- Sahoo et al. (2024). *Simple and Effective Masked Diffusion Language Models (MDLM)*. arXiv:2406.07524.
- Sainz et al. (2023). *NLP Evaluation in Trouble: On the Need to Measure LLM Data Contamination for each Benchmark*. arXiv:2310.18018.
- Shi et al. (2024). *Simplified and Generalized Masked Diffusion for Discrete Data (MD4)*. arXiv:2406.04329.
- Ye et al. (2025). *Dream 7B: Diffusion Large Language Models*. arXiv.
- Zhang et al. (2024). *xLAM: A Family of Large Action Models to Empower AI Agent Systems*. arXiv:2409.03215.
- Zheng et al. (2023). *Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena*. arXiv:2306.05685.