Vela-2.0-9B / MODIFICATIONS.md
adaamko's picture
Vela 2.0 9B: code, config, calibration, card
2f41850 verified
|
Raw History Blame Contribute Delete
2.5 kB

Modifications

This repository is derived from the Decision 2.0 package Decision-2.0-Lux-9B (its MODEL_MANIFEST.json sha256 and decision_config.json are recorded in this repository's MODEL_MANIFEST.json under base). Through it, the backbone and tokenizer come from Qwen3.5-9B at revision c202236235762e1c871ad0ccb60c8ee5ba337b9a (see ATTRIBUTIONS.md).

  • Backbone weights. Every backbone weight of Decision-2.0-Lux-9B was fine-tuned further with the Vela 2.0 routing recipe (full fine-tune, no adapter; the released weights are the merged, full parameters). They are stored as backbone.* in the safetensors shards: the Linear and conv1d weights in bf16 (the values bf16 autocast computes with), all other tensors in FP32.
  • Decision head. The Decision-2.0-Lux-9B CandidateHead (head.*) was trained further and reads choice, noul and score questions as in Decision 2.0; set questions use the same scores with one added scalar (set_bias).
  • New parameters. The span head (span2.*: layer norms, label-slot embedding, bilinear + MLP word × label scorer, outside-label readout) was added and trained in a second stage with the backbone and decision head frozen. The first-stage span head of the training checkpoint (7 tensors) is not used and is not included.
  • Prompt layout. The Decision 2.0 prompt form (context, task type, question, <option> blocks, Decision:) is extended with typed <segment role="..."> parts, a Target: line, set and span question blocks, and a tree forward in which every question block attends only to the parts and to itself. The code is in modeling_vela2.py and vela2_inference.py; the Decision 2.0 runtime (decision2/) is not included.
  • Tokenizer files. tokenizer.json and tokenizer_config.json are the Decision-2.0-Lux-9B files, unchanged.
  • Licence files. LICENSE, NOTICE, ATTRIBUTIONS.md and LICENSES/ are the Decision-2.0-Lux-9B files, passed on unchanged. They describe Decision-2.0-Lux-9B; here they apply to the material inherited from it and, through it, from Decision 1.0 and Qwen3.5.
  • Optional noul calibration. calibration.json carries a noul_calibration block (one temperature and bias for SystemOne noul answers, fitted on general-decision noul questions; see PARITY.md). It is off by default and is enabled with Vela2(..., noul_calibration=True) or VELA2_NOUL_CALIBRATION=1 in vela2_inference.py; with it off, outputs are those of the uncalibrated release.