reign-base-l3_gn-gte-base_dapfam-ftwarm-c512s384

REIGN base-l3 cross-chunk encoder over a frozen GTE-base guidance network, fine-tuned on the DAPFAM patent retrieval task (warm-start fine-tune, chunk 512 / stride 384).

Configuration

REIGN encoder	base-l3 — 3 layers, d = 768, 12 heads, FFN 3072, 22.45M trainable parameters
Guidance network (frozen)	thenlper/gte-base — GTE-base, 110M
Chunk size K	512
Stride S	384
Variant	warm-start fine-tune
Initialisation	warm (from the GoodWiki-Long-trained checkpoint)
Warm-start source	reign-base-l3_gn-gte-base_val-selected
Fine-tuning data	DAPFAM (patent family retrieval, FullText)
Training recipe	DAPFAM fine-tuning recipe: InfoNCE, τ = 0.07 (below)

Reported results

nDCG@100 on DAPFAM, top-k = 100 over the full FullText corpus with self-matches removed. These are the values the paper reports for this exact checkpoint (Appendix J).

Split	nDCG@100
test	32.31
test_in	36.88
test_out	5.62

Fine-tuning does not exceed zero-shot on this task. The paper's finding is that no cell in the sweep improves on the matched zero-shot backbone, and that naive fine-tuning at lr 1e-5 degrades it by 0.4–1.5 points. This family is released so the negative result is inspectable, not as a recommended starting point — for patent retrieval, prefer the zero-shot GoodWiki-Long checkpoints.

Usage

The checkpoint holds only the REIGN cross-chunk encoder. The guidance network is loaded separately and stays frozen, so both must be named at construction time.

pip install git+https://github.com/devrimcavusoglu/reign.git

import numpy as np
from huggingface_hub import snapshot_download
from reign.encoders.reign import ReignBaselineEncoder

checkpoint_path = snapshot_download("devrim/reign-base-l3_gn-gte-base_dapfam-ftwarm-c512s384")

encoder = ReignBaselineEncoder(
    checkpoint_path=checkpoint_path,
    gn_model="thenlper/gte-base",
    chunk_size=512,
    stride=384,
)
docs = [open("doc_a.txt").read(), open("doc_b.txt").read()]
emb = encoder.encode(docs, batch_size=8)   # (2, hidden_size), L2-normalised
print(float(np.dot(emb[0], emb[1])))       # cosine similarity

ReignBaselineEncoder returns L2-normalised vectors, so the cosine is a dot product. chunk_size is the guidance network's sliding-window size — 512 for every released checkpoint, matching its context window — and stride controls the overlap, with stride == chunk_size giving non-overlapping chunking. The evaluation-time stride is a runtime argument, and the paper's headline tables report the best-performing stride per guidance network.

For the lower-level surface, ReignModel (a PreTrainedModel consuming inputs_embeds) and ReignFeatureExtractor (the guidance-network wrapper, with the on-disk embedding cache) are importable from reign and reign.feature_extractor.

Operating regime

REIGN targets multi-chunk inputs and primarily document-to-document retrieval. Inputs shorter than the chunk size collapse to a single chunk embedding, leaving the cross-chunk encoder nothing to aggregate — that regime is served by the guidance network alone, and this checkpoint should not be used for it.

Training recipe

DAPFAM's relevance labels are binary, so this family uses a standard query/positive/negative contrastive path rather than the graded three-way cosine objective of the GoodWiki-Long checkpoints.

Setting	Value
Objective	InfoNCE, temperature 0.07, false-negative masking, partial policy ignore
Negatives	4 provided score-0 families per sample plus in-batch negatives
Optimiser	AdamW, cosine schedule
Learning rate	1e-5
Weight decay	1e-4
Epochs	15, validating every 3
Batch size	2
Precision	16-mixed
Seed	42
Chunk / stride	512 / 384

The full sweep covers lr ∈ {1e-5, 5e-6, 2e-6, 1e-6} and weight decay ∈ {1e-4, 1e-2, 1e-1}. See docs/TRAINING.md in the code repository.

Because 16-mixed training is not bit-reproducible even at a fixed seed, a retrained checkpoint will not match these weights bit-for-bit; compare metrics, not weights.

Files

• config.json — ReignModel configuration
• model.safetensors — encoder weights (float32)

Links

• Code: https://github.com/devrimcavusoglu/reign
• Project page: https://devrimcavusoglu.github.io/reign
• Dataset: https://huggingface.co/datasets/devrim/goodwiki_long_synthetic_ir
• Paper: REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling, Findings of the Association for Computational Linguistics: EMNLP 2026 (to appear).

Citation

@inproceedings{cavusoglu2026reign,
  title     = {{REIGN}: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling},
  author    = {{\c{C}}avu{\c{s}}o{\u{g}}lu, Devrim and Akba{\c{s}}, Emre},
  booktitle = {Findings of the Association for Computational Linguistics: {EMNLP} 2026},
  year      = {2026},
  publisher = {Association for Computational Linguistics},
  note      = {To appear}
}

License

Apache License 2.0. The devrim/goodwiki_long_synthetic_ir dataset is released under CC BY-SA 4.0, preserving the share-alike licensing and attribution of GoodWiki and the underlying Wikipedia text.