--- license: gemma base_model: google/embeddinggemma-300m pipeline_tag: sentence-similarity tags: - embeddinggemma - dense-retrieval - fine-tuned - matryoshka - onnx --- # EmbeddingGemma-300M memory retriever v1 ## Summary This model is the dense lane of a local memory-retrieval pipeline. It is a fine-tune of [`google/embeddinggemma-300m`](https://huggingface.co/google/embeddinggemma-300m) for finding passages of working notes and documentation that answer a natural-language question, trained on 116,101 graded query-passage pairs. On a fully judged 376-query development panel it reached Hit@1 0.657 against 0.598 for the conservative first fit, a paired gain of 0.059 with a family-clustered 95% interval of [+0.014, +0.104], and Recall@50 0.547 against 0.516 [+0.005, +0.058]. In the protected promotion read, as the first stage of the fused pipeline, it lifted useful Hit@1 from 0.484 to 0.574 over the previous production retriever on 895 answerable queries. Every relevance grade behind these numbers is a frontier-model judgment under a frozen two-plus-one protocol, and the corpus is mostly generated documents; the limitations section says what that does and does not license. ## Model details | Item | Value | |---|---| | Architecture | EmbeddingGemma-300M, mean pooling, dense projection head, merged attention LoRA (rank 16, alpha 32) | | Embedding widths | 768 (qualified), 512, 256, 128 (Matryoshka-trained, storage-only) | | Query prefix | `task: search result \| query: ` | | Passage prefix | `title: none \| text: ` | | Sequence limits | 128 query tokens, 1,024 passage tokens | | Serving precision | FP32 through ONNX Runtime, the qualified realization; no FP16 realization is qualified | | Deployment state | FP32 ONNX export qualified on CPU, CUDA and DirectML; installation activation is separate | ## Intended use Semantic candidate discovery inside a hybrid first stage: this model and BM25 each retrieve, reciprocal-rank fusion (rank constant 60) merges them, and a fixed pool of 50 passages goes to a cross-encoder reranker. The model is tuned for rich, question-shaped memory queries. Terse, exact, or pasted queries are the lexical lane's job; the model was neither trained nor selected on them alone. ## Training data | Item | Value | |---|---| | Queries in gradients | 2,612 answerable queries plus the 261-query tuning fold | | Unique positive pairs (grades 2 and 3) | 116,101, each seen exactly once | | Unique scheduled negatives (grade 0) | 53,645 | | Repeated supervised pairs | 0 | | Protected queries or families in gradients | 0 | Queries were authored by language models from document situations without seeing answers or candidates. Grades run 0 to 3; grade 3 positives carry 1.5 times the weight of grade 2. Same-family co-batched positives are masked so a related passage is never punished as a false negative, and sparse "needle" queries receive one direct positive-versus-hard-negative lesson. ## Recipe Effective contrastive batch 32, learning rate 5e-5 with 10% warmup and cosine decay, FP16 with dynamic loss scaling, 3,685 optimizer updates over one coverage epoch, Matryoshka objectives at 768/512/256/128 with weights 1, 0.25, 0.125, and 0.0625, temperature 0.05. The adapter was merged into the base; merging moved probe embeddings by at most 1.4e-7 and a save-and-reload by 0. ## Evaluation **Development panel.** 453 queries (376 answerable, 77 with no useful passage in any judged pool), 104 query families, 75,261 grades, zero unjudged passages through rank 50 for every compared system. Dense-only, no reranker. | Model | Hit@1 | Hit@5 | Hit@10 | Hit@20 | Hit@50 | Recall@50 | |---|---:|---:|---:|---:|---:|---:| | conservative first fit (G0) | 0.5984 | 0.7766 | 0.8404 | 0.8910 | 0.9654 | 0.5164 | | this model, batch 32 | 0.6569 | 0.7926 | 0.8617 | 0.9016 | 0.9575 | 0.5474 | | batch-64 finalist | 0.6463 | 0.8085 | 0.8590 | 0.9016 | 0.9575 | 0.5507 | The two finalists tie within uncertainty at every cutoff; the pre-registered efficiency rule selected batch 32. Real-source queries are a thin slice (43 of 376) on which this model scored Hit@1 0.744 against 0.646 on generated worlds; the slice saturates at Hit@20 and cannot discriminate deeper. ![Hit@k, Recall@k, and paired Hit@k gains over G0 on the development panel](../figures/dense_development_panel.svg) **Matryoshka widths** on the same panel, same vectors truncated and re-normalized. Smaller widths cut stored bytes but not the forward pass, and their changed pools were not fully judged. | Width | Hit@1 | Hit@10 | Hit@50 | Recall@50 | Unjudged@50 | Bytes per vector | |---:|---:|---:|---:|---:|---:|---:| | 768 | 0.6569 | 0.8617 | 0.9575 | 0.5474 | 0.000 | 3,072 | | 512 | 0.6516 | 0.8484 | 0.9415 | 0.5169 | 0.061 | 2,048 | | 256 | 0.6250 | 0.8404 | 0.9415 | 0.4812 | 0.131 | 1,024 | | 128 | 0.5851 | 0.8112 | 0.9388 | 0.4239 | 0.222 | 512 | **Protected gate, first stage only.** 990 sealed queries, two source-disjoint draws frozen before training, every top-50 pool of both arms fully graded. On the 895 queries answerable in the matched union, useful Hit@1 was 0.5743 against 0.4838 for the previous retriever, +0.0905 with a one-sided 95% lower bound of +0.0590 against a registered 0.02 margin. Every critical slice passed and no gate query or family appears in training. ## Runtime Serves through FastEmbed 0.8.0 and ONNX Runtime 1.24.4 without a Torch stack. The original CUDA-qualified export reproduced the merged model with a difference from the Torch reference of at most 9.2e-7 per vector element with identical top-1 and top-10 sets at every width. Re-measured on 2026-09-04 on an otherwise idle RTX 3060 Ti over 50 repetitions, a query embeds in 0.015 s median and 0.017 s at the 95th percentile, and a batch of 16 passages takes 0.82 s at the 95th percentile. ## Limitations, ranked 1. **Model-judged relevance.** No human grades exist; agreement between independent model sessions was 0.858 with kappa 0.682 on a 419-query audit. Scores measure agreement with that instrument. 2. **Generated corpus.** 80 fictional document worlds plus one operator's own projects. Behavior on other real corpora is unmeasured. The generated worlds were chosen for their breadth of domains and document shapes. 3. **One training run.** No seed replicate was affordable; the batch-32 versus batch-64 tie and the 0.059 gain over G0 are single-run estimates whose intervals capture query sampling only. 4. **Tuning fold in gradients.** The 261-query fold that chose the recipe was added to the final fit for more data once it had saturated; the selection read cannot be repeated on it. 5. **Rich queries only.** Terse and exact queries were left to the lexical lane and to the reranker's training mix. 6. **Per-query gate evidence lost.** The gate judgments and report were destroyed in a local storage incident after the aggregates above were recorded; they cannot be re-audited. ## Reproducibility - Model tree SHA-256 `ae68b190bca3a5e9ac90e45ca4c1449c823899ca593d65526a96398c0e607703`; base tree `24b175205b9fc9a4ba6480a54ee8ea8c26178d52ef33752d6f82d53921e96b4d`. - Training data manifest `8c0ec3eba6b641a6dc7a95886e838f6062525c4b064eb1d2ebd8b2ebbf793326`, schedule `250eb68b21aa6f9332b9f08fc69d90553321784b76200b7cef8bc4804166e8d4`. - Recipe id `final-coverage-lr5e5-b32`; registry manifest `../model_registry/embeddinggemma-300m-memory-ft-v1.json`; experiment log `../history/README.md`. - Build stack: Transformers 4.57.6, Sentence Transformers 5.5.1, PyTorch 2.10.0; export opset 18. ## License Gemma Terms of Use and the Gemma Prohibited Use Policy apply; the distribution includes the required notice and a modification disclosure. See . ## Portable ONNX execution The graph uses standard inferred reshape dimensions that execute on ONNX Runtime 1.24.4 with CPU, CUDA or DirectML. The correction changes 96reshape attributes; model weights, tokenization and output dimensions are unchanged. On an RTX 3060 Ti, all 74 reference vectors stayed within 1.5e-6 of the retained Torch reference. Ranking agrees at every tested width after identical passages are counted once. Concurrent query/passage calls and DirectML worker crash recovery passed through the managed adapter. This is one Windows/NVIDIA qualification; AMD and Intel devices were not measured. Batch sizes depend on available memory and qualified limits, so throughput depends on the host and passage lengths.