Mistral-7B-Instruct-v0.3-gsm8k-ties-d0p2-lam1p0-a0p9-v1

Merged model produced by TIES-Merging (Yadav et al., NeurIPS 2023).

• Reference: https://arxiv.org/abs/2306.01708

Construction

• Base model: mistralai/Mistral-7B-Instruct-v0.3
• Task vectors:
  • safety = wvnvwn/Mistral-7B-Instruct-v0.3-hhrlhf-v1, alpha = 0.9
  • downstream = wvnvwn/Mistral-7B-Instruct-v0.3-gsm8k-v1, alpha = 0.09999999999999998

Hyperparameters

• Method: ties
• Scaling coefficient (lambda): 1.0
• TIES Trim density: 0.2
• dtype: bfloat16

Merge formula

$${\theta }_{\text{merged}}={\theta }_{\text{base}}+\lambda \cdot M(\alpha \cdot {\tau }_{\text{safety}}+(1-\alpha )\cdot {\tau }_{\text{downstream}}),\backslash theta\_\backslash text\{merged\}\; =\; \backslash theta\_\backslash text\{base\}\; +\; \backslash lambda\; \backslash cdot\; M\backslash bigl(\backslash alpha\; \backslash cdot\; \backslash tau\_\backslash text\{safety\}\; +\; (1-\backslash alpha)\; \backslash cdot\; \backslash tau\_\backslash text\{downstream\}\backslash bigr),$$

where tau_i = theta_i_sft - theta_base, and M(.) is the identity for TA, the Trim-Elect-Disjoint operator for TIES, and DARE's drop-and-rescale preprocessing (composed with TA) for DARE.

Reproduction

Produced by the GTM merge_cli (see commit history of the GTM repo). Full metadata is shipped as merge_metadata.json alongside the weights.

Intended use

Research only. This checkpoint is part of a sweep designed to probe the safety / downstream-utility Pareto frontier; individual α values are NOT recommended for deployment without further evaluation.