---
license: apache-2.0
base_model:
- Qwen/Qwen3-0.6B
tags:
- draft
- speculative-decoding
---

A `0.75B` parameter draft (speculative decoding) model for use with [Qwen3-Coder-480B-A35B-Instruct](https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct).

---

I've included the `Q4_0` quants for 4 different [YaRN](https://arxiv.org/abs/2309.00071) extended context lengths:

- [Qwen3-Coder-Instruct-DRAFT-0.75B-32k-Q4_0.gguf](https://huggingface.co/jukofyork/Qwen3-Coder-Instruct-DRAFT-0.75B-GGUF/resolve/main/Qwen3-Coder-Instruct-DRAFT-0.75B-32k-Q4_0.gguf)
- [Qwen3-Coder-Instruct-DRAFT-0.75B-64k-Q4_0.gguf](https://huggingface.co/jukofyork/Qwen3-Coder-Instruct-DRAFT-0.75B-GGUF/resolve/main/Qwen3-Coder-Instruct-DRAFT-0.75B-64k-Q4_0.gguf)
- [Qwen3-Coder-Instruct-DRAFT-0.75B-128k-Q4_0.gguf](https://huggingface.co/jukofyork/Qwen3-Coder-Instruct-DRAFT-0.75B-GGUF/resolve/main/Qwen3-Coder-Instruct-DRAFT-0.75B-128k-Q4_0.gguf)
- [Qwen3-Coder-Instruct-DRAFT-0.75B-256k-Q4_0.gguf](https://huggingface.co/jukofyork/Qwen3-Coder-Instruct-DRAFT-0.75B-GGUF/resolve/main/Qwen3-Coder-Instruct-DRAFT-0.75B-256k-Q4_0.gguf)

**NOTE**: Because `llama.cpp` uses "static-YaRN" the scaling factor remains constant regardless of input length:

- Only use the YaRN extended versions when **processing long contexts** is required.
- Use the **smallest** YaRN-extension possible.

---

# How these were created

## 1. The initial model was created from [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) using [transplant-vocab](https://github.com/jukofyork/transplant-vocab):

```sh
> python3 transplant_vocab.py Qwen3-0.6B Qwen3-Coder-480B-A35B-Instruct Qwen3-Coder-Instruct-DRAFT-0.75B

Loading config from 'Qwen3-0.6B'... Done.
Loading config from 'Qwen3-Coder-480B-A35B-Instruct'... Done.
Loading tokenizer from 'Qwen3-0.6B'... Done.
Loading tokenizer from 'Qwen3-Coder-480B-A35B-Instruct'... Done.
Loading model from 'Qwen3-0.6B'... Done.

Input model configuration:
- Target vocabulary size    : 151936 (used = 151669, unused = 267)
- Donor vocabulary size     : 151936
- Donor num layers          : 28 (tied embeddings = True)
- Donor hidden size         : 1024
- Donor attention heads     : 16
- Donor intermediate size   : 3072 (ratio = 1:3.0)
- Donor total parameters    : 596049920 (0.60B)
-- Embedding parameters     : 155582464 (0.16B)
-- Non-embedding parameters : 440467456 (0.44B)

Processing 3 automatic token overrides:
✘ 'bos_token_id' : Not found for target model
✔ 'eos_token_id' : 151645 '<|im_end|>' → [151645] '<|im_end|>'
✔ 'pad_token_id' : 151643 '<|endoftext|>' → [151643] '<|endoftext|>'

NOTE: Using an "untied" copy of 'embed_tokens.weight' as new 'lm_head.weight' tensor...

Transplanting tokens: 100%|█████████████████████████████████████████████████████████████| 151669/151669 [00:40<00:00, 3751.23token/s]

Transplant mappings:
- 1 to 1  : 149829 (99%)
- 2 to 1  : 816 (0.54%)
- 3 to 1  : 506 (0.33%)
- 4 to 1  : 331 (0.22%)
- 5 to 1  : 118 (0.078%)
- 6 to 1  : 38 (0.025%)
- 7 to 1  : 22 (0.015%)
- 8 to 1  : 8 (0.0053%)
- 9 to 1  : 1 (0.00066%)

Head initialized with:
- Copies : 149829 (99%)
- Means  : 1840 (1.2%)
- Zeros  : 267 (0.18%)

Output model configuration:
- Output vocabulary size    : 151936
- Output num layers         : 28 (tied embeddings = False)
- Output hidden size        : 1024
- Output attention heads    : 16
- Output intermediate size  : 3072 (ratio = 1:3.0)
- Output total parameters   : 751632384 (0.75B)
-- Embedding parameters     : 311164928 (0.31B)
-- Non-embedding parameters : 440467456 (0.44B)

Saving model and tokenizer to 'Qwen3-Coder-Instruct-DRAFT-0.75B' folder

Patching 'torch_dtype' in 'Qwen3-Coder-Instruct-DRAFT-0.75B/config.json' based on actual saved tensors
- Updated 'torch_dtype' to 'bfloat16' based on actual tensor dtype

Operation completed successfully (ignore any 'segmentation fault' that follows!!!)
```

**NOTE**: No subsequent fine-tuning has been performed (due to the 99% "1 to 1" mapping...).

## 2. The context was extended using YaRN:

```json
  "max_position_embeddings": 65536,
  ...
  "rope_scaling": {
    "factor": 2.0,
    "original_max_position_embeddings": 32768,
    "type": "yarn"
  },
```

## 3. Converted and quantized:

```sh
./llama.cpp/convert_hf_to_gguf.py --outtype auto --outfile Qwen3-Coder-Instruct-DRAFT-0.75B-64k-BF16.gguf Qwen3-Coder-Instruct-DRAFT-0.75B
./llama.cpp/build/bin/llama-quantize Qwen3-Coder-Instruct-DRAFT-0.75B-64k-BF16.gguf Qwen3-Coder-Instruct-DRAFT-0.75B-64k-Q4_0.gguf Q4_0 44
```

---

See [here](https://huggingface.co/jukofyork/Qwen3-0.6B-YaRN-GGUF#how-to-patch-the-gguf-files-for-other-context-lengths) for information on how to patch the GGUF files for other context lengths.