davidtai-eigenlabs commited on
Commit
a210166
·
verified ·
1 Parent(s): eda45ab

Model card: conversion provenance, architecture, MTP head pointer, challenge role

Browse files
Files changed (1) hide show
  1. README.md +40 -0
README.md CHANGED
@@ -2,6 +2,46 @@
2
  language: en
3
  library_name: mlx
4
  pipeline_tag: text-generation
 
 
5
  tags:
6
  - mlx
 
 
7
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  language: en
3
  library_name: mlx
4
  pipeline_tag: text-generation
5
+ license: apache-2.0
6
+ base_model: Qwen/Qwen3.8-27B
7
  tags:
8
  - mlx
9
+ - 4-bit
10
+ - qwen3_5
11
  ---
12
+
13
+ # Qwen3.8-27B-4bit
14
+
15
+ A 4-bit MLX conversion of [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)
16
+ (revision `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`), quantized affine 4-bit with
17
+ group size 64 under a pinned **mlx 0.32.0** toolchain via `mlx_lm.convert`.
18
+
19
+ ## Architecture
20
+
21
+ - 64 layers on a 4-layer hybrid repeat: three linear-attention layers, then one
22
+ full-attention layer (`full_attention_interval: 4`)
23
+ - hidden size 5120, 24 attention heads / 4 KV heads, head dim 256
24
+ - vocabulary 248,320
25
+ - `model_type: qwen3_5`
26
+
27
+ ## Multi-token-prediction head
28
+
29
+ The upstream checkpoint ships a native MTP (multi-token-prediction) head as 15
30
+ `mtp.*` tensors. Those tensors are **not** in this repository — they are published
31
+ separately, in their released bf16 dtype, as
32
+ [EigenLabs/Qwen3.8-27B-MTP-bf16](https://huggingface.co/EigenLabs/Qwen3.8-27B-MTP-bf16).
33
+
34
+ ## Role
35
+
36
+ This is the pinned reference checkpoint for the Qwen 3.8 native-MTP
37
+ speculative-decode challenge track (`qwen3.8-27b-mtp-v1`) at
38
+ [Layr-Labs/qwen-3.8-mtp-challenge](https://github.com/Layr-Labs/qwen-3.8-mtp-challenge).
39
+ The track pins this repository by revision; scoring artifacts are byte-verified
40
+ against it.
41
+
42
+ ## Usage
43
+
44
+ ```bash
45
+ pip install mlx-lm
46
+ mlx_lm.generate --model EigenLabs/Qwen3.8-27B-4bit --prompt "Hello"
47
+ ```