Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -9,42 +9,33 @@ tags:
|
|
| 9 |
- gated-deltanet
|
| 10 |
---
|
| 11 |
|
| 12 |
-
# Qwen 3.5 2B β TFLite
|
| 13 |
|
| 14 |
-
Qwen 3.5 2B
|
| 15 |
|
| 16 |
-
**
|
| 17 |
|
| 18 |
-
##
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
##
|
| 23 |
|
| 24 |
| | |
|
| 25 |
|---|---|
|
| 26 |
| **Base model** | [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) |
|
| 27 |
-
| **
|
| 28 |
| **Quantization** | int8 dynamic |
|
| 29 |
| **Format** | TFLite (.tflite) |
|
| 30 |
| **Size** | ~1.9 GB |
|
| 31 |
|
| 32 |
-
## Architecture Notes
|
| 33 |
-
|
| 34 |
-
Qwen 3.5 uses a hybrid architecture that no other converter previously supported:
|
| 35 |
-
|
| 36 |
-
- **Linear attention (GatedDeltaNet):** Recurrent state-space model with A_log decay, conv1d, output gating β 18 layers
|
| 37 |
-
- **Full attention (every 4th layer):** Grouped query attention with asymmetric Q/KV head dims (Q=512 with partial rotary, KV=256) β 6 layers
|
| 38 |
-
|
| 39 |
-
Both layer types implemented using standard TFLite ops (matmul, element-wise, conv1d) β no custom kernels.
|
| 40 |
-
|
| 41 |
## Files
|
| 42 |
|
| 43 |
-
- `qwen35_2b.tflite` β The converted model
|
| 44 |
-
- `tokenizer.json` β BPE tokenizer (
|
| 45 |
- `tokenizer_config.json` β Tokenizer configuration
|
| 46 |
- `config.json` β Original model config
|
| 47 |
|
| 48 |
## Conversion
|
| 49 |
|
| 50 |
-
|
|
|
|
| 9 |
- gated-deltanet
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Qwen 3.5 2B β TFLite (.tflite)
|
| 13 |
|
| 14 |
+
Qwen 3.5 2B as raw TFLite format for on-device inference with the TFLite Interpreter API.
|
| 15 |
|
| 16 |
+
> **For LiteRT-LM Engine usage, use the bundled version instead: [paulsp94/Qwen3.5-2B-LiteRT-LM](https://huggingface.co/paulsp94/Qwen3.5-2B-LiteRT-LM)**
|
| 17 |
|
| 18 |
+
## What's this
|
| 19 |
|
| 20 |
+
Raw `.tflite` model file β use this if you're building your own inference pipeline with the TFLite Interpreter API directly. If you want the ready-to-use LiteRT-LM bundle with tokenizer included, use the [LiteRT-LM version](https://huggingface.co/paulsp94/Qwen3.5-2B-LiteRT-LM) instead.
|
| 21 |
|
| 22 |
+
## Architecture
|
| 23 |
|
| 24 |
| | |
|
| 25 |
|---|---|
|
| 26 |
| **Base model** | [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) |
|
| 27 |
+
| **Layers** | 24 total: 18Γ GatedDeltaNet linear + 6Γ GQA full attention |
|
| 28 |
| **Quantization** | int8 dynamic |
|
| 29 |
| **Format** | TFLite (.tflite) |
|
| 30 |
| **Size** | ~1.9 GB |
|
| 31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
## Files
|
| 33 |
|
| 34 |
+
- `qwen35_2b.tflite` β The converted model
|
| 35 |
+
- `tokenizer.json` β BPE tokenizer (you'll need to handle tokenization yourself)
|
| 36 |
- `tokenizer_config.json` β Tokenizer configuration
|
| 37 |
- `config.json` β Original model config
|
| 38 |
|
| 39 |
## Conversion
|
| 40 |
|
| 41 |
+
Source: [allot/tools/model-export](https://github.com/paulsp94/allot/tree/main/tools/model-export)
|