paulsp94 commited on
Commit
f3c9832
Β·
verified Β·
1 Parent(s): 750215a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +10 -19
README.md CHANGED
@@ -9,42 +9,33 @@ tags:
9
  - gated-deltanet
10
  ---
11
 
12
- # Qwen 3.5 2B β€” TFLite
13
 
14
- Qwen 3.5 2B converted to **TFLite** format (.tflite) for on-device inference.
15
 
16
- **First working conversion of Qwen 3.5's hybrid GatedDeltaNet + full attention architecture to TFLite.**
17
 
18
- ## Format
19
 
20
- This is a **raw .tflite file**, not a .litertlm bundle. To use with LiteRT-LM's `Engine`, you'll need to bundle it with a tokenizer into .litertlm format. It can also be used directly with the TFLite Interpreter API.
21
 
22
- ## Model Details
23
 
24
  | | |
25
  |---|---|
26
  | **Base model** | [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) |
27
- | **Architecture** | 24 layers β€” 18x GatedDeltaNet linear attention + 6x standard GQA |
28
  | **Quantization** | int8 dynamic |
29
  | **Format** | TFLite (.tflite) |
30
  | **Size** | ~1.9 GB |
31
 
32
- ## Architecture Notes
33
-
34
- Qwen 3.5 uses a hybrid architecture that no other converter previously supported:
35
-
36
- - **Linear attention (GatedDeltaNet):** Recurrent state-space model with A_log decay, conv1d, output gating β€” 18 layers
37
- - **Full attention (every 4th layer):** Grouped query attention with asymmetric Q/KV head dims (Q=512 with partial rotary, KV=256) β€” 6 layers
38
-
39
- Both layer types implemented using standard TFLite ops (matmul, element-wise, conv1d) β€” no custom kernels.
40
-
41
  ## Files
42
 
43
- - `qwen35_2b.tflite` β€” The converted model (1.9 GB)
44
- - `tokenizer.json` β€” BPE tokenizer (from original Qwen 3.5 2B)
45
  - `tokenizer_config.json` β€” Tokenizer configuration
46
  - `config.json` β€” Original model config
47
 
48
  ## Conversion
49
 
50
- Converted using custom litert-torch model authoring with GatedDeltaNet linear attention implementation. Source code: [allot/tools/model-export](https://github.com/paulsp94/allot/tree/main/tools/model-export)
 
9
  - gated-deltanet
10
  ---
11
 
12
+ # Qwen 3.5 2B β€” TFLite (.tflite)
13
 
14
+ Qwen 3.5 2B as raw TFLite format for on-device inference with the TFLite Interpreter API.
15
 
16
+ > **For LiteRT-LM Engine usage, use the bundled version instead: [paulsp94/Qwen3.5-2B-LiteRT-LM](https://huggingface.co/paulsp94/Qwen3.5-2B-LiteRT-LM)**
17
 
18
+ ## What's this
19
 
20
+ Raw `.tflite` model file β€” use this if you're building your own inference pipeline with the TFLite Interpreter API directly. If you want the ready-to-use LiteRT-LM bundle with tokenizer included, use the [LiteRT-LM version](https://huggingface.co/paulsp94/Qwen3.5-2B-LiteRT-LM) instead.
21
 
22
+ ## Architecture
23
 
24
  | | |
25
  |---|---|
26
  | **Base model** | [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) |
27
+ | **Layers** | 24 total: 18Γ— GatedDeltaNet linear + 6Γ— GQA full attention |
28
  | **Quantization** | int8 dynamic |
29
  | **Format** | TFLite (.tflite) |
30
  | **Size** | ~1.9 GB |
31
 
 
 
 
 
 
 
 
 
 
32
  ## Files
33
 
34
+ - `qwen35_2b.tflite` β€” The converted model
35
+ - `tokenizer.json` β€” BPE tokenizer (you'll need to handle tokenization yourself)
36
  - `tokenizer_config.json` β€” Tokenizer configuration
37
  - `config.json` β€” Original model config
38
 
39
  ## Conversion
40
 
41
+ Source: [allot/tools/model-export](https://github.com/paulsp94/allot/tree/main/tools/model-export)