turboderp commited on
Commit
3760820
·
verified ·
1 Parent(s): c52c086

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -1
README.md CHANGED
@@ -9,7 +9,17 @@ tags:
9
 
10
  EXL3 quants of [gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it)
11
 
12
- ⚠️ Requires ExLlamaV3 v0.0.40 (or v0.0.39 `dev` branch)
 
 
 
 
 
 
 
 
 
 
13
 
14
  [2.00 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/2.00bpw)
15
  [2.50 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/2.50bpw)
 
9
 
10
  EXL3 quants of [gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it)
11
 
12
+ mul1 quants (recommended, improved performance on exllamav3>=v1.0.0):
13
+
14
+ [3.00 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/3.00bpw_mul1)
15
+ [3.50 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/3.50bpw_mul1)
16
+ [4.00 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/4.00bpw_mul1)
17
+ [4.50 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/4.50bpw_mul1)
18
+ [5.00 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/5.00bpw_mul1)
19
+ [6.00 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/6.00bpw_mul1)
20
+ [8.00 bits per weight / H8](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/8.00bpw_mul1)
21
+
22
+ MCG quants (legacy):
23
 
24
  [2.00 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/2.00bpw)
25
  [2.50 bits per weight](https://huggingface.co/turboderp/gemma-4-12B-it-exl3/tree/2.50bpw)