flowchart-processing
Browse files
README.md
CHANGED
|
@@ -37,9 +37,26 @@ Thank you so much, "crazy chef" and "mad scientist", **Nitral**!
|
|
| 37 |
|
| 38 |
<br>
|
| 39 |
|
| 40 |
-
```
|
| 41 |
# Using the latest llama.cpp ...
|
| 42 |
release version at the time: b6258.
|
| 43 |
# Imatrix was based on the full ...
|
| 44 |
FP16 precision GGUF.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
```
|
|
|
|
| 37 |
|
| 38 |
<br>
|
| 39 |
|
| 40 |
+
```text
|
| 41 |
# Using the latest llama.cpp ...
|
| 42 |
release version at the time: b6258.
|
| 43 |
# Imatrix was based on the full ...
|
| 44 |
FP16 precision GGUF.
|
| 45 |
+
|
| 46 |
+
START: BF16 HuggingFace Model
|
| 47 |
+
↓
|
| 48 |
+
(1) Conversion to Full-Precision GGUF
|
| 49 |
+
↓
|
| 50 |
+
FP16 GGUF (for Calibration Imatrix)
|
| 51 |
+
BF16 GGUF (for Quantization)
|
| 52 |
+
↓
|
| 53 |
+
(2) Generate Imatrix (from FP16 GGUF)
|
| 54 |
+
↓
|
| 55 |
+
imatrix.fp16.gguf
|
| 56 |
+
↓
|
| 57 |
+
(3) Quantize with Imatrix (using BF16 GGUF)
|
| 58 |
+
↓
|
| 59 |
+
Final Quantized GGUF Models
|
| 60 |
+
↓
|
| 61 |
+
END
|
| 62 |
```
|