aaardpark commited on
Commit
91dac2a
·
verified ·
1 Parent(s): 9a3c89c

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +29 -1
README.md CHANGED
@@ -48,6 +48,34 @@ On smaller models (7B): GPTQ 3-bit PPL = 12.576, our 3-bit PPL = 6.148. GPTQ is
48
  | Base Q3_K_M (this format) | 2.904 |
49
  | Instruct Q3_K_M | 3.962 |
50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
  ## Why This Quant is Different
52
 
53
  Standard 3-bit quantization (RTN) rounds each weight to the nearest grid point uniformly. This destroys the precise weight values that control multi-step reasoning — GSM8K drops from 90% to 16%.
@@ -55,7 +83,7 @@ Standard 3-bit quantization (RTN) rounds each weight to the nearest grid point u
55
  Our method uses calibration data to identify which weights are critical for model quality, then allocates quantization precision accordingly. The result: 88% GSM8K at 3-bit, nearly matching FP16.
56
 
57
  ## Details
58
- - **Method**: Importance-weighted per-group grid search quantization
59
  - **Group size**: 128
60
  - **Quantization time**: ~20 minutes on a single GPU
61
  - **GGUF format**: Q3_K_M (converted via llama.cpp)
 
48
  | Base Q3_K_M (this format) | 2.904 |
49
  | Instruct Q3_K_M | 3.962 |
50
 
51
+ ## Example Outputs
52
+
53
+ **Game theory proof:**
54
+ > Player 1 chooses a=1. For ANY b chosen by Player 2, Player 1 picks c ≤ b²/4. Discriminant = b² - 4c ≥ b² - b² = 0 for all b. Player 1 has a universal winning strategy.
55
+
56
+ **100 prisoners problem:**
57
+ > Each prisoner follows the cycle starting from their own box number. Success probability ≈ 31% (1 - ln 2). The strategy works because random permutations have no cycle longer than 50 with probability ≈ 0.31.
58
+
59
+ **Math (bat and ball):**
60
+ > The ball costs $0.05. Let x = ball. Bat = x + 1. Total: 2x + 1 = 1.10 → x = 0.05.
61
+
62
+ **Code (Sieve of Eratosthenes):**
63
+ ```python
64
+ def sieve_of_eratosthenes(n: int) -> list[int]:
65
+ if n < 2: return []
66
+ is_prime = [True] * (n + 1)
67
+ is_prime[0] = is_prime[1] = False
68
+ p = 2
69
+ while p * p <= n:
70
+ if is_prime[p]:
71
+ for i in range(p * p, n + 1, p):
72
+ is_prime[i] = False
73
+ p += 1
74
+ return [i for i in range(n + 1) if is_prime[i]]
75
+ ```
76
+
77
+ All generated at ~5 tok/s on Apple Silicon with Metal. 35 GB file.
78
+
79
  ## Why This Quant is Different
80
 
81
  Standard 3-bit quantization (RTN) rounds each weight to the nearest grid point uniformly. This destroys the precise weight values that control multi-step reasoning — GSM8K drops from 90% to 16%.
 
83
  Our method uses calibration data to identify which weights are critical for model quality, then allocates quantization precision accordingly. The result: 88% GSM8K at 3-bit, nearly matching FP16.
84
 
85
  ## Details
86
+ - **Method**: Importance-weighted per-group optimization
87
  - **Group size**: 128
88
  - **Quantization time**: ~20 minutes on a single GPU
89
  - **GGUF format**: Q3_K_M (converted via llama.cpp)