sajalmadan09 commited on
Commit
efa0a2e
·
verified ·
1 Parent(s): 221ca55

Fix: link real GitHub repo instead of dangling local paths

Browse files
Files changed (2) hide show
  1. README.md +11 -4
  2. benchmark_results.json +2 -2
README.md CHANGED
@@ -24,8 +24,12 @@ cite both papers if you use this model** (citations below).
24
  **What Sajal Labs added**: a from-scratch native C++ port of the encoder
25
  and the WordPiece tokenizer — no PyTorch, no `transformers`, no Python at
26
  inference time — with rigorous equivalence and benchmark validation against
27
- the original. See `research/experiments/exp11-real-pretrained-transformer`
28
- through `exp14-wordpiece-native` in the Sajal Labs repo for full methodology.
 
 
 
 
29
 
30
  ## Model details (unchanged from the original)
31
 
@@ -66,12 +70,15 @@ Worth knowing before you read too much into "ONNX Runtime": its own cold-start
66
  number depends heavily on which tokenizer library it's paired with — using
67
  `transformers` for convenience costs ~25x more than using the lean, standalone
68
  `tokenizers` library for the exact same token IDs. Native sidesteps that
69
- whole dependency-choice question by construction. Full discussion in exp14.
 
70
 
71
  **Honest scope note on the warm-loop numbers**: native's advantage is
72
  *not* unconditional the way cold-invocation is — Sajal Labs found it
73
  depends on model width (`hidden_size`), with a measured crossover around
74
- `hidden≈250` on this hardware (exp12/exp13). This model's `hidden=128`
 
 
75
  sits comfortably below that, so native keeps a real warm-loop edge too —
76
  but that's a property of this model's size, not a general claim.
77
 
 
24
  **What Sajal Labs added**: a from-scratch native C++ port of the encoder
25
  and the WordPiece tokenizer — no PyTorch, no `transformers`, no Python at
26
  inference time — with rigorous equivalence and benchmark validation against
27
+ the original. See [Sajal Labs](https://github.com/Sajalmadan09/sajal-labs),
28
+ experiments
29
+ [exp11](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp11-real-pretrained-transformer)
30
+ through
31
+ [exp14](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp14-wordpiece-native),
32
+ for full methodology.
33
 
34
  ## Model details (unchanged from the original)
35
 
 
70
  number depends heavily on which tokenizer library it's paired with — using
71
  `transformers` for convenience costs ~25x more than using the lean, standalone
72
  `tokenizers` library for the exact same token IDs. Native sidesteps that
73
+ whole dependency-choice question by construction. Full discussion in
74
+ [exp14](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp14-wordpiece-native).
75
 
76
  **Honest scope note on the warm-loop numbers**: native's advantage is
77
  *not* unconditional the way cold-invocation is — Sajal Labs found it
78
  depends on model width (`hidden_size`), with a measured crossover around
79
+ `hidden≈250` on this hardware
80
+ ([exp12](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp12-warm-latency-width-depth)/[exp13](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp13-width-threshold)).
81
+ This model's `hidden=128`
82
  sits comfortably below that, so native keeps a real warm-loop edge too —
83
  but that's a property of this model's size, not a general claim.
84
 
benchmark_results.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
- "hardware": "Apple M4, 10 cores, 16GB RAM, macOS (see research/environment.md)",
3
- "methodology": "research/experiments/exp11-real-pretrained-transformer/results.md, exp12-warm-latency-width-depth/results.md, exp13-width-threshold/results.md, exp14-wordpiece-native/results.md",
4
  "equivalence_native_vs_pytorch": {
5
  "note": "native encoder + tokenizer vs. HF BertModel + BertTokenizerFast, 10 real sentences (lengths 4-25 tokens)",
6
  "hidden_state_max_abs_error": 9.5367431640625e-06,
 
1
  {
2
+ "hardware": "Apple M4, 10 cores, 16GB RAM, macOS (see https://github.com/Sajalmadan09/sajal-labs/tree/main/research/environment.md)",
3
+ "methodology": "https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp11-real-pretrained-transformer/results.md, https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp12-warm-latency-width-depth/results.md, https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp13-width-threshold/results.md, https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp14-wordpiece-native/results.md",
4
  "equivalence_native_vs_pytorch": {
5
  "note": "native encoder + tokenizer vs. HF BertModel + BertTokenizerFast, 10 real sentences (lengths 4-25 tokens)",
6
  "hidden_state_max_abs_error": 9.5367431640625e-06,