Fix: link real GitHub repo instead of dangling local paths
Browse files- README.md +11 -4
- benchmark_results.json +2 -2
README.md
CHANGED
|
@@ -24,8 +24,12 @@ cite both papers if you use this model** (citations below).
|
|
| 24 |
**What Sajal Labs added**: a from-scratch native C++ port of the encoder
|
| 25 |
and the WordPiece tokenizer — no PyTorch, no `transformers`, no Python at
|
| 26 |
inference time — with rigorous equivalence and benchmark validation against
|
| 27 |
-
the original. See
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
## Model details (unchanged from the original)
|
| 31 |
|
|
@@ -66,12 +70,15 @@ Worth knowing before you read too much into "ONNX Runtime": its own cold-start
|
|
| 66 |
number depends heavily on which tokenizer library it's paired with — using
|
| 67 |
`transformers` for convenience costs ~25x more than using the lean, standalone
|
| 68 |
`tokenizers` library for the exact same token IDs. Native sidesteps that
|
| 69 |
-
whole dependency-choice question by construction. Full discussion in
|
|
|
|
| 70 |
|
| 71 |
**Honest scope note on the warm-loop numbers**: native's advantage is
|
| 72 |
*not* unconditional the way cold-invocation is — Sajal Labs found it
|
| 73 |
depends on model width (`hidden_size`), with a measured crossover around
|
| 74 |
-
`hidden≈250` on this hardware
|
|
|
|
|
|
|
| 75 |
sits comfortably below that, so native keeps a real warm-loop edge too —
|
| 76 |
but that's a property of this model's size, not a general claim.
|
| 77 |
|
|
|
|
| 24 |
**What Sajal Labs added**: a from-scratch native C++ port of the encoder
|
| 25 |
and the WordPiece tokenizer — no PyTorch, no `transformers`, no Python at
|
| 26 |
inference time — with rigorous equivalence and benchmark validation against
|
| 27 |
+
the original. See [Sajal Labs](https://github.com/Sajalmadan09/sajal-labs),
|
| 28 |
+
experiments
|
| 29 |
+
[exp11](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp11-real-pretrained-transformer)
|
| 30 |
+
through
|
| 31 |
+
[exp14](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp14-wordpiece-native),
|
| 32 |
+
for full methodology.
|
| 33 |
|
| 34 |
## Model details (unchanged from the original)
|
| 35 |
|
|
|
|
| 70 |
number depends heavily on which tokenizer library it's paired with — using
|
| 71 |
`transformers` for convenience costs ~25x more than using the lean, standalone
|
| 72 |
`tokenizers` library for the exact same token IDs. Native sidesteps that
|
| 73 |
+
whole dependency-choice question by construction. Full discussion in
|
| 74 |
+
[exp14](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp14-wordpiece-native).
|
| 75 |
|
| 76 |
**Honest scope note on the warm-loop numbers**: native's advantage is
|
| 77 |
*not* unconditional the way cold-invocation is — Sajal Labs found it
|
| 78 |
depends on model width (`hidden_size`), with a measured crossover around
|
| 79 |
+
`hidden≈250` on this hardware
|
| 80 |
+
([exp12](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp12-warm-latency-width-depth)/[exp13](https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp13-width-threshold)).
|
| 81 |
+
This model's `hidden=128`
|
| 82 |
sits comfortably below that, so native keeps a real warm-loop edge too —
|
| 83 |
but that's a property of this model's size, not a general claim.
|
| 84 |
|
benchmark_results.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
-
"hardware": "Apple M4, 10 cores, 16GB RAM, macOS (see research/environment.md)",
|
| 3 |
-
"methodology": "research/experiments/exp11-real-pretrained-transformer/results.md, exp12-warm-latency-width-depth/results.md, exp13-width-threshold/results.md, exp14-wordpiece-native/results.md",
|
| 4 |
"equivalence_native_vs_pytorch": {
|
| 5 |
"note": "native encoder + tokenizer vs. HF BertModel + BertTokenizerFast, 10 real sentences (lengths 4-25 tokens)",
|
| 6 |
"hidden_state_max_abs_error": 9.5367431640625e-06,
|
|
|
|
| 1 |
{
|
| 2 |
+
"hardware": "Apple M4, 10 cores, 16GB RAM, macOS (see https://github.com/Sajalmadan09/sajal-labs/tree/main/research/environment.md)",
|
| 3 |
+
"methodology": "https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp11-real-pretrained-transformer/results.md, https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp12-warm-latency-width-depth/results.md, https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp13-width-threshold/results.md, https://github.com/Sajalmadan09/sajal-labs/tree/main/research/experiments/exp14-wordpiece-native/results.md",
|
| 4 |
"equivalence_native_vs_pytorch": {
|
| 5 |
"note": "native encoder + tokenizer vs. HF BertModel + BertTokenizerFast, 10 real sentences (lengths 4-25 tokens)",
|
| 6 |
"hidden_state_max_abs_error": 9.5367431640625e-06,
|