Korean Hallucination Leaderboard
๐ฅ
View Korean Hallucination Resistance Leaderboard
ginipick
ginipick
AI & ML interests
None yet
Recent Activity
updated a Space about 10 hours ago
ginigen/Korean-Hallucination-Leaderboard published a Space about 10 hours ago
ginigen/Korean-Hallucination-Leaderboard updated a dataset about 10 hours ago
ginigen/Korean-Hallucination-BenchOrganizations
updated a
Space about 10 hours ago
published a
Space about 10 hours ago
updated a
dataset about 10 hours ago
published a
dataset about 10 hours ago
updated a
Space about 13 hours ago
liked a
model 3 days ago
upvoted an article 3 days ago
Article
FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods
FINAL-Bench
โข โข 17 reacted to SeaWolf-AI's post with ๐ฅ 3 days ago
Post
6705
๐ Introducing FINAL-Bench Quantum โ an open, neutral benchmark that finally puts quantum-computing methods on one fair yardstick.
Quantum results are notoriously hard to compare. The same "logical error rate" or "query fidelity" means very different things depending on the code, noise model, hardware, and shot count. FINAL-Bench Quantum fixes that: five events judged under identical, published protocols, where every number is labeled as either measured here or quoted from a source.
Five events: โ QEC Decoder โก Optimization (Max-Cut) โข VQE โฃ QRAM โค Quantum Simulation
The rules are simple and strict:
โ Track A (measured here, with 95% confidence intervals) is kept separate from Track B (quoted from papers, not directly comparable).
๐ฌ Simulation and real hardware are clearly distinguished, and no quantum-advantage claims are made.
๐ Methods from Google, IBM, NVIDIA, USTC, Riverlane and more sit side by side, with origin flags and author credits.
๐ค Anyone can submit their own method via the Submit tab for review and listing.
Already on the board: real IBM Heron r2 measurements (repetition-code distance boundary, 29โ175ร error reduction from d3 to d5), a real-chip QRAM query fidelity of 0.92, and Hโ VQE at chemical accuracy โ always labeled honestly as simulation vs hardware.
A leaderboard is only useful if you can trust it, so neutrality is the whole point: strong competitors stay in even when they beat the host, sources are quoted faithfully, and a simulation is never rounded up into a hardware claim.
Leaderboard: FINAL-Bench/quantum-bench-leaderboard
Article: https://huggingface.co/blog/FINAL-Bench/quantum-leaderboard
#quantum #QEC #QuantumComputing #benchmark
Quantum results are notoriously hard to compare. The same "logical error rate" or "query fidelity" means very different things depending on the code, noise model, hardware, and shot count. FINAL-Bench Quantum fixes that: five events judged under identical, published protocols, where every number is labeled as either measured here or quoted from a source.
Five events: โ QEC Decoder โก Optimization (Max-Cut) โข VQE โฃ QRAM โค Quantum Simulation
The rules are simple and strict:
โ Track A (measured here, with 95% confidence intervals) is kept separate from Track B (quoted from papers, not directly comparable).
๐ฌ Simulation and real hardware are clearly distinguished, and no quantum-advantage claims are made.
๐ Methods from Google, IBM, NVIDIA, USTC, Riverlane and more sit side by side, with origin flags and author credits.
๐ค Anyone can submit their own method via the Submit tab for review and listing.
Already on the board: real IBM Heron r2 measurements (repetition-code distance boundary, 29โ175ร error reduction from d3 to d5), a real-chip QRAM query fidelity of 0.92, and Hโ VQE at chemical accuracy โ always labeled honestly as simulation vs hardware.
A leaderboard is only useful if you can trust it, so neutrality is the whole point: strong competitors stay in even when they beat the host, sources are quoted faithfully, and a simulation is never rounded up into a hardware claim.
Leaderboard: FINAL-Bench/quantum-bench-leaderboard
Article: https://huggingface.co/blog/FINAL-Bench/quantum-leaderboard
#quantum #QEC #QuantumComputing #benchmark
- 2 replies
liked a
Space 3 days ago
liked a
model 4 days ago
liked a
model 10 days ago
reacted to SeaWolf-AI's post with ๐ง 20 days ago
Post
4245
Darwin-60B-DUO: Two SOTAs, One Endpoint โ 88.38% on GPQA Diamond ๐
We're excited to release Darwin-60B-DUO, the Darwin family's first DUO model. Take two domain-verified specialists, hide them behind a single OpenAI-compatible endpoint, and let a router decide which one (or both) answers. You see one model, one API โ but get the best of both.
The number that matters: on the full 198-question GPQA Diamond, Darwin-60B-DUO hits 88.38%. The constituents alone land at 69.70% (Darwin-28B-REASON) and 77.27% (AWAXIS-Think-31B); a naive cascade only reaches 83.84%. The DUO clears them all. Two small specialists, intelligently routed, beat one big generalist on cost and quality. Both are independently verified โ Darwin-28B-REASON is #3 on the HF GPQA Diamond leaderboard, AWAXIS-Think-31B is #1 on Korea's national K-AI Leaderboard (MSIT).
The brains is a Hybrid-A router picking one of five strategies on the fly. Korean โ AWAXIS, English/STEM โ Darwin (single-backend, ~70% of traffic at 1ร cost). When a Korean answer needs rigorous English reasoning, split_refine fires โ Darwin drafts, AWAXIS polishes; MCQ/short-answer runs both with self-consistency + cross-verify. Net effective cost: only ~1.3ร a single 30B model.
The part the community will care about: the gateway is model-agnostic and Apache-2.0. Point it at any two OpenAI-compatible backends and you've got a DUO in minutes โ teach router.py when to use which, and parallel calls, response merging, and routing transparency via _duo_route are handled for you. Fork it and tell us what you built.
Painless deploy: docker compose up for both vLLM backends + gateway; FP8 ~30GB colocates on a single B200/H100. One git clone (~120GB). Text-only for now, streaming in v1.1.
Two SOTAs, one endpoint. Come build your own on the Community tab.
๐
๐ FINAL-Bench/Darwin-60B-DUO
We're excited to release Darwin-60B-DUO, the Darwin family's first DUO model. Take two domain-verified specialists, hide them behind a single OpenAI-compatible endpoint, and let a router decide which one (or both) answers. You see one model, one API โ but get the best of both.
The number that matters: on the full 198-question GPQA Diamond, Darwin-60B-DUO hits 88.38%. The constituents alone land at 69.70% (Darwin-28B-REASON) and 77.27% (AWAXIS-Think-31B); a naive cascade only reaches 83.84%. The DUO clears them all. Two small specialists, intelligently routed, beat one big generalist on cost and quality. Both are independently verified โ Darwin-28B-REASON is #3 on the HF GPQA Diamond leaderboard, AWAXIS-Think-31B is #1 on Korea's national K-AI Leaderboard (MSIT).
The brains is a Hybrid-A router picking one of five strategies on the fly. Korean โ AWAXIS, English/STEM โ Darwin (single-backend, ~70% of traffic at 1ร cost). When a Korean answer needs rigorous English reasoning, split_refine fires โ Darwin drafts, AWAXIS polishes; MCQ/short-answer runs both with self-consistency + cross-verify. Net effective cost: only ~1.3ร a single 30B model.
The part the community will care about: the gateway is model-agnostic and Apache-2.0. Point it at any two OpenAI-compatible backends and you've got a DUO in minutes โ teach router.py when to use which, and parallel calls, response merging, and routing transparency via _duo_route are handled for you. Fork it and tell us what you built.
Painless deploy: docker compose up for both vLLM backends + gateway; FP8 ~30GB colocates on a single B200/H100. One git clone (~120GB). Text-only for now, streaming in v1.1.
Two SOTAs, one endpoint. Come build your own on the Community tab.
๐
๐ FINAL-Bench/Darwin-60B-DUO
liked a
model 20 days ago
liked a
Space about 1 month ago
upvoted a paper about 1 month ago
updated a
Space about 1 month ago
updated a
Space 2 months ago
published a
Space 2 months ago
reacted to SeaWolf-AI's post with ๐ 2 months ago
Post
3056
Why This Matters โ David Defeats Goliath
MODEL: FINAL-Bench/Darwin-4B-David
SPACE: FINAL-Bench/Darwin-4B-david
We're releasing Darwin-4B-David, the first second-generation model in the Darwin Opus family. By evolving an already-evolved model, it achieves 85.0% on GPQA Diamond โ surpassing its 58.6% original ancestor and even gemma-4-31B (84.3%) โ with just 4.5B parameters.
Second-Generation Evolution
Most merges start from a base model and produce a single offspring. Darwin-4B-David breaks this pattern. The Father (Darwin-4B-Opus) was already evolved from gemma-4-E4B-it with Claude Opus reasoning distillation โ a Gen-1 model. The Mother (DavidAU's DECKARD-Expresso-Universe) brings Unsloth deep tuning across 5 in-house datasets with thinking mode by default. Crossbreeding these two produced the first Gen-2 Darwin model.
Darwin V6's Model MRI scanned both parents across all 42 layers, assigning independent optimal ratios per layer. The Mother's creativity and Korean language hotspot (Layer 22-25, weight 0.95) was maximally absorbed, while the Father's reasoning core (Layer 30-40, weight 0.48) was preserved. This is "Merge = Evolve" applied recursively โ evolution of evolution.
Benchmarks
Darwin-4B-David scores 85.0% on GPQA Diamond (+26.4%p over original 58.6%), evaluated generatively with maj@8 (8 generations per question, majority vote), Epoch AI prompt format, thinking mode enabled, 50 sampled questions. On ARC-Challenge (25-shot, loglikelihood), both score 64.93% โ expected, as loglikelihood doesn't capture thinking-mode reasoning differences.
Why This Matters
gemma-4-31B (30.7B) scores 84.3%. Darwin-4B-David surpasses it at 1/7th the size โ no training, no RL, just 45 minutes of MRI-guided DARE-TIES on one H100. The name "David" honors Mother creator DavidAU and evokes David vs. Goliath.
MODEL: FINAL-Bench/Darwin-4B-David
SPACE: FINAL-Bench/Darwin-4B-david
We're releasing Darwin-4B-David, the first second-generation model in the Darwin Opus family. By evolving an already-evolved model, it achieves 85.0% on GPQA Diamond โ surpassing its 58.6% original ancestor and even gemma-4-31B (84.3%) โ with just 4.5B parameters.
Second-Generation Evolution
Most merges start from a base model and produce a single offspring. Darwin-4B-David breaks this pattern. The Father (Darwin-4B-Opus) was already evolved from gemma-4-E4B-it with Claude Opus reasoning distillation โ a Gen-1 model. The Mother (DavidAU's DECKARD-Expresso-Universe) brings Unsloth deep tuning across 5 in-house datasets with thinking mode by default. Crossbreeding these two produced the first Gen-2 Darwin model.
Darwin V6's Model MRI scanned both parents across all 42 layers, assigning independent optimal ratios per layer. The Mother's creativity and Korean language hotspot (Layer 22-25, weight 0.95) was maximally absorbed, while the Father's reasoning core (Layer 30-40, weight 0.48) was preserved. This is "Merge = Evolve" applied recursively โ evolution of evolution.
Benchmarks
Darwin-4B-David scores 85.0% on GPQA Diamond (+26.4%p over original 58.6%), evaluated generatively with maj@8 (8 generations per question, majority vote), Epoch AI prompt format, thinking mode enabled, 50 sampled questions. On ARC-Challenge (25-shot, loglikelihood), both score 64.93% โ expected, as loglikelihood doesn't capture thinking-mode reasoning differences.
Why This Matters
gemma-4-31B (30.7B) scores 84.3%. Darwin-4B-David surpasses it at 1/7th the size โ no training, no RL, just 45 minutes of MRI-guided DARE-TIES on one H100. The name "David" honors Mother creator DavidAU and evokes David vs. Goliath.
reacted to SeaWolf-AI's post with ๐ฅ 3 months ago
Post
4696
๐ World Model Bench โ does your world model actually think?
FID measures realism. FVD measures smoothness. But neither tells you whether the model understood the scene.
We just released WM Bench โ the first benchmark for cognitive intelligence in world models. The core question: when a beast charges from 3 meters away, does the model know to sprint โ not walk? Does it respond differently to a human vs an animal? Does it remember the left corridor was blocked two steps ago?
Those are cognitive questions. No existing benchmark asks them. So we built one.
3 Pillars ยท 10 Categories ยท 100 Scenarios ยท 1,000-point scale
- ๐ P1 Perception (25%) โ Can it read the scene?
- ๐ง P2 Cognition (45%) โ Does it predict threats, escalate emotions, utilize memory?
- ๐ฅ P3 Embodiment (30%) โ Does the body respond with the right motion?
All evaluation is via simple JSON I/O โ no 3D engine, no special hardware. Any model with an API can participate.
We also built PROMETHEUS as a live reference implementation โ runs in your browser on a T4, no install needed. Combines FloodDiffusion motion generation with a LLM cognitive brain (Perceive โ Predict โ Decide โ Act). Scored 726/1000 (Grade B) on Track C โ the only directly verified model so far. Submissions from other teams very welcome.
---
๐ Dataset โ FINAL-Bench/World-Model
๐ Demo โ FINAL-Bench/World-Model
๐ Leaderboard โ FINAL-Bench/worldmodel-bench
๐ Article โ https://huggingface.co/blog/FINAL-Bench/world-model
Part of the FINAL Bench Family โ alongside FINAL Bench (Feb 2026). Feedback on rubrics and missing models always welcome!
FID measures realism. FVD measures smoothness. But neither tells you whether the model understood the scene.
We just released WM Bench โ the first benchmark for cognitive intelligence in world models. The core question: when a beast charges from 3 meters away, does the model know to sprint โ not walk? Does it respond differently to a human vs an animal? Does it remember the left corridor was blocked two steps ago?
Those are cognitive questions. No existing benchmark asks them. So we built one.
3 Pillars ยท 10 Categories ยท 100 Scenarios ยท 1,000-point scale
- ๐ P1 Perception (25%) โ Can it read the scene?
- ๐ง P2 Cognition (45%) โ Does it predict threats, escalate emotions, utilize memory?
- ๐ฅ P3 Embodiment (30%) โ Does the body respond with the right motion?
All evaluation is via simple JSON I/O โ no 3D engine, no special hardware. Any model with an API can participate.
We also built PROMETHEUS as a live reference implementation โ runs in your browser on a T4, no install needed. Combines FloodDiffusion motion generation with a LLM cognitive brain (Perceive โ Predict โ Decide โ Act). Scored 726/1000 (Grade B) on Track C โ the only directly verified model so far. Submissions from other teams very welcome.
---
๐ Dataset โ FINAL-Bench/World-Model
๐ Demo โ FINAL-Bench/World-Model
๐ Leaderboard โ FINAL-Bench/worldmodel-bench
๐ Article โ https://huggingface.co/blog/FINAL-Bench/world-model
Part of the FINAL Bench Family โ alongside FINAL Bench (Feb 2026). Feedback on rubrics and missing models always welcome!