Demo for this model on Spaces
Hi @autotrust 🤗
I'm Apolinario, from the open-source team at Hugging Face. Congrats and thanks for open-sourcing autotrust/JEV-9B on the Hub! We were excited about this work and built with an agent an interactive demo app of it on Hugging Face Spaces, running on a free ZeroGPU infrastructure.
Here's a link to the demo: https://huggingface.co/spaces/hugging-apps/jev-9b-decision-demo
We would love to transfer this demo to you or your organization. Would you like this demo to live under your own account or organization? If so just let me know here which username to transfer to, and we'll transfer the Space over to you, we hope it can give your work more visibility, discoverability and allows folks to try it out.
(If you have any questions or just want to chat more about this, you can find me on Twitter, LinkedIn or apolinario @ huggingface.co)
Cheers,
Poli
Hi Poli @multimodalart 🤗
Wow, thank you, Poli, and the whole Hugging Face open-source team! Seeing JEV running as a live, interactive demo on ZeroGPU is a real thrill for us.
Yes please, we'd love to have the Space transferred to our organization: autotrust (https://huggingface.co/autotrust).
And great timing: we've just released the upgrade, JEV-27B 👉 https://huggingface.co/autotrust/JEV-27B
Same recipe and API as the 9B, now on a Qwen3.8-27B backbone:
• Mean KL to the teacher 0.019, noul AUROC 0.996, calibration ECE 0.0009
• Out-of-distribution KL halved vs. the 9B (0.234 → 0.104)
• 96–98% of the teacher's accuracy on an independent, human-labelled benchmark it never saw in training
• Generation head untouched: HumanEval 78.0%, byte-identical to the base model
• Apache-2.0, runs on vLLM, ~140 ms per request on a single B200
If ZeroGPU can host the 27B as well, we'd love to add it to the demo so people can try both sizes side by side.
Thanks again for the support, it means a lot to a small lab like ours! 🚀
Cheers,
Josh Liu
AutoTrust AI
Hi Poli @multimodalart 🤗
Quick follow-up with some exciting news! Our team just ran JEV-27B through all the relevant benchmarks, and the results are in (chart and a short demo reel attached 📊):
• Across six benchmarks (JevBench, Kev, OpenJev text, Nimble, VitaminC, MASSIVE-en), our open-weights JEV-27B averages 84.07%. That's ahead of the closed TypeSafe Jev 1.13 teacher itself (83.85%), and it beats the teacher on 4 of the 6.
• It also leads every other notable open-source JEV-style model we compared: NeoHorse-Jev, Open-Jev, Kev and Laya English. Its average is 6+ points clear of the closest one.
We're really proud that an open, Apache-2.0 student now edges past its own closed teacher on average!
We've also put up a live demo so anyone can try it 👉 https://huggingface.co/spaces/autotrust/JEV-27B-Demo
Model: https://huggingface.co/autotrust/JEV-27B
Please check it out, we'd love to hear what you think! And whenever it's convenient, the JEV-9B Space can go to our autotrust org as mentioned.
Thanks again for all the support 🚀
Cheers,
Josh Liu
AutoTrust AI
Hey @autotrustailab !
Thank you for open sourcing JEV-9B!
Transferred to: https://huggingface.co/spaces/autotrust/jev-9b-decision-demo with a ZeroGPU grant
Free to post about, put the demo on the project page, reference it on repos, etc. as you wish
Also feel more than free to take ownership and make modifications as you see fit. For future releases from you, would be great if they already came with a demos! You can use this one as a blueprint to build by yourself or with the help of an agent, you can load the huggingface-spaces skill on Claude Code, Codex, Hermes, Pi, etc.
Cheers,
Poli
Thanks so much, Poli @multimodalart ! 🤗 Transfer received, and thank you for the ZeroGPU grant! Great advice on demos: JEV-27B already has one (https://huggingface.co/spaces/autotrust/JEV-27B-Demo), and every future release will too. Much appreciated! 🚀
Hi Poli @multimodalart
cc: @cloudyu
I'm Josh Liu, co-founder of AutoTrust AI in Singapore. Thanks for transferring JEV-98 demo to us. Also congratulations on building the Jev Decision Index which is the clearest head-to-head view of the open Jev reproductions we've found.
We'd love to see how our two Apache-2.0 open-weight Jev students score on your frozen suite:
JEV-9B: https://huggingface.co/autotrust/JEV (Qwen3.5-9B backbone)
JEV-27B: https://huggingface.co/autotrust/JEV-27B (Qwen3.8-27B backbone; also serves text generation from the untouched base lm_head)
Both were distilled from TypeSafe Jev 1.13's full output distributions (SargeDev/jev-distill-corpus-v3) by training a LoRA plus a 24-slot decision head. On the 29,955-question held-out set, AutoTrust JEV-27B picks the same option as Jev 1.13 on 90.3% of choice questions (mean KL 0.019). In our re-run of gazelle93's decision-models-under-pressure, it scores 0.740 at 16 options against Jev 1.13's published 0.769.
Following your note in the Space discussions, we'll self-score with apolinario/decision-index: run the full frozen suite, publish the run directories as a Hub dataset, and open a PR adding both models to submissions/README.md. Two things to flag up front:
Our server exposes /v1/decisions rather than /v1/systemone, so the PR will include a small Engine subclass to make the run reproducible.
The choice head takes at most 16 options. Questions with more options, and inputs beyond our configured context length, will be declared Unsupported rather than truncated.
Does that approach work for you? In the meantime, could both models go on the News tab under Trained? Happy to send that PR as well.
Best,
Josh Liu
Chairman & Co-Founder, AutoTrust AI