kshitijthakkar/moe-141m-89m-8x2-10L-small-250m-8exp
0.2B • Updated
Mobile MoE Architecture Search
32 MoE models from 41 experiments exploring expert count, routing, and learning rates for mobile deployment.
-
-
kshitijthakkar/moe-161m-123m-4x2-12L-4exp-large-experts
0.2B • Updated -
kshitijthakkar/moe-198m-114m-8x2-12L-8exp-balanced
0.3B • Updated -
kshitijthakkar/moe-202m-104m-12x2-10L-medium-300m-12exp
0.3B • Updated -
kshitijthakkar/moe-225m-108m-12x2-10L-baseline-330m
0.3B • Updated -
kshitijthakkar/moe-229m-111m-12x2-10L-full-attention-no-gqa
0.3B • Updated -
kshitijthakkar/moe-240m-103m-12x2-16L-deep-narrow-16l
0.3B • Updated -
kshitijthakkar/moe-241m-111m-12x2-12L-balanced-350m-12exp
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-bs2-ctx1024
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-bs2-ctx2048
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-bs2-ctx512
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-bs4-ctx1024
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-bs4-ctx512
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-bs8-ctx512
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr1e-03
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr1e-04
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr1e-05
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr2e-04
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr3e-04
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr3e-05
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr5e-04
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr5e-05
0.3B • Updated -
kshitijthakkar/moe-255m-114m-12x2-12L-full-attention-no-gqa-lr5e-06
0.3B • Updated -
kshitijthakkar/moe-270m-132m-12x2-8L-shallow-wide-8l
0.4B • Updated -
kshitijthakkar/moe-274m-132m-16x4-12L-top4-routing
0.4B • Updated -
kshitijthakkar/moe-284m-119m-12x2-14L-aggressive-gqa-1kv
0.4B • Updated -
kshitijthakkar/moe-312m-114m-16x2-12L-baseline-390m
0.4B • Updated -
kshitijthakkar/moe-340m-107m-24x2-12L-24exp-specialized
0.4B • Updated -
kshitijthakkar/moe-350m-102m-16x1-12L-top1-routing
0.4B • Updated -
kshitijthakkar/moe-353m-130m-16x2-12L-large-400m-16exp
0.4B • Updated -
kshitijthakkar/moe-415m-147m-16x2-12L-xlarge-450m-16exp
0.5B • Updated -
kshitijthakkar/moe-99m-70m-8x2-8L-tiny-200m-8exp
0.2B • Updated -
kshitijthakkar/moe-inference-benchmark
Viewer • Updated • 256 • 16 • 1