fusion-embedding-2-k3-vision

Kimi K3's vision encoder, made language-searchable.

This is a small trained projector that maps Kimi K3's frozen vision encoder (MoonViT-V2, about 401M parameters) into the frozen fusion-embedding-2 text space. K3's own visual features become searchable in plain language, and land in the same space as fusion-embedding's text, image, video, and audio, plus its thermal (Ember) and motion (Tremor) sensor packs.

It builds on only the ~401M vision tower of Kimi K3, never the 2.8T language model. The tower runs on a single GPU. The projector is the only trained weight, about 3.15M parameters.

Results

Text-to-image retrieval on Flickr30k (test), 1000-image gallery, querying in the fusion-embedding-2 text space:

Metric	fusion-embedding-2-k3-vision	same recipe, DINOv2 backbone
R@1	0.432	0.318
R@5	0.740	0.613
R@10	0.836	0.749

MoonViT-V2 was trained alongside a language model, so its features align to text more readily than a self-supervised vision backbone. On the identical projector recipe it beats the DINOv2 version by nearly 9 points at R@10.

Usage

The projector ships here; the vision tower is pulled from moonshotai/Kimi-K3 on first use (only the vision_tower.* tensors, never the 2.8T language model).

from inference import K3VisionEmbedder
m = K3VisionEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-k3-vision")
v = m.embed_image("photo.jpg")             # 2048-d, in the fusion-embedding-2 text space

Query it with text using fusion-embedding-2 itself:

from inference import FusionEmbedder      # from the fusion-embedding-2 repo
fe = FusionEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-2b-preview")
q = fe.embed_text("a dog on a beach")
score = float(q @ v)                       # cosine, both unit-norm

How it works

1. Kimi K3's MoonViT-V2 vision tower (frozen) encodes an image into patch tokens (1024-d).
2. The tokens are pooled into one image descriptor.
3. A trained projector (LayerNorm -> Linear -> GELU -> Dropout -> Linear) maps that descriptor into the 2048-d fusion-embedding-2 text space.
4. Trained with image-caption InfoNCE on Flickr30k. Same recipe as the Fusion Perception DINO to fusion-embedding projector, with K3's vision tower as the backbone.

What this is and is not

• It is a projector on a frozen vision encoder, not a new foundation model, and not a fine-tune of Kimi K3's language model.
• The base_model tag points at moonshotai/Kimi-K3 because the vision tower is a component of that model. We use only MoonViT-V2, not the 2.8T language model.
• It is a research preview.

License and attribution

The projector weights and code in this repository are released under CC-BY-NC-4.0.

The vision tower is part of Kimi K3 (Moonshot AI), used under the Kimi K3 License, which permits derivative works with its notice retained. This model is a derivative that uses K3's frozen MoonViT-V2 vision encoder. Copyright 2026 Moonshot AI.

Family

• fusion-embedding-2 (the base multimodal embedding model)
• Ember (thermal) and Tremor (motion) sensor packs
• Cookbook: give a Kimi K3 agent a multimodal memory
• Code and paper: github.com/Eximius-Labs, arXiv:2607.18666