---
license: cc-by-nc-4.0
base_model: moonshotai/Kimi-K3
base_model_relation: adapter
library_name: fusion-embedding
pipeline_tag: feature-extraction
tags:
- multimodal
- embeddings
- retrieval
- vision
- kimi-k3
- moonvit
---
# fusion-embedding-2-k3-vision
Kimi K3's vision encoder, made language-searchable.
[](https://huggingface.co/moonshotai/Kimi-K3)
[](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview)
[](https://creativecommons.org/licenses/by-nc/4.0/)
[](https://github.com/Eximius-Labs/cookbook)
[](https://arxiv.org/abs/2607.18666)
This is a small trained projector that maps Kimi K3's frozen vision encoder
(**MoonViT-V2**, about 401M parameters) into the frozen
[fusion-embedding-2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview) text
space. K3's own visual features become searchable in plain language, and land in the same
space as fusion-embedding's text, image, video, and audio, plus its thermal (Ember) and
motion (Tremor) sensor packs.
It builds on **only the ~401M vision tower of Kimi K3, never the 2.8T language model**. The
tower runs on a single GPU. The projector is the only trained weight, about 3.15M
parameters.
## Results
Text-to-image retrieval on Flickr30k (test), 1000-image gallery, querying in the
fusion-embedding-2 text space:
| Metric | fusion-embedding-2-k3-vision | same recipe, DINOv2 backbone |
| --- | --- | --- |
| R@1 | 0.432 | 0.318 |
| R@5 | 0.740 | 0.613 |
| **R@10** | **0.836** | 0.749 |
MoonViT-V2 was trained alongside a language model, so its features align to text more
readily than a self-supervised vision backbone. On the identical projector recipe it beats
the DINOv2 version by nearly 9 points at R@10.
## Usage
The projector ships here; the vision tower is pulled from `moonshotai/Kimi-K3` on first use
(only the `vision_tower.*` tensors, never the 2.8T language model).
```python
from inference import K3VisionEmbedder
m = K3VisionEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-k3-vision")
v = m.embed_image("photo.jpg") # 2048-d, in the fusion-embedding-2 text space
```
Query it with text using fusion-embedding-2 itself:
```python
from inference import FusionEmbedder # from the fusion-embedding-2 repo
fe = FusionEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-2b-preview")
q = fe.embed_text("a dog on a beach")
score = float(q @ v) # cosine, both unit-norm
```
## Deploy on RunPod
[](https://www.runpod.io/console/hub/Eximius-Labs/k3-vision)
One-click deploy the endpoint from the
[RunPod Hub](https://www.runpod.io/console/hub/Eximius-Labs/k3-vision) (serverless,
scales to zero when idle). Once it is running, call it:
```bash
curl -s https://api.runpod.ai/v2//runsync \
-H "Authorization: Bearer $RUNPOD_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": {"image": ""}}'
```
Kimi K3 is gated, so set `HF_TOKEN` (with the K3 license accepted) on the endpoint.
Returns 2048-d vectors in the fusion-embedding-2 text space; query them with text
through the fusion-embedding-2 endpoint.
## How it works
1. Kimi K3's MoonViT-V2 vision tower (frozen) encodes an image into patch tokens (1024-d).
2. The tokens are pooled into one image descriptor.
3. A trained projector (`LayerNorm -> Linear -> GELU -> Dropout -> Linear`) maps that
descriptor into the 2048-d fusion-embedding-2 text space.
4. Trained with image-caption InfoNCE on Flickr30k. Same recipe as the Fusion Perception
DINO to fusion-embedding projector, with K3's vision tower as the backbone.
## Composability
K3's projected vision does not sit alone. It shares the fusion-embedding-2 space with the Ember
thermal pack and the Tremor motion pack, so one text query reaches across senses in a single index.
Below, each query returns its top match from vision, thermal, and motion at once. Vision and thermal
retrieve cleanly (ask for "a running dog" and it returns a photo of a running dog and a thermal image
of a dog); motion (Tremor) is in the same space, though its text-to-motion retrieval is research
preview.
Real data throughout: Flickr photos (vision, via this model), thermal infrared (Ember), recorded IMU
(Tremor).
## What this is and is not
- It is a projector on a frozen vision encoder, not a new foundation model, and not a
fine-tune of Kimi K3's language model.
- The `base_model` tag points at `moonshotai/Kimi-K3` because the vision tower is a component
of that model. We use only MoonViT-V2, not the 2.8T language model.
- It is a research preview.
## License and attribution
The projector weights and code in this repository are released under **CC-BY-NC-4.0**.
The vision tower is part of **Kimi K3** (Moonshot AI), used under the **Kimi K3 License**,
which permits derivative works with its notice retained. This model is a derivative that
uses K3's frozen MoonViT-V2 vision encoder. Copyright 2026 Moonshot AI.
## Family
- [fusion-embedding-2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview) (the base multimodal embedding model)
- [Ember](https://huggingface.co/EximiusLabs/fusion-embedding-2-ember) (thermal) and [Tremor](https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor) (motion) sensor packs
- Cookbook: [give a Kimi K3 agent a multimodal memory](https://github.com/Eximius-Labs/cookbook)
- Code and paper: [github.com/Eximius-Labs](https://github.com/Eximius-Labs), [arXiv:2607.18666](https://arxiv.org/abs/2607.18666)