---
library_name: transformers
tags: []
---

Moondream 3 (Preview) is vision language model with a mixture of experts architecture (9B total parameters, 2B active).

Architecture details:

1. 24 layers; the first four are dense, the rest have MoE FFNs with 64 experts, 8 activated per token
2. MoE FFNs have GeGLU architecture, with inner/gate dim of 1024. The model's hidden dim is 2048.
3. Usable context length increased to 32K, with [a custom efficient SuperBPE tokenizer](https://huggingface.co/moondream/starmie-v1)
4. Multi-headed attention with learned position- and data-dependent temperature scaling
5. SigLIP-based vision encoder, with multi-crop channel concatenation for token-efficient high resolution image processing

For more details, please refer to our ||coming soon release blog post||. Or try the model out in our [playground demo](https://moondream.ai/c/playground).

## Usage

Load the model and prepare it for inference. We use [FlexAttention for inference](https://pytorch.org/blog/flexattention-for-inference/), so calling `.compile()` is critical for fast decoding. Our `compile` implementation also handles warmup, so you can start making requests directly once it returns.

```python
import torch
from transformers import AutoModelForCausalLM

moondream = AutoModelForCausalLM.from_pretrained(
    "moondream/moondream3-preview",
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map={"": "cuda"},
)
moondream.compile()
```

The model comes with four skills, tailored towards different visual understanding tasks.

### Query

The `query` skill can be used to ask open-ended questions about images.

||TK -- code example for simple VQA||

By default, `query` runs in reasoning mode, allowing the model to "think" about the question before generating an answer. This is helpful for more complicated tasks, but sometimes the task you're running is simple and doesn't benefit from reasoning. To save on inference cost when this is the case, you can disable reasoning:

||TK -- example without reasoning||

If you want to stream outputs, pass in `stream=True`. You can control the temperature, top-p, and maximum number of tokens generated by passing in optional settings.

||TK -- stream + settings example||

Note that this isn't just for images; Moondream is also a strong general-purpose text model.

||TK -- text only example||

### Caption

Whether you want short, normal-sized or long descriptions of images, the `caption` skill has you covered.

||TK -- captioning example||

It accepts the same streaming and temperature etc. settings as the `query` skill.

## Point

TK

## Detect

TK