---
license: apache-2.0
base_model: google/embeddinggemma-2
pipeline_tag: feature-extraction
library_name: coreai
tags:
- coreai
- apple
- embeddings
- multimodal
- image-feature-extraction
- audio-feature-extraction
- video-feature-extraction
- experimental
---

# EmbeddingGemma 2 — Apple Core AI

Apple Core AI `.aimodel` conversion of [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2/tree/914f7f89142e33e77833254d9c9b90c3cef7303b), revision `914f7f89142e33e77833254d9c9b90c3cef7303b`, Apache-2.0. This is the new 740M multimodal model, distinct from EmbeddingGemma 300M.

Supports text/code, images, video frames, audio, and interleaved combinations within the limits below. Returns normalized 768-dimensional embeddings. All neural inference, projection, masked mean pooling and initial L2 normalization run in Core AI. The host uses Transformers for tokenization/media preprocessing and NumPy for feature placement and optional truncated-vector normalization. No source PyTorch checkpoint is needed for inference.

## Limits and platform

- Batch size one; **1,024 combined text and media tokens**, including special tokens. The source supports 8,192; this conversion is a fixed smaller export. Oversized inputs raise an error instead of being truncated.
- Vision: 2,520 patches per image/frame, at most 280 soft tokens. Preserves the upstream aspect-ratio-aware image processor and padding. Each image/frame is processed sequentially; total context capacity still applies.
- Audio: one mono 16 kHz clip per `encode` call, maximum **11.2 seconds** (1,120 mel frames / 280 soft tokens). No resampling in the adapter. Longer clips must be handled separately by the caller; this does not implement the source's minute-scale audio context.
- Video accepts an already sampled NumPy `[frames,height,width,channels]` array. Validation used two frames. Decode and sample video files before calling the adapter; no file decoding, temporal sampling or audio-track extraction is implemented by this adapter. The upstream per-frame image processing is preserved.
- Text task prefixes are preserved via `prompt_name`, e.g. `SearchQuery`, `Document`, `CodeRetrieval`. For mixed content, manually supply placeholder text (`<|image|>`, `<|audio|>`, `<|video|>`) in the desired order and omit `prompt_name`.
- Optional Matryoshka truncation re-normalizes the vector; dimensions 128, 256 and 512 were exercised.
- FP32 inference with the source weights promoted from BF16; assets total approximately 3 GB. Tested on macOS 27.0.1 / Apple Silicon with default Core AI specialization. No independent iOS, ANE or GPU placement claim.
- Experimental community conversion. Numerical fixtures are conversion tests, not a retrieval-quality benchmark.

## Download and run

```bash
hf download maxffarrell/embeddinggemma-2-CoreAI --local-dir embeddinggemma-2-CoreAI
cd embeddinggemma-2-CoreAI
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

```python
import asyncio
import numpy as np
from PIL import Image
from coreai_embeddinggemma2 import EmbeddingGemma2CoreAI

async def main():
    model = await EmbeddingGemma2CoreAI.load(".")
    query = await model.encode("What causes the northern lights?", prompt_name="SearchQuery")
    document = await model.encode("Charged particles from the sun cause auroras.", prompt_name="Document")
    print(float(query @ document))
    image = await model.encode(images=Image.open("photo.jpg"))
    audio = await model.encode(audio=np.zeros(16000, dtype=np.float32), sampling_rate=16000)
    small = await model.encode("An example sentence", truncate_dim=256)

asyncio.run(main())
```

## Validation

`validation.json`: **21 cases** against the full pinned PyTorch FP32 model with native-length source inputs, testing task prefixes, code, Spanish, Japanese, short/long/near-capacity text, images, mixed text/images, two images, short/long/limit audio, text/audio, video, text/video and all four modalities together. Worst elementwise error: **9.76026058e-07**; minimum cosine similarity: **0.999999881**. Predeclared final limits: cosine > 0.99999 and maximum elementwise error < 0.00005, plus finite unit vectors.

`smoke.json`: separate source-checkpoint-free native runtime smoke in five text/media cases with PyTorch neural model loading explicitly disabled, plus invalid-input rejection. Task-independent retrieval accuracy has not been benchmarked.

## Reproduce

From a parent directory containing this bundle under `bundles/embeddinggemma-2-CoreAI`, download the pinned source into `upstream/embeddinggemma-2`, install `requirements.txt`, and run `convert_gemma2.py`, `validate_gemma2.py`, then `smoke_gemma2.py`. The validation/smoke scripts default to that directory layout. The converter uses the exact official Transformers commit in requirements because stable 5.17 predates support. `coreai-torch` 0.4.3 warns it was validated through PyTorch 2.13; this export was tested on 2.14.

Graphs: `embedding.aimodel` (text backbone + projection + pooling + normalization), `vision.aimodel` (vision encoder + projection + valid-token mask), `audio.aimodel` (audio encoder + projection + valid-token mask). Integer inputs are int32. `manifest.json` declares signatures, static shapes and output names. An equivalent slice/stack decomposition replaces unsupported `aten.unfold` in audio; the source weights are unchanged. The text mask preserves the source's inclusive bidirectional sliding window.

`SOURCE.json` records the verified upstream weight hash; `SHA256SUMS` covers every release file except itself. Original model documentation is in `UPSTREAM_README.md`. For Clef Flash, an existing Core AI conversion was found and verified to use current source weight hashes; see `RELATED_MODEL_RESEARCH.md`.