---
title: MAOAM GLaMM Object & Material Selection
emoji: 🎯
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 5.46.0
app_file: app.py
pinned: false
python_version: "3.12"
short_description: Click/text object & material selection (GLaMM + SAM ViT-H).
preload_from_hub:
  - MBZUAI/GLaMM-GranD-Pretrained
  - openai/clip-vit-large-patch14-336
  - jpark677/maoam_ckpts
---

# MAOAM: Unified Object & Material Selection with VLMs (GLaMM backend)

Faithful port of the authors' `GLaMM/demo.py` from
[adobe-research/obj-and-mat-selection](https://github.com/adobe-research/obj-and-mat-selection),
adapted for ZeroGPU. Select **objects** or **materials** in an image with star
clicks, a text prompt, or both. Output is a cyan selection-mask overlay plus the
raw binary mask, and a downloadable zip of all artifacts.

## Selection types

- **Material: click** — drop 1-5 star points; `<COLOR>` is auto-filled from the
  first star's auto-contrast color.
- **Material: text** — describe a material (e.g. *shiny chrome metal*). No stars.
- **Material: click + text** — combine a star cue with a material description.
- **Object: text** — a RefCOCO-style object expression (e.g. *the man in a red shirt*).

Regions with the same base material but different colors count as different
materials; lighting/shadow variation of the same material counts as the same.

## Backend

**GLaMM** = LLaVA-Llama (GranD pretrained) as the vision-language model +
**SAM ViT-H** as the grounding / mask encoder. Segmentation is a single
teacher-forced forward pass that emits a `[SEG]` token whose hidden state is
decoded by SAM's mask decoder (no autoregressive generation).

Weights:
- Base VLM: `MBZUAI/GLaMM-GranD-Pretrained`
- MAOAM fine-tuned checkpoint: `jpark677/maoam_ckpts` (`glamm/mp_rank_00_model_states.pt`)
- SAM ViT-H: `sam_vit_h_4b8939.pth` (fetched at runtime; set `SAM_REPO` to use an HF mirror)
- Vision tower / CLIP processor: `openai/clip-vit-large-patch14-336`

## Citation

```bibtex
@inproceedings{park2026maoam,
  title     = {MAOAM: Unified Object and Material Selection with Vision-Language Models},
  author    = {Park, Jaden and Deschaintre, Valentin and Kuen, Jason and
               Liu, Kangning and Georgiev, Iliyan and Singh, Krishna Kumar and
               Lee, Yong Jae and Fischer, Michael},
  booktitle = {ACM SIGGRAPH 2026 Conference Papers},
  year      = {2026},
  publisher = {ACM},
  doi       = {10.1145/3799902.3811186},
}
```

Project page: https://jadenpark0.github.io/project_pages/maoam/

License: **Adobe Research**.