---
license: mit
tags: [clip, embeddings, recommendation, skincare]
---

# DermaGlow Embeddings — Recommendation Engine (Part 3)

Precomputed **image embeddings** for the [DermaGlow dataset](https://huggingface.co/datasets/havaferber/dermaglow-skin-faces),
powering the "people with skin like yours" recommender in the [DermaGlow app](https://huggingface.co/spaces/havaferber/dermaglow).

## Model selection — 3 candidates evaluated

Metric: do the top-5 visually-nearest faces share the same skin labels? (random baseline: 19.6%)

| Model | Mean label match | Dim | Embed time (1,100 imgs) |
|---|---|---|---|
| **openai/clip-vit-base-patch32 (winner)** | **43.5%** | 512 | 27s |
| google/siglip-base-patch16-224 | 41.5% | 768 | 82s |
| facebook/dinov2-small | 36.2% | 384 | 32s |

On the final dataset, CLIP top-5 matches: skin_tone 61.2%, age_group 65.3%, skin_type 27.5%, main_concern 19.7%.

**Design insight:** visual embeddings capture tone/age strongly but skin concerns only weakly (they're subtle in photos).
The app therefore uses a **hybrid matcher**: CLIP visual similarity + boosting by the user's questionnaire answers.

## Files

- `clip_embeddings.parquet` — `id` + 512-dim L2-normalized vector per dataset row
- `eval_results.json` — full evaluation numbers

## Usage

```python
import pandas as pd, numpy as np
from huggingface_hub import hf_hub_download
df = pd.read_parquet(hf_hub_download("havaferber/dermaglow-embeddings", "clip_embeddings.parquet"))
emb = np.array(df["embedding"].tolist(), dtype=np.float32)   # (1100, 512), pre-normalized
# similarity = emb @ query_vec  (cosine)
```

Embedding model to encode new queries: `openai/clip-vit-base-patch32` (`get_image_features`).