Initial release: PaGeR-normals (surface normals only)

README.md

@@ -0,0 +1,85 @@
+---
+license: apache-2.0
+tags:
+- surface-normals
+- normal-estimation
+- panoramic-images
+- equirectangular
+- high-resolution
+- computer-vision
+- in-the-wild
+- zero-shot
+pipeline_tag: image-to-image
+---
+
+<h1 align="center"> 📟 PaGeR-Normals Model Card</h1>
+
+<p align="center">
+<a title="Github" href="https://github.com/prs-eth/PaGeR" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
+    <img src="https://img.shields.io/github/stars/prs-eth/PaGeR?label=GitHub%20%E2%98%85&logo=github&color=C8C" alt="Github">
+</a>
+<a title="Website" href="https://prs-eth.github.io/PaGeR/" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
+    <img src="https://img.shields.io/badge/%E2%99%A5%20Project%20-Website-blue" alt="Website">
+</a>
+<a title="arXiv" href="https://arxiv.org/abs/TBD" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
+    <img src="https://img.shields.io/badge/%F0%9F%93%84%20Read%20-Paper-AF3436" alt="arXiv">
+</a>
+<a title="Hugging Face" href="https://huggingface.co/spaces/prs-eth/PaGeR" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
+    <img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-FFD21E" alt="Hugging Face Spaces">
+</a>
+<a title="PanoInfinigen dataset" href="https://huggingface.co/datasets/prs-eth/PanoInfinigen" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
+    <img src="https://img.shields.io/badge/Dataset-PanoInfinigen-7e57c2" alt="PanoInfinigen dataset">
+</a>
+<a title="License" href="LICENSE" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
+  <img src="https://img.shields.io/badge/License-Apache%202.0-green" alt="License">
+</a>
+</p>
+
+`PaGeR-Normals` is the **normals-only** variant of PaGeR released with our paper:
+
+- **Paper:** *Unified Panoramic Geometry Estimation via Multi-View Foundation Models* — [arXiv (TBD)](https://arxiv.org/abs/TBD)
+
+From a single equirectangular (ERP) panorama, it produces a **dense surface-normal map** (unit vectors in the panorama's world frame) in a single forward pass. If you want depth, metric scale, and sky masking from the same model, use the unified [`prs-eth/PaGeR`](https://huggingface.co/prs-eth/PaGeR) checkpoint instead.
+
+You can also browse the rest of our [PaGeR HF collection](https://huggingface.co/collections/prs-eth/pager) or try the [interactive demo](https://huggingface.co/spaces/prs-eth/PaGeR).
+
+## Model Details
+
+- **Developed by:** [Vukasin Bozic](https://vulus98.github.io/), [Isidora Slavkovic](https://linkedin.com/in/isidora-slavkovic), [Dominik Narnhofer](https://scholar.google.com/citations?user=tFx8AhkAAAAJ&hl=en), [Nando Metzger](https://nandometzger.github.io/), [Denis Rozumny](https://rozumden.github.io/), [Konrad Schindler](https://scholar.google.com/citations?user=FZuNgqIAAAAJ), [Nikolai Kalischek](https://scholar.google.com/citations?user=XwzlnZoAAAAJ&hl=de).
+- **Model type:** Feed-forward, multi-view foundation-model adaptation for single-image panoramic surface-normal estimation.
+- **Backbone:** [Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3) (`da3-giant`, ViT-Giant), repurposed for cubemap-based multi-view processing of the panorama.
+- **Inputs:** A single ERP panorama, internally projected onto a 6-face cubemap at 504 px per face.
+- **Output:** Dense surface-normal map at panoramic resolution, returned as unit vectors in the panorama's world frame.
+- **Resolution:** Designed for high-resolution ERP inputs, up to 3K.
+- **License:** [Apache License 2.0](LICENSE). The DA3 backbone weights are released under their own license; please refer to the [Depth Anything 3 repository](https://github.com/ByteDance-Seed/Depth-Anything-3) for those terms.
+- **Resources for more information:** [Project Website](https://prs-eth.github.io/PaGeR/), [Paper](https://arxiv.org/abs/TBD), [Code](https://github.com/prs-eth/PaGeR).
+
+### Other released checkpoints
+
+| Checkpoint | Hugging Face id | Depth | Normals | Sky |
+|---|---|---|---|---|
+| PaGeR *(unified, recommended)* | [`prs-eth/PaGeR`](https://huggingface.co/prs-eth/PaGeR) | ✅ | ✅ | ✅ |
+| PaGeR-Metric-Depth | [`prs-eth/PaGeR-metric-depth`](https://huggingface.co/prs-eth/PaGeR-metric-depth) | ✅ (metric) |  |  |
+| **PaGeR-Normals** *(this card)* | [`prs-eth/PaGeR-normals`](https://huggingface.co/prs-eth/PaGeR-normals) |  | ✅ |  |
+
+## Usage
+
+```python
+import torch
+from huggingface_hub import hf_hub_download
+from omegaconf import OmegaConf
+
+from src.pager import Pager
+
+checkpoint = "prs-eth/PaGeR-normals"
+device = torch.device("cuda")
+
+config_path = hf_hub_download(repo_id=checkpoint, filename="config.yaml")
+cfg = OmegaConf.load(config_path)
+
+pager = Pager(checkpoint, cfg=cfg, device=device)
+pager.get_intrinsics_extrinsics(image_size=cfg.face_size, fov=getattr(cfg, "cube_fov", 90.0))
+pager.model.to(device).eval()
+```
+
+See the [GitHub repository](https://github.com/prs-eth/PaGeR) for the full inference / evaluation pipeline (ERP → cubemap projection, batch inference via `inference.py`, the Gradio demo `app.py`, and the normals evaluation script).

config.yaml

@@ -0,0 +1,7 @@
+# Released PaGeR-Normals.
+pretrained_model_name: da3-giant
+modalities:
+  - normals
+face_size: 504
+log_depth: true
+valid_conv_padding: true

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:295682261c88ae4a6301b68b2c38cdaf87cb6c4ac7fa3ebbc263f5f837388ab5
+size 5221983184