grKnight
/

pointcal-c

@@ -1,168 +1,189 @@
----
-license: cc-by-4.0
-base_model: laion/CLIP-ViT-B-32-laion2B-s34B-b79K
-pipeline_tag: zero-shot-classification
-library_name: pointcal-c
-tags:
-- 3d
-- point-cloud
-- modelnet40-c
-- clip
-- calibration
-- selective-prediction
-- uncertainty
-- zero-shot
----
-# PointCal-C — selective zero-shot 3D recognition under corruption
-A post-hoc **calibration and abstention head** on top of a frozen OpenCLIP
-ViT-B/32, for zero-shot 3D point-cloud recognition under corruption, plus the
-complete evaluation artifacts backing it.
-Code: <https://github.com/crimsonKn1ght/pointcal-c>
-## What the model is
-The backbone is **never trained**. `laion/CLIP-ViT-B-32-laion2B-s34B-b79K` is
-loaded frozen, in eval mode, with `requires_grad_(False)`.
-The fitted model is **four scalars**, in `runs/full/results/calibration.json`,
-fit on clean calibration objects only — corrupted labels, corruption identity
-and severity never touch a fitted parameter:
-| parameter | value | role |
-|---|---|---|
-| `temperature` | 1.2309 | divides logits before softmax |
-| `bias` | -1.5723 | combined-score intercept |
-| `weight_confidence` | +1.7071 | on `logit(p_calibrated)` |
-| `weight_disagreement` | +4.9158 | on cross-view disagreement `d` |
-Combined score: `sigmoid(bias + w_conf * logit(p_cal) + w_dis * d)`, where `d`
-is `1 -` mean pairwise Jensen-Shannon divergence across the six views.
-Converged, unclamped, not degenerate. Fit in 0.94 s on CPU.
-## How inference works
-```
-point cloud (N,1024,3)
-  -> centroid-center, unit-radius normalize
-  -> 6 orthographic cameras  [front right back left top bottom]
-  -> 64x64 splat raster + nearest-depth z-buffer -> upsample to 224x224
-  -> frozen OpenCLIP ViT-B/32 image encoder
-  -> per-view logits vs a fixed 8-template prompt ensemble   <-- cached here
-  -> mean over views  ->  the prediction
-  -> four confidence scores over that one prediction
-```
-All four confidence methods (`msp`, `temperature`, `disagreement`, `combined`)
-rank the **same** predictions. A confidence method that changed a prediction
-would be a different classifier, and `assert_predictions_unchanged` raises if
-one ever does.
-## Results
-All 75 corrupted conditions pooled against clean, 1975 evaluation objects,
-95% bootstrap intervals resampled over base object IDs:
-| | accuracy | ECE (MSP) | ECE (combined) | AURC (MSP) | AURC (combined) |
-|---|---|---|---|---|---|
-| clean | 0.2896 | 0.1160 | 0.0245 | 0.4339 | 0.4147 |
-| corrupted | 0.2354 | 0.1416 | 0.0229 | 0.5217 | 0.5008 |
-Accuracy degrades under corruption and MSP calibration degrades with it. The
-clean-fit combined score holds ECE near **0.023** under shift and lowers AURC
-by 0.021.
-The ECE improvement is large and consistent. **The AURC gain is small** and
-should be read against the intervals in `results.csv`, not as a headline. This
-is a reliability audit and a bounded abstention baseline, not a
-state-of-the-art claim.
-Accuracy by severity: 0.2896 (clean), 0.2593, 0.2515, 0.2388, 0.2252, 0.2023 —
-monotonically non-increasing.
-## Repository contents
-The expensive artifact is `runs/*/logits/` — per-view logits for every
-condition under both prompt modes. Every metric, ablation, bootstrap interval
-and figure here was computed on CPU from those caches, so **any re-analysis is
-free**: a different aggregation, disagreement statistic, coverage level or
-bootstrap depth costs no GPU time, because CLIP never runs again.
-```
-runs/<tier>/
-  logits/<condition>.npz     per-view logits, both prompt modes (~1.4 MB each)
-  results/calibration.json   the fitted scalars above
-  results/results.csv|.json  metrics table with intervals
-  results/ablations.csv      view-count, prompt-mode, disagreement ablations
-  results/predictions.npz    per-sample confidences and correctness
-  results/results_summary.md auto-generated tables + hypothesis checks
-  figures/fig1..fig4         accuracy, ECE, risk-coverage, cost
-  provenance/                data manifest, run manifest, split audit
-  ledger_inference.json      measured GPU-hours, dollars, throughput, memory
-artifacts/split.json         the frozen calibration/evaluation object split
-env/requirements.lock.txt    exact resolved environment (171 packages)
-```
-Tiers: `xs` (5 conditions, 100 objects), `s` (13 conditions), `full` (all 76 =
-clean + 15 corruptions x 5 severities).
-## Training / evaluation data
-Evaluated on **ModelNet40-C** (2468 objects, 40 classes, 15 corruption types,
-5 severities). The calibration scalars are fit on the clean split of 493
-calibration objects; 1975 evaluation objects are held out.
-The split is over **base object IDs**, not samples, and every corruption array
-is verified row-aligned with the clean array, so holding an object out of
-calibration holds it out under all 76 conditions. Audited: `overlap: 0`,
-0 row-misaligned conditions, all 40 classes present on both sides.
-## Compute
-| tier | conditions | GPU-hours | USD | views/s | peak VRAM |
-|---|---|---|---|---|---|
-| xs | 5 | 0.001 | 0.0003 | 1091 | 2.5 GB |
-| s | 13 | 0.015 | 0.004 | 2816 | 12.1 GB |
-| full | 76 | 0.089 | 0.025 | 2808 | 12.1 GB |
-**0.105 GPU-hours and about $0.03** of inference in total, on a single Runpod
-RTX 4000 Ada at $0.28/hr (2026-08-16).
-## Limitations
-- `full` uses **200** bootstrap replicates, not the preregistered 1000. The CPU
-  bootstrap dominated runtime (~2 h at 1000 vs ~5 min of GPU work). Deeper
-  intervals recompute from the cached logits at no GPU cost.
-- NLL and Brier require a distribution over all 40 classes, so they are reported
-  for `msp` and `temperature` only. For the two ranking scores they are `null`,
-  not a lookalike substitute.
-- Cross-view disagreement is prior art, and calibration degrading under 3D
-  corruption is a confirmation rather than a discovery.
-- Absolute accuracy is low (~0.29 clean). This is zero-shot transfer of a 2D
-  model to depth renders, with no 3D training whatsoever.
-- Severity is reported **1-indexed**, matching the on-disk release.
-## Provenance
-- Frozen spec hash: `ca487597c978ab17217bd640ab8bf95c8a21db0eaacd51ae5ba4b00618dcc820`
-- Backbone checkpoint SHA-256:
-  `1bd3c7172de5b207ceac554f5ab5266166f3b9baccc9af5989bc801016d080ad`
-- Split fingerprint: `cc5ed4b3ce7108154039f497d474f3ac6bdbfe7eb7dbfe4d0f90785e1829bb31`
-## Licence and attribution
-Cached logits and metrics are **derived from ModelNet40-C** and redistributed
-under **CC BY 4.0**, as the source record declares.
-- ModelNet40-C — Zenodo [6017834](https://zenodo.org/records/6017834),
-  DOI `10.5281/zenodo.6017834`, CC BY 4.0
-- Generation code — <https://github.com/jiachens/ModelNet40-C>, BSD-3-Clause
-- Underlying ModelNet40 carries its own original terms
-- OpenCLIP — MIT · PointCal-C code — MIT
-The multi-view depth-projection approach follows the PointCLIP paper (CVPR
-2022); the projector is an independent reimplementation from the paper text,
-with no PointCLIP repository code used.

+---
+license: cc-by-4.0
+base_model: laion/CLIP-ViT-B-32-laion2B-s34B-b79K
+pipeline_tag: zero-shot-classification
+library_name: pointcal-c
+tags:
+- 3d
+- point-cloud
+- modelnet40-c
+- clip
+- calibration
+- selective-prediction
+- uncertainty
+- zero-shot
+---
+# PointCal-C — selective zero-shot 3D recognition under corruption
+A post-hoc **calibration and abstention head** on top of a frozen OpenCLIP
+ViT-B/32, for zero-shot 3D point-cloud recognition under corruption, plus the
+complete evaluation artifacts backing it.
+Code: <https://github.com/crimsonKn1ght/pointcal-c>
+## What the model is
+The backbone is **never trained**. `laion/CLIP-ViT-B-32-laion2B-s34B-b79K` is
+loaded frozen, in eval mode, with `requires_grad_(False)`.
+The fitted model is **four scalars**, in `runs/full/results/calibration.json`,
+fit on clean calibration objects only — corrupted labels, corruption identity
+and severity never touch a fitted parameter:
+| parameter | value | role |
+|---|---|---|
+| `temperature` | 1.2309 | divides logits before softmax |
+| `bias` | -1.5723 | combined-score intercept |
+| `weight_confidence` | +1.7071 | on `logit(p_calibrated)` |
+| `weight_disagreement` | +4.9158 | on cross-view disagreement `d` |
+Combined score: `sigmoid(bias + w_conf * logit(p_cal) + w_dis * d)`, where `d`
+is `1 -` mean pairwise Jensen-Shannon divergence across the six views.
+Converged, unclamped, not degenerate. Fit in 0.94 s on CPU.
+## How inference works
+```
+point cloud (N,1024,3)
+  -> centroid-center, unit-radius normalize
+  -> 6 orthographic cameras  [front right back left top bottom]
+  -> 64x64 splat raster + nearest-depth z-buffer -> upsample to 224x224
+  -> frozen OpenCLIP ViT-B/32 image encoder
+  -> per-view logits vs a fixed 8-template prompt ensemble   <-- cached here
+  -> mean over views  ->  the prediction
+  -> four confidence scores over that one prediction
+```
+All four confidence methods (`msp`, `temperature`, `disagreement`, `combined`)
+rank the **same** predictions. A confidence method that changed a prediction
+would be a different classifier, and `assert_predictions_unchanged` raises if
+one ever does.
+## Results
+All 75 corrupted conditions pooled against clean, 1975 evaluation objects,
+95% bootstrap intervals resampled over base object IDs:
+| | accuracy | ECE (MSP) | ECE (combined) | AURC (MSP) | AURC (combined) |
+|---|---|---|---|---|---|
+| clean | 0.2896 | 0.1160 | 0.0245 | 0.4339 | 0.4147 |
+| corrupted | 0.2354 | 0.1416 | 0.0229 | 0.5217 | 0.5008 |
+Accuracy degrades under corruption and MSP calibration degrades with it. The
+clean-fit combined score holds ECE near **0.023** under shift and lowers AURC
+by 0.021.
+The ECE improvement is large and consistent. **The AURC gain is small** and
+should be read against the intervals in `results.csv`, not as a headline. This
+is a reliability audit and a bounded abstention baseline, not a
+state-of-the-art claim.
+Accuracy by severity: 0.2896 (clean), 0.2593, 0.2515, 0.2388, 0.2252, 0.2023 —
+monotonically non-increasing.
+### Pre-registered hypotheses
+Of the four pre-registered hypotheses, three are mechanically checked. H2 and H3
+resolve True. **H1 does not hold strictly:** accuracy, NLL, Brier and AURC all
+worsen monotonically with severity, but ECE dips at severities 3→4 and again
+from clean→1. A paired object-grouped bootstrap places both reversals within
+noise of zero, so H1 is supported in direction but not as strict monotonicity.
+H4 is read off the family-scope rows rather than reduced to a verdict.
+## Repository contents
+The expensive artifact is `runs/*/logits/` — per-view logits for every
+condition under both prompt modes. Every metric, ablation, bootstrap interval
+and figure here was computed on CPU from those caches, so **any re-analysis is
+free**: a different aggregation, disagreement statistic, coverage level or
+bootstrap depth costs no GPU time, because CLIP never runs again.
+```
+runs/<tier>/
+  logits/<condition>.npz     per-view logits, both prompt modes (~1.4 MB each)
+  results/calibration.json   the fitted scalars above
+  results/results.csv|.json  metrics table with intervals
+  results/ablations.csv      view-count, prompt-mode, disagreement ablations
+  results/predictions.npz    per-sample confidences and correctness
+  results/results_summary.md auto-generated tables + hypothesis checks
+  figures/fig1..fig4         accuracy, ECE, risk-coverage, cost
+  provenance/                data manifest, run manifest, split audit
+  ledger_inference.json      measured GPU-hours, dollars, throughput, memory
+artifacts/split.json         the frozen calibration/evaluation object split
+env/requirements.lock.txt    exact resolved environment (171 packages)
+```
+Tiers: `xs` (5 conditions, 100 objects), `s` (13 conditions), `full` (all 76 =
+clean + 15 corruptions x 5 severities).
+## Training / evaluation data
+Evaluated on **ModelNet40-C** (2468 objects, 40 classes, 15 corruption types,
+5 severities). The calibration scalars are fit on the clean split of 493
+calibration objects; 1975 evaluation objects are held out.
+The split is over **base object IDs**, not samples, and every corruption array
+is verified row-aligned with the clean array, so holding an object out of
+calibration holds it out under all 76 conditions. Audited: `overlap: 0`,
+0 row-misaligned conditions, all 40 classes present on both sides.
+## Compute
+| tier | conditions | GPU-hours | USD | views/s | peak VRAM |
+|---|---|---|---|---|---|
+| xs | 5 | 0.001 | 0.0003 | 1091 | 2.5 GB |
+| s | 13 | 0.015 | 0.004 | 2816 | 12.1 GB |
+| full | 76 | 0.089 | 0.025 | 2808 | 12.1 GB |
+**0.105 GPU-hours and about $0.03** of inference in total, on a single Runpod
+RTX 4000 Ada at $0.28/hr (2026-08-16).
+## Known defects
+See **[ERRATUM.md](ERRATUM.md)** for the full list. In short: the `ledger` block
+embedded in `runs/full/provenance/run_manifest.json` is wrong — it records a
+cache-hit re-invocation, not the real run. **Trust `ledger_inference.json`,**
+which every figure and the compute table above are derived from. `git_dirty` in
+that manifest is not meaningful either, because the dirtiness check counts
+expected untracked artifacts.
+## Limitations
+- `full` uses **200** bootstrap replicates, not the preregistered 1000. The CPU
+  bootstrap dominated runtime (~2 h at 1000 vs ~5 min of GPU work). Deeper
+  intervals recompute from the cached logits at no GPU cost.
+- NLL and Brier require a distribution over all 40 classes, so they are reported
+  for `msp` and `temperature` only. For the two ranking scores they are `null`,
+  not a lookalike substitute.
+- `delta_*_vs_clean` rows in `results.csv` carry `NaN` intervals: degradation is
+  reported as point estimates without paired bootstrap intervals. Method
+  differences should likewise not be read from the marginal intervals alone.
+- Cross-view disagreement is prior art, and calibration degrading under 3D
+  corruption is a confirmation rather than a discovery.
+- Absolute accuracy is low (~0.29 clean). This is zero-shot transfer of a 2D
+  model to depth renders, with no 3D training whatsoever.
+- Severity is reported **1-indexed**, matching the on-disk release.
+## Provenance
+- Frozen spec hash: `ca487597c978ab17217bd640ab8bf95c8a21db0eaacd51ae5ba4b00618dcc820`
+- Backbone checkpoint SHA-256:
+  `1bd3c7172de5b207ceac554f5ab5266166f3b9baccc9af5989bc801016d080ad`
+- Split fingerprint: `cc5ed4b3ce7108154039f497d474f3ac6bdbfe7eb7dbfe4d0f90785e1829bb31`
+## Licence and attribution
+Cached logits and metrics are **derived from ModelNet40-C** and redistributed
+under **CC BY 4.0**, as the source record declares.
+- ModelNet40-C — Zenodo [6017834](https://zenodo.org/records/6017834),
+  DOI `10.5281/zenodo.6017834`, CC BY 4.0
+- Generation code — <https://github.com/jiachens/ModelNet40-C>, BSD-3-Clause
+- Underlying ModelNet40 carries its own original terms
+- OpenCLIP — MIT · PointCal-C code — MIT
+The multi-view depth-projection approach follows the PointCLIP paper (CVPR
+2022); the projector is an independent reimplementation from the paper text,
+with no PointCLIP repository code used.