Evaluation Inspector The Sterkfontein Caves Dataset
Version D0.9-E0.9 — dataset release 0.9, evaluation protocol E0.9
Companion page to The Sterkfontein Caves Dataset (ECCV 2026): ten cave scenes captured in near-darkness with a smartphone. Each method is trained per scene and renders one held-out evaluation view; this page shows every method's saved render beside that view's ground-truth reference, under evaluation protocols you can switch: qualitative results · quantitative results.
The reference is a merge of a ~100-frame burst captured at the evaluation view's camera pose — an HDR+-style burst merge that significantly reduces noise compared to any single frame. References come from dataset release 0.9.
raw-nerfacto and raw-splatfacto are the paper's methods; the others are baselines as run for the paper (references at the bottom). A raw chip on a card means the method reconstructs the scene in the sensor's native raw color space; srgb means it works on developed 8-bit images.
Controls
- click a render — toggle ("blink") it against the reference
- ⛶ — fullscreen at full resolution
- 1–4 — switch evaluation protocol (anywhere)
- q / w — every card shows the reference / the renders
- fullscreen: wheel zoom at cursor (nearest-neighbor when magnified) · drag pan · 0 reset · click blink · ←/→ walk methods · Esc close
- the number on each card is PSNR under the selected protocol; the small delta is the change versus calibrated
- grid thumbnails are compressed previews for navigation; every fullscreen image is a lossless PNG, loaded on demand
Qualitative results
D01 Deposit1
low light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD02 Deposit2
low light + collocated light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD03 Deposit3
spot light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD04 Deposit4
spot light + collocated light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD05 Tunnel1
extreme low light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD06 Tunnel2
extreme low light + artificial light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD07 Protrusion1
natural light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
renderD08 Protrusion2
natural + artificial light · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD09 RoofPendant1
slow shutter · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
render
renderD10 RoofPendant2
fast shutter · ⛶ opens fullscreen at full resolution
render
render
render
render
render
render
render
render
renderQuantitative results
Renders are scored under the calibrated protocol, which brings each render into the reference's geometry and color space, and under two fitted radiometric variants that we keep separate because they forgive different things: cal + WB allows only what a physical capture error can produce (a per-channel white-balance gain and black-level offset), while cal + affine is an unconstrained per-channel gain and offset that can also absorb color differences no camera setting explains. The gap between the two measures exactly that extra forgiveness. A raw-space evaluation further below compares the raw-output methods directly against the sensor measurements, with no development or exposure normalization. One caveat throughout: a reconstruction that integrates many views can be lower-noise than the reference itself, and pixel metrics then penalize the render for not reproducing the reference's residual noise — take the numbers with a pinch of salt 🧂 (see d07 raw-nerfacto in fullscreen: the render's shadows are cleaner than the reference's).
Linear sRGB evaluation
protocol details
| method | metric | d01 | d02 | d03 | d04 | d05 | d06 | d07 | d08 | d09 | d10 | mean |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| raw-nerfacto | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| raw-splatfacto | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| rawnerf | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| hdrsplat | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| le3d | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| nerfacto | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| splatfacto | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| 3dgs_dk | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| aleth_nerf | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ | |||||||||||
| luminance-gs | PSNR↑SSIM↑LPIPS-1K↓LPIPS-4K↓ |
Flags (grey cells and card chips) — diverged: the training run collapsed. diverged-exposure: trained, but at a wildly wrong brightness. superposition / watch-superposition: the d08 model was trained with mixed image orientations and contains both (the d08 evaluation view was captured with the phone physically inverted). enc-ambiguous: the method's output encoding is inconsistent across scenes. no saved render: not present in the paper's result archive. Flagged pairs remain in the means; missing pairs are excluded.
Raw-space evaluation
Raw-output methods only (the five with a raw chip); protocol-independent. The float32 raw render is compared directly against the reference burst-merge mosaic, sampled at each pixel's color-filter-array (CFA/Bayer) channel, in DN. A DN (digital number) is one integer step of the sensor's analog-to-digital output — the native unit of the raw files; these are 10-bit, so values run 0–1023 with white level 1023. No development pipeline and no exposure normalization touches either side; d08's inverted capture is handled as in the calibrated protocol.
| method | metric | d01 | d02 | d03 | d04 | d05 | d06 | d07 | d08 | d09 | d10 | mean |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| raw-nerfacto | L1 (DN)↓rel err↓exposure × | 0.8817.4%0.96 | 0.7820.8%1.09 | 20.6019.7%0.85 | 19.9578.7%1.14 | 1.9158.9%1.22 | 2.9083.7%1.40 | 1.7435.6%1.53 | 4.4458.0%1.62 | 43.7292.7%1.64 | 3.9356.3%1.58 | 10.0852.2%1.30 |
| raw-splatfacto | L1 (DN)↓rel err↓exposure × | 1.0519.2%0.87 | 1.0136.8%1.04 | 18.8825.2%0.97 | 24.05165.8%0.94 | 6.24225.7%1.47 | 2.7463.1%0.52 | 2.2485.2%1.29 | 16.64808.9%0.93 | 33.81112.6%1.38 | 3.7568.6%0.84 | 11.04161.1%1.03 |
| rawnerf | L1 (DN)↓rel err↓exposure × | 0.9817.9%0.87 | 0.6518.1%1.01 | 88.4380.0%0.14 | 18.2738.7%0.84 | 1.3637.4%1.10 | 2.0646.0%1.21 | 1.7440.2%1.38 | 4.2761.4%0.94 | 37.2670.0%0.80 | 4.0045.6%0.40 | 15.9045.5%0.87 |
| hdrsplat | L1 (DN)↓rel err↓exposure × | 1.2526.0%0.82 | 0.8929.1%0.99 | 16.9316.6%0.97 | 9.4422.5%0.95 | 1.2838.4%0.81 | 1.3737.1%0.85 | 1.9662.5%1.30 | 3.2159.9%0.95 | 25.8770.8%1.19 | 2.2443.0%0.88 | 6.4440.6%0.97 |
| le3d | L1 (DN)↓rel err↓exposure × | 1.1621.7%0.90 | 0.7922.8%0.87 | 19.0119.0%0.94 | 15.4232.8%0.87 | 0.9526.8%0.90 | 1.2329.8%1.00 | — | 3.2481.4%1.01 | 26.5433.3%0.65 | — | 8.5433.4%0.89 |
exposure × = median render/reference over pixels ≥5 DN (1.00 = absolute photometric agreement). L1 = mean |error| in DN. rel err = mean |error| / max(reference, 1 DN). Raw PSNR is not reported: measured against the full 0–1023 range it comes out above 55 dB for every method — not because the reconstructions are near-perfect, but because these scenes occupy only the bottom few percent of the range — and it does not compare across scenes of different brightness.
Downloads
- metrics_all.json — PSNR / SSIM / LPIPS-1K / LPIPS-4K for every scene–method pair under all four protocols
- raw_metrics.json — raw-space exposure ratio, L1 (DN), relative error per pair
- rescore_v3.csv — per-pair scores with flags and registration diagnostics
- manifest.txt —
URLs of every lossless full-resolution PNG on this page; fetch them all with
wget -i manifest.txt
Full-resolution images: eval-inspector assets D0.9-E0.9 (paper-run renders scored against dataset 0.9 references under evaluation protocol E0.9 — the "calibrated" protocol on this page), served from the dataset's public Globus collection. Asset versions name the (dataset release, evaluation protocol) pair and are immutable once published. Protocol versions are defined in the code repository's EVAL_PROTOCOL.md; E0.9 is current, and E1.0 is reserved for the declared-stable protocol.
The float32 raw-space method outputs — the exact files scored here, suitable for new evaluation protocols — and the dataset itself are distributed through the dataset's Globus collection; see the project page for access.
References
- raw-nerfacto, raw-splatfacto — I. Liu, B. Xu, S. James, D. Stratford, R. Klein, J. Tompkin. The Sterkfontein Caves Dataset: A Novel View Synthesis Challenge from the Cradle of Humankind. ECCV 2026. project page
- nerfacto, splatfacto — Tancik et al. Nerfstudio: A Modular Framework for Neural Radiance Field Development. SIGGRAPH 2023. nerf.studio
- rawnerf — Mildenhall et al. NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images. CVPR 2022. code
- hdrsplat — Singh et al. HDRSplat: Gaussian Splatting for High Dynamic Range 3D Scene Reconstruction from Raw Images. BMVC 2024. code
- le3d — Jin et al. Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis. NeurIPS 2024. code
- 3dgs_dk (Gaussian-DK) — Ye et al. Gaussian in the Dark: Real-Time View Synthesis from Inconsistent Dark Images Using Gaussian Splatting. Computer Graphics Forum 43, e15213 (Pacific Graphics), 2024. code
- aleth_nerf — Cui et al. Aleth-NeRF: Illumination Adaptive NeRF with Concealing Field Assumption. AAAI 2024. code
- luminance-gs — Cui et al. Luminance-GS: Adapting 3D Gaussian Splatting to Challenging Lighting Conditions with View-Adaptive Curve Adjustment. CVPR 2025. code