Marigold V2: turn a diffusion transformer into sharp monocular depth

A methods brief on affordable DiT fine-tuning for depth (and other dense maps): single-step inference, Sinkhorn sharpening, and 16–26% AbsRel gains on KITTI/ETH3D versus prior best.

Primary source: arXiv:2609.08084: https://arxiv.org/abs/2609.08084

What’s new: Marigold V2 (arXiv:2609.08084) revisits the Marigold idea: take a modern image generation/editing diffusion transformer and fine-tune it into a monocular depth estimator that stays sharp on out-of-distribution photos. The recipe targets single-step inference from pretrained multi-step flow-matching models (with quantization when needed), so you keep capacity without multi-step sampling cost.

The authors diagnose artifacts from naive training and add two fixes: align internal representations with semantic features from ground truth, and a two-stage fine-tune built around a novel Sinkhorn-based loss (SinkLoss) for edge sharpness while keeping semantic detail. Reported results include 16–26% AbsRel improvement over the previous best on KITTI and ETH3D, plus qualitative recovery of fur, foliage, and hair-thin edges. The same protocol transfers to other dense regression tasks (surface normals, intrinsic decomposition), with weights and a demo on Hugging Face (huawei-bayerlab/marigold-v2-0). Training is framed as feasible on a single ~32 GB GPU.

Why it matters: Depth is plumbing for robotics, reconstruction, and scientific imaging pipelines. A methods story that is affordable to reproduce (one GPU, open DiT backbone, single-step inference) matters more than another closed SOTA leaderboard spike. SinkLoss-plus-semantic alignment is a concrete recipe other dense predictors can steal.

Caveats: Preprint. AbsRel gains are on named benchmarks under the paper’s protocol; check baselines and metrics before citing as universal. Affine-invariant depth still needs metric calibration for many applications. See-through / layered variants are separate heads and datasets. Computer-vision methods, not medical imaging validation. Use it as an open dense-prediction recipe with strong qualitative edges, then measure on your domain.