Skip to content

depth

depth #

Learned depth — monocular and multi-view models over captured images, as recipes.

The layer order is types → sfm → depth: this package reads the type system, cgr.sfm's camera geometry and the GPU driver machinery. splats sits beside it, not on it — a target refines frames with learned depth before it trains a splat. Learned depth is expensive once and free ever after, which is the case for caching it: a prediction is keyed on the images and the model — its driver's bytes and its weights' commit — so a model evaluates a capture once and an edited driver or newer weights evaluate it again.

  • [.models][] — a model as a value: Model is a Driver (the script, hashed by content) and Weights (a Hugging Face repository at one commit); DA3_LARGE and PI3X are the built-in values, snapshot downloads a model's weights once into the scratch, and the driver scripts live beside them.
  • [.inference][] — running a model: the contract a driver honours, predict (a model's RelativeDepth per image) and multiview (its cameras, points and depths for a set posed together, a scale-free Reconstruction beside cgr.sfm.reconstruct).
  • [.fitting][] — relative depth made metric: fitted (an affine map in inverse depth onto a measured depth) and refined (frames whose LiDAR is replaced by the model's depth fitted to it).

A driver is a capturegraph-free script run by path in the GPU interpreter (cgr.GPU_PYTHON_ENV) under cgr.gpu_lock(), honouring one contract: --images DIR --weights DIR --output DIR [--multiview], where --images holds NNNNN.jpg files, --weights the model's snapshot, and --output receives NNNNN.npy (float32 depth) and NNNNN_conf.npy (float32 confidence in [0, 1]) per image at the image's size, plus with --multiview a cameras.json (OpenCV camera-to-world 4×4 and pixel intrinsics per image) and points.npy/colors.npy.

This package is the cgr.depth namespace:

frames = cgr.depth.refined(frames, cgr.depth.DA3_LARGE)
seen = cgr.depth.multiview(images, cgr.depth.PI3X)