Skip to content

affine

affine #

fitted — a model's relative depth made metric by a measured depth map.

  1. Compare. The prediction is area-averaged down to the measured map's grid; a pixel counts where the measurement is finite, further than NEAREST_METERS and both confidences are above zero.
  2. Solve. Monocular models are invariant up to an affine map in inverse depth, so 1/d_metric ≈ a·x + b is fitted by iteratively reweighted least squares: the weight is the measured confidence times the model's, times a Huber weight on the residual, rescaled from the residuals' median absolute deviation each round.
  3. Apply. The map is applied to the full-resolution prediction, the inverse depth clamped to 1/FARTHEST_METERS so it stays positive; the result carries the model's confidence.

fitted(prediction, measured) #

prediction in metres, its inverse depth affinely mapped onto measured.

One global affine map for the whole image is the model here; the fit is weighted by both confidences and robust to outliers in either map.

depth = cgr.depth.fitted(prediction, frame.depth)

Parameters:

Name Type Description Default
prediction RelativeDepth

A model's depth, in its own units, with its confidence.

required
measured Depth

The metric reference, a captured LiDAR map at any size.

required

Returns:

Type Description
Depth

Metric depth at the prediction's size, carrying the model's confidence.

Raises:

Type Description
ValueError

If fewer than MIN_SAMPLES measured pixels are usable.

Source code in capturegraph-recipes/capturegraph_recipes/depth/fitting/affine.py
@cg.pure_function("depth.fitted/1")
def fitted(prediction: RelativeDepth, measured: cg.Depth) -> cg.Depth:
    """``prediction`` in metres, its inverse depth affinely mapped onto ``measured``.

    One global affine map for the whole image is the model here; the fit is
    weighted by both confidences and robust to outliers in either map.

    ```python
    depth = cgr.depth.fitted(prediction, frame.depth)
    ```

    Args:
        prediction: A model's depth, in its own units, with its confidence.
        measured: The metric reference, a captured LiDAR map at any size.

    Returns:
        Metric depth at the prediction's size, carrying the model's confidence.

    Raises:
        ValueError: If fewer than ``MIN_SAMPLES`` measured pixels are usable.
    """
    values = prediction.to_array().astype(np.float32)
    model_confidence = prediction.confidence()
    metric = measured.to_array().astype(np.float32)
    measured_confidence = measured.confidence()
    height, width = metric.shape
    coarse = cv2.resize(values, (width, height), interpolation=cv2.INTER_AREA)
    weight = np.ones_like(metric) if measured_confidence is None else measured_confidence
    if model_confidence is not None:
        weight = weight * cv2.resize(
            model_confidence.astype(np.float32),
            (width, height),
            interpolation=cv2.INTER_AREA,
        )
    usable = np.isfinite(metric) & (metric > NEAREST_METERS) & np.isfinite(coarse) & (weight > 0)
    if int(usable.sum()) < MIN_SAMPLES:
        raise ValueError(
            f"fitted needs at least {MIN_SAMPLES} usable measured pixels, found {int(usable.sum())}"
        )
    x = coarse[usable].astype(np.float64)
    y = 1.0 / metric[usable].astype(np.float64)
    scale, shift, inlier = _affine(x, y, weight[usable].astype(np.float64))
    inverse = np.maximum(scale * values + shift, 1.0 / FARTHEST_METERS)
    residual = np.abs(1.0 / np.maximum(scale * x + shift, 1.0 / FARTHEST_METERS) - 1.0 / y)
    cg.logging.info(
        "Fitted to measured depth",
        scale=scale,
        shift=shift,
        inlier_fraction=float(inlier.mean()),
        median_residual_meters=float(np.median(residual[inlier])) if inlier.any() else float("nan"),
    )
    return cg.Depth.of_arrays((1.0 / inverse).astype(np.float32), model_confidence)