Skip to content

refine

refine #

refined — frames whose measured depth is replaced by a model's, fitted to it.

  1. Predict. predict runs the model over every frame's image at once.
  2. Fit. A frame with a measured depth map gets fitted(prediction, depth): denser and sharper, in the measurement's metres.
  3. Keep. A frame without one has no metric reference, so its depth stays Missing; image and pose pass through unchanged.

refined(frames, model) #

frames with each measured depth replaced by model's prediction fitted to it.

Downstream stages (depth_seed, train) see denser metric depth with no change of their own.

frames = cgr.depth.refined(frames, cgr.depth.DA3_LARGE)

Parameters:

Name Type Description Default
frames Array[PosedImage]

The posed images, some with a captured depth map.

required
model Model

The model whose prediction is fitted.

required
Source code in capturegraph-recipes/capturegraph_recipes/depth/fitting/refine.py
@cg.pure_function("depth.refined/1")
def refined(frames: cg.Array[PosedImage], model: Model) -> cg.Array[PosedImage]:
    """``frames`` with each measured depth replaced by ``model``'s prediction fitted to it.

    Downstream stages (``depth_seed``, ``train``) see denser metric depth
    with no change of their own.

    ```python
    frames = cgr.depth.refined(frames, cgr.depth.DA3_LARGE)
    ```

    Args:
        frames: The posed images, some with a captured depth map.
        model: The model whose prediction is fitted.
    """
    predictions = predict(frames.image, model)
    result = [
        PosedImage(
            image=frame.image,
            pose=frame.pose,
            depth=fitted(prediction, frame.depth),
        )
        for frame, prediction in zip(frames, predictions, strict=True)
    ]
    without = sum(cg.is_missing(frame.depth) for frame in frames)
    cg.logging.info(
        "Refined depth",
        model=model.weights.repo_id,
        fitted=len(frames) - without,
        without_lidar=without,
    )
    return cg.Array[PosedImage](result)