Skip to content

similarity

similarity #

aligned and georeferenced — the similarity that carries a model onto its priors.

  1. Correspondence. Every registered camera (a key of model.cameras) is matched to its prior, centre to centre: aligned to the pose at that index of an Array, georeferenced to the entry under that key of a Map — sparse fixes, as EXIF gives them.
  2. Solve. umeyama finds the scale, rotation and translation that map the model's centres onto the priors' in the least-squares sense: in three dimensions for aligned, whose priors carry altitude and gravity; in the horizontal plane alone for georeferenced, whose model was levelled first and whose fixes know nothing of up.
  3. Apply. transformed moves every camera pose (rotation composed, centre mapped) and every point by a similarity; observations are index-only and carry over.

aligned(model, priors) #

model carried by the similarity mapping its camera centres onto priors.

The gauge lock for a scale-free model — one reconstructed from nothing, or a multi-view network's output — before it is adjusted against priors or used as a seed. A camera's key is its position in priors.

Parameters:

Name Type Description Default
model Reconstruction

The reconstruction to move.

required
priors Array[Pose]

The poses whose centres the model's cameras should land on, indexed like the images the model was built from.

required
Source code in capturegraph-recipes/capturegraph_recipes/sfm/alignment/similarity.py
@cg.pure_function("sfm.aligned/1")
def aligned(model: Reconstruction, priors: cg.Array[cg.Pose]) -> Reconstruction:
    """``model`` carried by the similarity mapping its camera centres onto ``priors``.

    The gauge lock for a scale-free model — one reconstructed from nothing, or
    a multi-view network's output — before it is adjusted against priors or
    used as a seed. A camera's key is its position in ``priors``.

    Args:
        model: The reconstruction to move.
        priors: The poses whose centres the model's cameras should land on,
            indexed like the images the model was built from.
    """
    indices = [camera_index(key) for key in model.cameras.keys()]
    source = np.array([position(model.cameras[camera_key(index)]) for index in indices])
    target = np.array([position(priors[index]) for index in indices])
    source, target = source.reshape(-1, 3), target.reshape(-1, 3)
    scale, rotation, translation = umeyama(source, target)
    residual = source @ (scale * rotation).T + translation - target
    cg.logging.info(
        "Aligned to priors",
        scale=scale,
        rotation_degrees=math.degrees(
            math.acos(min(1.0, max(-1.0, (np.trace(rotation) - 1.0) / 2.0)))
        ),
        mean_residual_meters=float(np.linalg.norm(residual, axis=1).mean()) if indices else 0.0,
    )
    return transformed(model, scale, rotation, translation)

georeferenced(model, priors) #

model carried by the horizontal similarity landing its cameras on their fixes.

A yaw about y, one scale and a shift in x and z, solved over the cameras both model and priors hold; heights are scaled with the rest but never lifted or tilted, so level the model first. Fixes without altitude are the point: only their horizontal positions count.

Parameters:

Name Type Description Default
model Reconstruction

The levelled reconstruction to move.

required
priors Map[String, Pose]

A prior pose per image position, sparse: typically exif_priors.

required

Raises:

Type Description
ValueError

If fewer than two cameras have a fix.

Source code in capturegraph-recipes/capturegraph_recipes/sfm/alignment/similarity.py
@cg.pure_function("sfm.georeferenced/1")
def georeferenced(model: Reconstruction, priors: cg.Map[cg.String, cg.Pose]) -> Reconstruction:
    """``model`` carried by the horizontal similarity landing its cameras on their fixes.

    A yaw about ``y``, one scale and a shift in ``x`` and ``z``, solved over
    the cameras both ``model`` and ``priors`` hold; heights are scaled with
    the rest but never lifted or tilted, so level the model first. Fixes
    without altitude are the point: only their horizontal positions count.

    Args:
        model: The levelled reconstruction to move.
        priors: A prior pose per image position, sparse: typically ``exif_priors``.

    Raises:
        ValueError: If fewer than two cameras have a fix.
    """
    keys = sorted(cg.String.key_encode(key) for key in model.cameras.keys() if key in priors)
    if len(keys) < 2:
        raise ValueError(f"georeferencing needs two cameras with a fix; {len(keys)} found")
    source = np.array([position(model.cameras[key])[[0, 2]] for key in keys])
    target = np.array([position(priors[key])[[0, 2]] for key in keys])
    scale, yaw, shift = umeyama(source, target)
    rotation = np.array(
        [
            [yaw[0, 0], 0.0, yaw[0, 1]],
            [0.0, 1.0, 0.0],
            [yaw[1, 0], 0.0, yaw[1, 1]],
        ]
    )
    translation = np.array([shift[0], 0.0, shift[1]])
    residual = source @ (scale * yaw).T + shift - target
    cg.logging.info(
        "Georeferenced",
        fixes=len(keys),
        scale=scale,
        yaw_degrees=math.degrees(math.atan2(yaw[1, 0], yaw[0, 0])),
        horizontal_residual_meters=float(np.linalg.norm(residual, axis=1).mean()),
    )
    return transformed(model, scale, rotation, translation)

transformed(model, scale, rotation, translation) #

model under the similarity x ↦ scale · rotation · x + translation.

Every camera's centre is mapped and its orientation composed with rotation; every point is mapped; the observations carry over.

Source code in capturegraph-recipes/capturegraph_recipes/sfm/alignment/similarity.py
def transformed(
    model: Reconstruction,
    scale: float,
    rotation: np.ndarray,
    translation: np.ndarray,
) -> Reconstruction:
    """``model`` under the similarity ``x ↦ scale · rotation · x + translation``.

    Every camera's centre is mapped and its orientation composed with
    ``rotation``; every point is mapped; the observations carry over.
    """
    similarity = np.eye(4)
    similarity[:3, :3] = scale * rotation
    similarity[:3, 3] = translation
    cameras: dict[str, cg.Pose] = {}
    for key in model.cameras.keys():
        pose = model.cameras[key]
        moved = similarity @ camera_to_world(pose)
        moved[:3, :3] = rotation @ camera_to_world(pose)[:3, :3]
        cameras[cg.String.key_encode(key)] = pose_from_matrix(moved, pose)
    positions, colors = model.points.points()
    return Reconstruction(
        cameras=cg.Map[cg.String, cg.Pose](cameras),
        points=cg.PointCloud.of_points(
            (positions @ (scale * rotation).T + translation).astype(np.float32), colors
        ),
        observations=model.observations,
    )

umeyama(source, target, with_scale=True) #

The least-squares similarity target ≈ scale · R · source + t (Umeyama, 1991).

Parameters:

Name Type Description Default
source ndarray

N×D points to map.

required
target ndarray

N×D points to land on, row for row.

required
with_scale bool

Whether to solve the scale too, else it stays 1.

True

Returns:

Type Description
float

(scale, rotation, translation): a float, a D×D proper rotation and

ndarray

a D-vector. Fewer than two distinct points make the problem degenerate

ndarray

and yield a pure translation of the means.

Source code in capturegraph-recipes/capturegraph_recipes/sfm/alignment/similarity.py
def umeyama(
    source: np.ndarray,
    target: np.ndarray,
    with_scale: bool = True,
) -> tuple[float, np.ndarray, np.ndarray]:
    """The least-squares similarity ``target ≈ scale · R · source + t`` (Umeyama, 1991).

    Args:
        source: N×D points to map.
        target: N×D points to land on, row for row.
        with_scale: Whether to solve the scale too, else it stays 1.

    Returns:
        ``(scale, rotation, translation)``: a float, a D×D proper rotation and
        a D-vector. Fewer than two distinct points make the problem degenerate
        and yield a pure translation of the means.
    """
    source = np.asarray(source, dtype=np.float64)
    target = np.asarray(target, dtype=np.float64)
    count, dimension = source.shape
    source_mean = source.mean(axis=0)
    target_mean = target.mean(axis=0)
    source_centered = source - source_mean
    target_centered = target - target_mean
    variance = float((source_centered**2).sum() / count)
    if count < 2 or variance < 1e-12:
        return 1.0, np.eye(dimension), target_mean - source_mean
    covariance = (target_centered.T @ source_centered) / count
    u, singular, vt = np.linalg.svd(covariance)
    reflection = np.ones(dimension)
    if np.linalg.det(u) * np.linalg.det(vt) < 0.0:
        reflection[-1] = -1.0
    rotation = u @ np.diag(reflection) @ vt
    scale = float((singular * reflection).sum() / variance) if with_scale else 1.0
    translation = target_mean - scale * rotation @ source_mean
    return scale, rotation, translation