Skip to content

sfm

sfm #

Structure from motion — the geometry of images, as recipes.

The layer order is types → sfm → depth and types → sfm → splats: this package reads the type system and the memoized-call machinery and nothing above itself; learned depth and splat training each build on it and it never reaches up.

A stage that needs a derived value calls the stage that produces it — match calls keypoints, tracks calls match, triangulate calls keypoints — and every such call is a cache hit, so a stage's key conditions on images, never on features threaded through its signature.

The folders are the stages, in order:

  • [.camera][] — the projective model the stages share, exported as cgr.sfm.camera: a Pose in the y-up metric world as a matrix, pixels as rays, the way back from a solved matrix; the OpenCV-frame solvers it builds on stay internal.
  • [.correspondence][] — which features agree across which images: keypoints (SIFT with RootSIFT descriptors), the pairs worth comparing (exhaustive_pairs, sequential_pairs, nearby_pairs), match (mutual ratio-test correspondences verified by a fundamental matrix), and tracks (matches chained across images into the keypoint sets that see one point).
  • [.reconstruction][] — points and cameras from correspondence: triangulate (tracks placed by the frames' poses, checked by angle, depth and reprojection), bundle_adjust (cameras and points refined together, with position priors or a held first camera), and reconstruct (a model from images and tracks alone, grown one registered image at a time).
  • [.alignment][] — the model in the world: posed and reposed (images carrying the poses a model solved for them), exif_priors (the poses a burst's EXIF implies), leveled (a free reconstruction given gravity, its dominant plane made the ground), the local metric frame a Location maps into, and aligned / georeferenced (the similarity carrying a model onto its priors, in full or in the horizontal plane).

This package is the cgr.sfm namespace:

pairs = cgr.sfm.nearby_pairs(frames.pose)
model = cgr.sfm.triangulate(frames, cgr.sfm.tracks(frames.image, pairs))
model = cgr.sfm.bundle_adjust(frames, model, position_sigma_meters=0.05)
frames = cgr.sfm.reposed(frames, model)