Skip to content

train

train #

train — a splat from posed images, on the GPU one call at a time.

  1. Toolchain. The trainer runs in the GPU interpreter gpu_run finds: torch, gsplat and nerfstudio live there, never in the library's own environment.
  2. Manifest. The frames are handed over explicitly as frames.json under the work directory: one record per frame, in input order, holding the image's path, the Pose's JSON and the paths of the depth and confidence planes staged beside it (null where none was measured). A depth plane is float32 metres at its native resolution, an absent return 0; its confidence float32 in [0, 1], 1 when the map carries none. The trainer resizes them to the image on load and supervises its rendered depth at depth_weight; frames stage no depth at weight 0.
  3. Lock. The whole run holds cgr.gpu_lock(), so calls from every page worker, process and script on the machine train one after another and a waiting call says so in its progress.
  4. Progress. The driver writes progress.json as it trains; a watcher thread reads it into a Training count, so the step, loss and gaussian count reach a page or terminal while the run is live, and finally how far camera refinement moved the cameras.

train(frames, seed, iterations=15000, optimize_cameras=False, depth_weight=0.2) #

A 3D Gaussian splat trained on frames from seed with nerfstudio's splatfacto.

frames = cgr.splats.scene_dataset(session.capture)
splat = cgr.splats.train(frames, cgr.splats.depth_seed(frames))

Parameters:

Name Type Description Default
frames Array[PosedImage]

The posed images to fit.

required
seed PointCloud

The coloured points the gaussians start from.

required
iterations int

How many optimizer steps to run.

15000
optimize_cameras bool

Whether to refine the camera poses while training, for poses a sensor estimated rather than a tracker. The splat is then carried back by the similarity that realigns the refined cameras onto the given ones, so it stays where the poses put it, and the progress says how far the cameras moved.

False
depth_weight float

The weight of the depth term, pulling the rendered depth toward each frame's measured depth, relative to the photometric loss; 0, or frames that measured no depth, is stock splatfacto.

0.2

Raises:

Type Description
ScratchError

If no GPU interpreter can be found.

CalledProcessError

If training fails; the exception carries the tail of the trainer's log.

Source code in capturegraph-recipes/capturegraph_recipes/splats/training/train.py
@cg.pure_function("splats.train/4")
def train(
    frames: cg.Array[PosedImage],
    seed: cg.PointCloud,
    iterations: int = 15_000,
    optimize_cameras: bool = False,
    depth_weight: float = 0.2,
) -> Splat:
    """A 3D Gaussian splat trained on ``frames`` from ``seed`` with nerfstudio's splatfacto.

    ```python
    frames = cgr.splats.scene_dataset(session.capture)
    splat = cgr.splats.train(frames, cgr.splats.depth_seed(frames))
    ```

    Args:
        frames: The posed images to fit.
        seed: The coloured points the gaussians start from.
        iterations: How many optimizer steps to run.
        optimize_cameras: Whether to refine the camera poses while training,
            for poses a sensor estimated rather than a tracker. The splat is
            then carried back by the similarity that realigns the refined
            cameras onto the given ones, so it stays where the poses put it,
            and the progress says how far the cameras moved.
        depth_weight: The weight of the depth term, pulling the rendered
            depth toward each frame's measured depth, relative to the
            photometric loss; ``0``, or frames that measured no depth, is
            stock splatfacto.

    Raises:
        ScratchError: If no GPU interpreter can be found.
        subprocess.CalledProcessError: If training fails; the exception carries
            the tail of the trainer's log.
    """
    work = cg.ManagedDir.new()
    splat = Splat.new("ply")
    records = [_record(work, index, frame, depth_weight > 0) for index, frame in enumerate(frames)]
    supervised = any(record["depth"] is not None for record in records)
    manifest = work.put(MANIFEST, json.dumps(records))
    arguments = [
        "--frames",
        manifest,
        "--output",
        work,
        "--iterations",
        str(iterations),
        "--seed",
        seed,
        "--splat",
        splat,
        "--progress",
        work / PROGRESS,
        "--depth-weight",
        str(depth_weight if supervised else 0),
        *(["--optimize-cameras"] if optimize_cameras else []),
    ]
    with gpu_lock(), cg.logging.progress(total=iterations, label="Training") as training:
        with _watching(training, work / PROGRESS):
            gpu_run(work, TRAINER, arguments)
    return splat