capture
capture
#
World-tracked capture procedures backed by ARKit.
Two sampling regimes over the same posed RGB-D substrate, both feeding offline 3D reconstruction (e.g. gaussian splatting on a server):
ARKitCaptureScene— a sparse, curated set of posed keyframes, novelty- and sharpness-gated. Maximal data per viewpoint; a keyframe carries everything the tracker can deliver, so derived products (meshes, point clouds, trajectories) are reconstructed server-side rather than captured as lesser substitutes.ARKitCaptureVideo— the dense, continuous counterpart: one clip with the 6-DoF pose baked into every frame and best-effort depth on the (sparser) frames a depth sensor reached.
Example
import capturegraph as cg
class SceneTarget(cg.Struct):
world_map: cg.ARWorldMap
sessions: cg.Map[cg.Date, cg.SceneCapture]
@cg.procedure(SceneTarget)
def scene(root: cg.Procedure[cg.Path[SceneTarget]]):
session = root.sessions[cg.CaptureTime()]
capture = cg.ARKitCaptureScene(
initial_world_map=root.world_map.load_or(cg.ARKitCreateScene()),
)
# The full capture per session (each doubles as a dated world-map
# backup); the shared slot carries the current map forward.
session &= capture
root.world_map &= capture.world_map
ARKitCaptureScene
#
Bases: Procedure[SceneCapture]
Capture a world-anchored set of posed keyframes with ARKit world tracking.
The user walks the scene while keyframes fire automatically whenever the
camera has moved or rotated enough past the frames already captured (the
translation and rotation fractions of the thresholds combine as a Euclidean
norm against the nearest kept keyframe, so partial motion on both axes
accumulates) — and is moving slowly enough for a sharp frame (the
translational and angular speeds, as fractions of max_speed_mps and
max_rotation_dps, combine the same way; while their norm exceeds 1 the
shutter is held, so an overspeeding camera cannot fire a motion-blurred
keyframe. The speeds are peak-held with a short exponential decay, so a
momentary dip mid-swing does not read as steady — captures wait for a
genuine settle). Each keyframe reduces from a single tracked frame: the
image,
its 6-DoF world pose with pinhole intrinsics, its exposure/light estimate,
a best-effort metric depth map captured in the same instant (present on
devices that deliver scene depth), and a best-effort GPS fix (absent with
geotagging disabled or no signal) — pose/fix pairs let a server solve the
scene's geographic alignment, and the rest is the raw substrate it
reconstructs offline. Finishing also serializes the session's
ARWorldMap, so a later session (any participant) relocalizes into the
same coordinate frame and extends the scene.
Attributes:
| Name | Type | Description |
|---|---|---|
initial_world_map |
Input[ARWorldMap]
|
The scene's world map to relocalize into. Wire it as
|
min_translation_meters |
Input[Number]
|
Metres of camera movement that alone earn the next keyframe (default: 0.1). |
min_rotation_degrees |
Input[Number]
|
Degrees of camera rotation that alone earn the next keyframe (default: 10.0). |
max_speed_mps |
Input[Number]
|
Metres per second of camera movement past which a capture would motion-blur; the shutter is held until the camera slows (default: 0.5 — ~8 px of blur on the 4K stream at a 1/60 s indoor exposure and ~3 m scene depth, which is ~4 px at the roughly halved resolution reconstruction runs at; translational blur scales inversely with depth, so tighten this for close-up scenes). |
max_rotation_dps |
Input[Number]
|
Degrees per second of camera rotation past which a capture would motion-blur, holding the shutter the same way (default: 10.0 — the same budget, depth-independent: the ~65° wide camera smears ~0.9 px per °/s at 4K and 1/60 s, and OIS recovers much of so small an angular excursion; matches the panorama gate's rate limit). |
max_frames |
Input[Number]
|
Soft cap on keyframes per session (default: 300). |
format |
Input[PosedVideoFormat]
|
The frame-rate floor / resolution priority for the AR stream
each keyframe is cut from (default:
|
Keyframes are always 8-bit sRGB JPEG — the reconstruction toolchain (COLMAP/FreeImage, OpenCV) reads JPEG directly, and the fixed sRGB target keeps every frame colorimetrically uniform. The session always prefers an HDR-capable video format (best-effort): ARKit's video HDR is exposure fusion within that same 8-bit feed, so it improves highlight retention without changing the on-disk format.
Example
Source code in capturegraph-lib/capturegraph/procedures/nodes/collection/arkit/capture.py
ARKitCaptureVideo
#
Bases: Procedure[PosedVideo]
Record a world-anchored posed RGB-D video with ARKit world tracking.
The dense, continuous counterpart to ARKitCaptureScene. Where a scene
scan keeps a sparse, sharpness-gated set of posed keyframes, this records
one continuous clip and bakes the 6-DoF camera pose into every frame,
plus a best-effort metric depth map on the frames a depth sensor reaches
(LiDAR/depth-camera devices). Depth is sparser than the RGB stream, so it is
kept keyed by the exact frame it was measured at — the result is honest about
which frames carry fresh depth rather than holding a stale map forward.
Finishing archives the session's ARWorldMap the same way
ARKitCaptureScene does, so the video shares a scene's coordinate frame
and later sessions relocalize into it and extend the capture.
The result is a PosedVideo: the clip, its per-frame poses and
depth (both keyed by frame instant so they line up), and the refreshed
world_map. Reach into the result's fields to store or forward them —
capture.video, capture.poses, capture.depth, capture.world_map.
Attributes:
| Name | Type | Description |
|---|---|---|
initial_world_map |
Input[ARWorldMap]
|
The scene's world map to relocalize into. Wire it as
|
dynamic_range |
Input[DynamicRange]
|
Whether the session engages ARKit's video HDR —
exposure fusion within the unchanged 8-bit BT.709 feed, better
highlight retention — when the AR video format offers it
(default: |
format |
Input[PosedVideoFormat]
|
The frame-rate floor / resolution priority (default:
|
Example
Source code in capturegraph-lib/capturegraph/procedures/nodes/collection/arkit/capture.py
ARKitScanScene
#
Bases: Procedure[ARWorldMap]
Scan a world-anchored scene into an ARWorldMap without capturing images.
The registration-only counterpart to ARKitCaptureScene: the user walks
the scene while ARKit extends its world map, and finishing archives the map
alone — no keyframes, no depth, no clips. A study seeds each scene with one
scan so the map (a few tens of megabytes) reaches the server before any
image-bearing capture exists; later sessions relocalize into it immediately
instead of queueing behind an image-upload backlog. The capture UI reads
ARKit's live mapping status as a coverage indication, encouraging the user
to keep scanning until the map is fully established.
Attributes:
| Name | Type | Description |
|---|---|---|
initial_world_map |
Input[ARWorldMap]
|
The scene's world map to relocalize into and extend.
Wire it as |