CaptureGraph Sync#

CaptureGraph Sync is a command-line tool for downloading captured data from a CaptureGraph server to your local machine.

Overview#

After users capture data with the iOS app and it reaches the server, researchers run capturegraph-sync to mirror that data locally for analysis:

iOS App → Server → capturegraph-sync → Local Analysis

A synced folder is a complete capture target on disk. Its layout is the procedure's schema, so it loads directly with the data framework — reuse the same schema class the procedure was authored with:

import capturegraph as cg

survey = cg.load("./data/MyStudy", Survey)

Key Features#

Schema-Driven Sync#

Sync never blindly walks server directories. It works from the target's procedure definition, whose embedded schema describes every position that can exist:

• Fetches the target's definition {schema, root, nodes} from the server (GET /api/v1/targets/{target}/definition)
• Walks the embedded schema to know every position the data can occupy
• Enumerates dynamic Map/Array directories through a server directory-listing endpoint (GET /api/v1/targets/{target}/directories, .../directories/{path})
• Writes files straight into the schema's positions, so the synced folder loads immediately

Incremental, Version-Based Downloads#

On each run, sync sends the server the opaque content version it already holds for each leaf, in batches, and the server resolves each one as unchanged, present, or missing. Only present leaves are downloaded — and a small one arrives inline with the verdict, sparing a round trip:

• Skip unchanged files: one cheap batch request replaces thousands of downloads
• Skip whole entries: a Map/Array entry folder (a session, say) that already exists locally is not re-checked at all (see --refresh-entries)
• Resume interrupted syncs: each file's version is recorded in a local store the moment the file is written, so a killed sync picks up where it left off

Parallel and Responsive#

Changed files download concurrently across a thread pool (--jobs, default 16), pipelined behind the resolve so bytes start moving while later batches are still being checked; dynamic session folders are likewise listed a whole depth level at a time. Each file streams in chunks behind its own live progress bar under an overall meter (files, bytes, transfer speed, ETA), transient network errors retry with backoff, and the run ends in a compact two-line summary. Interrupting is safe — progress is recorded as it lands, and rerunning resumes.

Guided CLI#

The tool is built to help a newcomer succeed. It adds a URL scheme if you omit one, catches a mistyped target with a Did you mean …? suggestion, turns a connection failure into an actionable hint, and lists the available targets whenever a choice is missing. It writes a .gitignore into each synced folder (only the generated schema.py loader stays visible to git), and offers — never forces — to clear files an updated schema no longer accounts for.

Reference Client#

capturegraph-sync is also the worked example of consuming the server's read API — one client method per endpoint, and a sync engine that reads top-to-bottom as the canonical list → definition → resolve → download flow.

Optional DNG Conversion#

RAW .dng captures can be converted to JPEG, PNG, or HEIF as a post-sync step, with the originals deleted on success. The version store ignores file extensions, so converted files are not re-downloaded on the next run.

Section Contents#