Containers#
A loaded target is made of two broadcasting containers — cg.Array and
cg.Map — that project attribute access across every element. This is what
turns analysis into vectorized attribute access.
cg.Array#
cg.Array is a homogeneous
sequence whose attribute access broadcasts to all elements, NumPy-style (it wraps
its items rather than subclassing list):
import capturegraph as cg
survey = cg.load("/path/to/MySurvey", Survey)
sessions = survey.sessions.values() # a Map's values broadcast too
# Instead of [s.location.latitude for s in sessions]:
latitudes = sessions.location.latitude # chains through nested objects
Indexing returns an element, a slice returns an Array:
Transforms#
.map(fn) applies a function element-wise (a Map keeps its keys);
.pmap(fn) does the same across the process's one thread pool, sized to the
CPU count (with cg.parallelism(n): caps how many elements are in flight at
once), which pays off when fn releases the GIL (decoding an image, NumPy, a
subprocess). A pmap inside a worker runs
sequentially, the outer pool being the parallelism. .map_leaves(fn) descends
into nested arrays and transforms only the leaves.
labels = sessions.rating.value.map(lambda r: f"{r}/5")
tooltips = sessions.photo.pmap(lambda p: p.tooltip())
Element types#
A container knows its element type where one exists: Array[Image] indexes and
iterates as Image, a Map's values() is an Array of its values, and
map/pmap carry the function's return type, so a mapped array reaches a typed
parameter unaided.
frames = capture.frames # Array[SceneFrame]
thumbnails = frames.image.map(cgui.thumbnail) # Array[Thumbnail]
A broadcast (frames.image) or a projection (frames["image"]) names a field
whose type no checker can compute, so its elements are Any: the call that
follows types them, as map(cgui.thumbnail) does above.
NumPy#
.to_numpy() converts to an array (Missing/None become nan):
cg.Map#
cg.Map keeps its keys and broadcasts attribute
access over its values, so you can index by key or project a field:
sessions = survey.sessions
sessions.keys() # every key (e.g. capture times, as cg.Time)
sessions.values() # the entries (an Array, broadcasts)
sessions.items() # (key, value) pairs
sessions[some_time] # one entry by key, a datetime or a cg.Time
sessions.rating # broadcast a field over all entries
A typed cg.Map[K, V] holds its keys as values of the key scalar K: a
cg.Map[cg.Time, V] key is a cg.Time, whose .value is the datetime. A
raw key given to the map (a lookup, in, an assignment, the constructor) is
converted to that scalar first, so sessions[noon] and
sessions[cg.Time(noon)] are the same entry. A bare cg.Map({...}) built by
hand has no key type and keeps the keys it is given.
Order#
A Map's and an Array's entries always enumerate in ascending lexicographic
order of their on-disk name — the same rule on every platform, so keys(),
values(), and PathValues/PathKeys agree across the Python reader, the iOS
client, and the sync tool. Because a Map[Time]/Map[Date] key is stored as a
fixed-width hexadecimal-microsecond folder and an Array index as a zero-padded
number ([cg.Array.index_name(index)][capturegraph.Array.index_name]),
lexicographic order is chronological / numeric order — so the last entry is
the most recent session. Keys compare by their folder name too, so the
largest key is the latest capture time:
latest = survey.sessions.values()[-1] # the most recent session
newest = max(survey.sessions.keys(), default=None) # its key, a cg.Time (None if empty)
A Map[String] key sorts by the key text itself.
Missing data#
A per-element lookup that fails yields cg.Missing
for that element instead of crashing the whole projection:
moods = sessions.mood # sparse field: some entries Missing
for mood in moods:
if not cg.is_missing(mood):
print(mood.value)
Missing absorbs further attribute access, so deep chains stay safe;
.to_numpy() turns it into nan. To fill the gaps instead, | substitutes a
default for every Missing/None leaf (recursing into nested Arrays):
A Missing element can be stored. An array element, a map value or a struct
field that is Missing is written as the marker file <entry>.cg-missing
holding its reason (the exception class name, then its message), beside its
siblings, so an array keeps its numbering and a map its keys; it loads back as
Missing carrying that reason. The marker is an analysis-side convention: a
capture platform never writes one, and a leaf it did not capture is simply
absent.
Failing loudly vs. absent data#
Every broadcast surface — attribute access, string/tuple projection (arr["x"],
arr["a", "b"]), .map/.pmap/.map_leaves, and calling the elements
(arr()) — distinguishes a programmer error from legitimately absent data. A
surface raises when every element fails for a non-data reason (a typo, a
wrong key, a broken .map lambda — an AttributeError, KeyError,
ZeroDivisionError, TypeError, …), so a mistake in an analysis script fails
loudly instead of yielding a silent all-Missing result. The raised exception
chains from the first element's failure.
The one carve-out is absence: a field the schema declares but a given entry
never captured loads as Missing carrying a FileNotFoundError. When that is the
reason on every entry, the projection stays all-Missing — so photos.exif()
over a column of never-captured images is Missing, not an error. Any present
element (a partial success) likewise never raises; only sparse data keeps
chaining.
Complete Example#
import capturegraph as cg
import numpy as np
import pandas as pd
survey = cg.load("/path/to/MySurvey", Survey)
sessions = survey.sessions
df = pd.DataFrame({
"date": [instant.value for instant in sessions.keys()],
"rating": sessions.rating.value.to_numpy(),
"notes": list(sessions.notes.value),
})
print(f"Sessions: {len(sessions)}")
print(f"Mean rating: {np.nanmean(df['rating']):.1f}")
See Also#
- Loading Captures —
cg.loadover a schema - Types and Nodes — the composite type forms