Skip to content

Containers#

A loaded target is made of two broadcasting containers — cg.Array and cg.Map — that project attribute access across every element. This is what turns analysis into vectorized attribute access.

cg.Array#

cg.Array is a homogeneous sequence whose attribute access broadcasts to all elements, NumPy-style (it wraps its items rather than subclassing list):

import capturegraph as cg

survey = cg.load("/path/to/MySurvey", Survey)
sessions = survey.sessions.values()      # a Map's values broadcast too

# Instead of [s.location.latitude for s in sessions]:
latitudes = sessions.location.latitude    # chains through nested objects

Indexing returns an element, a slice returns an Array:

sessions[0]        # one element
sessions[1:3]      # an Array

Transforms#

.map(fn) applies a function element-wise; .map_leaves(fn) descends into nested containers and transforms only the leaves. Both have a thread-pooled .pmap(fn, workers=...) for I/O-bound work (decoding images, network):

labels = sessions.rating.value.map(lambda r: f"{r}/5")
tooltips = sessions.photo.pmap(lambda p: cg.Image.tooltip(p), workers=8)

NumPy#

.to_numpy() converts to an array (Missing/None become nan):

import numpy as np

ratings = sessions.rating.value.to_numpy()
print(np.nanmean(ratings))

cg.Map#

cg.Map keeps its keys and broadcasts attribute access over its values, so you can index by key or project a field:

sessions = survey.sessions

sessions.keys()                  # every key (e.g. capture datetimes)
sessions.values()                # the entries (an Array, broadcasts)
sessions.items()                 # (key, value) pairs

sessions[some_time]              # one entry by key
sessions.rating                  # broadcast a field over all entries

Order#

A Map's and an Array's entries always enumerate in ascending lexicographic order of their on-disk name — the same rule on every platform, so keys(), values(), and PathValues/PathKeys agree across the Python reader, the iOS client, and the sync tool. Because a Map[Time]/Map[Date] key is stored as a fixed-width hexadecimal-microsecond folder and an Array index as a zero-padded number, lexicographic order is chronological / numeric order — so the last entry is the most recent session:

latest = survey.sessions.values()[-1]   # the most recent session

A Map[String] key sorts by the key text itself.

Missing data#

A per-element lookup that fails yields cg.Missing for that element instead of crashing the whole projection:

moods = sessions.mood            # sparse field: some entries Missing

for mood in moods:
    if not cg.is_missing(mood):
        print(mood.value)

Missing absorbs further attribute access, so deep chains stay safe; .to_numpy() turns it into nan. To fill the gaps instead, | substitutes a default for every Missing/None leaf (recursing into nested Arrays):

ratings = sessions.rating.value | 0       # Missing entries become 0

Failing loudly vs. absent data#

Every broadcast surface — attribute access, string/tuple projection (arr["x"], arr["a", "b"]), .map/.pmap/.map_leaves, and calling the elements (arr()) — distinguishes a programmer error from legitimately absent data. A surface raises when every element fails for a non-data reason (a typo, a wrong key, a broken .map lambda — an AttributeError, KeyError, ZeroDivisionError, TypeError, …), so a mistake in an analysis script fails loudly instead of yielding a silent all-Missing result. The raised exception chains from the first element's failure.

The one carve-out is absence: a field the schema declares but a given entry never captured loads as Missing carrying a FileNotFoundError. When that is the reason on every entry, the projection stays all-Missing — so photos.exif() over a column of never-captured images is Missing, not an error. Any present element (a partial success) likewise never raises; only sparse data keeps chaining.

Complete Example#

import capturegraph as cg
import numpy as np
import pandas as pd

survey = cg.load("/path/to/MySurvey", Survey)
sessions = survey.sessions

df = pd.DataFrame({
    "date": list(sessions.keys()),
    "rating": sessions.rating.value.to_numpy(),
    "notes": list(sessions.notes.value),
})

print(f"Sessions: {len(sessions)}")
print(f"Mean rating: {np.nanmean(df['rating']):.1f}")

See Also#