Pages and Visualizations#

A page function takes in data and returns a website as a folder, all the data's inside. The server hosts that folder. Usually the folder is a cg.Page that a @cg.ui.PageBuilder() function built from cg.ui components; it may also be any directory value holding an index.html, such as a report a tool wrote. A page is declared beside the target it reads, at a schema position, the way an interceptor is; the server calls the root page function with the loaded target, memoized by the input's content hash, so an unchanged target is instant, and hosts the folder it returned at /pages/<target>/~/. The root page links to /pages/<target>/sessions/<16-hex>/, a URL known ahead of time from the position, and visiting it calls the session page function with that session, cached the same way. Nothing in a sealed folder is ever modified. The walkthrough adds two pages to a target step by step.

Pages are where the thesis becomes visible. Because the schema is declared before capture begins, analysis can run during the study rather than after it: a page is that analysis, addressed at a schema position and recomputed as data lands. Participants and investigators see coverage — which conditions the dataset still lacks — while there is still time to change what gets captured, which is the same information the interceptors use to decide whom to interrupt.

Quick start#

cd capturegraph-server && uv run capturegraph-server serve --root examples --port 4433

uv run python -c "from PIL import Image; Image.new('RGB', (640, 480), (90, 120, 40)).save('$HOME/photo.jpg')"
KEY=$(printf '%016X' $(( $(date +%s) * 1000000 )))
curl -X PUT -H 'X-CG-File-Extension: jpeg' --data-binary @"$HOME/photo.jpg" \
  "http://localhost:4433/api/v1/targets/PlantJournal/files/sessions/$KEY/photo"

Open http://localhost:4433/pages/PlantJournal/sessions/$KEY/, then http://localhost:4433/pages/PlantJournal/, then http://localhost:4433/pages. From a shell:

BASE=http://localhost:4433/pages/PlantJournal/sessions/$KEY
curl -s -o /dev/null "$BASE/"
until curl -s "$BASE/~status" | grep -q '"state": *"ready"'; do sleep 1; done
curl -s "$BASE/~/"

Declaring a page#

@cgserver.page(Schema, /, select=None, *, version)

A page is as public as the target's files: whatever access rule the server applies to /api/v1/targets/<target>/files/… applies to its pages, and nothing else gates them.

The function takes one parameter, named as you like and annotated with the type at the selected position, and returns a directory value whose folder holds an index.html. The usual form is a @cg.ui.PageBuilder() function: its body is annotated -> None and builds components, and the wrapper the decorator returns is annotated -> cg.Page, which is what @cgserver.page sees. A plain function may instead return any directory scalar itself (below). Anything else, two pages at one pattern, or a selector the schema lacks is a DefinitionError when the module loads, reported through the target's status.

@cgserver.page(PlantJournal, version=1)
@cg.ui.PageBuilder()
def journal(target: PlantJournal) -> None: ...          # the root: ""


@cgserver.page(PlantJournal, lambda s: s.sessions, version=1)
@cg.ui.PageBuilder()
def sessions(sessions: cg.Map[cg.Date, Session]) -> None: ...   # the map: sessions


@cgserver.page(PlantJournal, lambda s: s.sessions.for_each(), version=1)
@cg.ui.PageBuilder()
def session(session: Session) -> None: ...              # each session: sessions/*


@cgserver.page(PlantJournal, lambda s: s.sessions.for_each().photo, version=1)
@cg.ui.PageBuilder()
def photo(photo: cg.Image) -> None: ...                 # each photo: sessions/*/photo

Under @cg.ui.PageBuilder() the body reads as the page, top to bottom: every component it constructs attaches to the innermost open stack in construction order, the way procedure nodes attach to the recording context under @cg.procedure; with cg.ui.HStack(): or with cg.ui.VStack(): opens a nested stack for the block; cg.ui.Title(...) names the page and renders nothing in the body. Ordinary Python does the rest: a for loop builds one PageLink per session, an if decides whether to say something.

@cgserver.page(PlantJournal, version=1)
@cg.ui.PageBuilder()
def journal(target: PlantJournal) -> None:
    cg.ui.Title("Plant Journal")
    cg.ui.Markdown("## Sessions")
    with cg.ui.HStack():
        cg.ui.Stat("Sessions", len(target.sessions))
        cg.ui.Stat("Days", 4)
    cg.ui.Gallery.of(target.sessions, lambda s: session_thumbnail(s.photo))
    for instant, session in target.sessions.to_dict().items():
        cg.ui.PageLink(session, title=instant.date().isoformat())

A page function is a cg.pure_function, so the recipes guide holds: it is an ordinary synchronous call, its result seals into the scratch, and editing the body without bumping the version keeps serving the old render. The server loads the position's value with cg.load and passes it.

Selectors#

A selector is a plain function of the schema, run once at decoration against a stand-in that records the fields it walks. The grammar and its errors are the interceptors': for_each() is the only way past a Map or Array, a field of a container without it is a load error naming the fix, and indexing is a load error. What differs is where a page may stop.

• A page may stop anywhere. lambda s: s.sessions.for_each() is one page per session (sessions/*, parameter Session); lambda s: s.sessions.for_each().photo one per photograph (sessions/*/photo, parameter cg.Image), answering at …/photo/.
• A page may be declared at the container itself. lambda s: s.sessions is the single page of the whole map (sessions, parameter cg.Map[cg.Date, Session]), answering at …/sessions/; it exists as soon as the directory does.
• No data, no page. A position whose file or directory is absent is 404, whatever its type.

What a page is on disk#

A @cg.ui.PageBuilder() function returns a cg.Page: the components its body built, in a root VStack, and the title a cg.ui.Title set. Nothing is written until the page is built, which the store does when the page function returns. The built folder is:

The title is the page's name wherever it is listed: the browser tab, the card at /pages, ~status. Derive it from the data rather than repeating the function's name. A page with no file props has no files/; one using no library has no lib/.

Outside a builder, a stack takes its children outright and cg.Page.declare(root, title=None) makes the page from the tree: cg.Page.declare(cg.ui.VStack(children=[cg.ui.Markdown("# Hi")]), title="Hi"). Inside a builder, children= is not needed and doubles a component that was already attached to the open stack.

Returning a folder that already exists#

A page function need not use cg.ui at all. It may return any directory scalar — a cg.DirScalar subclass, such as a directory value a tool wrote its own HTML report into — as long as the folder holds an index.html. The server hosts the folder as it stands at the position's ~/, serving every file inside it, and reads the page's title from the <title> of that index.html. cg.Page is one such value, and cg.ui is one way to make one. A returned folder without index.html fails the render with an error naming index.html.

class Report(cg.DirScalar, name="edu.example.report"): ...


@cg.pure_function("coverage_report/1")
def coverage_report(sessions: cg.Map[cg.Date, Session]) -> Report:
    report = Report.new()
    (report / "index.html").write_text(
        f"<title>Coverage</title><p>{len(sessions)} sessions captured.</p>"
    )
    return report


@cgserver.page(PlantJournal, lambda s: s.sessions, version=1)
def sessions(sessions: cg.Map[cg.Date, Session]) -> Report:
    return coverage_report(sessions)

Such a folder links however its author wrote it; PageLink and the of constructors are how a cg.ui page links, and a hand-made folder holding an index.html is a cg.Page as it stands: cg.Page.load(folder) takes it, stores it byte for byte, and reads page.title from the <title>.

Links#

A page links to another value's page by the relative path between the two values' on-disk positions, plus one ../ for the ~/ segment the server hosts the folder under. The server sets which value a page is for (page.at) after the page function returns, so authors never do. From the root page a session links as ../sessions/<key>/; from a session page the root is ../../../, a sibling ../../<key>/, the page itself ../. A link is a position URL, and only a position with a page declared exactly there answers, so link to values whose pattern the target declares a page at. Only PageLink and the of constructors link; every other component shows its data and links nowhere.

• cg.ui.PageLink(value, title=None) is a card linking to value's page, captioned title or the value's folder name. value must have a position in the target: the page function's argument and everything reached through it, or anything a cg.load returns. A value built in memory, changed since it was loaded, or computed by a recipe fails the build with a message naming its type; so does a PageLink in a page with no at.
• Pictures and markers link through the of constructors. Gallery.of(container, image), Timelapse.of(container, image) and MapViewer.of(container, location, label=None, anchor=None, marker="pin", outline=None) walk a loaded Map or Array with pmap, so the per-entry recipes run in parallel, and link each result to its entry's page, not to wherever the result lives. A thumbnail computed per session has no position of its own; Gallery.of(target.sessions, thumbnail) links it to its session. A function returning None or Missing leaves that entry out, so Gallery.of(target.sessions, lambda s: thumbnail(s.photo)) skips a session whose photo never landed: a recipe called with a Missing argument returns Missing. A function that raises on an entry, or returns a Missing carrying an error, leaves that entry out the same way; the walk fails only when every entry fails. Over a container built in memory, of draws the items unlinked.
• A Scatter plots columns, not entries. Its columns are one quantity per point — frames.iso, sun.altitude, a plain list — every column the same length, and the dict's keys are the axis menus and the axis titles. A value that was never recorded is absent (a Missing from a broadcast column reads as one), so that point sits off the axes showing it and a column with nothing recorded is left out of the menus altogether. Broadcast access is what builds a column: frames.exif.iso reads the field off every element, and frames["latitude", "iso"] builds rows of several at once.
• Without of, nothing links: ImageViewer(session.photo), Gallery(target.sessions.photo), Timelapse(...) and MapViewer(locations) show their pictures and markers as they are. A keyed collection still captions each item by its key.

URLs#

Every URL under /pages is a target name (which may nest, Plants/Tulip), a target-relative path spelled as the file API spells it, and an optional ask after a ~ segment. A trailing slash is optional.

A component that could escape a directory is 400; a path that is not a position of the schema, a key that is not canonical, an unknown ask, or an unknown target is 404.

Components#

Every component is a class in cg.ui; its annotated fields are its props, checked when it is built. A component given Missing for any prop renders as nothing: cg.ui.Stat("Wind", session.weather.wind_speed_mps) leaves the page when no weather was recorded, and so does cg.ui.ImageViewer(web_jpeg(session.photo)), because the recipe returns Missing for a missing photo. Page code therefore reads the value it wants and lets absence fall out, rather than guarding with cg.is_missing; guard only where absence changes what to say. A page is self-contained except for the map tiles, which MapViewer fetches from OpenStreetMap.

cg.ui.PageBuilder() is the decorator that turns a component-building function into a page function: under it, each component below attaches to the innermost open stack as it is built, with on a stack nests, and Title names the page. The builder's root is a VStack, so a page with no stacks of its own is its components top to bottom.

A suite declares its own by subclassing cg.ui.Component, annotating the props, and implementing html(self, render). cg.ui.element and cg.ui.text write escaped HTML; the cg.ui.Render handed in stages files (render.stage), copies a shipped library (render.lib), emits a script once per page (render.once), and resolves links (render.href). Built inside a builder function, a custom component attaches to the open stack like a standard one, and a Missing prop makes it render nothing, with no code of its own for either.

class Badge(cg.ui.Component):
    text: str

    def html(self, render: cg.ui.Render) -> str:
        return cg.ui.element("span", {"class": "badge"}, cg.ui.text(self.text))


@cgserver.page(PlantJournal, lambda s: s.sessions.for_each(), version=1)
@cg.ui.PageBuilder()
def session(session: Session) -> None:
    cg.ui.Title("Session")
    Badge("photographed" if not cg.is_missing(session.photo) else "pending")
    cg.ui.ImageViewer(session.photo)

Thin assembly over cached per-session calls#

A page's key is the content of the value at its position, so the root page is recomputed on the first visit after any of the target's files change. That is cheap as long as the root's body is assembly: it walks the loaded data and calls memoized per-session recipes, so a new upload costs one session's work plus the assembly, never the whole study again.

def _thumbnail(session: Session) -> cg.Thumbnail:
    small = cg.Thumbnail.new()            # rendered inside the root's call
    session.photo.pil(max_axis=256).save(small)
    return small


@cgserver.page(PlantJournal, version=1)
@cg.ui.PageBuilder()
def journal(target: PlantJournal) -> None:
    cg.ui.Title("Plant Journal")
    cg.ui.Gallery.of(target.sessions, _thumbnail)

@cg.pure_function("session_thumbnail/1")
def session_thumbnail(photo: cg.Image) -> cg.Thumbnail:
    thumbnail = cg.Thumbnail.new()
    photo.pil(max_axis=256).save(thumbnail)
    return thumbnail


@cgserver.page(PlantJournal, version=1)
@cg.ui.PageBuilder()
def journal(target: PlantJournal) -> None:
    cg.ui.Title("Plant Journal")
    cg.ui.Gallery.of(target.sessions, lambda s: session_thumbnail(s.photo))

Anything that scales with the number of sessions belongs in a per-session cg.pure_function, or in a page declared at the session, whose key is that session alone. Anything that scales with the whole dataset is either cheap enough for the root page or its own memoized call over exactly the data it needs.

What computing looks like#

The first visit to a position with no sealed folder for its input shows Calculating…, polls ~status four times a second, and reloads once the folder is ready. The panel draws the step tree from the scratch's steps table (recipes): the page's own call at the root, every memoized call, map, pmap and cg.step block beneath it nested, each with its elapsed seconds, a bar and count where it has a total, and its latest note; a line above sums the counts into 17 of 40 steps done. A step that finishes stays listed as done, so the tree grows as the page computes, and the last lines of the deepest running call's exec.log follow it. A gallery of forty thumbnails reads as one Gallery step filling up, with the thumbnail calls computing right now beneath it. A recipe waiting for the GPU reads as a gpu lock step whose note names what holds it (global locks), so a page queued behind another target's training says so rather than sitting silent. The state is shared across worker processes, because the runner asks the scratch rather than its own memory. Each target has one worker thread, so a slow page holds up only its own target, and two overlapping first visits compute once.

A visit always calls the memoized page function. A page that raises seals nothing; once the attempt ends the computing page shows the error text and its notes, the exec.log tail among them, and the traceback is in the server log under the page's URL. ~status reports failed with that error until the next visit, which calls the function again, so reloading a failed page tries again; pending means nothing is computed for these inputs and nothing is in flight.

Eviction and recompute#

A render is an object in the recipe scratch like any other result. The runner holds each position's current render under the scratch name page:<target>/<path>, pinned while the server runs; the rest is evictable history. A position computes only when someone visits it, and then only when no sealed folder answers for the value at its position: it has changed, its render was evicted, the version was bumped, or the last attempt raised. An evicted render recomputes to the same hash, so old links heal. Nothing computes on a restart: an unchanged position serves from the scratch at once.

The app's button#

A hosted target in the app fetches ~pages once per open and shows a Visualizations button in the explorer's actions toolbar whenever the shown value's path matches a declared pattern, * standing for exactly one component. It opens <server>/pages/<target>/<path…>/ in the browser, ready or not. A target without pages answers {"patterns": []} and the button never appears.

Troubleshooting#

• 404 at a position URL. No page is declared at that path's pattern, or nothing has been captured there yet: ~pages lists the patterns, and the file API says what is on disk.
• RuntimeError: no page is being built. A cg.ui.Title or a with on a stack ran outside a @cg.ui.PageBuilder() function: a helper that builds part of a page must be called from inside one, or return the components for the caller to place. A stack built outside a builder takes children= instead of with.
• The build fails naming a PageLink. with no position in a target: the value was built in memory, computed by a recipe, or changed since it was loaded; link the loaded value instead. the page has no at: the page was built outside the server without at=.
• The render fails naming index.html. The page function returned a directory value whose folder holds no index.html; write one there, or return a cg.Page.
• A component appears twice. It was built inside a builder function and then also passed as children= to a stack; inside a builder, build it under the with block instead.
• A gallery's pictures do not link. The pictures are derived and the gallery was built with Gallery(...) rather than Gallery.of(...), or the container came from memory rather than the loaded target.
• 404 under ~/ that used to work. The render was evicted or replaced by a newer one. Open the position URL: an evicted render recomputes to the same hash.
• The root page recomputes after every upload. By design. If each recompute is slow, move per-session work into cg.pure_functions (above) or into a page at the session.
• A target declares a page but its URL says it declares none. The module failed to load; the target's status in the management panel and the server log carry the error.
• The index shows the target's name, not the page's title. Its root page has not been rendered yet, or it declares no root page.