Pages and Visualizations#

A page is a function in the target module that takes the value at one schema position and returns a website as a folder, with all the data inside. The server calls it on the first visit, memoizes the folder by the value's content, and hosts it; a page links to another page by position, so every URL is known before anything is rendered. Usually the folder is a cgui.Page that a @cgui.PageBuilder() function built from the components of capturegraph-ui (Pages); it may also be any directory value holding an index.html, such as a report a tool wrote. The walkthrough adds two pages to a target step by step.

Pages are where the thesis becomes visible. Because the schema is declared before capture begins, analysis can run during the study rather than after it: a page is that analysis, addressed at a schema position and recomputed as data lands. Participants and investigators see coverage — which conditions the dataset still lacks — while there is still time to change what gets captured, which is the same information the interceptors use to decide whom to interrupt.

Every example on this page assumes three aliases. cgserver is the server's own authoring module, not a separate package:

import capturegraph as cg
import capturegraph_ui as cgui
from server import authoring as cgserver

A position is the target-relative path a value sits at, spelled as the file API spells it — sessions/00063B40E29D696A/photo, a hidden field _metadata as the dotfile .metadata. A pattern is a position with * for each container crossed: sessions/*/photo. The root is the empty path. A session's key is its capture instant as sixteen hex digits (sessions).

Quick start#

cd capturegraph-server && uv run capturegraph-server serve --root examples --port 4433

Upload one photograph as a new session (KEY mints a capture instant, sixteen hex digits of microseconds), visit its page, wait for the render, and fetch the hosted folder:

uv run python -c "from PIL import Image; Image.new('RGB', (640, 480), (90, 120, 40)).save('$HOME/photo.jpg')"
KEY=$(printf '%016X' $(( $(date +%s) * 1000000 )))
curl -X PUT -H 'X-CG-File-Extension: jpeg' --data-binary @"$HOME/photo.jpg" \
  "http://localhost:4433/api/v1/targets/PlantJournal/files/sessions/$KEY/photo"

BASE=http://localhost:4433/pages/PlantJournal/sessions/$KEY
curl -s -o /dev/null "$BASE/"
until curl -s "$BASE/~status" | grep -q '"state": *"ready"'; do sleep 1; done
curl -s "$BASE/~/"

In a browser, $BASE/ is the session's page, http://localhost:4433/pages/PlantJournal/ the journal's root page, and http://localhost:4433/pages the index of every page the server hosts.

Declaring a page#

@cgserver.page(Schema, /, select=None, *, version)

The capture API is unauthenticated, and so are pages: anyone who can reach the server can read any page. Only the management panel is password-gated — do not put a page on an open port you would not put the capture data on.

The function takes one parameter, named as you like and annotated with the type at the selected position, and returns a directory value whose folder holds an index.html. The usual form is a @cgui.PageBuilder() function: its body is annotated -> None and builds components, and the wrapper the decorator returns is annotated -> cgui.Page, which is what @cgserver.page sees. A plain function may instead return any directory scalar itself (below). Anything else is a DefinitionError, and where it is raised decides the damage. A selector the schema lacks, or a parameter or return annotation that does not fit, fails at decoration and so fails the whole module load, which the panel shows as a load error. Two pages at one pattern, or a page declared against a Struct other than the target's own, is caught after the module runs: the target keeps serving its procedure, but every page and every interceptor it declares is dropped, with the reason in the server log and under the target's interceptor status (Troubleshooting).

@cgserver.page(PlantJournal, version=1)
@cgui.PageBuilder()
def journal(target: PlantJournal) -> None: ...          # the root: ""


@cgserver.page(PlantJournal, lambda s: s.sessions, version=1)
@cgui.PageBuilder()
def sessions(sessions: cg.Map[cg.Date, Session]) -> None: ...   # the map: sessions


@cgserver.page(PlantJournal, lambda s: s.sessions.for_each(), version=1)
@cgui.PageBuilder()
def session(session: Session) -> None: ...              # each session: sessions/*


@cgserver.page(PlantJournal, lambda s: s.sessions.for_each().photo, version=1)
@cgui.PageBuilder()
def photo(photo: cg.Image) -> None: ...                 # each photo: sessions/*/photo

Under @cgui.PageBuilder() the body reads as the page, top to bottom: components attach in construction order, with on a stack nests, and cgui.Title names the page (Pages). Ordinary Python does the rest: a for loop builds one PageLink per session, an if decides whether to say something.

@cgserver.page(PlantJournal, version=1)
@cgui.PageBuilder()
def journal(target: PlantJournal) -> None:
    sessions = target.sessions.to_dict()
    cgui.Title("Plant Journal")
    cgui.Markdown("## Sessions")
    with cgui.HStack():
        cgui.Stat("Sessions", len(sessions))
        cgui.Stat("Days", len({instant.value.date() for instant in sessions}))
    cgui.Gallery.of(target.sessions, lambda s: s.photo)
    for instant, session in sessions.items():
        cgui.PageLink(session, title=instant.value.date().isoformat())

The gallery stages every photograph at full size and links each to its session's page; Thin assembly replaces the photographs with a cached thumbnail per session.

A page function is a cg.pure_function whose identity is "<function name>/<version>". A memo entry is keyed by that identity together with the content hash of the one argument, so a render serves until the value at the position changes or the version is bumped; editing the body without bumping the version keeps serving the old render, deliberately (caching guide). The call is ordinary and synchronous and its result seals into the scratch. The server loads the position's value with cg.load against the schema and passes it.

Selectors#

A selector is a plain function of the schema, run once at decoration against a stand-in that records the fields it walks. The grammar and its errors are the interceptors': for_each() is the only way past a Map or Array, a field of a container without it is a load error naming the fix, and indexing is a load error. What differs is where a page may stop.

• A page may stop anywhere. lambda s: s.sessions.for_each() is one page per session (sessions/*, parameter Session); lambda s: s.sessions.for_each().photo one per photograph (sessions/*/photo, parameter cg.Image), answering at …/photo/.
• A page may be declared at the container itself. lambda s: s.sessions is the single page of the whole map (sessions, parameter cg.Map[cg.Date, Session]), answering at …/sessions/; it exists as soon as the directory does.
• No data, no page. A position whose file or directory is absent is 404, whatever its type.

What a page is on disk#

A @cgui.PageBuilder() function returns a cgui.Page: the components its body built, in a root VStack, and the title a cgui.Title set. Nothing is written until the page is built, which the store does when the page function returns. The built folder is:

The title is the page's name wherever it is listed: the browser tab, the card at /pages, ~status. Derive it from the data rather than repeating the function's name. A page with no file props has no files/; one using no library has no lib/. Nothing in a sealed folder is ever modified.

Returning a folder that already exists#

A page function need not use cgui at all. It may return any directory scalar — a cg.DirScalar subclass, such as a directory value a tool wrote its own HTML report into — as long as the folder holds an index.html. The server hosts the folder as it stands at the position's ~/, serving every file inside it, and reads the page's title from the <title> of that index.html. cgui.Page is one such value, and cgui is one way to make one. A returned folder without index.html fails the render with an error naming index.html.

class Report(cg.DirScalar, name="edu.example.report"): ...


@cg.pure_function("coverage_report/1")
def coverage_report(sessions: cg.Map[cg.Date, Session]) -> Report:
    report = Report.new()
    (report / "index.html").write_text(
        f"<title>Coverage</title><p>{len(sessions)} sessions captured.</p>"
    )
    return report


@cgserver.page(PlantJournal, lambda s: s.sessions, version=1)
def sessions(sessions: cg.Map[cg.Date, Session]) -> Report:
    return coverage_report(sessions)

name="edu.example.report" is the scalar's reverse-DNS identifier, which names its byte layout; a format you define lives under your own domain (The name).

Such a folder links however its author wrote it; PageLink and the of constructors are how a cgui page links, and a hand-made folder holding an index.html is a cgui.Page as it stands: cgui.Page.load(folder) takes it, stores it byte for byte, and reads page.title from the <title>.

Links#

A page links to another value's page by the relative path between the two values' positions, plus one ../ for the ~/ segment the server hosts the folder under: from the root page a session is ../sessions/<key>/. The server sets which value a page is for (page.at) after the page function returns, so authors never do. A link is a position URL, and only a position with a page declared exactly at its pattern answers, so link to values whose pattern the target declares a page at.

The only components that link to another value's page are PageLink and the of constructors. PageLink(value) is a card to the page of a value loaded from the target. Gallery.of, Timelapse.of and MapViewer.of walk a loaded container and link each derived item — a thumbnail computed per session, which has no position of its own — to the page of the entry it came from. Every other component shows its data and links nowhere; a [text](url) in cgui.Markdown is an ordinary hyperlink you wrote, not a position link. The rules in full are in Pages.

URLs#

Every URL under /pages is a target name (which may nest, Plants/Tulip), a target-relative path spelled as the file API spells it, and an optional ask after a ~ segment. A trailing slash is optional.

/pages is the index: every page position as a tree of links, under the server's configured name. Targets nest under their group directories. A target with a root page links to it, captioned by that page's <title> once it has a render and by the target's name until then. Beneath a target, each field with a page is a link, and a Map or Array folds open into its entries on disk, each captioned by its key and linked when a page is declared at that pattern. Only positions with data are listed.

A component that could escape a directory is 400; a path that is not a position of the schema, a key that is not canonical, an unknown ask, or an unknown target is 404.

Components#

The components come from capturegraph-ui. The standard vocabulary, and the rule that a component given Missing for any prop renders as nothing, are in Pages; a component of your own is Adding a component.

Thin assembly over cached per-session calls#

A page's key is the content of the value at its position, so the root page is recomputed on the first visit after any of the target's files change. That is cheap as long as the root's body is assembly: it walks the loaded data and calls memoized per-session recipes, so a new upload costs one session's work plus the assembly, never the whole study again.

def _thumbnail(session: Session) -> cg.Thumbnail:
    small = cg.Thumbnail.new()            # rendered inside the root's call
    session.photo.pil(max_axis=256).save(small)
    return small


@cgserver.page(PlantJournal, version=1)
@cgui.PageBuilder()
def journal(target: PlantJournal) -> None:
    cgui.Title("Plant Journal")
    cgui.Gallery.of(target.sessions, _thumbnail)

@cg.pure_function("session_thumbnail/1")
def session_thumbnail(photo: cg.Image) -> cg.Thumbnail:
    thumbnail = cg.Thumbnail.new()
    photo.pil(max_axis=256).save(thumbnail)
    return thumbnail


@cgserver.page(PlantJournal, version=1)
@cgui.PageBuilder()
def journal(target: PlantJournal) -> None:
    cgui.Title("Plant Journal")
    cgui.Gallery.of(target.sessions, lambda s: session_thumbnail(s.photo))

pil(max_axis=256) opens an image as a PIL image with its longest side resized to 256 pixels, and cg.Thumbnail is the JPEG file scalar the app's galleries read. Anything that scales with the number of sessions belongs in a per-session cg.pure_function, or in a page declared at the session, whose key is that session alone. Anything that scales with the whole dataset is either cheap enough for the root page or its own memoized call over exactly the data it needs.

What computing looks like#

The first visit to a position with no sealed folder for its input shows Calculating… and subscribes to ~events. The page function runs under a log sink of the runner's own, and the view draws the tree as it streams: the page's own call at the root, every memoized call, map, pmap, lock and tool run beneath it nested, each scope collapsible, each count a bar, each tool's last lines of output beneath its command, a line logged every iteration of a loop revised in place rather than repeated. A gallery of forty thumbnails reads as one count filling up, a worker count per thread beneath it, each showing the thumbnail call it is on right now or the session_thumbnail: cached line it just served. A recipe waiting for the GPU reads as a gpu lock scope whose message names what holds it (global locks), so a page queued behind another target's training says so rather than sitting silent. When the final state arrives the view reloads into the page. Each target has one worker thread, so a slow page holds up only its own target, and two overlapping first visits compute once.

A visit always calls the memoized page function. A page that raises seals nothing; once the attempt ends the view shows the error text and its notes, the exec.log tail among them, above the full tree with the failed scope marked, and the traceback is in the server log under the page's URL. ~status reports failed with that error and log until the next visit, which calls the function again, so reloading a failed page tries again; pending means nothing is computed for these inputs and nothing is in flight.

The build log#

When the attempt ends, the runner stores the tree under <root>/.cg-pages/<target>/<path…>/ and serves it at ~log, which every hosted page's banner links as Build log; while a build runs, ~log shows the live view instead. The sealed page itself never carries the log, so a recompute seals to the same hash. The server root's .gitignore excludes .cg-pages/ alongside the scratch.

Across workers#

The live log lives in the process computing the page. With workers > 1 (configuration) a browser served by another worker learns only that the call is claimed — the runner reads the key's call lock — and polls ~status until the folder is ready. ~log follows the same split: the worker running the build shows it live, any other worker shows the last stored build (or 404 if there is none) until this one finishes; afterwards the stored log is on disk for every worker. If the process computing a page dies, its ~events stream ends without a state event and the kernel releases the call lock with it: the view falls back to polling ~status, which now says pending, and nothing recomputes until the position URL is visited again.

Eviction and recompute#

A render is an object in the recipe scratch like any other result. The runner holds each position's current render under the scratch name page:<target>/<path>, pinned while the server runs; the rest is evictable history. A position computes only when someone visits it, and then only when no sealed folder answers for the value at its position: it has changed, its render was evicted, the version was bumped, or the last attempt raised. An evicted render recomputes to the same hash, so old links heal. Nothing computes on a restart: an unchanged position serves from the scratch at once.

The app's button#

A hosted target in the app fetches ~pages when the target opens and shows a Visualizations button in the explorer's actions toolbar whenever the shown value's path matches a declared pattern, * standing for exactly one component. It opens <server>/pages/<target>/<path…>/ in the browser, ready or not. A target without pages answers {"patterns": []} and the button never appears.

Troubleshooting#

• 404 at a position URL. No page is declared at that path's pattern, or nothing has been captured there yet: ~pages lists the patterns, and the file API says what is on disk.
• RuntimeError: no page is being built. A cgui.Title or a with on a stack ran outside a @cgui.PageBuilder() function: a helper that builds part of a page must be called from inside one, or return the components for the caller to place. A stack built outside a builder takes children= instead of with.
• The build fails naming a PageLink. with no position in a target: the value was built in memory (a .clone() included) or computed by a recipe; link the loaded value instead. the page has no at: the page was built outside the server without at=.
• The render fails naming index.html. The page function returned a directory value whose folder holds no index.html; write one there, or return a cgui.Page.
• A component appears twice. It was built inside a builder function and then also passed as children= to a stack; inside a builder, build it under the with block instead.
• A gallery's pictures do not link. The pictures are derived and the gallery was built with Gallery(...) rather than Gallery.of(...), or the container came from memory rather than the loaded target.
• 404 under ~/ that used to work. The render was evicted or replaced by a newer one. Open the position URL: an evicted render recomputes to the same hash.
• The root page recomputes after every upload. By design. If each recompute is slow, move per-session work into cg.pure_functions (above) or into a page at the session.
• A target declares a page but ~pages says it declares none, and its interceptors are dead too. The module's declarations were rejected. Two pages at one pattern, or a page declared against a Struct other than the target's own, drops every page and every interceptor in the module while the target keeps serving its procedure — one cause for both symptoms. The server log carries the Invalid declarations: … line, the panel's code editor shows it as a failing badge in the interceptor strip rather than as a load error, and Save & go live refuses the same module. A selector the schema lacks, or a parameter or return annotation that does not fit, fails the whole module load instead, which does show as a load error.
• The index shows the target's name, not the page's title. Its root page has not been rendered yet, or it declares no root page.