Skip to content

Adding a log sink#

cg.logging records a tree of entries; a sink is where the tree shows. Three ship, one per strategy: the core package's Terminal appends lines to a stream, the ui package's cgui.Notebook redraws a cell, and the server's LiveLog coalesces changes onto an event loop for the browser. A sink of yours is a subclass of cg.logging.Sink:

Method When it is called
start(log) Once, as the cg.logging.to() block opens, with the Log about to be written. A no-op on the base class.
emit(entry) Once when an entry is made and once per change after that, in order, on the thread that logged, under the log's lock. Abstract — the one method to implement.
finish() Once, as the block ends; flush whatever is pending. A no-op on the base class.

start and finish are concrete no-ops, so a sink that needs only emit overrides nothing else. The real split is by strategy: an Entry holds no reference to its log and emit is handed nothing else, so a sink that redraws the tree must keep the Log from start, while a sink that appends works from the entry alone.

Two guarantees shape emit. The lock is an RLock held by the thread that logged, so emit may call log.entries() or log.children() without deadlocking — but keep it quick, since every logging call waits on it. An emit that raises is caught and logged; a broken sink degrades the display, never the computation.

What emit sees#

The entry carries every key of the wire shape as an attribute. The ones a sink decides from:

  • id — a string the log assigns in creation order ("1", "2", …), unique within the log.
  • parent — the parent's id, or None at the top level. It is an id, not an entry: a sink that walks upward keeps its own {id: entry} map, as window.cgLog.upsert does; a sink holding the Log descends instead with log.children(log.root) for the top level and log.children(entry) below it.
  • revision — 0 when the entry is made, one more on every change: a scope opening or closing, update, advance, each turn of a loop.
  • status — done, open or failed. Every entry is born done at revision 0; a with block, or iterating a Progress, turns it open, and the block's end sets done or failed, with error holding the exception's text.
  • message and fields — the line and its keyword data: cg.logging.warning("no depth", frame=3) is message == "no depth" and fields == {"frame": 3}.
  • done and total — the count of a Progress, None on a plain line; total is None for a count that does not know its length. Both shipped renderers draw a bar from them.
  • level, iteration, started and ended complete the shape; ended is None while a scope is open.

One block, as a sink sees it:

with cg.logging.info("scope"):
    cg.logging.info("line")
emit id parent revision status Why
1 "1" None 0 done info("scope") made the entry
2 "1" None 1 open with opened it
3 "2" "1" 0 done info("line") beneath it
4 "1" None 2 done the block ended

So a sink that keeps its own tree upserts by id, and a sink that prints decides from revision and status together — revision alone cannot tell a plain line from a scope about to open, since both arrive done at revision 0.

The smallest useful sink collects messages:

import capturegraph as cg


class Collect(cg.logging.Sink):
    def __init__(self) -> None:
        self.messages: list[str] = []

    def emit(self, entry: cg.logging.Entry) -> None:
        if entry.revision == 0:
            self.messages.append(entry.message)


sink = Collect()
with cg.logging.to(sink):
    for name in cg.logging.progress(["a", "b", "c"], label="Thumbnails"):
        cg.logging.info(f"thumbnail {name}")

print(sink.messages)  # ['Thumbnails', 'thumbnail a']

Two messages, not four: inside an iterated progress, a line logged from the same call site each iteration revises one entry instead of making a new one, so the sink sees a change, not a creation — one message per slot, not per call (Loops).

Redraw the whole tree#

A sink that shows the tree — a cell, a window, a web page — has an easier contract than one that prints: ignore what changed and render the whole log again from the Log kept in start. log.entries() is every entry, parents before children; log.children(log.root) is the top level and log.children(entry) descends. Rendering is a function of the log alone, so a redraw is idempotent and a skipped emit costs nothing; throttle by time, not by entry, so a step that logs ten thousand lines repaints ten times a second. Every shipped sink uses that figure — REFRESH_SECONDS = 0.1 in Notebook, REDRAW_SECONDS = 0.1 in Terminal, FLUSH_SECONDS = 0.1 in the server's LiveLog. The mechanism is a time.monotonic() stamp of the last redraw, compared in emit (Notebook and Terminal), or one deferred flush scheduled on the first change after a quiet moment (LiveLog); there is no base-class helper.

That is how cgui.Notebook works. cgui.log.html(log) renders a Log as one self-contained fragment of nested <details>, and the sink shows it through IPython's display handle, then replaces it. No redraw thread — IPython routes output by the thread's cell — so a step that stays silent repaints on its next entry. Trimmed to the contract:

import time

import capturegraph as cg
import capturegraph_ui as cgui
from IPython.display import HTML, display

REFRESH_SECONDS = 0.1


class Notebook(cg.logging.Sink):
    def start(self, log: cg.logging.Log) -> None:
        self._log = log
        self._handle = display(HTML(cgui.log.html(log)), display_id=True)
        self._shown_at = time.monotonic()

    def emit(self, entry: cg.logging.Entry) -> None:
        if time.monotonic() - self._shown_at >= REFRESH_SECONDS:
            self._refresh()

    def finish(self) -> None:
        self._refresh()

    def _refresh(self) -> None:
        self._handle.update(HTML(cgui.log.html(self._log)))
        self._shown_at = time.monotonic()

The shipped sink differs in one respect: it takes a show callable that displays the first fragment and returns the function that replaces it, with IPython's display handle as the default, so a test drives it without a kernel.

Beside html, cgui.log ships stylesheet() and script(): the stylesheet every rendering shares, and the browser renderer that builds the identical tree from a stream of JSON entries, which the server's computing view uses. window.cgLog.mount(container, entries, now) draws the snapshot — now is the server's clock, so an open entry's elapsed time reads correctly against a skewed browser clock — and upsert(container, entry) applies each entry as it streams in. A frontend of your own needs only the wire shape: Entry.json() per entry, Log.json() for the tree.

Append: what Terminal does instead#

A stream cannot take a line back, so the core sink decides per entry from revision and status. On a file each entry is one line as it is made, one more as a scope closes, and a revised line at most once a second (REPRINT_SECONDS = 1.0). On a tty the open entries are redrawn in place beneath the lines already printed, and a daemon thread started in start repaints them every REDRAW_SECONDS even while nothing logs, so an open scope's elapsed time keeps moving.

The sequence above has one consequence for an appending sink: a scope's creation emit arrives marked done, indistinguishable from a plain line. On a file that is harmless — the line is printed as made, and a closing line follows if it turns out to be a scope. On a tty a line is printed above the redrawn block for good while an open scope is drawn inside it, so the sink holds the creation one beat and commits it as a line only if the next emit does not open it. Terminal is the reference for a sink that must append: its file path is append-only.

Coalesce onto an event loop: the server's LiveLog#

The third strategy neither redraws nor appends. The page runner's LiveLog (server.pages.live) keeps the Log from start; emit marks the entry dirty by id under a lock of its own and, while someone is subscribed and no flush is scheduled, schedules one on the asyncio loop FLUSH_SECONDS later — nothing crosses to the loop per entry. The flush sends each dirty entry's Entry.json() to every subscriber's queue as an event; a subscriber that arrives mid-computation gets Log.json() as a snapshot first, then the stream. A sink feeding any async frontend has that shape: a dirty set, one scheduled flush, snapshot then stream.

Testing#

cg.logging.to() drives a sink with real entries, so a test asserts on what the sink collected or rendered:

def test_collect_keeps_one_message_per_entry() -> None:
    sink = Collect()
    with cg.logging.to(sink):
        with cg.logging.info("outer"):
            cg.logging.warning("inner", count=3)
    assert sink.messages == ["outer", "inner"]