Adding a log sink#
cg.logging records a tree of entries; a sink is
where the tree shows. Three ship, one per strategy: the core package's
Terminal appends lines to
a stream, the ui package's
cgui.Notebook redraws a
cell, and the server's LiveLog coalesces changes onto an event loop for the
browser. A sink of yours is a subclass of
cg.logging.Sink:
| Method | When it is called |
|---|---|
start(log) |
Once, as the cg.logging.to() block opens, with the Log about to be written. A no-op on the base class. |
emit(entry) |
Once when an entry is made and once per change after that, in order, on the thread that logged, under the log's lock. Abstract — the one method to implement. |
finish() |
Once, as the block ends; flush whatever is pending. A no-op on the base class. |
start and finish are concrete no-ops, so a sink that needs only emit
overrides nothing else. The real split is by strategy: an
Entry holds no reference to
its log and emit is handed nothing else, so a sink that redraws the tree
must keep the Log from start, while a sink that appends works from the
entry alone.
Two guarantees shape emit. The lock is an RLock held by the thread that
logged, so emit may call log.entries() or log.children() without
deadlocking — but keep it quick, since every logging call waits on it. An
emit that raises is caught and logged; a broken sink degrades the display,
never the computation.
What emit sees#
The entry carries every key of the wire shape as an attribute. The ones a sink decides from:
id— a string the log assigns in creation order ("1","2", …), unique within the log.parent— the parent'sid, orNoneat the top level. It is an id, not an entry: a sink that walks upward keeps its own{id: entry}map, aswindow.cgLog.upsertdoes; a sink holding theLogdescends instead withlog.children(log.root)for the top level andlog.children(entry)below it.revision—0when the entry is made, one more on every change: a scope opening or closing,update,advance, each turn of a loop.status—done,openorfailed. Every entry is borndoneat revision 0; awithblock, or iterating aProgress, turns itopen, and the block's end setsdoneorfailed, witherrorholding the exception's text.messageandfields— the line and its keyword data:cg.logging.warning("no depth", frame=3)ismessage == "no depth"andfields == {"frame": 3}.doneandtotal— the count of aProgress,Noneon a plain line;totalisNonefor a count that does not know its length. Both shipped renderers draw a bar from them.level,iteration,startedandendedcomplete the shape;endedisNonewhile a scope is open.
One block, as a sink sees it:
emit |
id |
parent |
revision |
status |
Why |
|---|---|---|---|---|---|
| 1 | "1" |
None |
0 | done |
info("scope") made the entry |
| 2 | "1" |
None |
1 | open |
with opened it |
| 3 | "2" |
"1" |
0 | done |
info("line") beneath it |
| 4 | "1" |
None |
2 | done |
the block ended |
So a sink that keeps its own tree upserts by id, and a sink that prints
decides from revision and status together — revision alone cannot tell a
plain line from a scope about to open, since both arrive done at revision 0.
The smallest useful sink collects messages:
import capturegraph as cg
class Collect(cg.logging.Sink):
def __init__(self) -> None:
self.messages: list[str] = []
def emit(self, entry: cg.logging.Entry) -> None:
if entry.revision == 0:
self.messages.append(entry.message)
sink = Collect()
with cg.logging.to(sink):
for name in cg.logging.progress(["a", "b", "c"], label="Thumbnails"):
cg.logging.info(f"thumbnail {name}")
print(sink.messages) # ['Thumbnails', 'thumbnail a']
Two messages, not four: inside an iterated progress, a line logged from the
same call site each iteration revises one entry instead of making a new one,
so the sink sees a change, not a creation — one message per slot, not per
call (Loops).
Redraw the whole tree#
A sink that shows the tree — a cell, a window, a web page — has an easier
contract than one that prints: ignore what changed and render the whole log
again from the Log kept in start. log.entries() is every entry, parents
before children; log.children(log.root) is the top level and
log.children(entry) descends. Rendering is a function of the log alone, so
a redraw is idempotent and a skipped emit costs nothing; throttle by time,
not by entry, so a step that logs ten thousand lines repaints ten times a
second. Every shipped sink uses that figure — REFRESH_SECONDS = 0.1 in
Notebook, REDRAW_SECONDS = 0.1 in Terminal, FLUSH_SECONDS = 0.1 in
the server's LiveLog. The mechanism is a time.monotonic() stamp of the
last redraw, compared in emit (Notebook and Terminal), or one deferred
flush scheduled on the first change after a quiet moment (LiveLog); there
is no base-class helper.
That is how cgui.Notebook works.
cgui.log.html(log) renders a
Log as one self-contained fragment of nested <details>, and the sink
shows it through IPython's display handle, then replaces it. No redraw
thread — IPython routes output by the thread's cell — so a step that stays
silent repaints on its next entry. Trimmed to the contract:
import time
import capturegraph as cg
import capturegraph_ui as cgui
from IPython.display import HTML, display
REFRESH_SECONDS = 0.1
class Notebook(cg.logging.Sink):
def start(self, log: cg.logging.Log) -> None:
self._log = log
self._handle = display(HTML(cgui.log.html(log)), display_id=True)
self._shown_at = time.monotonic()
def emit(self, entry: cg.logging.Entry) -> None:
if time.monotonic() - self._shown_at >= REFRESH_SECONDS:
self._refresh()
def finish(self) -> None:
self._refresh()
def _refresh(self) -> None:
self._handle.update(HTML(cgui.log.html(self._log)))
self._shown_at = time.monotonic()
The shipped sink differs in one respect: it takes a show callable that
displays the first fragment and returns the function that replaces it, with
IPython's display handle as the default, so a test drives it without a
kernel.
Beside html, cgui.log ships stylesheet() and script(): the
stylesheet every rendering shares, and the browser renderer that builds the
identical tree from a stream of JSON entries, which the server's computing
view uses.
window.cgLog.mount(container, entries, now) draws the snapshot — now is
the server's clock, so an open entry's elapsed time reads correctly against a
skewed browser clock — and upsert(container, entry) applies each entry as
it streams in. A frontend of your own needs only the
wire shape: Entry.json() per entry,
Log.json() for the tree.
Append: what Terminal does instead#
A stream cannot take a line back, so the core sink decides per entry from
revision and status. On a file each entry is one line as it is made, one
more as a scope closes, and a revised line at most once a second
(REPRINT_SECONDS = 1.0). On a tty the open entries are redrawn in place
beneath the lines already printed, and a daemon thread started in start
repaints them every REDRAW_SECONDS even while nothing logs, so an open
scope's elapsed time keeps moving.
The sequence above has one consequence for an appending sink: a scope's
creation emit arrives marked done, indistinguishable from a plain line. On
a file that is harmless — the line is printed as made, and a closing line
follows if it turns out to be a scope. On a tty a line is printed above the
redrawn block for good while an open scope is drawn inside it, so the sink
holds the creation one beat and commits it as a line only if the next emit
does not open it. Terminal is the reference for a sink that must append:
its file path is append-only.
Coalesce onto an event loop: the server's LiveLog#
The third strategy neither redraws nor appends. The page runner's LiveLog
(server.pages.live) keeps the Log from start; emit marks the entry
dirty by id under a lock of its own and, while someone is subscribed and no
flush is scheduled, schedules one on the asyncio loop FLUSH_SECONDS later —
nothing crosses to the loop per entry. The flush sends each dirty entry's Entry.json() to every
subscriber's queue as an event; a subscriber that arrives mid-computation
gets Log.json() as a snapshot first, then the stream. A sink feeding any
async frontend has that shape: a dirty set, one scheduled flush, snapshot
then stream.
Testing#
cg.logging.to() drives a sink with real entries, so a test asserts on what
the sink collected or rendered: