Skip to content

resource

resource #

lock — exclusive use of a machine-wide resource, such as the GPU, for a block.

  1. Wait. lock(name) opens a log scope <name> lock and polls the resource's flock in the active scratch, updating the scope's message with who holds it and for how long (gpu lock: waiting for train (pid 4242), held 5m12s), so a page or terminal shows the wait.
  2. Hold. Once acquired, the lock file names the holder — the purpose it locked for, its pid and when — the scope's message becomes <name> lock: held and the scope stays open for the block, and the name is recorded in the context so a nested lock of the same name, or a pmap worker of the block, passes straight through instead of waiting on itself.
  3. Release. The block's end, or its exception, releases the flock and closes the scope; the kernel releases it on any kind of death.

held_resources = ContextVar('cg_held_resources', default=(frozenset())) module-attribute #

The resources the current context holds; a nested lock of one passes through.

lock(name) #

Hold the machine-wide resource name for the block, one holder at a time.

Holders are serialized across every thread and process sharing the active scratch; the wait and the hold are a scope in the log, so a computing page reads gpu lock: waiting for train (pid 4242), held 5m until the block gets its turn.

with cg.lock("gpu"):
    work.exec(["ns-train", "splatfacto", "--data", dataset])

Parameters:

Name Type Description Default
name str

The resource, "gpu" for the GPU.

required
Source code in capturegraph-lib/capturegraph/scratch/calls/resource.py
@contextmanager
def lock(name: str) -> Iterator[None]:
    """Hold the machine-wide resource ``name`` for the block, one holder at a time.

    Holders are serialized across every thread and process sharing the active
    scratch; the wait and the hold are a scope in the log, so a computing page
    reads ``gpu lock: waiting for train (pid 4242), held 5m`` until the block
    gets its turn.

    ```python
    with cg.lock("gpu"):
        work.exec(["ns-train", "splatfacto", "--data", dataset])
    ```

    Args:
        name: The resource, ``"gpu"`` for the GPU.
    """
    if name in held_resources.get():
        yield
        return
    purpose = _purpose()
    flock = active_scratch().lock_resource(name)
    with info(f"{name} lock") as holding:
        try:
            _wait(name, flock, holding)
            flock.claim(f"{purpose} (pid {os.getpid()}){SINCE}{time.time()}")
            holding.update(f"{name} lock: held")
            token = held_resources.set(held_resources.get() | {name})
            try:
                yield
            finally:
                held_resources.reset(token)
        finally:
            flock.release()