Skip to content

resource

resource #

lock — exclusive use of a machine-wide resource, such as the GPU, for a block.

  1. Wait. lock(name) opens a step labelled <name> lock beneath the ambient step and polls the resource's flock in the active scratch, noting who holds it and for how long, so a page or terminal shows the wait.
  2. Hold. Once acquired, the step notes held and stays open for the block, the lock file names the holding step, and the name is recorded in the context so a nested lock of the same name, or a pmap worker of the block, passes straight through instead of waiting on itself.
  3. Release. The block's end, or its exception, releases the flock and closes the step; the kernel releases it on any kind of death.

GPU = 'gpu' module-attribute #

The one resource every GPU-bound recipe locks.

held_resources = ContextVar('cg_held_resources', default=(frozenset())) module-attribute #

The resources the current context holds; a nested lock of one passes through.

lock(name) #

Hold the machine-wide resource name for the block, one holder at a time.

Holders are serialized across every thread and process sharing the active scratch; the wait and the hold are a step in the progress tree, so a computing page reads gpu lock waiting for train (pid 4242), held 5m until the block gets its turn.

with cg.lock("gpu"):
    work.exec(["ns-train", "splatfacto", "--data", dataset])

Parameters:

Name Type Description Default
name str

The resource, "gpu" for the GPU.

 required

Raises:

Type Description
ScratchNotConfigured

If no scratch is configured.
Source code in capturegraph-lib/capturegraph/recipes/calls/resource.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
@contextmanager
def lock(name: str) -> Iterator[None]:
    """Hold the machine-wide resource ``name`` for the block, one holder at a time.

    Holders are serialized across every thread and process sharing the active
    scratch; the wait and the hold are a step in the progress tree, so a
    computing page reads ``gpu lock  waiting for train (pid 4242), held 5m``
    until the block gets its turn.

    ```python
    with cg.lock("gpu"):
        work.exec(["ns-train", "splatfacto", "--data", dataset])
    ```

    Args:
        name: The resource, ``"gpu"`` for the GPU.

    Raises:
        ScratchNotConfigured: If no scratch is configured.
    """
    if name in held_resources.get():
        yield
        return
    scratch = active_scratch()
    flock = scratch.lock_resource(name)
    with opened(Step(f"{name} lock")) as holding:
        try:
            _wait(flock, scratch, holding)
            flock.claim(holding.id)
            holding.note("held")
            token = held_resources.set(held_resources.get() | {name})
            try:
                yield
            finally:
                held_resources.reset(token)
        finally:
            flock.release()