parallel
parallel
#
The one thread pool every pmap runs on, and the parallel map rule.
- Pool. One process-wide pool, sized to the CPU count. A
pmapmade from inside a worker runs sequentially, the outer pool being the parallelism, so nested fan-out never oversubscribes and never waits on the pool it occupies. - Cap.
parallelism(workers)bounds how many elements eachpmapin the block has in flight at once; the default is the pool's size. - Map.
pmappedisarray.mappedacross the pool: the same results and failure rule, a logged count of the elements as they finish with oneworkerentry per pool thread beneath it, each element one turn of its worker; every element runs in a copy of the caller's context so the active scratch, workspace and log scope carry over. The first element to raise propagates as soon as it finishes, and elements not yet started are cancelled.
POOL_SIZE = os.cpu_count() or 1
module-attribute
#
How many threads the pool runs, and the default cap on elements in flight.
parallelism(workers)
#
Cap how many elements each pmap in the block has in flight at once.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
workers
|
int
|
The cap; |
required |
Source code in capturegraph-lib/capturegraph/types/containers/parallel.py
pmapped(items, function, label, on_error)
#
mapped across the pool: the parallel rule Array and Map share.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
items
|
Iterable[I]
|
The elements to apply |
required |
function
|
Callable[[I], R]
|
The per-element operation. |
required |
label
|
str | None
|
What the count shows; |
required |
on_error
|
OnError
|
What an element that raises does to the walk. |
required |
Returns:
| Type | Description |
|---|---|
list[R]
|
What |