Parallel Execution: Design

This document describes how --parallel N works internally. For the user-facing guide, see Parallel Execution.

Goal

behave --parallel N runs a spec suite across N worker subprocesses concurrently. With no flag, behave keeps its single-process, single-worker behavior unchanged.

The design holds to four properties:

• A suite that takes T seconds serially takes roughly T / N seconds on N cores, modulo per-worker startup overhead and load imbalance.

• Failure output, summary counts, profile/benchmark sections, exit code, and seed reporting stay meaningful and reproducible.

• User code that assumes a fresh process per worker (DB connection per worker, file per worker, port per worker) can discover its worker identity.

• Tests that cannot safely run concurrently have an opt-out (:serial).

Process-based, not thread-based

Threads in Raku are real (MoarVM has no GIL), but the runner uses processes, not threads:

• Spec files declare top-level class / role / enum / subset / constant symbols. Loading two spec files into the same process can collide on those symbols. Loading them concurrently would turn collisions into races.

• User code commonly uses thread-hostile global state: DB connection objects, %*ENV mutation in before-each, chdir, signal handlers, framework singletons. Most of this is process-safe but not thread-safe.

• EVAL / EVALFILE and the surrounding compiler state are not designed to run multiple unrelated programs concurrently.

• --bisect and --coverage already shell out to subprocesses for isolation. Parallel execution follows the same pattern, so the operational model is uniform.

Each worker is a separate raku bin/behave subprocess.

Worker lifecycle

parent (orchestrator)
  ├── discovery pass (load spec tree, plan buckets)
  ├── fork worker 0  ── runs assigned buckets, streams events back
  ├── fork worker 1  ── …
  ├── …
  ├── fork worker N-1
  ├── wait, aggregate
  └── serial phase (run any :serial examples on a single worker)

A worker:

1. Receives a manifest of the buckets it owns plus its BEHAVE_WORKER_INDEX and BEHAVE_WORKER_COUNT.

2. Loads every spec file it owns at least one bucket of (load cost is paid once per worker).

3. Walks the same Runner the serial path uses, with an additional filter that gates each example on "is this example in one of my assigned buckets?"

4. Emits one JSON line per spec-tree event on stdout. stderr is forwarded to the parent's stderr verbatim.

5. Exits 0 on a clean run regardless of test failures (failures are reported via events). A non-zero exit means the worker itself crashed: an uncaught exception escaping the runner, OOM, or a signal.

Workers are launched with --worker-manifest <path> (static distribution) or --queue-worker (queue distribution). Both are hidden flags. Users never type them.

Spec discovery

Discovery is preceded by a precompile pass: one behave --no-config --compile-only <spec-files> subprocess loads every spec file, writing the whole module dependency graph's precompilation from a single process. Discovery and the workers then only read that cache. Units written concurrently by processes with differing load states carry conflicting repossession records, and loading that mix segfaults the deserializer. One writer per graph is the invariant the pass exists to hold. --no-precompile skips it.

Distribution requires the parent to know what groups exist so it can bucket them. The parent runs a short-lived discovery subprocess (behave --no-config --list-examples --list-examples-format=json), parses the emitted JSON tree, and rebuilds a skeleton Suite / ExampleGroup / Example tree used only for bucketing and event lookup. The parent never EVALFILEs a user spec file, so user-declared class / role definitions never run in the parent. Each bucket is keyed by (file, group-path), where group-path is the chain of describe / context descriptions from the suite root down to (but not including) the leaf example.

In the pool modes (lpt / queue), affinity is enforced by bucketing on the top-level describe / context (or a top-level it). Keeping a group's examples on one worker lets before-all / after-all / around-all amortize correctly instead of re-running per example.

Work distribution

--parallel-mode selects the execution model:

• isolated (default): one subprocess per spec file, with concurrency capped at --parallel N (or $*KERNEL.cpu-cores). A file's examples run together in their own process. There is no cross-file bucket packing and no persistent worker pool. Each running file leases a BEHAVE_WORKER_INDEX from a recycled pool of 0 .. N-1 slots (returned when the file finishes), so the index is bounded by the concurrency cap and concurrently-running files always hold distinct indices, letting user code key a per-worker resource (e.g. a per-worker database) off the index with only N resources to provision. This is the strongest isolation and the default.

• lpt: a fixed pool of N persistent workers. The parent sorts buckets by example count descending (a cost proxy in the absence of historical timing data), then greedily assigns each bucket to the worker with the smallest current load (longest-processing-time-first). This is a 4/3-approximation of optimal makespan. With N much smaller than the bucket count it lands within a few percent of perfect.

• queue: a fixed pool of N workers pulling buckets from a shared queue, so a worker that finishes early picks up the next available bucket. This wins when bucket costs are uneven and the bucket count comfortably exceeds the worker count. It costs a per-bucket coordination round-trip.

isolated is the default for maximal isolation. The lpt and queue pool modes share the group-affinity bucketing above and trade some isolation for fewer, longer-lived processes. queue further trades LPT's static balance for dynamic rebalancing.

:serial phase

Examples or groups tagged :serial are filtered out of the parallel manifests during discovery. After all parallel workers exit cleanly, the parent forks one more worker (index 0, count 1) with a manifest containing only the serial examples and runs them sequentially.

:serial combines with existing filters using AND semantics, identical to :tag:

• --tag foo + :serial: an example runs only if it has both foo and :serial.

• --exclude-tag serial skips them entirely.

• --example PATTERN and focus mode apply normally.

Failures in the serial phase merge into the same RunResult as the parallel phase, and the summary stays a single line. If --parallel is omitted or is 1, :serial is a no-op: everything is already serial.

IPC: parent ⇆ worker protocol

Workers emit one JSON line per event on stdout. Events cover the spec-tree lifecycle (suite-start, group-start, example-start, example-pass, example-fail, example-pending, example-skipped, group-end, suite-end, run-summary), worker bookkeeping (worker-ready, bucket-done, load-error), and the optional reporting streams (profile-record, benchmark-record, retry-record) when their flags are requested.

The parent reads these line by line, feeds them to the existing formatter through a thin adapter, and aggregates counts into a single RunResult.

JSON-lines was chosen over a binary protocol or Raku-native serialization because it is:

1. Trivially streamable: line-buffered, no length-prefix framing.

2. Debuggable: worker stdout can be dumped to a file and read directly.

3. Robust to stdout flushing edge cases: a half-line is detectable as malformed and reported, not silently lost.

stderr from workers is forwarded to the parent's stderr unchanged, so user note / say *.error calls reach the user as-is.

Output rendering

The parent owns the terminal. Workers never write directly to the user's stdout.

For the default progress formatter this is direct: the parent prints one character per example event as it arrives: . for example-pass, F for example-fail, * for example-pending, S for example-skipped. Interleaved order across workers is fine. progress makes no per-line ordering promise.

For the verbose and documentation formatters the parent buffers per suite: it accumulates events for a (worker, file) pair until that worker emits suite-end for the file, then flushes the whole suite to the formatter as one block. This keeps a suite's nested output contiguous. The cost is bounded memory per worker (one suite's worth of events). The formatter sees the same call sequence it would in serial mode. The parent reorders the worker stream before calling into it.

Worker identity API

BDD::Behave::Worker exposes the current worker's identity:

class BDD::Behave::Worker {
  method id    (--> Int) { %*ENV<BEHAVE_WORKER_INDEX> // 0 }
  method count (--> Int) { %*ENV<BEHAVE_WORKER_COUNT> // 1 }
}

Both env vars are always set: in serial mode BEHAVE_WORKER_INDEX=0 and BEHAVE_WORKER_COUNT=1, so user config that interpolates BDD::Behave::Worker.id into a DB name (myapp_test_{id}) behaves identically in serial and parallel modes.

The values are seeded by the parent into each child's environment at fork time. The accessors read the env vars on each call, so user mutation does not change the reported identity.

Seed mode

--seed-mode controls how --seed N combines with --parallel K:

• xor (default): each worker gets a derived seed seed XOR worker-index. The summary still prints the root seed. Reproducible only when the worker count also matches.

• stable: every bucket gets a deterministic hash of (file, group-path, seed). Buckets are sorted globally by that hash and the bucket at sorted position i is assigned to worker i mod K. The global hash-sorted order is identical regardless of K. Workers run --order=defined within their buckets. This reproduces a run across different worker counts.

Interaction with existing flags

Known limitations

• Discovery cost on huge suites. A worker that owns buckets in every file pays that file's load cost, so a very large suite loads spec files more than once across the worker set.

• No live progress totals. The default progress formatter does not print "423 / 5000" mid-run. Parallel mode keeps that behavior.

Non-goals

• Persistent workers across runs (would require detecting spec changes between runs).

• Distributed multi-host execution.