dry-run

Dry run and listing

Behave can load spec files and report what would run without actually executing any example. Two CLI flags surface this from different angles:

  • --dry-run is for humans: a hierarchical, indented listing followed by an N example(s) count.

  • --list-examples is for tools: one line per example (or a JSON document) suitable for editor integrations.

Both flags honor --tag, --exclude-tag, --example, --only-example, focus mode (fit/fdescribe), and skip metadata (xit/xdescribe).

--dry-run

$ behave --dry-run specs/cart-spec.raku
Cart
  adding items
    increments the count
    updates the total price
    persists across reloads (PENDING)
  removing items
    decrements the count
    is intentionally skipped (SKIPPED)

5 examples

Statuses appear in parentheses after the description:

  • (PENDING): declared with pending

  • (SKIPPED): declared with xit / xdescribe (or :skipped metadata)

  • (FOCUSED): declared with fit / fdescribe (or :focused metadata)

The trailing N example(s) line reports the count after filters.

--dry-run --verbose

Adds each example's file:line and effective tag list (tags inherited from ancestor describe/context blocks are included):

Cart
  adding items
    increments the count
      /path/to/cart-spec.raku:5
      tags: fast
    updates the total price
      /path/to/cart-spec.raku:6

Exit code

--dry-run exits 0 when every spec file loaded cleanly, and 1 when at least one spec file failed to load. The exit code is independent of whether the filters matched any examples. Zero matching examples is still a successful dry run.

--list-examples

Plain-text mode prints one line per matching example. Each line is:

<absolute spec file>:<line><TAB><full nested description>
$ behave --list-examples specs/cart-spec.raku
/path/to/cart-spec.raku:5	Cart adding items increments the count
/path/to/cart-spec.raku:6	Cart adding items updates the total price
/path/to/cart-spec.raku:7	Cart adding items persists across reloads
/path/to/cart-spec.raku:11	Cart removing items decrements the count
/path/to/cart-spec.raku:12	Cart removing items is intentionally skipped

The tab separator makes the format easy to parse with cut -f1 (locations) or cut -f2 (descriptions). Pair with xargs behave --only-example or your editor's "run this test" action.

--list-examples-format=json

For tooling integrations the JSON output exposes the same fields as the programmatic SpecRegistry query API:

$ behave --list-examples --list-examples-format=json specs/cart-spec.raku
{
  "version": 1,
  "count": 5,
  "examples": [
    {
      "description": "increments the count",
      "full-description": "Cart adding items increments the count",
      "file": "/path/to/cart-spec.raku",
      "line": 5,
      "tags": ["fast"],
      "metadata": { "tags": ["fast"] },
      "pending": false,
      "focused": false,
      "skipped": false
    }
  ]
}

Field reference:

FieldTypeNotes
descriptionstringThe example's own description.
full-descriptionstringAncestor groups + example, joined with a space.
filestringAbsolute path to the spec file.
lineintegerLine of the it (or fit/xit/pending).
tagslist of stringsEffective tags (own + inherited from ancestor groups).
metadataobjectThe example's own metadata (excluding the internal lets).
pendingbooleanDeclared with pending or marked pending.
focusedbooleanEffectively focused (own or via an ancestor fdescribe).
skippedbooleanEffectively skipped (own or via an ancestor xdescribe).

Unknown future fields may appear, so keep your parser tolerant.

Hierarchical suites field

Alongside the flat examples list, the JSON document also includes a top-level suites key with the full discovered tree (suites → groups → examples, with each node carrying type, description, file, line, metadata, and, for examples, pending). Unlike examples, the suites tree is unfiltered: focus mode and tag filters don't prune it. Behave's --parallel parent uses this field to rebuild the spec tree without EVALFILE-ing user code in the parent process. External tools can use it for the same purpose, or ignore it and read examples as before.

{
  "version": 1,
  "count": 5,
  "examples": [ /* filtered flat list, as above */ ],
  "suites": [
    {
      "type": "suite",
      "description": "cart-spec.raku",
      "file": "/path/to/cart-spec.raku",
      "line": 0,
      "metadata": {},
      "children": [
        {
          "type": "group",
          "description": "Cart",
          "file": "/path/to/cart-spec.raku",
          "line": 3,
          "metadata": { "tags": ["unit"] },
          "children": [
            {
              "type": "example",
              "description": "increments the count",
              "file": "/path/to/cart-spec.raku",
              "line": 5,
              "metadata": { "tags": ["fast"] },
              "pending": false
            }
          ]
        }
      ]
    }
  ]
}

Programmatic query API

Tooling that wants to drive Behave from inside a Raku process (without parsing CLI output) should use BDD::Behave::SpecRegistry's query methods directly.

use BDD::Behave::SpecRegistry;

# After your spec files have been loaded (e.g. via EVALFILE), the
# global registry holds every Suite/ExampleGroup/Example node.
my $reg = BDD::Behave::SpecRegistry::registry();

# Every example across every loaded suite, as ExampleQueryResult records:
my @all = $reg.all-examples;

# Filtered query: every kwarg below is optional and ANDed:
my @hits = $reg.query-examples(
  description-pattern => '/^Cart\s/',     # substring or /regex/
  file                => 'cart-spec.raku',# basename, relative, or absolute
  line                => 5,
  include-tags        => ['fast'],         # OR across listed tags
  exclude-tags        => ['slow'],         # ANDed with the rest
  metadata            => { type => 'unit' },
  metadata-exclude    => { type => 'integration' },
  pending             => False,
  focused             => True,
  skipped             => False,
);

# Count without materializing the records:
my $n = $reg.count-examples(include-tags => ['fast']);

Each ExampleQueryResult exposes:

  • .description / .full-description

  • .file (IO::Path) / .line / .location

  • .suite-file (IO::Path) / .suite-description

  • .group-descriptions (list of strings, outermost first)

  • .tags (effective tags)

  • .metadata (own metadata hash with the internal lets key removed)

  • .pending / .focused / .skipped (Bool)

  • .to-hash for serialization (used by --list-examples-format=json)

This API is stable for editor and IDE integrations. The CLI --list-examples output is a thin wrapper over it.

BDD::Behave v0.9.4

Behavior driven development framework

Authors

  • Greg Donald

License

Artistic-2.0

Dependencies

Provides

  • BDD::Behave
  • BDD::Behave::Benchmark
  • BDD::Behave::Benchmark::Baseline
  • BDD::Behave::Benchmark::Format
  • BDD::Behave::Bisect
  • BDD::Behave::Colors
  • BDD::Behave::Configuration
  • BDD::Behave::Coverage
  • BDD::Behave::DSL
  • BDD::Behave::Diff
  • BDD::Behave::DocExtractor
  • BDD::Behave::DryRun
  • BDD::Behave::Expectation
  • BDD::Behave::Failure
  • BDD::Behave::FailureStore
  • BDD::Behave::Failures
  • BDD::Behave::Files
  • BDD::Behave::Formatter
  • BDD::Behave::Formatter::Documentation
  • BDD::Behave::Formatter::HTML
  • BDD::Behave::Formatter::JSON
  • BDD::Behave::Formatter::JUnit
  • BDD::Behave::Formatter::JsonEvents
  • BDD::Behave::Formatter::Progress
  • BDD::Behave::Formatter::Registry
  • BDD::Behave::Formatter::TAP
  • BDD::Behave::Formatter::Tree
  • BDD::Behave::LetRuntime
  • BDD::Behave::Matcher
  • BDD::Behave::Matcher::Async
  • BDD::Behave::Matcher::Boolean
  • BDD::Behave::Matcher::Change
  • BDD::Behave::Matcher::Collection
  • BDD::Behave::Matcher::Core
  • BDD::Behave::Matcher::Custom
  • BDD::Behave::Matcher::Exception
  • BDD::Behave::Matcher::Numeric
  • BDD::Behave::Matcher::String
  • BDD::Behave::Matcher::Type
  • BDD::Behave::Mock::Allow
  • BDD::Behave::Mock::ArgMatcher
  • BDD::Behave::Mock::Double
  • BDD::Behave::Mock::HaveReceived
  • BDD::Behave::Mock::Spy
  • BDD::Behave::Mock::Stub
  • BDD::Behave::Parallel
  • BDD::Behave::Parallel::Distribution
  • BDD::Behave::Parallel::EventStream
  • BDD::Behave::Parallel::Manifest
  • BDD::Behave::Parallel::Queue
  • BDD::Behave::Parallel::WorkerPool
  • BDD::Behave::Runner
  • BDD::Behave::SharedContexts
  • BDD::Behave::SharedExamples
  • BDD::Behave::Slang
  • BDD::Behave::SpecLoader
  • BDD::Behave::SpecRegistry
  • BDD::Behave::SpecTree
  • BDD::Behave::SpecTree::Core
  • BDD::Behave::SpecTree::Example
  • BDD::Behave::SpecTree::ExampleGroup
  • BDD::Behave::SpecTree::Suite
  • BDD::Behave::Time
  • BDD::Behave::TypeName
  • BDD::Behave::Version
  • BDD::Behave::Watch
  • BDD::Behave::Watch::Session
  • BDD::Behave::Watch::SmartSelector
  • BDD::Behave::Watch::UI
  • BDD::Behave::Watch::Watcher
  • BDD::Behave::Worker

The Camelia image is copyright 2009 by Larry Wall. "Raku" is trademark of the Yet Another Society. All rights reserved.