=begin rakudoc

Overview

Sources and destinations connect Qwiratry pipelines to external locations.

Use this extension point when the new thing is a place Qwiratry receives data from or outputs data to: files, Postgres, SQLite, HTTP APIs, message queues, object stores, search indexes, and similar systems.

If the work is only about converting between external data representations and Qwiratry-visible data, use a format extension instead.

To add support for a new source or destination:

• 1. Setup: decide whether the extension receives data, outputs data, or does both.

• 2. Backend module: create a location backend module under Qwiratry::Location.

• 3. Source: define how Qwiratry receives data from the external location.

• 4. Destination: define how Qwiratry outputs data to the external location.

• 5. Testing: test validation, receive behavior, output behavior, errors, and pipeline composition.

Setup

Decide whether the extension is source/destination behavior or a format. A database, API, queue, or object store is a source/destination even if it also uses a format internally.

Define validation rules for locations, connection strings, credentials, or other connection details. Decide whether the extension returns data, rows, streams, or another structured value.

Location backends are loaded through Qwiratry::Location. The backend name is inferred from the location scheme: postgres://example/db resolves to Qwiratry::Location::Postgres, s3://bucket/key resolves to Qwiratry::Location::S3, and local paths or file:// URLs resolve to the built-in Qwiratry::Location::File backend.

Backend Module

A backend module groups the source and destination implementations for one location family. A backend can implement only source, only destination, or both.

use X::Qwiratry;
use Qwiratry::Location::Base;

class Qwiratry::Location::Postgres::Source is Qwiratry::Location::Base::Source {
    method read(Str $location --> Str) {
        # Connect to $location and return text, rows encoded as text,
        # or another representation expected by the rest of the pipeline.
    }
}

class Qwiratry::Location::Postgres::Destination is Qwiratry::Location::Base::Destination {
    method write(Str $location, Mu $content --> Mu) {
        # Connect to $location and write $content.
        $content
    }
}

No registration step is required. Once the module is available as Qwiratry::Location::Postgres, the factory can discover Qwiratry::Location::Postgres::Source and Qwiratry::Location::Postgres::Destination.

Source

Source Responsibilities

• Validate the location or connection details early.

• Receive data from the external system without changing query semantics.

• Return data in a form that later operators can parse, walk, or transform.

• Throw X::Qwiratry::IO::LocationError for location failures.

Minimal File Source Example

This example uses SourceOperator to receive file data, ready to be passed to a parser.

use Qwiratry::Operator::IO;
use Qwiratry::Operator::Capability;

my $source = SourceOperator.new(:location<input.csv>);
my $data = execute($source);

Under the operator surface, Qwiratry resolves the concrete source through:

Qwiratry::Location.make(:type<Source>, :location<input.csv>).read('input.csv')

Destination

Destination Responsibilities

• Validate the destination location or connection details.

• Accept rendered content or structured values as appropriate.

• Preserve pipeline return behavior so callers can observe what was output.

• Throw X::Qwiratry::IO::LocationError for output failures.

Minimal File Destination Example

This example uses DestinationOperator to output already-rendered data. The operator returns the same content it outputs, which keeps pipeline behavior observable.

use Qwiratry::Operator::IO;
use Qwiratry::Operator::Capability;

my $rendered = "name,score\nalpha,10\nbeta,5\n";
my $destination = DestinationOperator.new(
    :location<output.csv>,
    :subject($rendered),
);

my $output = execute($destination);

Under the operator surface, Qwiratry resolves the concrete destination through:

Qwiratry::Location.make(:type<Destination>, :location<output.csv>).write('output.csv', $rendered)

Testing

Add tests that prove:

• Invalid locations or connection strings are rejected.

• Unavailable locations throw X::Qwiratry::IO::LocationError.

• Sources receive expected data.

• Destinations output expected data.

• Qwiratry::Location.backends(:type<Source>) or Qwiratry::Location.backends(:type<Destination>) discovers the backend.

• Pipelines compose source, parse, query, render, and destination steps correctly.

Reference Material

Current Surface

The current source/destination surface is represented by IO operators:

• Qwiratry::Operator::IO::SourceOperator

Receives data from a location.

• Qwiratry::Operator::IO::DestinationOperator

Outputs data to a location.

• Qwiratry::Location

Factory that discovers location backends and creates source or destination implementation objects.

• Qwiratry::Location::Base

Defines Qwiratry::Location::Base::Source and Qwiratry::Location::Base::Destination, the abstract contracts for backend implementations.

• Qwiratry::Location::File

Built-in local file backend. Local paths and file:// URLs resolve here.

• X::Qwiratry::IO::LocationError

Exception for invalid, unavailable, or unsupported locations.

Backend Shape

Database-like sources should not be implemented as formats. They usually need connection state, authentication, schema discovery, query pushdown, streaming, or transaction behavior.

A source/destination extension introduces backend-specific classes such as:

Qwiratry::Location::Postgres::Source
Qwiratry::Location::Postgres::Destination
Qwiratry::Location::HTTP::Source
Qwiratry::Location::S3::Destination

Example Direction

A Postgres source is a source/destination extension, not a format extension. The format layer might still be used after receiving data, but Postgres itself is a live backend with query and connection semantics.

my $source = SourceOperator.new(:location<postgres://example/db>);

=end rakudoc