Qwiratry Core Hygiene

Plan/Spec for core hygiene work formerly listed under Post-SQL §F.2 (Post-SQL-TaOP-Improvements.md): small, independent cleanups that do not need the SQL DatabaseStorage sibling.

Status: Will not do (postponed; archived).Parent backlog: Post-SQL §F.11 (items 11.1–11.3); paging is Post-SQL §F.5.4 (was §F.2 2.4).Related sources: Post-SQL-TaOP-Improvements.md §B.7, §C.4, §D.1, §E.1.1; Qwiratry-TaOP-Plan.md §11 item 10; Qwiratry-TaOP-Basics.md DQ10 / Non-goals; Qwiratry-TaOP-Prerequisites.md §2 / §5.4 (SourceOperator walkable-only); SQL-Features-Plan.md §B.3 (paging → Post-SQL §D.1 / §F.5.4).

Question IDs are stable: Q1, Q2, … stay the same when an item moves from §Open Questions into §Decisions.

1. Goals

1. Untangle plugin discovery so Qwiratry is not carrying a private fork of loader / singleton concerns that belong in a separate module again (§3 / Post-SQL 11.1 / E.1.1).

2. Revisit (or explicitly reaffirm) whether SourceOperator may return a pull Iterator when that would simplify pipelines (§4 / 11.2 / B.7).

3. Prefer QueryIterator.of advertising Tuple when cheap (§5 / 11.3 / C.4 / Basics DQ10).

4. Specify paging for in-memory / language use — likely array element ranges (§6 / Post-SQL 5.4 / D.1).

2. Non-goals

• SQL backend MVP and WalkCursor mapping (Post-SQL §F.3).

• TOP parity Textish work (Post-SQL §F.1 — done; archived Parity spec).

• Greenfield TOP front-end (View, atom, … — §F.5).

• Full Format / Location Supply push and model converters (Streaming-v2 / Post-SQL §F.7).

• Aggregate catalogue expansion (Post-SQL §F.8).

• Language-level SQL LIMIT/OFFSET dialect encoding (belongs with multi-dialect SQL / §F.3–§F.6 once paging shape is clear).

3. Extract Implementation::Loader / Singleton (§F.11 11.1)

(Post-SQL §E.1.1; TaOP-Plan §11 item 10)

3.1. Background

TaOP-Plan asked to update Implementation::Loader (and possibly Singleton) that were imported into Qwiratry but shouldn’t have been, so they become a separate module again.

Today:

• External package Implementation::Loader still exists and is used directly in places (e.g. Walker::Factory, Tree::Navigator).

• Core also ships Qwiratry::Implementation::Loader: a role that does external Implementation::Loader and Qwiratry::Singleton, with Qwiratry-specific !lib-paths, caches, and ensure/make helpers.

• Qwiratry::Format and Qwiratry::Location compose Qwiratry::Implementation::Loader.

• Qwiratry::Singleton lives in core (lib/Qwiratry/Singleton.rakumod) and was recently fixed for Rakudo object-hash //= behaviour.

3.2. Intent

Stop treating loader/singleton glue as an accidental in-tree fork of something that should be publishable / versioned separately (or use the external package consistently). Resolve Q1.

3.3. Likely work (after Q1)

• Choose packaging (separate dist vs thin core wrapper vs all-callers use external Implementation::Loader only).

• Align Format / Location / Walker / Navigator on one discovery path.

• Keep plugin discovery behaviour (sibling use lib / !lib-paths) unless deliberately changed.

• Tests: Format/Location make / discovery still work; no Singleton bind regressions.

4. Consider: SourceOperator → Iterator (§F.11 11.2)

(Post-SQL §B.7; Prerequisites §2 / §5.4; Basics Non-goal)

4.1. Current behaviour

SourceOperator.evaluate always uses read-walkable (whole payload: typically Str / path / directory-result shapes). It does not return an Iterator.

Pull is composed by ParseOperator when both the Location (SourcePull) and the Format (ParsePull) advertise pull. Prerequisites rejected bare Source → Iterator for now so directory/glob walkable shapes stay stable.

4.2. Why revisit

Post-SQL §B.7 keeps a Consider: revisit only if the walkable-Source + pull-Parse split proves awkward in real pipelines (or Streaming-v2 operator policy makes Source-level pull clearly better).

4.3. Scope for this cut

Decide Q2: leave walkable-only (document “won’t revisit until …”), or design a Source pull path that preserves dir/glob semantics. Do not silently change evaluate without that call.

5. DQ10: QueryIterator.of → Tuple (§F.11 11.3)

(Post-SQL §C.4; Basics DQ10)

5.1. Current behaviour

QueryIterator does TypedIterator with default .of = Mu. Basics DQ10: if easy, advertise Tuple once it exists; Mu is fine; not high priority.

5.2. Direction

When table/query iterators already yield Tuple-shaped rows, set .of so downstream ElementType checks and docs match reality. Avoid lying about .of for tree / mixed walks that are not Tuples.

Resolve Q3 for when / which iterators advertise Tuple.

6. Paging (Post-SQL §F.5 5.4; was §F.2 2.4)

(Post-SQL §D.1; SQL-Features §B.3)

6.1. Sketch from SQL-Features

Original note: paging “can probably just be done with array element ranges.”

6.2. Direction for this cut

Prefer an in-memory / language story first (e.g. @rows[10..^20], or a thin helper / operator that documents the same idea) rather than inventing a full SQL LIMIT/OFFSET surface before the SQL sibling exists.

SQL dialect paging can follow once §F.3 multi-dialect encoding is in play — out of scope here except to avoid painting into a corner (Q4).

7. Implementation checklist

• 1. Resolve Q1; extract / realign Implementation::Loader + Singleton packaging (11.1).

• 2. Resolve Q2; either keep Source walkable-only with an explicit deferral note, or implement a considered Source pull path (11.2).

• 3. Resolve Q3; advertise QueryIterator.of as Tuple where appropriate (11.3).

• 4. Resolve Q4; land paging via ranges (or documented equivalent) (5.4).

• 5. Docs: Changelog, Extending where relevant; mark Post-SQL §F.11 / §F.5.4 items done; cross-link this plan.

8. Open Questions

• Q1. How should loader / singleton packaging look? (a) Move Qwiratry::Implementation::Loader (+ maybe Singleton) to a separate dist that depends on external Implementation::Loader; (b) Drop the Qwiratry wrapper and teach Format/Location to use external Implementation::Loader + shared helpers in core; (c) Keep the wrapper in core but document it as intentional Qwiratry glue (re-interpret “separate module” as already-external Implementation::Loader only).

• Q2. Should SourceOperator gain an Iterator-returning path now? (a) No — reaffirm Prerequisites / Basics; leave §B.7 closed as deferred until Streaming-v2 operator policy forces it; (b) Yes — design Source pull that still handles directory/glob walkable shapes without breaking Parse composition.

• Q3. When should QueryIterator.of be Tuple? (a) Only table / Relation walks that already yield Tuples; (b) All QueryIterators once a Tuple type exists (even if some are Mu in practice — avoid); (c) Leave .of = Mu until a broader typed-stream pass (Streaming-v2 §E.2 typing).

• Q4. What is the public paging surface for this cut? (a) Document / rely on ordinary Raku subscript ranges only; (b) Add a small Qwiratry helper or operator that slices Positional / lazy Seq results; (c) Also sketch SQL LIMIT/OFFSET now (even if implementation waits for §F.3).

9. Decisions

(None yet — fill from §8 as answers land.)

10. References

• Post-SQL-TaOP-Improvements.md §F.11, §F.5.4, §B.7, §C.4, §D.1, §E.1.1

• Qwiratry-TaOP-Plan.md §11 item 10

• Qwiratry-TaOP-Basics.md DQ10; Non-goals (SourceOperator)

• Qwiratry-TaOP-Prerequisites.md §2, §5.4

• SQL-Features-Plan.md §B.3

• Code: lib/Qwiratry/Implementation/Loader.rakumod, lib/Qwiratry/Singleton.rakumod, lib/Qwiratry/Format.rakumod, lib/Qwiratry/Location.rakumod, Source/Parse operators, QueryIterator