Required Raku Modifications

Language/runtime changes that would better support data-oriented programming

=begin rakudoc

Overview

Qwiratry can be developed as a module on today's Raku. If Qwiratry, or the data-oriented programming ideas behind it, were ever merged into Raku itself, the changes below, though not required, would make that integration much easier.

Dispatch Changes

Multiple Dispatch Ordering

Traditional Raku objects should continue to use normal Raku multiple dispatch ordering. Data-oriented objects should be able to choose a different ordering when the type needs one.

The dispatch ordering should depend on the type of the invocant or matched object. For example, ordinary Raku objects would keep the existing candidate ordering, while Mold objects could use a custom ordering designed for data-oriented transformation rules.

This would let molds remain dispatch-like and composable without forcing them through the same precedence rules used for ordinary methods.

If we did this right, we could also have a dispatch that would work like gather/take; subsequent calls to the function would continue from where it left off, and return the next item taken. Also, when a dispatch matches, it may return its own value, but then continue on with the next item in the queue; might need a special return for this. Would probably need a way to use proceed/succeed with these as well.

Orthogonal Redispatch

Nice to have, but less helpful would be Orthogonal Redispatch.

Grammar Capabilities

The useful grammar changes are the ones that let grammars participate in data-oriented input and output pipelines without requiring the whole data set to be resident in memory.

Progressive And Streamable Grammars

It should be possible to mark Grammars as progressive/streamable. They should be able to parse input incrementally instead of requiring the whole input to be present in memory.

Important use cases include:

  • Parsing network input as it arrives.

  • Parsing files larger than available memory.

This also might require some signalling back and forth with the downstream objects, for example GrammarActions or whatever replaces it.

Probably the main thing to do is to be able to draw some boundary where, if the grammar fails, then it walks back to that point, and waits on more data.

Reversible Grammars

Grammars should ideally be reversible, so the same declarative structure can be used for rendering as well as parsing.

This would let a data-oriented program describe a representation once and use that description both to read data into a structure and to render a structure back out.

I'm thinking in particular that it should be possible to provide a Match tree and have the grammar render it as a string.

Chaining Replacing Actions

Instead of just having a Grammar Actions object, chain the grammar to the next item in a pipeline. This might be a Grammar Actions object, but might be a Transformer, or something else. There could easily be more than one item in the pipeline. The idea is that we could stream tokens from the grammar to whatever is next in the pipeline.

Path Operators

The idea here is operators that can take a label. This would mean that things like:

``` $html⪪body⪪h1{.id eq "red"} ``` The ⪪ operator will look for matching children, and the {} is just a block, and that part works, it's the part between the ⪪ and {} that needs to be adjusted.

The elements such as body and h1 are supposed to be undeclared labels; you could equally well substitute in filenames if the tree in question happened to be a filesystem instead of a document.

=end rakudoc

Qwiratry v0.0.9

Declarative query and data-walking architecture for Raku, with transformers, molds, walkers, and I/O pipelines.

Authors

  • Tim Nelson

License

Dependencies

SlangifyImplementation::Loader:ver<0.0.9+>Glob::Grammar

Test Dependencies

Provides

  • Qwiratry
  • Qwiratry::Context
  • Qwiratry::Format
  • Qwiratry::Format::Base
  • Qwiratry::Format::CSVdemo
  • Qwiratry::Format::JSONdemo
  • Qwiratry::Location
  • Qwiratry::Location::Base
  • Qwiratry::Location::File
  • Qwiratry::Mold
  • Qwiratry::Mold::Compiler
  • Qwiratry::Mold::Registry
  • Qwiratry::Mold::Slang
  • Qwiratry::Operator::Capability
  • Qwiratry::Operator::IO
  • Qwiratry::Operator::MapReduce
  • Qwiratry::Operator::Navigation
  • Qwiratry::Operator::Set
  • Qwiratry::Query::Evaluator::Eager
  • Qwiratry::Query::Evaluator::Filter
  • Qwiratry::Query::Evaluator::Join
  • Qwiratry::Query::Evaluator::Lazy
  • Qwiratry::Query::Evaluator::MapReduce
  • Qwiratry::Query::Evaluator::Navigation
  • Qwiratry::Query::Evaluator::Relational
  • Qwiratry::Query::Evaluator::Row
  • Qwiratry::Query::Evaluator::Set
  • Qwiratry::Query::Evaluator::Union
  • Qwiratry::Query::Extract
  • Qwiratry::Query::NamedJoins
  • Qwiratry::Query::RelationCommon
  • Qwiratry::Query::Runtime
  • Qwiratry::Query::Selector
  • Qwiratry::Query::Slang
  • Qwiratry::Query::Specificity
  • Qwiratry::Query::Topic
  • Qwiratry::QueryIterator
  • Qwiratry::QueryMatch
  • Qwiratry::Setup
  • Qwiratry::Strategy
  • Qwiratry::Strategy::ControlSignal
  • Qwiratry::Strategy::FinishResult
  • Qwiratry::Strategy::RewriteSpec
  • Qwiratry::Strategy::Traversal
  • Qwiratry::Suggest
  • Qwiratry::Table
  • Qwiratry::Table::Schema
  • Qwiratry::Transformer
  • Qwiratry::Transformer::Copy
  • Qwiratry::Transformer::TreeRewrite
  • Qwiratry::Tree::Navigator
  • Qwiratry::Tree::Navigator::Base
  • Qwiratry::Tree::Navigator::Filesystem
  • Qwiratry::Tree::Navigator::Match
  • Qwiratry::Tree::Navigator::RakuAST
  • Qwiratry::Tree::Replace
  • Qwiratry::Walker
  • Qwiratry::Walker::Capabilities
  • Qwiratry::Walker::Factory
  • Qwiratry::Walker::Implementation::Table
  • Qwiratry::Walker::Implementation::Tree
  • Qwiratry::Walker::Master
  • Qwiratry::Walker::Providing
  • X::Qwiratry

The Camelia image is copyright 2009 by Larry Wall. "Raku" is trademark of the Yet Another Society. All rights reserved.