TokenProcessing

TokenProcessing

TokenProcessing package has grammar classes and action classes for the parsing Raku Perl 6 grammar files and making a file with token grammar rules that are extended to have fuzzy matching function calls.

Synopsis

use TokenProcessing;
    enhance-token-specs( $fileName, output, :add-protos, sym-name => 'English', :add-exclusions);

Grammar::TokenProcessing v0.1.10

Processing grammar files in order to extract tokens, to replace token names, or to add fuzzy matching expressions in token definitions.

Authors

  • Anton Antonov

License

Artistic-2.0

Dependencies

Data::Generators:ver<0.1.6+>Data::Reshapers:ver<0.2.7+>Lingua::EN::Stem::Porter:ver<1.2+>Text::Levenshtein::Damerau:ver<0.3.0+>

Provides

  • Grammar::TokenProcessing
  • Grammar::TokenProcessing::Actions::EBNF
  • Grammar::TokenProcessing::Actions::EnhancedTokens
  • Grammar::TokenProcessing::Actions::RandomSentence
  • Grammar::TokenProcessing::Actions::TokenNames
  • Grammar::TokenProcessing::Actions::Tokens
  • Grammar::TokenProcessing::Actions::TokensHash
  • Grammar::TokenProcessing::ComprehensiveGrammar
  • Grammar::TokenProcessing::Grammar

The Camelia image is copyright 2009 by Larry Wall. "Raku" is a trademark of the Yet Another Society. All rights reserved.

Built with Podlite — the markup and publishing tools behind this site.