Incremental-grammar-enhancement-work
Incremental grammar enhancement
Introduction
This document demonstrates how to use Large Language Models (LLMs) and Extended Backus-Naur Form (EBNF) to generate and incrementally develop grammars for Domain Specific Languages (DSLs).
Procedure outline
If LLM speaks Raku
TBD...
Using BNF
Come up with sentences from a certain Domain Specific Language (DSL).
Request a certain Large Language Model (LLM) -- for example, ChatGPT or PaLM -- to generate a corresponding grammar in Backus-Naur Form (BNF).
Using the obtained BNF string create a corresponding Raku object that can be used generate new random sentences. One of:
Raku class for "FunctionalParsers"
Raku grammar
With Raku object generate a set of random sentences.
Request LLM to come up with, say, 5-10 variations of each sentence.
Request BNF for the new, enhanced set of sentences.
Is the obtained grammar large or comprehensive enough?
If not then go to step 2.
If yes finish.
Setup
Here are the packages we are going to use:
use Grammar::TokenProcessing;
use EBNF::Grammar;
use FunctionalParsers;
use WWW::OpenAI;
use WWW::PaLM;Several iterations using OpenAI
my @startSentences = [
'I hate R', 'I really love WL', 'We really hate WL', 'I love R',
'I love Julia', 'I really hate R', 'We hate R', 'I often hate WL', 'I often hate Perl',
'I like Perl', 'We often hate R'
];my $request1 = "Generate BNF grammar for the sentences: {@startSentences.join(', ')}";
my $variations1 = openai-completion($request1, format=>'values', temperature => 0.15, max-tokens => 600);
$variations1my $variations2 = $variations1.lines.grep({ EBNF::Grammar::Relaxed.parse($_, rule => 'rule') }).join("\n");my $grCode = ebnf-interpret($variations2, style => 'inverted', name => 'First', rule-type => 'rule');
say $grCode;my $gr = ebnf-interpret($variations2, style => 'inverted', name=>'First', rule-type => 'rule'):eval;my $grTopRule = "<{grammar-top-rule($grCode)}>";
say $grTopRule;Generate random sentences:
my @genSentences = (^12).map({ random-sentence-generation($gr, $grTopRule) }).sort;
.say for @genSentences;Make variations for each sentence:
my $k = 1;
my $request2 = "Make 4 variations of each of the sentences: {@genSentences.map({ "{$k++}) $_"}).join("\n")}";
$request2 my $answer2 = openai-completion($request2, format=>'values', temperature => 0.65, max-tokens => 600);
$answer2Split the sentences:
my @varSentences = $answer2.lines;
.say for @varSentences;Generate new BNF:
my $request3 = "Generate BNF grammar for the sentences: {[|@genSentences, |@varSentences].join(', ')}";
my $answer3 = openai-completion($request1, format=>'values', temperature => 0.15, max-tokens => 600);
$answer3