ONNX Runtime Raku Bindings - Summary

What Works

Standalone Scripts (✅ WORKING)

• onnx-working.raku - Basic functionality demonstration

• mnist-final.raku - Complete MNIST inference example

• Both scripts work perfectly and can load models and run inference

Why Modules Don't Work

Raku's module precompilation system has a bug when:

1. Modules use NativeCall to load native libraries

2. Function pointers are obtained at runtime (not direct exports)

3. These function pointers are called during module initialization

This causes the precompilation process to hang indefinitely.

Attempted Solutions

1. Traditional Module Approach (❌ FAILED)

• Created various module architectures (Simple, Direct, Lazy, etc.)

• All hang during precompilation when trying to initialize ONNX Runtime

2. Server-Client Approach (⚠️ PARTIAL)

• Created onnx-runtime-server.raku - A standalone server that handles ONNX operations

• Created wrapper modules (ONNX::Wrapper, ONNX::Runtime) to communicate with server

• Works in principle but has complexity issues with process management

3. Script Generation Approach (⚠️ PARTIAL)

• Created ONNX::ScriptWrapper that generates temporary scripts

• Avoids precompilation by running standalone scripts

• Works but has overhead and complexity

Recommendations

For production use:

1. Use the standalone scripts directly - They work perfectly

2. Wait for Raku to fix the precompilation bug - This needs to be reported upstream

3. Consider creating a C wrapper library - With direct function exports instead of function pointers

Key Learning

The ONNX Runtime C API's design (function pointer table via OrtGetApiBase()) is fundamentally incompatible with Raku's current module precompilation system. This is a Raku bug, not an issue with our implementation.

Files Overview

Working Scripts:
├── onnx-working.raku          # Basic ONNX Runtime demo
├── mnist-final.raku           # Complete MNIST inference
└── onnx-runtime-server.raku   # Server for wrapper approach

Module Attempts:
├── lib/
│   ├── ONNX/
│   │   ├── Simple.rakumod     # Minimal module (exports types only)
│   │   ├── Wrapper.rakumod    # Async server wrapper
│   │   ├── Runtime.rakumod    # Sync server wrapper
│   │   └── ScriptWrapper.rakumod # Script generation wrapper
│   └── ONNX/Runtime/
│       ├── API/Simple.rakumod # Various architecture attempts
│       ├── Direct.rakumod
│       ├── Lazy.rakumod
│       └── ... (many more attempts)

Documentation:
├── README-ONNX-Runtime.md     # User documentation
├── README-precompilation-issue.md # Technical details of the bug
└── SUMMARY.md                 # This file

Next Steps

1. Report the bug to Raku developers with our minimal test cases

2. Use the working scripts for any immediate needs

3. Consider alternative approaches like creating a simplified C wrapper library