DB::DuckDB

A DuckDB driver for Raku's DB interface, implemented as a direct NativeCall binding over the DuckDB C API — no external process, no hand-maintained FFI layer. The C in the distribution is three small self-contained files under resources/, none of which includes duckdb.h or links libduckdb: a ~45-line pointer bridge (the two remaining by-value duckdb_result accessors cannot be called safely from NativeCall on SysV platforms; built on first use, skipped entirely on Windows), a VARCHAR bulk decoder, and an Arrow release bridge. The two shims are optional — without a compiler the driver uses slower pure Raku paths.

use DB::DuckDB;

my $db = DB::DuckDB.new;                              # in-memory
# my $db = DB::DuckDB.new(path => 'data.duckdb');     # file backed

say $db.query('select 42').value;                     # 42

$db.execute('create table foo (id integer, name varchar)');
$db.execute('insert into foo values (?, ?)', 1, 'a');

for $db.query('select * from foo').hashes -> %row {
    say %row<id>, %row<name>;
}

Status

Early but working. Implemented and covered by tests:

• driver: new, connect, query, execute, close, library-version

• connection: prepare, prepare-nocache, query, execute, begin, commit, rollback, ping, finish

• statement: positional ? and named $name placeholders (DuckDB syntax; :name is not supported), binding and rebinding, execute

• result: value, row, array, hash, arrays, hashes, column, column-view, column-views, column-arrays, :parallel, :degree, :stream, chunks, is-streaming, names, rows, rows-changed, free

• writes: appender, insert-rows, execute-many, and the DB::DuckDB::Appender handle (append-row / append-rows / flush / close / rows / width), all routed through DuckDB's appender

• parquet: read-parquet (a file, a glob or a list; :union-by-name, :filename, :hive-partitioning, :columns, :stream), write-parquet (a table or a query, with compression / row-group-size / partitioning), parquet-metadata, parquet-schema, parquet-row-groups

• DB::DuckDB::Arrow: arrow-chunk, with-arrow-chunks — Arrow C data interface views, no Arrow extension and no NanoArrow

Results are read chunk by chunk out of the vectors and decoded in Raku (DB::DuckDB::Decoder), because 2.0 stopped rendering several types through duckdb_value_varchar. VARCHAR columns are unpacked in bulk by a small C shim (see Design notes) and fall back to the per-value decoder when the shim cannot be built.

Every chunk the driver fetches is destroyed again once its vector has been consumed or copied, including the chunks column-views holds for lazy decoding (released in free). Chunks that were never destroyed cost ~235 KB each, which is what made a 200k-row query leak ~23 MB per round; the whole suite now runs flat across hundreds of rounds.

Not implemented (yet): callbacks/UDFs, extension loading, and anything that needs a by-value struct return (see Design notes).

Requirements

• Raku 6.d / Rakudo (tested on Rakudo 2026.09, Windows 11 and WSL Ubuntu)

• DB, plus Alien::Libduckdb, which puts a libduckdb where this module can find it: a prebuilt archive in seconds, a cmake source build as fallback (5-20 minutes). zef resolves both automatically.

• Optional: a C compiler for the VARCHAR and Arrow shims. It is not required - without one the driver just logs a note and uses the slower pure Raku decoders (and the pure Raku Arrow release callback, which caches its callable per release pointer since 0.8.1). Set DB_DUCKDB_CC if the compiler is not called cc/gcc/clang and not in one of the usual Windows toolchain directories.

The VARCHAR shim

resources/duckdb-varchar-shim.c is a self-contained C file: it does not include duckdb.h and does not link libduckdb, it only knows the byte layout of duckdb_string_t. It is compiled on first use (one gcc -O2 -shared) into ~/.local/share/DB-DuckDB and reloaded from there afterwards; a cached library is reused only when the per-file stamp next to it records the ABI this driver expects, so a newer source or a bumped ABI means a rebuild.

The two paths are cross-checked in t/04-varchar-shim.rakutest: both digests of the same table have to match byte for byte.

The cache directory is swept once per process, after a shim has been located: .bak files from older releases, stamps whose library is gone, per-process builds older than an hour, and a shared-name library with no stamp that predates a stamped one. Everything in that directory is rebuildable from resources/, so the sweep is safe; it still refuses to empty a cache that has no usable library left, and never touches the library the process resolved. DB::DuckDB::ShimCache holds the logic, t/15-shim-cache.rakutest pins it.

Why the Alien dependency is pinned

The DuckDB C API is not ABI-stable across majors: duckdb_result is passed by value, the DUCKDB_TYPE enum was renumbered, and the by-value accessors moved. This binding is written and tested against DuckDB 2.x, so META6.json asks for exactly Alien::Libduckdb:ver<1.1.0> - the release that ships libduckdb 2.0.0 - instead of whatever is newest. Bumping that pin is a deliberate act: read duckdb.h, check the result struct layout, run t/, then bump.

As a second line of defence, the driver calls duckdb_library_version() right before it opens its first database handle and refuses to continue when the loaded library is not 2.x:

DB::DuckDB: libduckdb v3.0.0 is not ABI-compatible with this binding, which
targets DuckDB 2.x; library in use: C:\rakudo\bin\duckdb.dll. ...

That catches what the pin cannot: a DB_DUCKDB_LIB override, a stale duckdb.dll sitting next to raku, or a system libduckdb.

How the library is located

1. Alien::Libduckdb::shared-lib() - the private prefix that zef install set up (this is the normal path)

2. %DB_DUCKDB_LIB environment variable, if it points at an existing file

3. resources/libraries/duckdb inside the distribution, if installed with one

4. next to the running raku executable - this covers C:\rakudo\bin\duckdb.dll

5. /usr/local/lib, /usr/lib, /opt/homebrew/lib

6. bare duckdb, letting the OS loader search

$env:DB_DUCKDB_LIB = 'C:\rakudo\bin\duckdb.dll'

Install

zef install .

Or run straight from a checkout:

raku -Ilib -e 'use DB::DuckDB; say DB::DuckDB.new.query("select version()").value'

Bulk writes

Multi-row INSERT statements pay DuckDB's parser for every row. The appender API does not, and the driver exposes it at three levels:

my $db = DB::DuckDB.new;

# 1. the low-level handle
my $ap = $db.appender('measurements');
$ap.append-row(1, 'x', 1.5);
$ap.append-row([2, 'y', 2.5]);    # a single Positional is unpacked
$ap.free;                         # flush + close; raises on a failed cast

# 2. whole batches (transactional by default)
$db.insert-rows('measurements', ((3, 'z', 3.5), (4, Any, 4.5)));

# 3. reuse an existing INSERT statement
$db.execute-many('insert into measurements values (?, ?, ?)',
                 ((5, 'w', 5.5), (6, 'v', 6.5)));

insert-rows and execute-many wrap the load in a transaction, so a batch that fails halfway leaves nothing behind. When the caller already has a transaction open (an explicit begin), the batch joins it instead of opening its own and nothing is committed early:

$db.execute('begin');
$db.execute('insert into measurements values (7, NULL, 7.5)');
$db.insert-rows('measurements', ((8, 'u', 8.5),));   # joins the transaction
$db.execute('rollback');                              # both rows are gone

execute-many only routes to the appender when the statement is a plain INSERT INTO t [(cols)] VALUES (?...). A batch that adds ON CONFLICT is staged instead: from 32 rows up, the values go into a TEMP table through the appender and are replayed with one INSERT ... SELECT that keeps the clause, so a 20k-row upsert costs 0.25 s rather than 55.6 s. DO NOTHING always takes that path; DO UPDATE takes it only when the clause names its conflict target and the batch has no duplicate key -- the bulk form updates a key once where the prepared path updates it per row, so anything else stays sequential. Everything left (RETURNING, expressions, sub-selects, ON CONSTRAINT, narrow batches) still runs as one prepared statement per row inside the same transaction, and a failure in a staged batch rolls back with no staging table left behind.

Session options

threads, memory_limit and preserve_insertion_order are pushed to every connection the driver opens, and can be changed on a live session:

my $db = DB::DuckDB.new(:threads(8), :memory-limit('2GB'),
                        :preserve-insertion-order(False));

$db.set-session-option('threads', 4);        # applied immediately

preserve_insertion_order => False is only sent when it is turned off: True is DuckDB's own default, so the driver stays out of the engine's way. Set any other option with set-session-option.

Zero-boxing column views

column() always builds a plain Array and boxes every value into a Raku scalar. When you only want to scan or aggregate a column, column-view keeps DuckDB's own vector buffer and reads through it in place:

my $r = $conn.query('SELECT id, amount, note FROM sales');
my $ids = $r.column-view(0);

say $ids.type;      # BIGINT
say $ids.elems;     # 5000
say $ids.int64(0);  # no Mu allocation per value
say $ids.sum;       # walks the raw buffer, skips NULLs
say $ids.min;       # same
say $ids.count;     # non-NULL values
say $ids.to-list;   # List, built with a typed loop (fastest full read)
say $dates.at(0, :typed);  # Date, for DATE / TIMESTAMP columns
say $ids.slice(0, 100);   # a narrower view, no copy

int64 / num64 / is-null / sum / min / max / count / mean / slice work on the numeric types without boxing. at($i) returns a proper Raku value for every type, but for VARCHAR, dates, wide DECIMAL and nested types it goes through the same decoder the boxed path uses, so to-list is the better choice there. $r.column-views returns all columns at once.

Temporal columns can skip the text form: at($i, :typed) and to-list(:typed) answer DATE as a Date and every TIMESTAMP_* as a UTC DateTime (fractional seconds kept as a Rat), built from the integer payload rather than formatted and re-parsed. TIME and TIMESTAMPTZ keep their text, because there is no lossless Raku value for a bare clock or an explicitly zoned stamp. A sliced or materialised view rebuilds the value from DuckDB's own text, so (BC) and infinity come back as the text they went in as instead of a wrong date.

The view borrows the result's memory: it is only valid while $r is alive, and reading it after $result.finish raises.

Zero-copy text columns

Text is the one type the view above still boxes, because at($i) on a VARCHAR has to return a Str. .text gives the column back as bytes instead: one payload buffer, plus a byte offset and a length per row, filled by the shim in a single copy per chunk.

my $t = $conn.query('SELECT note FROM sales').column-view(0).text;

say $t.type;          # VARCHAR
say $t.elems;         # 1_000_000
say $t.byte-size;     # 41_896_900 -- one buffer, one allocation
$t.bytes;             # the whole payload, back to back (Buf, borrowed)
$t.offset(7);         # byte offset of row 7 inside .bytes
$t.length(7);         # byte length (UTF-8 bytes, not characters)
$t.bytes-at(7);       # Buf slice -- no decode, no Str
$t.at(7);             # Str, one allocation, only when you ask for it
$t.is-null(7);        # signals NULL; an empty string is length 0 but not NULL
$t.to-list;           # List of Str/Any, for compatibility with column()
$t.slice(100, 500);   # rows 100..599 as another TextColumn -- no copy

slice is the cheap window: the new column borrows the same payload buffer and the same three tables and only remembers where it starts, so it costs one small object however big the parent is -- a thousand 100k-row slices of a million-row column take 23 ms in total. .bytes on the slice still reports the whole parent payload (trimming it would be the copy the view avoids); .byte-size reports the window's own extent. Slices of slices add their start rows, an out-of-range window raises, and a NULL inside the window still reads back as Any.

Use it for anything that does not need a string object per row: grouping, hashing, length filters, writing the column back out, feeding a parser. On one million 40-byte strings the export costs 67 ms against 1.9 s for to-list (3.9 s before it learned to decode whole ASCII windows), and the rows read back byte for byte identical across chunk seams (t/16-text-column.rakutest).

The tables are borrowed: offset-buffer / length-buffer / null-buffer expose the raw CArrays so a hot loop can index them without a method call per row. Treat them as read-only, and keep the result alive while the column is in use -- a streamed result recycles its chunks, so .text on a streaming view is a detached copy.

BLOB columns work the same way; the payload is the bytes as stored. Other types are refused: numeric columns already have int64 / num64 / sum, and temporal and DECIMAL values are rendered from their integer payload. When the C shim is unavailable, .text refuses with an explanation rather than silently falling back to the slow path -- check DB::DuckDB::TextColumn::text-column-available() first if you want to branch.

Parallel decoding

column-arrays decodes the columns of a materialised result on several threads by default, once the result is big enough to pay for them:

my $r = $conn.query('SELECT * FROM wide_table');

my @cols = $r.column-arrays;                        # parallel when it pays
my @four = $r.column-arrays(:degree(4));            # capped at 4 workers
my @one  = $r.column-arrays(:!parallel);            # force the serial path

A worker is only used if it gets at least 20k rows, so a 25k-row result stays on one thread and a million-row one spreads over the pool; :degree caps the count on top of that. DB_DUCKDB_NO_COLUMN_PARALLEL=1 switches the default off for a whole process.

The work is cut into (column, chunk-span) tasks and each worker writes into that column's preallocated array, so no memory is shared between workers and no value is copied twice. A result with more columns than workers spreads the columns over the workers; a result with fewer columns splits each column's chunks into contiguous spans and stitches the spans back in chunk order, so row order is preserved and the output is identical to the serial path (checked value by value in t/08-parallel.rakutest).

DB_DUCKDB_DEGREE in the environment sets the default worker count.

Measured on this machine, 1M rows, wall clock for the whole decode (median of four runs per shape; this is a 24-core box and the spread between runs is wide because GC dominates). The numbers predate the weight-based split below; the one- and two-column rows are exactly the lopsided cases that motivated it, so treat them as a lower bound:

Parallelism pays where the per-value cost is high (VARCHAR unpacking, DECIMAL formatting) or where there are many columns to spread. Results with only one or two columns are bound by the boxing step and by Raku's GC -- both shared between threads -- so the gain there is small. Below 20 000 rows the driver stays serial, because setting the workers up costs more than the decode.

Splitting by column weight

Cutting every column into the same number of spans spreads the work evenly only if the columns cost the same. They do not: a VARCHAR value costs about four BIGINTs to decode, a nested value about three, DECIMAL/HUGEINT about two. With two such columns and four workers, an even split of spans leaves the cheap column's worker idle while the expensive one still runs, and a lopsided result can end up slower than the serial path.

So the span count per column is ceil(nt * w / sum-of-weights), capped at both nt and the chunk count, with weights VARCHAR 4, LIST/STRUCT/MAP/ ARRAY/UNION 3, DECIMAL/HUGEINT 2, everything else 1. Plain numeric columns are left whole unless they are the only column, because stitching a numeric column back costs a copy that cancels the gain.

Measured on 200k rows of BIGINT + VARCHAR(40), 20 interleaved rounds in one process, paired median against the serial path:

The point is the second row: adding workers no longer costs speed. Absolute ratios move by up to 20% with machine load.

Two cases cannot be split and fall back to the serial path automatically: a streaming result (its chunks belong to the stream, not to a chunk list) and a result that was already partially read.

Streaming results

By default a result is materialised -- DuckDB runs the query to completion before the first row is available. :stream keeps the query pulling row batches on demand:

my $r = $conn.query('SELECT * FROM huge', :stream);

$r.is-streaming;         # True

for $r.chunks -> $chunk {
    # $chunk.size         -- rows in this vector (<= 2048, the last one shorter)
    # $chunk.column-count
    my $view = $chunk.column-view(0);
    ... $view.sum ...
    my @row  = $chunk.row(0);      # boxed values, for the odd row
    my @rows = $chunk.to-rows;     # boxed, for small chunks
}

A chunk is valid only until the next one is pulled. DuckDB recycles the vector, so the driver expires the previous chunk (and every view it handed out) before fetching the next one; any read through a stale chunk or view raises instead of returning another row's data. That is the trade-off for not copying: consume inside the loop body and let the chunk go. $chunk.is-valid reports whether the driver has moved on; .size and .column-count stay readable either way, because they are plain numbers rather than pointers.

Measured on this machine, 1M rows x 3 columns (bigint, double, varchar): first usable chunk after 12 ms, versus 5.8 s to materialise the whole result through column-arrays; summing the bigint column while streaming takes 3.9 s and agrees exactly with SELECT sum(a).

Prefetching

chunks used to take turns with the engine: fetch, decode, fetch. The engine computes while you decode nothing, and sits idle while you decode. A worker thread now pulls the next chunk while the caller works on the current one.

The queue is two slots deep, not more: a chunk is ~235 KB and decoding it costs several times what fetching it costs, so a deeper queue would only raise time to first chunk and memory. Two slots are enforced with a token channel -- the worker takes a token before fetching, the caller returns one when it drops the previous chunk -- so at most two chunks are alive.

Nothing changes at the API: free stops the worker, joins it and destroys anything still queued before the result goes away, so a stream abandoned after one chunk is safe. Measured 300k rows, 7-9 interleaved rounds: 1.06-1.13x on a decode-bound full scan, 1.03-1.18x on an engine-bound streaming aggregation, time to first chunk unchanged, RSS plateau +17 MB. DB_DUCKDB_NO_PREFETCH=1 restores the one-at-a-time path.

Arrow views

DB::DuckDB::Arrow exposes a chunk of a materialised result through the Arrow C data interface, so the buffers can be handed to another component instead of being boxed into Raku values here:

use DB::DuckDB::Arrow;

my $r = $conn.query('SELECT id, tags FROM items');

my $chunk = arrow-chunk($r, 0);          # 0-based chunk index, 2048 rows each
my $col   = $chunk.column(1);            # LIST<VARCHAR>
say $chunk.length;                       # rows in this chunk
say $chunk.columns;
say $col.null-count;
say $col.children[0].strs;               # the child vector of the list
$chunk.free;                             # idempotent

with-arrow-chunks($r, -> $c, $i { ... });  # frees every chunk for you

with-arrow-chunks hands each chunk and its index to the block and frees the chunk on the way out, so nothing leaks if the block dies. Chunks are numbered the same way as arrow-chunk.

Column gives typed access without boxing: .ints, .int32s, .nums, .bools, .strs (each returns a List), .is-null($i), .children for recursing into LIST/STRUCT, plus the raw .buffer($j) pointer and .length for talking to something that wants the buffers themselves. Chunk also exposes .addr for the ArrowArray* struct.

There is no LOAD arrow and no NanoArrow dependency: the Arrow C data interface is two ten-field structs, so ArrowArray / ArrowSchema are declared directly as repr('CStruct').

This is not a faster read path. strs has to follow the offset buffer and copy each value out, which on 200k rows is 6.3 s against 1.7 s for the bulk join-varchar decoder used by column() -- about 3.8x slower. Use it for nested types, for passing data to other libraries, or when you want the Arrow buffers as they are.

Releasing an Arrow array means calling a callback pointer stored inside the struct. Doing that from Raku naively leaked on Windows (~174 MB to ~711 MB over 30k releases, because each nativecast to a callable builds a stub the VM never reclaims). Two things fix it: resources/duckdb-arrow-shim.c moves the indirect call to C, and -- for machines without a compiler -- the in-Raku path now caches one callable per release pointer instead of building one per chunk. DuckDB uses a single release function for every chunk it produces, so that cache holds exactly one entry; the no-shim path measured +69 MB over ~1500 releases before and +2 MB after (GC noise). It is built on demand next to the VARCHAR shim, under a per-process name that is renamed into place only when the shared name is free, so several processes can build at once (same hardening the VARCHAR shim received in 0.7.1).

DB_DUCKDB_ARROW_SHIM points at a prebuilt library, DB_DUCKDB_NO_ARROW_SHIM=1 forces the in-Raku path, DB_DUCKDB_ARROW_DEBUG=1 logs which release path is taken. When no compiler is available the driver checks that a shim library actually exists before trusting it -- an earlier version only required the binding module, which compiles fine without the library and then failed on the first release call.

A chunk fetched from the result is owned by the caller: arrow-chunk destroys it right after the conversion has copied the data into Arrow-owned buffers, so the Arrow view does not keep the DuckDB result alive.

Parquet

DuckDB 2.0's C API has no Parquet entry point -- there is nothing named parquet in duckdb.h. The format is reachable only through SQL, so the driver wraps read_parquet(...) and COPY ... TO ... (FORMAT PARQUET) instead of pretending a binding exists. Rows read from a file travel through the same chunk/vector path a table scan uses, which is why the VARCHAR, temporal, DECIMAL and ENUM C channels apply unchanged.

# read: a file, a glob, or a list of files
for $db.read-parquet('sales.parquet').hashes -> %row { ... }
$db.read-parquet('data/*.parquet', :union-by-name, :filename).column-views[0].sum;
$db.read-parquet(<a.parquet b.parquet>, :union-by-name);

# write: a table, or a query
$db.write-parquet('out.parquet', 'events');                    # returns rows written
$db.write-parquet('out.parquet', 'select * from events', :sql);
$db.write-parquet('out.z.parquet', 'events', :compression<zstd>, :row-group-size(250_000));
$db.write-parquet('out/part', 'events', :partition-by<day region>);

# footer only
$db.parquet-metadata('out.parquet')[0]<num_rows num_row_groups format_version>;
$db.parquet-schema('out.parquet');
$db.parquet-row-groups('out.parquet');

Read paths bind the file names as parameters, one placeholder per path, so a file whose name contains a quote needs no escaping. Writes have to interpolate the target and the table name, so both go through the same literal and identifier quoting used internally, and compression is validated against the codec list before the statement is built.

Options: :union-by-name, :filename, :hive-partitioning, :columns, :stream when reading; :sql, :schema, :compression, :row-group-size, :partition-by, :overwrite-or-ignore when writing. COPY replaces an existing file by default; :overwrite-or-ignore matters for partitioned directories that already hold files. :row-group-size is only honoured at 2048-row vector boundaries, so the real groups come out as multiples of that.

Measured on 1M rows x 4 columns (best of 3, single process, zstd):

The reader is not a separate decode path, and does not need to be: reading a Parquet file costs what reading the equivalent table costs, within noise.

Environment switches

The shim build variables (DB_DUCKDB_VARCHAR_SHIM, DB_DUCKDB_SHIM_CACHE, DB_DUCKDB_CC, DB_DUCKDB_SHIM_TRACE) are listed under The VARCHAR shim.

Type mapping (DuckDB → Raku)

Raku → DuckDB (bind): Bool → BOOLEAN, Int → BIGINT, Num / Rat / FatRat → DOUBLE, Str → VARCHAR, Blob → BLOB, undefined → NULL, anything else → its .Str.

Dates and decimals are deliberately not auto-coerced into Raku Date/DateTime/Rat: DuckDB's own textual rendering is exact for every type, while the duckdb_value_* family that would return real date/decimal structures is deprecated and partly by-value. Exactness beats convenience here — convert at the call site when you need it.

Design notes

• One database instance per driver. connect creates connections that all share a single duckdb_database. Opening :memory: twice would silently create two independent databases, and opening a file path twice trips DuckDB's instance/file locking, so the driver opens once and guards it with $!opened.

• duckdb_result is mapped as a fixed-layout struct (48 bytes on LP64: three idx_t plus three pointers) and its size is asserted at load time, so an ABI change fails loudly instead of corrupting memory.

• Results are read through the chunk/vector API. duckdb_result_chunk_count / duckdb_fetch_chunk hand back a duckdb_data_chunk pointer, and each vector's data buffer is read in place; the structs living inside those buffers are dissected field by field through uint32/uint64 views, since NativeCall faults on CStruct array element access on Windows. No struct is ever returned or taken by value, with one deliberate exception: the two accessors that still declare a by-value duckdb_result are called through DB::DuckDB::Bridge, which passes the same 48 bytes as a pointer. NativeCall's by-value marshalling happens to work under MS x64 but is miscompiled under SysV, where it surfaced as unique_ptr NULL crashes inside DuckDB.

• Values are rendered exactly like DuckDB's own CAST(x AS VARCHAR), not coerced into Raku Date/DateTime/Rat. TIMESTAMP_TZ is the one piece of session state involved: the vector holds UTC microseconds, so the instant is shifted by the session zone's offset (synced from strftime(..., '%z') on construction and refreshed after SET TimeZone) and only then labelled with that offset.

• Prepared statements are cached per connection by the DB role; statements are destroyed when their connection is dropped. close also tears down connections the caller forgot to finish, because DuckDB will not release the file lock until every connection and prepared statement is gone.

• VARCHAR is unpacked in bulk by a C shim. Decoding a string column one duckdb_string_t at a time and Buf.push-ing each payload was the slowest thing in result reading (and pure-Raku batching was slower still, because every payload still had to be copied byte by byte). duckdb-varchar-shim.c snapshots the vector's duckdb_string_t array once, then copies every payload into one flat buffer while writing an offsets table, a lengths table and a NULL bitmap; Raku decodes that buffer once and slices values out with substr. Measured 2.7-3.4x on column() and 1.5-1.8x on arrays()/hashes() at 100k-1M rows. The shim never trusts the pointers it is handed: the string_t array is copied into memory it owns before being walked, out-of-line payloads are read through a probing copy, a payload longer than 16 MiB (the signature of a selection buffer mistaken for a string vector) and any unreadable pointer make it return DKD_NOT_FLAT, which is the signal to fall back to the per-value decoder for that chunk.

• Temporal columns are formatted by the driver, not by DuckDB. A DATE is an int32 day count and the TIMESTAMP* / TIME* types are int64 counters, so the text can be computed directly from the payload instead of asking DuckDB to CAST each value. dkd_temporal_join in the same shim renders a whole chunk into one 0x01-separated buffer (a NULL becomes 0x02) that Raku splits once, and a pure Raku renderer produces exactly the same text when no compiler is around. Measured on 1M rows against DuckDB's own cast: DATE 8.38 → 1.38, TIMESTAMP 23.35 → 1.22, TIME 4.58 → 1.27 µs/value, and a three column arrays() read 35.88 → 4.78 µs/row (the cast itself costs 1.30 µs/value, so this is the floor). The text matches the CAST to the byte, including the (BC) suffix on pre-epoch dates, 24:00:00, and fractional seconds with trailing zeros removed. TIME_TZ / TIMESTAMP_TZ / TIMESTAMP_TZ_NS stay on DuckDB's own rendering because their text carries the session zone. Set DB_DUCKDB_NO_TEMPORAL_SHIM=1 to force the Raku renderer.

• DECIMAL is rendered from its unscaled integer, in C. A DECIMAL is an integer plus a scale, and DuckDB prints exactly scale digits, trailing zeros included. Doing that in Raku turned out to cost more per value than asking DuckDB to format it (1.71 µs just for the formatting), so dkd_decimal_join does the digit loop instead, for the 64 bit and the 128 bit widths alike (the latter behind __SIZEOF_INT128__; without it the Raku path stays). On 1M rows: DECIMAL(9,2) 2.45 → 1.23, DECIMAL(18,4) 3.52 → 1.24, DECIMAL(38,10) 9.42 → 1.48 µs/value, against 1.24–1.42 for CAST(x AS VARCHAR). One quirk is reproduced on purpose: the all-fraction widths print without an integer digit (DECIMAL(9,9) → .5), where DECIMAL(10,9) → 0.5. Set DB_DUCKDB_NO_DECIMAL_SHIM=1 to force the Raku renderer.

• ENUM is a dictionary lookup, not a render. The keys are read as the narrowest unsigned integer that fits the dictionary (duckdb_enum_internal_type says which) and the text comes from duckdb_enum_dictionary_value, read once per chunk. 2.71 → 0.61 µs/value on 1M rows, which is faster than the VARCHAR channel because rows sharing a label share one Str and nothing is allocated per value.

• Works on Windows without an import library: the DLL is loaded at runtime, so the MSVC .lib that ships with the C API is not needed.

Tests

raku -Ilib t/01-basic.rakutest    # types, placeholders, transactions, errors
raku -Ilib t/02-file.rakutest     # file database, multiple connections, reopen
raku -Ilib t/03-types.rakutest    # every 2.0 type vs. DuckDB's own text rendering,
                                  # NULLs, chunked results, session time zone, uuid
raku -Ilib t/04-varchar-shim.rakutest   # the VARCHAR shim, including its fallbacks
raku -Ilib t/05-appender.rakutest       # appender, insert-rows, type mapping
raku -Ilib t/06-column-view.rakutest    # column-views vs column-arrays, all types
raku -Ilib t/07-streaming.rakutest      # chunks, 2048-row vectors, chunk lifetime,
                                        # prefetch vs one-at-a-time
raku -Ilib t/08-parallel.rakutest       # parallel column decode vs the serial path,
                                        # including lopsided results at 2/3/4/8 workers
raku -Ilib t/09-lazy-view.rakutest      # lazily decoded views and their cleanup
raku -Ilib t/10-varchar-join.rakutest   # the join-buffer VARCHAR decoder and its fallbacks
raku -Ilib t/11-arrow.rakutest          # Arrow C data interface views, nested types, release
raku -Ilib t/12-temporal.rakutest       # date/time/timestamp rendering vs DuckDB's own cast,
                                        # pre-epoch and extreme values, NULLs, lazy views
raku -Ilib t/13-decimal-enum.rakutest   # decimal widths 4..38 incl. the 128 bit slot and the
                                        # all-fraction quirk; enum dictionaries and NULLs
raku -Ilib t/14-parquet.rakutest        # parquet round trip, path shapes (quotes, unicode,
                                        # glob, list), options, footer metadata, partitioning
raku -Ilib t/15-shim-cache.rakutest     # cache sweep: what is removed, what is kept,
                                        # and that it never empties a usable cache

No prove harness is required -- each file is a standalone Test script.

License

Artistic-2.0 （内容由AI生成，仅供参考） （内容由AI生成，仅供参考） （内容由AI生成，仅供参考）