Changelog

Versioning follows Semantic Versioning. Because the module is still pre-1.0, the minor number carries both new features and behaviour changes; api in META6.json is bumped only when the DB role contract is broken.

0.9.2 - 2026-11-01

Added

• TextColumn.slice($from, $count?): a row window over a text column that copies nothing. The view borrows the parent's payload buffer and its three tables and only carries a start row, so slice costs one small object whatever the column size -- a thousand 100k-row slices of a million-row column take 23 ms in total. elems, at, to-list, bytes-at, byte-size and is-slice all read through the window; .bytes still returns the whole parent payload, which is documented on the method rather than silently trimmed, because trimming it would be the copy the view exists to avoid. Slices of slices compose (the start rows add), a window that runs past the column end raises instead of reading past the table, and a NULL inside the window still answers Any from at / to-list.

Changed

• A duplicate key inside an execute-many batch no longer drops that batch back to one prepared execution per row. ON CONFLICT ... DO UPDATE is applied once per key by the bulk replay and once per row by the prepared path; the two agree as long as the batch holds no repeated key. Rather than refusing those batches, the staging load now collapses each key group to its last row (row_number() over (partition by <keys> order by rowid desc)), which is the row the prepared path would have left behind, and rows with a NULL in any key column are exempt because a UNIQUE constraint lets NULLs repeat. The window costs about a third of the replay, so a probe runs first and the window is only built when the batch really repeats a key. A 200k-row upsert batch with unique keys stages in 1.2 s; ON CONSTRAINT <name> still takes the prepared path, but only because DuckDB itself refuses that clause before any row is looked at. t/05-appender.rakutest grows to 51 assertions (duplicate keys, NULL keys, a 140k batch across the 128 Ki flush boundary).

• column-arrays decodes in parallel by default when the result is materialised and large enough to pay for the threads, and the worker count now adapts to the result instead of always taking the core count: a worker is only used if it gets at least 20k rows, so a 25k-row result stays on one thread while a million-row one spreads over the pool. One million rows by seven columns: 8.0 s serial, 4.6 s parallel. :!parallel still forces the serial path, and DB_DUCKDB_NO_COLUMN_PARALLEL=1 turns the default off for a whole process (both covered in t/08-parallel.rakutest, 31 assertions).

Notes

• TextColumn.build stays single-threaded, on measurement. The build was split into spans and run on the worker pool in this cycle; eight workers moved two million 200-byte rows from 355 ms to 320 ms (1.1x), were level at one million, and were slower than one thread on a narrow payload. Both passes are almost entirely inside the shim, and MoarVM runs one native call per process at a time, so the fan-out bought nothing but two span planners and a promise dance -- it was removed again. The measured cost is one Buf allocation and one copy per chunk: a million 40-byte strings build in 67 ms.

0.9.1 - 2026-10-31

Added

• at(:typed) / to-list(:typed) on ColumnView: temporal columns as Raku values. DATE comes back as a Date, every TIMESTAMP_* as a UTC DateTime with sub-second digits kept as a Rat, both built straight from the integer payload instead of formatting the text and parsing it back. A raw view decodes typed segments into a cache of their own -- the same column answers Date/DateTime for a typed read and text otherwise -- while a slice, a batch or a detached from-list view rebuilds the value from DuckDB's own text and keeps that text for (BC), infinity and anything else that is not the plain shape. TIME and TIMESTAMPTZ stay text: there is no lossless Raku value to map a bare clock or a zoned stamp onto. 53 assertions in t/12-temporal.rakutest.

• TextColumn.to-list decodes one Str per ASCII window. The payload is laid out back to back, so a window that starts and ends on row boundaries can be decoded in one piece and sliced with substr; a window whose byte count exceeds its character count (non-ASCII) falls back to the per-row subbuf + decode, and windows are capped at 8 MiB so the extra memory is one window rather than a second copy of the payload. One million 40-byte strings: 3.91 s before, 1.91 s after (best of 3). DB_DUCKDB_NO_TEXT_WHOLE=1 forces the old path.

Changed

• execute-many stages ON CONFLICT batches instead of running one prepared statement per row. For a batch of at least 32 rows whose statement is INSERT INTO t [(cols)] VALUES (?...) [ON CONFLICT ...], the rows are loaded into a __db_duckdb_stage_N TEMP table through the appender (created by CTAS ... limit 0, so the column types come from the target and the constraints do not) and replayed with a single INSERT ... SELECT that carries the original clause; the table is dropped on the way out. DO NOTHING always takes this path, because skipping a row gives the same result whether the rows go in one at a time or all at once. DO UPDATE takes it only when the clause names its conflict target explicitly and the batch holds no duplicate key: the bulk form applies DO UPDATE once per key, the prepared path once per row, so a batch with repeated keys keeps the prepared path and its last-row-wins order. Everything else -- RETURNING, expressions, sub-selects, ON CONSTRAINT, narrow batches -- is unchanged. The batch stays inside the usual transaction: a failing row rolls the whole thing back and leaves no staging table behind. A 20k-row upsert batch drops from 55.6 s to 0.25 s, 100k rows to 0.69 s.

0.9.0 - 2026-10-31

Added

• DB::DuckDB::TextColumn: a byte oriented export of a VARCHAR / BLOB column. column-view(2).text returns one payload buffer plus a per-row byte offset and length, filled by the shim with a single copy per chunk -- no Str per row, no boxing. bytes, offset, length, bytes-at, offset-buffer, length-buffer, null-buffer, is-null, at, to-list, byte-size and count are the accessors; at still returns one Str, but only for the rows you ask for. One million 40-byte strings: 156 ms to export, against 1535 ms for to-list, rows identical across chunk seams. Other types are refused -- numeric columns already have int64 / num64 / sum. When the shim is missing, .text refuses with an explanation instead of silently using the slow path; TextColumn::text-column-available() answers that question up front.

• dkd_varchar_unpack_base(), shim ABI 6: dkd_varchar_unpack numbers a chunk's payloads from 0, so a column spread over several chunks stored every chunk's offsets relative to its own slice -- row 2048 of a two-chunk column claimed to live at byte 0. The new entry point takes an offset_base and adds it to each entry, which keeps the fix at one store per row inside C; the same fixup in Raku measured 503 ms per million rows, more than the whole export. dkd_varchar_unpack stays as a wrapper with base 0, so the ABI of existing callers is unchanged, and the cache rebuilds anything below ABI 6.

• t/16-text-column.rakutest (45 assertions): payload layout, NULL versus empty string, the inline/pointer boundary at 12 bytes, multibyte payloads, chunk seams in a 5000-row result, BLOB columns, the refusals, and the offset[i] == sum(length[0..i-1]) invariant that makes byte slicing legal.

0.8.3 - 2026-10-29

Fixed

• The by-value result bridge rebuilt itself on every run. It decided a cached build was current with $out.modified >= $src.modified, and this Rakudo reports a negative Instant for files written seconds ago -- so a freshly compiled bridge looked older than its own source and was recompiled every single time. The cache stamp is now a content fingerprint (FNV-1a over the C file, 3 KB, no dependency: sha1 lives in Digest::SHA1, not in core), and a library is reused only when its own <lib>.abi stamp records that fingerprint.

• The bridge build had the same concurrency hole the other two shims had. It linked straight onto the shared name; two processes building at once fought over it, and a library mapped by another process cannot be replaced at all. Builds now land under libduckdb-result-shim-<pid>-<epoch>.<ext> and are renamed into place only when the shared name is free or holds an older fingerprint. Six concurrent first builds on Linux: 6/6 pass.

Added

• DB::DuckDB::ShimCache, a sweep for cache debris that has accumulated across the three naming schemes this project has been through: .bak files, stamps whose library is gone (including the old directory-wide libduckdb-x-shim.abi), per-process builds older than an hour, and a shared-name library with no stamp that predates a stamped one. It refuses to empty a cache that has no usable library left, never touches the library the process just resolved, and can be switched off with DB_DUCKDB_NO_CACHE_SWEEP=1. Real cache before: 8 files, 4 of them dead. After one query: 4 files, all live.

• t/15-shim-cache.rakutest (11 assertions) pins what is removed, what is kept, and the two conservative cases.

0.8.2 - 2026-10-29

Fixed

• Arrow's documented fallback was not actually a fallback. On a machine without a C compiler, shim-release-fn still returned the dkd-arrow-release sub: require succeeds because is native(&shim-library) resolves lazily, so the failure surfaced on the first native call -- after the array had been handed out, with no second release path left. The shim module is now asked whether a library exists before it is trusted; when it does not, the cached in-Raku callback takes over. Verified by making the build fail on purpose (a directory where the library file belongs): t/11 passes 17/17 on that path instead of raising.

• The Arrow shim build now has the concurrency hardening the VARCHAR shim got in 0.7.1. It used to link straight onto the shared file name, so two processes building at once fought over it -- and on Windows a library already mapped by another process cannot be replaced at all. Builds land under libduckdb-arrow-shim-<pid>-<epoch>.<ext> and are renamed onto the shared name only when it is free or holds an older ABI; the ABI stamp is per file (<lib>.abi) rather than one per directory. Six concurrent first builds: 6/6 pass, and the cache ends up with the shared library plus at most one private one, each carrying its stamp.

Changed

• Five em dashes in Appender.rakumod / Connection.rakumod comments had been mangled into 鈥? by a GBK round trip. Comments only, no behaviour.

0.8.1 - 2026-10-29

Fixed

• The Arrow release callback no longer grows the process on machines without a C compiler. ArrowArray.release is a function pointer, so it can only be called from Raku by nativecast-ing the address to a callable -- and every nativecast builds a fresh native call stub that the VM never reclaims. A 60-round loop over 50k-row results (about 1500 releases) grew by 69 MB with the shim disabled; the same loop now grows by 2 MB, which is GC noise. The fix is not a new dependency: DuckDB uses one release function for every chunk it converts (measured: a single distinct pointer across 245 chunks from five results), so the callback is now cast once per distinct pointer and cached, with a Lock around the fill. A library that genuinely uses several release functions still gets one stub per pointer.

• On Linux the same 40-round loop grows with and without the shim (+54 MB against +42 MB), so that growth is the allocator/GC holding pages rather than the release path. The Windows numbers are the ones that isolate the bug: +69 MB before, +2 MB after.

• tools/perf/bench-arrow-leak.raku reported rss=0 on Linux because /proc/self/status separates the label from the value with a tab and the pattern expected a single space.

Added

• DB_DUCKDB_NO_ARROW_SHIM=1, the Arrow counterpart of DB_DUCKDB_NO_VARCHAR_SHIM. It skips the build and the load entirely, which is what makes the in-Raku release path testable rather than only reachable by accident on a compiler-less box.

• t/11-arrow.rakutest now exercises that path: 10k rows are walked with the shim forced off, checking that every chunk still decodes and arrives in order (17 assertions, was 15).

0.8.0 - 2026-10-29

Added

• Parquet read and write, on top of the SQL layer. duckdb.h contains nothing named parquet in DuckDB 2.0 -- there is no C entry point to bind -- so DB::DuckDB::Parquet composes the two statements that do exist: the read_parquet(...) table function and COPY ... TO ... (FORMAT PARQUET). New methods on both the driver and the connection: read-parquet (a path, a glob, or a list of paths; :union-by-name, :filename, :hive-partitioning, :columns, :stream), write-parquet (a table or a query; :sql, :schema, :compression, :row-group-size, :partition-by, :overwrite-or-ignore), and parquet-metadata / parquet-schema / parquet-row-groups for the footer.

• Reads bind their file names, one placeholder per path (the list form is read_parquet([?, ?])), so a path containing a quote needs no escaping and cannot be spliced into SQL. Writes must interpolate the target path and the table name, so both go through the same literal/identifier quoting the rest of the driver uses, and compression is validated against the codec whitelist before the statement is built.

• t/14-parquet.rakutest (64 assertions): round trip for every value type, quote/space/unicode paths, glob and list reads, column pruning, footer metadata, overwrite semantics, error paths, partitioned writes, streaming and lazy views.

• tools/perf/bench-parquet.raku.

Notes

• A Parquet scan is not a second decode path. Rows arrive as the same chunks and vectors a table scan produces, so the VARCHAR, temporal and DECIMAL C channels apply unchanged. On 1M rows x 4 columns: BIGINT 0.58 s from Parquet against 0.58 s from the table, VARCHAR 1.15 against 1.20, all four columns 5.08 against 5.05 µs/row. The first streamed chunk is faster (19.8 ms against 37.7 ms) because less data is read off disk.

• Writing is COPY: 1M rows x 4 columns into a 2.05 MiB zstd file in 0.033 s, against 11.3 µs/row for the INSERT path through execute-many -- roughly 400x per row.

• row-group-size is honoured only at 2048-row vector boundaries, so real row groups come out as multiples of that. That is DuckDB's behaviour; it is recorded here because the option name suggests finer control.

• Reading a partitioned directory leaves the partition column out of the files and hive_partitioning appends it after the stored columns, which is worth knowing when reading by column index.

0.7.2 - 2026-10-29

Changed

• The README is back in step with the code. Status no longer lists Appender as unimplemented -- it has existed since 0.6.0 and t/05-appender.rakutest covers it -- and now lists the write API among the implemented surface. Requirements names the versions actually tested: Rakudo 2026.09 on Windows 11 and on WSL Ubuntu, instead of 2025.06 on Windows only. The same paragraph now says that Parquet is reachable through SQL (read_parquet, COPY ... TO) while no dedicated helper is bound for it, so the remaining gap is stated without implying the format is unusable.

Removed

• stamp-file and abi-stamp in DB::DuckDB::VarcharShim, both dead since the cache moved to per-library .abi stamps written by lib-stamp. The reason mtime comparison cannot be trusted on this Rakudo is kept as a note on lib-stamp, so the knowledge survives the code.

0.7.1 - 2026-10-29

Fixed

• Installers no longer invent a msvcrt dependency on Linux, or a libc one on Windows. Arrow.rakumod asked for memcpy through a literal is native('msvcrt'); zef and raku-pm scan the source for those literals and treat every hit as a hard dependency, so the module refused to install on Linux with "missing 1 native library: msvcrt". The name is now built in a constant out of platform-sized pieces, which leaves that scan empty while still resolving at compile time. On Linux the constant is libc.so.6, because NativeCall would have turned a bare libc into liblibc.so, which does not exist.

• Concurrent first-time shim builds no longer corrupt each other. Two processes compiling the shim at the same moment both wrote the shared file name, so one lost its output (Permission denied on Windows) and the cache stamp could describe a library that was no longer there. A build now lands under a private ...-<pid>-<epoch> name and is moved onto the shared name only when that name is free, so the cache is safe to share between processes. .abi stamps are checked per file, and a library built in the last hour is never pruned, since another process may be about to load exactly that one.

• t/10-varchar-join.rakutest looked for the cache stamp under the shared library name, so it reported a failure whenever the shim legitimately lived under a private name.

0.7.0 - 2026-10-29

Changed

• DECIMAL is now rendered in C from its unscaled integer. The Raku fixed point renderer cost 1.71 µs per value on its own -- more than DuckDB spends formatting the value and reading the column -- so the digit loop moved into dkd_decimal_join, covering the int16, int32, int64 and 128 bit widths. Measured on 1M rows: DECIMAL(9,2) 2.45 → 1.23, DECIMAL(18,4) 3.52 → 1.24, DECIMAL(38,10) 9.42 → 1.48 µs/value, against 1.24–1.42 for DuckDB's own ::VARCHAR cast. Values and text are unchanged.

• ENUM is decoded as a dictionary lookup instead of a per-value render. The keys are read with the width duckdb_enum_internal_type reports (UTINYINT, USMALLINT or UINTEGER) and the text comes from the type's dictionary, read once per chunk. 2.71 → 0.61 µs/value on 1M rows, and rows sharing a label share one Str, so nothing is allocated per value.

Added

• dkd_decimal_join in duckdb-varchar-shim.c (shim ABI 5), with 128 bit support behind __SIZEOF_INT128__; on a compiler without a native 128 bit integer the wide types keep the Raku renderer. DB_DUCKDB_NO_DECIMAL_SHIM=1 forces that fallback.

• t/13-decimal-enum.rakutest and t/decimal-enum-digest.raku, pinning nine decimal widths and scales, the 128 bit extremes, NULLs, the 400 entry dictionary (USMALLINT keys), the lazy view path, and three decode paths against DuckDB's own ::VARCHAR.

Fixed

• DECIMAL(19..38) lost negative values below -263.** push-decimals-huge treated a high word of -1 as the sign extension of an int64 and returned the low word as-is, so a value such as -10000000000000000000 came back as 8446744073709551616. The sign extension only applies when the low word is also negative; anything else now goes through the full 128 bit reconstruction.

• All-fraction widths printed a leading zero. DECIMAL(9,9) renders as .5 in DuckDB, not 0.5; the Raku renderer now matches, and the C renderer takes a flag derived from width == scale.

0.6.9 - 2026-10-29

Changed

• DATE / TIMESTAMP / TIMESTAMP_S / TIMESTAMP_MS / TIMESTAMP_NS / TIME / TIME_NS are decoded from their integer payload instead of DuckDB's VARCHAR cast. They used to come back as the text DuckDB formatted for each value; they are now rendered from the int32 day count (DATE) and the int64 counters (the rest), which removes DuckDB's per-value formatting and Raku's per-value string assembly. Measured on 1M rows: DATE 8.38 → 1.38 µs/value (6.1×), TIMESTAMP 23.35 → 1.22 µs/value (19.1×), TIME 4.58 → 1.27 µs/value (3.6×), and a three column .arrays read 35.88 → 4.78 µs/row (7.5×); DuckDB's own ::VARCHAR cast costs 1.30 µs/value, so the driver's rendering is now as cheap as having DuckDB format the text. The column contract is unchanged: the values are still Str, and TIMESTAMP_TZ / TIMESTAMP_TZ_NS still come from DuckDB because their text carries the session time zone.

Added

• dkd_temporal_join in duckdb-varchar-shim.c renders a whole chunk of temporary values into one separator-joined buffer (ABI 4). Without a C compiler the driver falls back to the pure Raku renderers and produces byte identical output; DB_DUCKDB_NO_TEMPORAL_SHIM=1 forces that fallback.

• t/12-temporal.rakutest and t/temporal-digest.raku pin the rendering against DuckDB's own VARCHAR cast, including pre-epoch dates (the (BC) suffix), 24:00:00, fractional seconds that end in zeros, the extreme timestamp, NULLs, the lazy column view and all seven type ids.

Fixed

• Temporal kind codes shared between C and Raku now start at 1. A zero code is false in Raku, so a DATE (coded 0 in the first cut) silently fell back to the generic per-value decoder.

0.6.8 - 2026-10-28

Fixed

• DB::DuckDB::Arrow was missing from META6.json. Neither Arrow nor the two ArrowShim modules were listed in provides, and duckdb-arrow-shim.c was not in resources, so installing 0.6.6/0.6.7 shipped a driver without the Arrow module -- and even the source file the shim is built from was absent from the distribution. All three modules and the C file are listed now.

• arrow-chunk / with-arrow-chunks were lexical subs. They were exported and worked when imported, but DB::DuckDB::Arrow::with-arrow-chunks(...) resolved to nothing -- the same shape as the release-shim bypass fixed in 0.6.6. Now our subs.

Tests

• t/11-arrow.rakutest +2: both exports called by their full name.

0.6.7 - 2026-10-28

Changed

• column-arrays(:parallel) splits by column weight instead of evenly. A result with two columns where one is a VARCHAR (SELECT id, s) is lopsided: a VARCHAR value costs about four times what a BIGINT costs to decode, so the old rule ("one worker per column, split every column the same number of times") let the cheap column finish early and left its worker idle while the expensive one ran on. Spans are now ceil(nt * w / Σw) per column, with weights VARCHAR 4 / nested 3 / DECIMAL·HUGEINT 2 / else 1; plain numeric columns stay whole unless they are the only column. On 200k rows of BIGINT + VARCHAR(40) with 20 interleaved rounds: deg=4 went from 0.92–0.93× (slower than serial) to 1.29–1.32×, deg=2 unchanged at 1.20–1.38×.

• Streaming results prefetch the next chunk. chunks used to take turns with the engine: fetch, decode, fetch. A worker thread now pulls the next chunk while the caller decodes the current one, with two slots (never a deeper queue) and a token channel to keep at most two chunks alive. Measured 1.06–1.13× on a decode-bound scan and 1.03–1.18× on an engine-bound streaming aggregation; time to first chunk is unchanged, RSS plateau +17 MB. DB_DUCKDB_NO_PREFETCH=1 restores the one-at-a-time path.

Fixed

• Span stitching died above 65535 elements. @col.append: |@parts[$k] flattens the span into arguments, so any lopsided split whose span crossed 65535 values raised "Too many arguments". Now spliced in place.

• A prefetch worker is joined before the result is destroyed. free stops the worker, awaits it and destroys anything still queued before calling duckdb_destroy_result; a stream abandoned after one chunk would otherwise read a freed result.

Tests

• t/08-parallel.rakutest +4: lopsided BIGINT+VARCHAR at 2/3/4/8 workers, compared value by value against the serial path.

• t/07-streaming.rakutest +9: prefetched and one-at-a-time streams agree, streams abandoned mid-way and streams dropped after one chunk leave the connection usable.

0.6.6 - 2026-10-28

Added

• DB::DuckDB::Arrow — Arrow C data interface view of a result. arrow-chunk($res, $i) converts one 2048-row chunk with duckdb_data_chunk_to_arrow and exposes it as a Chunk: .column($j) returns a typed view with .ints / .int32s / .nums / .bools / .strs, .is-null($i), .children for recursively walking LIST / STRUCT, and an idempotent .free; with-arrow-chunks($res, &code) releases every chunk for you. No LOAD arrow and no NanoArrow: the Arrow C data interface is just the ten-field ArrowArray / ArrowSchema structs, so they are declared directly as repr('CStruct'). The regular read path is untouched — Arrow is for nested types and for handing buffers to other components.

• resources/duckdb-arrow-shim.c + DB::DuckDB::ArrowShim. Releasing an Arrow array from Raku means calling the release callback pointer, and each such indirect call leaked on Windows (RSS 174 MB → 711 MB over 30 000 releases). The shim wraps the callback in C so the indirect call happens on the C side; with it, 30 000 release cycles stay flat (~175–238 MB). It is compiled on demand next to the VARCHAR shim, sharing the same compiler detection.

Fixed

• duckdb_result_get_chunk chunks were never destroyed. The duckdb.h contract is that the caller owns the returned chunk and must call duckdb_destroy_data_chunk, but all three call sites treated it as a borrowed handle. Cost: ~235 KB per 2048-row chunk, i.e. ~23 MB per 200 000-row query, ~4.9 GB after 200 such queries. Now arrow-chunk destroys the source chunk right after the conversion copies the data into Arrow-owned buffers, the parallel decoder destroys its chunk list after the workers are done, and column-views registers the chunks it keeps for lazy decoding in @!owned-chunks and releases them in free. 200 rounds of a 200k-row query now hold at ~270 MB (was 4.9 GB); the plain read path stays at 196 MB.

• The Arrow release shim was silently bypassed. Its entry points were lexical subs, so $mod.WHO<&dkd-arrow-release> yielded Any, the surrounding try swallowed the type error and every release fell back to the leaking Raku callback. They are our subs now. Set DB_DUCKDB_ARROW_DEBUG=1 to see which path is taken.

0.6.5 - 2026-10-28

Added

• VARCHAR decoding is one decode plus one split per chunk. The C shim grew dkd_varchar_join, which concatenates the payloads of a vector into a single buffer with a 0x01 separator between values and a 0x02 marker for NULLs. join-varchar then decodes that buffer once and splits it, instead of reading an offset/length pair and slicing out one substring per value. NULL rows come back as Any, so the marker never reaches the caller. Trailing empty strings are padded explicitly because Str.split drops them. When a payload really contains 0x01 or 0x02, the C side reports a conflict and the decoder falls back to the per-value path, so hostile data still round-trips exactly.

• t/10-varchar-join.rakutest covers the join path, the two fallback switches (DB_DUCKDB_NO_VARCHAR_JOIN, DB_DUCKDB_NO_VARCHAR_SHIM), empty strings, embedded NULs, multibyte text and payloads containing the separator bytes.

Changed

• Multibyte VARCHAR no longer decodes one subbuf per value. unpack-varchar keeps its ASCII fast path (decode once, slice by byte offset), but a column that is not pure ASCII is now handed to join-varchar instead. On 200k rows that is 5.14 us -> 1.27 us per value; the per-value loop survives only as the last resort for vectors the join refuses.

• A whole-column read of a lazy column decodes straight into the result. It used to build a 2048-element List per segment and copy it into the output, so every value was touched three times. Uncached segments now decode into the caller's array directly and the per-segment cache holds only what at() and slice() asked for. That removes the ~16% penalty the 0.6.4 note mentioned: at 1M rows a lazy VARCHAR read is now at parity with, or slightly faster than, eager column() (0.82 / 1.05 / 0.91 us per value for short / long / multibyte, against 0.85 / 1.28 / 0.98 eager).

Removed

• The temporary DB_DUCKDB_SHIM_TRACE hook in the shim loader, used to bisect a crash during the join work.

0.6.4 - 2026-10-28

Changed

• column-views is lazy for non-raw columns. VARCHAR, BLOB, temporal and nested columns used to be fully decoded while the views were built, so a wide result paid the whole string-assembly cost even when the caller read a single column. Such a column now keeps one owned chunk handle per segment and decodes a segment the first time a row in it is touched, caching the result; to-list reuses the same segment decode. On 1M rows x (2 VARCHAR + 1 BIGINT), building the views drops from ~5.0 s to 0.10 s, and reading a single value from 5.0 s to 0.09 s. Reading every column still costs the same as before (~16% more than one-shot eager decode for VARCHAR, because each of the 489 segments is dispatched separately). DB_DUCKDB_NO_COLUMN_VIEW=1 still falls back to column-arrays.

Fixed

• Point access was 25x slower than it needed to be. ColumnView::!seg copied the whole segment table (my @b = @!bases) on every value read, which runs once per at / int64 / num64 / is-null call. 1M int64() calls took 107 s (107 us per value); after reading the attributes in place it takes 3.7 s, and 1000 at() calls drop from 202 ms to 19 ms.

• to-list on raw columns ignored the element type. The CArray sat in an untyped variable, so every index re-dispatched. to-list now hands each NULL-free segment to one of ten typed loops, taking 1M BIGINTs from 1.71 s to 0.97 s (on par with column-arrays at 1.01 s).

Added

• t/09-lazy-view.rakutest - 22 assertions over the lazy path: shape, random access, NULLs, to-list, slice, single-column results, aggregates refusing a lazy column, and the eager fallback. All nine test files pass.

0.6.2 - 2026-10-28

Added

• Appender.append-rows(@rows) - bulk write path for the C appender. The target table's column types are read once (duckdb_appender_column_type, newly bound) and each column gets a filler that calls the matching duckdb_append_* directly, with the return code compared inline instead of through a Raku sub call. integer / float / text / boolean columns are specialised; DECIMAL, temporal, BLOB and nested columns keep the generic per-value type dispatch so exactness is preserved. insert-rows and execute-many were switched over to it, so both get the win without any API change.

Performance

• insert-rows, 100k rows x 4 columns, 0.6.1 vs 0.6.2 measured back to back on the same machine (median of five runs, one mode per process because Raku GC noise swings a single run by 2-3x):

  • int / num / bool / text values: ~11.6 us -> ~4.2 us per row (~2.8x);

  • values holding a Rat (e.g. $_ / 3 without .Num): ~15.5 us -> ~11.3 us per row (~1.4x) - turning each Rat into a num now dominates the per-row cost, so the specialised filler has less to win back.

  Writing the rows with an explicit append-row loop still costs ~8.7 us/row (num data) - use append-rows (or insert-rows) for bulk loads.

Fixed

• A row shorter than the table used to be appended as a single value (for $vals iterated the scalar holding the row once), so DuckDB tried to cast the row's string form - Could not convert string '2 x' to INT64 - and the batch failed. Short rows now go through the general path and leave the missing columns on their DEFAULT.

Tests

• t/05-appender.rakutest +5 assertions (30 total) - short rows through append-rows / insert-rows, DEFAULT on the missing columns, and an integer wider than int64 (2**80) surviving the specialised filler as an exact HUGEINT.

0.6.1 - 2026-10-28

Added

• column-arrays(:parallel) - the columns of a materialised result are now decoded by several threads at once (:degree caps the worker count, otherwise the core count / DB_DUCKDB_DEGREE). The work is cut into (column, chunk-span) tasks: a worker owns one column over a contiguous run of chunks and appends into that column's preallocated array, so workers share no memory and the result is byte-for-byte identical to the serial path. A result with fewer columns than workers splits the chunks of each column instead and stitches the spans back in chunk order, which is what makes a single wide VARCHAR column scale too. Measured at 1M rows (median of four runs): 5 numeric columns 2.2x, 4 numeric columns 2.0x, 7 mixed columns 2.1x, 1 VARCHAR column 2.4x; narrow two-column results barely move (1.1x), because boxing and GC are shared between the workers.

• Streaming results and already-partially-read results cannot be split and fall back to the serial path; results below 20 000 rows stay serial.

Docs

• duckdb-perf-next-steps.md: P2-1 (per-chunk parallel decode) marked implemented, with the numbers and the two dead ends written down.

Tests

• t/08-parallel.rakutest (22 assertions) - :parallel against the serial path for a 50 000-row x 7-column table at 0/2/4/8 workers, single-column results that have to be stitched back together (VARCHAR, BIGINT, DECIMAL), the below-threshold fallback, the streaming fallback, the partially-consumed fallback, and repeated runs.

0.6.0 - 2026-10-28

Added

• DB::DuckDB::ColumnView - a column handle that never boxes a value on the way in. It keeps the vector's data pointer and validity bitmap as they are, so int64 / num64 / is-null read straight out of DuckDB's buffers, and to-list walks them with a typed loop instead of going through Mu. at falls back to the boxed path for the types that have no scalar view (VARCHAR, dates, wide DECIMAL, nested types). sum / min / max / count / mean run over the raw buffers and skip NULLs; slice returns a narrower view. Reached through $result.column-view($i) and $result.column-views.

• Streaming results. $conn.query($sql, :stream) (and $stmt.execute(:stream)) switches the result over to duckdb_execute_prepared_streaming + duckdb_stream_fetch_chunk, and $result.chunks hands out one DB::DuckDB::Chunk at a time. A chunk is one DuckDB vector (2048 rows, the last one short), exposes size, column-count, column-view($i), row($i) and to-rows, and keeps only the current vector in memory -- 1M rows x 3 columns take 12 ms to the first usable chunk instead of 5.8 s to the last one.

• DB::DuckDB::Chunk documents and enforces its own lifetime: a chunk dies the moment the next one is pulled, and the views handed out by it die with it. Reading a stale view raises instead of returning whatever now lives at that address, which is what the old code did silently.

• Result.is-streaming.

Changed

• chunks expires the previous chunk before pulling the next one, and free expires and drops the chunk it still holds, so a Result can no longer be freed under a live view.

• Statement.execute and Connection.query default :stream to False. Without the default, an omitted :stream passed the Bool type object down to Result, which then reported an undefined is-streaming and made the streaming branch depend on how the value happened to be used.

Tests

• t/06-column-view.rakutest (36 assertions) - column-views against column-arrays for all 20 columns of a 5000-row table covering every numeric width, VARCHAR, unicode, DATE, TIMESTAMP, HUGEINT and DECIMAL(38,6), plus NULL counts, a single-chunk table, a zero-row result, the aggregates, and the "second read is refused" rule.

• t/07-streaming.rakutest (41 assertions) - chunk sizes follow the 2048-row vector capacity (2048/2048/904 for 5000 rows), streamed values equal the materialised ones, zero-row streams yield no chunks, row / to-rows box correctly, and both expiry paths (next chunk pulled, result freed) raise on use.

0.5.1 - 2026-10-27

Added

• DB::DuckDB::Appender - a binding for duckdb_appender_*. Rows are appended straight into the table's storage instead of going through the SQL parser, and the value types are mapped the same way the parameter binder maps them (Bool, Int, Num, Rat, Str, Blob, undefined -> NULL).

• $db.insert-rows(Str $table, @rows) - the batch form. One appender, one transaction, UInt back. Measured on this machine, 100k rows x 4 columns (bigint, double, varchar, boolean) into an in-memory database: 1.0-1.5 s versus 3.4-3.7 s for 1000-row multi-value INSERT statements, i.e. 2.5-3.3x across runs. The remaining gap to the C API's own 0.33 s is Raku-level dispatch and string marshalling per value.

• $db.execute-many(Str $sql, @rows) - runs a prepared INSERT once per row and routes plain INSERT INTO t [(cols)] VALUES (?...) statements through the appender instead. ON CONFLICT, RETURNING, expressions and sub-selectors keep the statement path.

• $db.appender(Str $table) - the raw handle, for code that wants to control row boundaries itself. An unfinished appender is flushed and closed by its destructor, but an explicit $ap.free is what reports a failed cast.

• Session options - DB::DuckDB.new(:threads(8), :memory-limit('2GB'), :preserve-insertion-order(False)) pushes them to every connection, and $db.set-session-option($name, $value) changes them on a live session. preserve_insertion_order is only sent when it is turned off, so the engine default stays untouched otherwise.

Fixed

• Appender opens the transaction only when it has to. The bulk paths check the connection's real transaction state, so a batch issued inside a caller's begin joins that transaction instead of nesting a second one and committing the caller's work.

• A failed batch leaves no rows behind. The appender is closed inside the CATCH block, before the rollback, so buffered rows are not flushed into a transaction that is about to be discarded. Verified by a regression test that feeds a bad cast as the last row of a batch and expects an empty table.

• execute-many no longer leaks an open transaction. A return inside the !atomically callback skipped the COMMIT; the callback now yields a value instead.

• t/04-varchar-shim.rakutest had no done-testing, which turned the whole suite's TAP plan into a failure even though every assertion passed.

• A reordered INSERT column list silently wrote DEFAULTs. The appender fast path maps the statement's column order back onto table order using the index list from the insert plan, which lives in a Hash slot and therefore comes back itemized ($(1, 2, 0)). Assigning that into an array without .List produced a one-element array holding the whole list, so each row was appended as one sliced value and every column fell back to its DEFAULT; execute-many('insert into t (d, i, s) values (?, ?, ?)', ...) returned the right row count while writing nothing. Both table-order and reordered statements are now covered by t/05-appender.rakutest.

0.5.0 - 2026-10-27

Added

• C shim for VARCHAR columns (resources/duckdb-varchar-shim.c, loaded through DB::DuckDB::VarcharShim). The vector's duckdb_string_t array is snapshotted once, then a single pass copies every payload into one contiguous Buf while filling an offsets table, a lengths table and a NULL bitmap. Raku side then decodes the flat buffer once and slices each value out with substr. This replaces the per-value duckdb_string_t dereference plus Buf.push loop that made wide string columns the slowest part of result reading. Measured against the per-value decoder on this machine:

  rows	column()	arrays()	hashes()
  100k	2.7x	1.5x	1.6x
  500k	3.1x	1.8x	1.7x
  1M	3.4x	1.7x	1.8x

• Soft dependency on a C compiler. The shim compiles on first use into ~/.local/share/DB-DuckDB (DB_DUCKDB_SHIM_CACHE overrides the directory, DB_DUCKDB_CC / CC pick the compiler, DB_DUCKDB_VARCHAR_SHIM points at a prebuilt library). With no compiler the module logs one note and keeps using the pure Raku decoder; nothing fails.

Changed

• VARCHAR no longer falls back to the per-value decoder whenever the shim is available: column(), column-arrays(), arrays(), row() and hashes() all share the batch result. The old path is kept and still used for non-flat vectors (dictionary or selection vectors), oversized payloads and unreadable pointers.

Safety

• Everything the shim reads is validated first: the duckdb_string_t array is snapshotted into memory the shim owns, out-of-line payloads are copied with a read-probing helper, and any payload longer than 16 MiB (the signature of a selection buffer being mistaken for a string vector) makes the shim return DKD_NOT_FLAT, which makes the driver fall back instead of crashing.

0.4.0 - 2026-10-27

Added

• column(Int $idx) returns every remaining value of a single column as one Array. It skips the per-row Array allocation, so pulling one column out of a wide result is roughly an order of magnitude cheaper than arrays() plus a map.

• column-arrays() drains the whole result column by column into a List of Arrays (one per column; named that way because $.columns is the column count accessor). Measured 1.7-2x faster than arrays() at 3-5 columns, and it is the cheapest way to consume a wide result in full.

Changed

• hashes() decodes a whole data chunk at a time: each column is appended to a typed Raku array with one native loop pass, then row hashes are built with nqp::hash/nqp::bindkey instead of going through Hash assignment. Measured 2.2-2.6x faster at 3-5 columns on 300k rows.

• Column decoding got typed fast paths for the integer, unsigned integer, float, double and boolean vector types. Columns carrying NULLs, or types without a fast path (VARCHAR in particular), still fall back to the per-value decoder, so behaviour is unchanged.

• row() and arrays() are unchanged in behaviour; their timings sit inside run-to-run noise on this machine.

0.3.1 - 2026-10-26

Changed

• Alien::Libduckdb dependency pinned to an exact version: Alien::Libduckdb:ver<1.1.0>. The C API is not ABI-stable across DuckDB majors - duckdb_result layout, DUCKDB_TYPE numbering and the by-value accessors all moved - so "whatever Alien is newest" was a silent memory-corruption risk. 1.1.0 is the release that ships libduckdb 2.0.0, which this binding targets.

• Alien::Libduckdb now installs libduckdb from the upstream prebuilt archive by default and falls back to a cmake source build; nothing changes on this side, the library simply appears in seconds instead of minutes.

Added

• ABI guard at first use. verify-library() (exported from DB::DuckDB::Native) compares duckdb_library_version() with BINDING-MAJOR (2) and the driver calls it before opening its first database handle. A 1.x/3.x library reached through %DB_DUCKDB_LIB, a stale duckdb.dll next to raku, or a system libduckdb now dies with an explanatory message instead of corrupting result reads.

0.3.0 — 2026-10-26

Fixed

• Crash on POSIX when a result needs more than one chunk. duckdb_result_chunk_count and duckdb_fetch_chunk take duckdb_result by value (48 bytes on LP64). NativeCall's by-value marshalling for a struct that large happens to work under the MS x64 convention, but under SysV AMD64 the callee received a corrupt copy and DuckDB aborted in unique_ptr-related asserts. Any query whose result spanned multiple data chunks died; small results happened to fit in one chunk and looked fine.

• Result reading on POSIX no longer relies on NativeCall taking or returning duckdb_result by value anywhere.

Added

• DB::DuckDB::Bridge — hands the same 48 bytes to the two remaining by-value accessors as a pointer instead. Enabled on POSIX only; Windows keeps binding the C API directly.

• resources/duckdb-result-shim.c — ~20-line pointer-in wrapper around those two accessors, compiled on first use into the user's data directory and cached there.

Changed

• DB::DuckDB::Native binds the chunk accessors from the bridge library where one is used, and from libduckdb on Windows.

• META6.json gained resources and the DB::DuckDB::Bridge entry in provides (resources paths are declared relative to the distribution root, without the resources/ prefix).

• README no longer claims the distribution is free of C code.

Notes

• api stays 1: the public DB interface is unchanged.

• Verified on Linux (DuckDB 1.5.x, SysV AMD64): 18 smoke assertions and the full 42-assertion test suite pass, including multi-chunk results; and on Windows (MS x64) as a regression check. zef install also builds the shim from the installed distribution.

0.2.0

• Initial NativeCall driver over the DuckDB C API: DB role, connections, prepared statements, all 2.0 result types rendered exactly like DuckDB's own CAST(x AS VARCHAR). （内容由AI生成，仅供参考）