vortex-data/vortex

💥 ⚠️ Breaks

• aggregate fns to have grouped aggregate kernels for sum and count (#8314) @onursatici
• Remove `len` parameter from `ScalarFnArray::try_new` (#8378) @connortsui20
• Remove all TurboQuant stuff (#8377) @connortsui20
• Fix Validity::mask_eq semantics for mixed variants (#8334) @joseph-isaacs
• Rename Validity::no_nulls to definitely_no_nulls and add execute_no_nulls (#8333) @joseph-isaacs
• Pass execution context through list view rebuild (#8274) @connortsui20
• Initialize LazyBitBufferBuilder from Mask if available, when appending masks pass them by reference (#8221) @robert3005
• layout reader ctx (#8037) @onursatici

🗑️ 🚧 Deprecation

• Refactor validity checks to require explicit ExecutionCtx (#8273) @joseph-isaacs
• Deprecate vortex-array public APIs that use the hidden LEGACY_SESSION (#8269) @joseph-isaacs

✨ ✨ Features

• feat(vortex-geo): native Point extension type and GeoDistance scalar function (#8372) @HarukiMoriarty
• Add arrow import/export for vortex-json (#8339) @AdamGS
• WKB import/export VTables to Arrow (#8325) @a10y
• feat[gpu]: export as Arrow device array binary (#8320) @0ax1
• feat[gpu]: sliced validity in Arrow device export (#8318) @0ax1
• add vortex-geo crate and WKB extension type (#7722) @a10y
• Expose Merge expression in vortex-jni (#8319) @robert3005
• feat(vortex-array): add Interleave array encoding (#8277) @joseph-isaacs
• + 15 more

⚡ 🚀 Performance

• perf: branchless primitive zip kernel (#8270) @joseph-isaacs
• perf[gpu]: coalesce streaming kernels (#8357) @0ax1
• perf[mask]: check AllTrue/AllFalse in eq (#8354) @joseph-isaacs
• BitBuffer binary operations support in place updates to lhs (#8162) @robert3005
• Optimize slice for dict and minor changes in other arrays (#8321) @AdamGS
• perf: remove unneeded scalar_at and valid_at and use arrays instead (#8307) @joseph-isaacs
• Allow caching LayoutReader in VortexFile (#8244) @myrrc
• perf: branchless boolean zip kernel (#8275) @joseph-isaacs
• + 9 more

🐛 🐛 Bug Fixes

• <details>
• <summary>13 changes</summary>
• fix: keep Arrow device schemas aligned with exports (#8360) @0ax1
• aggregate_fn: Return NaN instead of null for mean of all-null input (#8365) @dimitarvdimitrov
• fix[file]: read the one-row pruning result in can_prune instead of re… (#8369) @tomsanbear
• Fix generation of stat pruning expression for unsupported dtypes (#8326) @robert3005
• fix[gpu]: decode extension arrays on the GPU (#8353) @0ax1
• fix issue with FileSystem path handling (#8248) @robert3005
• + 8 more

📝 📖 Documentation

• Validate all rustdoc builds (#8386) @AdamGS

📦 🧰 Maintenance

• <details>
• <summary>40 changes</summary>
• chore: clean up Arrow device export (#8359) @0ax1
• chore: polish cuDF C FFI (#8358) @0ax1
• Remove the unused website and clean other dependencies (#8362) @AdamGS
• Minor touchups for vortex-datafusion (#8356) @AdamGS
• Add docs for StatsRewriteRule (#8355) @AdamGS
• Disable wide-columns benchmark to stop benchmarks from timing out (#8331) @AdamGS
• + 35 more

💥 ⚠️ Breaks

• ffi: remove vx_file scan (#8152) @myrrc

✨ ✨ Features

• Expose row_idx in vortex-jni bindings (#8211) @robert3005
• Support for importing Arrow extension arrays into core vortex arrays and ParquetVariant support. (#8125) @AdamGS
• feat: add experimental `onpair` string encoding (#8144) @joseph-isaacs
• Teach zoned pruning to lower StatFn (#7937) @gatesn
• TurboQuant: Add distortion benchmark (#8070) @connortsui20
• Add UUID literal expression to vortex-jni (#8154) @robert3005
• Introduce basic binary compressor schemes and refactor some other bits (#8153) @AdamGS

⚡ 🚀 Performance

• perf: trim zctl listview instead of full rebuild (#8164) @mhk197
• Use Vec<u64> instead of BTreeSet for splits (#8194) @myrrc
• Skip empty chunks when aggregating over chunked arrays (#8168) @robert3005
• perf: fast FixedSizeListArray canonicalization for chunked arrays (#8161) @joseph-isaacs
• Always use `FileStatsLayoutReader` if available for `VortexFile` reader (#8147) @connortsui20
• Use selection in repeated scans (#8137) @myrrc
• Reduce monomorphization by dispatching over unsigned integer widths (#8110) @joseph-isaacs

🐛 🐛 Bug Fixes

• fix(fuzz): VortexError in file_io (#8189) (#8201) @[vortex-claude[bot]](https://github.com/apps/vortex-claude)
• fix: onpair use crates verison (#8200) @joseph-isaacs
• Fix unsound Delta unsigned-widening cast (#8195) @joseph-isaacs
• fix[fastlanes]: remove unsound delta cast? (#8192) @joseph-isaacs
• Always try compressing extension array's storage directly (#8157) @AdamGS
• fix: code coverage ci task (#8142) @0ax1
• fix: speed up coverage tests and LLVM tools path (#8136) @0ax1
• Pin cargo-edit to version that still supports rust 1.91.0 (#8141) @robert3005

📝 📖 Documentation

• docs: link to Vortex Explorer (#8132) @danking

📦 🧰 Maintenance

• <details>
• <summary>22 changes</summary>
• Refactor: Split monolithic scheme files into modular submodules (#8207) @joseph-isaacs
• test: expand utf8view and binaryview cuda export coverage (#8159) @0ax1
• Update all patch updates (#8174) @[renovate[bot]](https://github.com/apps/renovate)
• don't account array deallocation in random access bench (#8160) @myrrc
• Update aws-actions/configure-aws-credentials digest to 99214aa (#8170) @[renovate[bot]](https://github.com/apps/renovate)
• Update docker/setup-qemu-action digest to 0611638 (#8171) @[renovate[bot]](https://github.com/apps/renovate)
• + 17 more

✨ ✨ Features

• feat: support nullable Arrow Device array export (#8104) @0ax1
• Add built-in stats rewrite rules (#7935) @gatesn
• Add FilterPushdown support to spark data source (#7785) @robert3005
• Explicit Precision::Absent variant instead of Option<Precision> (#8042) @AdamGS
• Expression pushdown for duckdb (#7727) @myrrc
• Benchmarks Website V3: Admin and Auto-Deploy (#7849) @connortsui20
• Add pruning aggregate functions (#8025) @gatesn
• Allow writing Variant to files and test parquet-variant IO (#7945) @AdamGS

⚡ 🚀 Performance

• TurboQuant: better centroid initialization (#8116) @connortsui20
• Use CachedId for ArrayExpr scalar function ID (#8124) @joseph-isaacs
• Add Sparse pushdown kernels for is_constant, sum, and compare (#8028) @joseph-isaacs
• perf: aggregate min/max (#8061) @joseph-isaacs
• fastlanes: streaming compare + between kernels for BitPacked (#8015) @joseph-isaacs
• perf: remove implicit `ListViewArray` rebuild during `take` and `filter` operations (#8048) @mhk197
• perf[array]: bool filter kernel optimisation (#7125) @joseph-isaacs
• perf[array]: cast to prim faster (#8062) @joseph-isaacs
• + 2 more

🐛 🐛 Bug Fixes

• <details>
• <summary>10 changes</summary>
• Correctly handle Nan/Inf comparison in ALP between reduce (#8126) @robert3005
• fsst like to respect sql escape codes in the pattern (#8107) @onursatici
• Fix native library publish for non amd64 platforms (#8108) @robert3005
• wipe duckdb source in build.rs if not fully extracted before (#8106) @onursatici
• like stat falsification respects escape characters (#8103) @onursatici
• Fix dtype mismatch in Filter::take logic (#8073) @robert3005
• + 5 more

📝 📖 Documentation

• Document existing TurboQuant types (#8053) @connortsui20

📦 🧰 Maintenance

• <details>
• <summary>31 changes</summary>
• Remove public-api lockfiles (#8099) @AdamGS
• Remove pre df 53.0 polarsignals benchmark workaround (#8122) @robert3005
• Use plain functions for copy function in duckdb (#8109) @myrrc
• `-isystem` for Duckdb headers, build with `-Werror` (#7747) @myrrc
• Update actions/stale digest to eb5cf3a (#8077) @[renovate[bot]](https://github.com/apps/renovate)
• Update anthropics/claude-code-action digest to 787c5a0 (#8078) @[renovate[bot]](https://github.com/apps/renovate)
• + 26 more

✨ ✨ Features

• Thread scope dtype through stats rewrites (#8024) @gatesn
• Add Samply and benchmark skills (#8021) @gatesn

⚡ 🚀 Performance

• Slice list/list_view elements in duckdb exporter (#8020) @myrrc

🐛 🐛 Bug Fixes

• Fix nullability of Constant BetweenReduce rule (#8029) @robert3005

📦 🧰 Maintenance

• <details>
• <summary>4 changes</summary>
• Avoid re-building vortex-duckdb twice (#8035) @AdamGS
• Bump DuckDB to 1.5.3 (#8031) @AdamGS
• Remove type coercion (#8032) @gatesn
• chore[gpu]: error out in case of mixed cpu/gpu execution (#8016) @0ax1
• </details>

💥 ⚠️ Breaks

• Switch python runtime to CurrentThreadRuntime (#7896) @robert3005

✨ ✨ Features

• pluggable registry for input/export arrow kernels (#7824) @a10y
• feat[fastlanes]: allow delta to support signed bases (#7923) @joseph-isaacs
• Updated Variant array and the new VariantGet expression (#7877) @AdamGS
• Add Arrow to Vortex conversion to C FFI (#7906) @robert3005
• Add statistic expression (#7854) @gatesn
• Add `DType::Union` variant carrying just `Nullability` (#7901) @connortsui20
• tracing logger for duckdb (#7892) @myrrc
• simplify projection expression for SELECT * in duckdb (#7885) @myrrc
• + 8 more

⚡ 🚀 Performance

• perf[array]: add the SimplifyCache to `optimize` (#7948) @joseph-isaacs
• Mimic duckdb's post-filter cardinality estimates (#7895) @myrrc
• Use many connections on the benchmarks server (#7852) @connortsui20
• Faster Mask::from_slices (#7857) @gatesn
• Improve intersect_by_rank performance (#7744) @robert3005
• perf[array]: small vec for slots (#7825) @joseph-isaacs
• perf[array]: outline the array parts data from the DynArray (#7750) @joseph-isaacs
• Remove more scalar_at from sparse arrays (#7439) @gatesn

🐛 🐛 Bug Fixes

• <details>
• <summary>21 changes</summary>
• BufferMut::zeroed_aligned stores actually allocated length instead of passed length (#7979) @robert3005
• move getrandom backend override to config.toml (#7983) @onursatici
• fix[gpu]: retain device buffers for dyn dispatch kernel (#7980) @0ax1
• fix[gpu]: handle sliced BP arrays in CUDA (#7912) @0ax1
• duckdb: flatten runend arrays on export if requested (#7951) @myrrc
• Fix segfault when calling vx CLI from Python bindings (#7947) @joseph-isaacs
• + 16 more

📦 🧰 Maintenance

• <details>
• <summary>74 changes</summary>
• Update Rust crate lance-encoding to v6 (#7974) @[renovate[bot]](https://github.com/apps/renovate)
• Lock file maintenance (#7976) @[renovate[bot]](https://github.com/apps/renovate)
• Update anthropics/claude-code-action digest to 51ea8ea (#7959) @[renovate[bot]](https://github.com/apps/renovate)
• Update cloudflare/wrangler-action action to v4 (#7972) @[renovate[bot]](https://github.com/apps/renovate)
• Lock file maintenance (#7975) @[renovate[bot]](https://github.com/apps/renovate)
• Update EmbarkStudios/cargo-deny-action digest to 6c8f9fa (#7960) @[renovate[bot]](https://github.com/apps/renovate)
• + 69 more

💥 ⚠️ Breaks

• Remove `PartialOrd` implementation for `ScalarValue` (#7742) @connortsui20
• New Java Scan API (#7527) @robert3005
• Make `StatsCache` and `ArrayAndStats` have interior mutability (#7583) @connortsui20
• Pass `ExecutionCtx` through the compressor (#7578) @connortsui20
• break: remove deprecated methods and plumb ExecutionCtx (#7512) @joseph-isaacs
• break: remove `try_from` struct array to record batch (#7488) @joseph-isaacs

🗑️ 🚧 Deprecation

• deprecate: into_arrow (#7577) @joseph-isaacs
• deprecate: execute_mask over to_mask (#7574) @joseph-isaacs
• depreacte non compute methods without a ctx (e.g. `to_canonical`) (#7473) @joseph-isaacs

✨ ✨ Features

• feat: add builder execution path with AppendChild step (#7677) @joseph-isaacs
• feat: iterative execution for DateTimePartsArray (#7698) @joseph-isaacs
• feat(ffi): add literal expression support (#7675) @m7kss1
• feat: teach Array<Struct> try_concat(&[Array<Struct>]) (#7632) @danking
• Late materialization support for duckdb (#7631) @myrrc
• feat: masked array iterative execution clean up (#7690) @joseph-isaacs
• feat(cuda): support f64 in alp dyn dispatch (#7666) @0ax1
• feat(cuda): prefer standalone kernels in `auto` mode (#7667) @0ax1
• + 16 more

⚡ 🚀 Performance

• Late materialization support for duckdb (#7721) @myrrc
• perf: zero copy validity export to duckdb (#7371) @joseph-isaacs
• Improve stats reporting to DF (#7628) @AdamGS
• Try and improve the perf of natural file splits in DF (#7609) @AdamGS
• Cache available parallelism (#7620) @AdamGS
• Faster BoolArray::min_max via true_count instead of set_slices (#7599) @robert3005
• perf: faster execution ctx and no opt (#7597) @joseph-isaacs
• Use layout file splits when DF re-partitions individual files (#7591) @AdamGS
• + 8 more

🐛 🐛 Bug Fixes

• <details>
• <summary>16 changes</summary>
• fix: reduce CUDA benchmark noise on codspeed (#7749) @0ax1
• Actually wire the pluggable expression convertor (#7730) @AdamGS
• Fix cosine similarity optimization bug (#7724) @connortsui20
• Prunning expressions can reference rowcount (#7589) @robert3005
• Fix dtype mismatch in FileStatsLayoutReader for stat scalars (#7639) @brainhart
• Validate zoned build (#7627) @connortsui20
• + 11 more

📝 📖 Documentation

• Add epic and tracking issue templates (#7680) @connortsui20
• C API scan examples (#7564) @myrrc
• Improve docs for DataFusion integration (#7442) @AdamGS

📦 🧰 Maintenance

• <details>
• <summary>85 changes</summary>
• Remove unused parameters from vtab functions (#7738) @myrrc
• Rename fixed shape tensor module and id (#7733) @connortsui20
• Validity casting doesn't hide usage of execution context (#7710) @robert3005
• ci[gpu]: run cuda micro-benchmarks with codspeed (#7696) @0ax1
• Fix java formatting (#7703) @robert3005
• Java CI is self contained and writes files through java instead of rust (#7702) @robert3005
• + 80 more

📋 What's Changed

• ALP decoding plugin by @a10y in https://github.com/vortex-data/vortex/pull/7370
• fix(cuda): handle validity in GPU kernels by @0ax1 in https://github.com/vortex-data/vortex/pull/7372
• Fix `AGENTS.md` sccache by @connortsui20 in https://github.com/vortex-data/vortex/pull/7377
• Replace `&PrimitiveArray` with `ArrayView` in a bunch of places by @connortsui20 in https://github.com/vortex-data/vortex/pull/7378
• Remove MaskMut and unused functions from BitBufferMut by @robert3005 in https://github.com/vortex-data/vortex/pull/7379
• AllInvalid arrays have non empty distinct info by @robert3005 in https://github.com/vortex-data/vortex/pull/7369
• duckdb: Reset validity instead of filling by @myrrc in https://github.com/vortex-data/vortex/pull/7365
• use expect over allow in clippy by @joseph-isaacs in https://github.com/vortex-data/vortex/pull/7373
• + 61 more

✨ New Contributors

• @xiaoxuandev made their first contribution in https://github.com/vortex-data/vortex/pull/6969
• Full Changelog: https://github.com/vortex-data/vortex/compare/0.68.0...0.69.0

💥 ⚠️ Breaks

• Move Array::serialize -> ArrayPlugin::Serialize (#7348) @gatesn
• Unify id creation for ScalarFns and rename is_null to vortex.is_null (#7325) @robert3005
• Rework `Scheme` estimation in compressor (#7230) @connortsui20
• block serialization for the existing aggregations (#7322) @blaginin
• break[array]: remove old `vtable!` macro (#7317) @joseph-isaacs
• Hoist Slots (#7287) @gatesn
• Fix Array::validity (#7285) @gatesn
• Array VTables Follow Up (#7276) @gatesn
• + 9 more

✨ ✨ Features

• Add IsNull expression to vortex-jni (#7358) @robert3005
• feat: add an arrays slots macro (#7360) @joseph-isaacs
• Allow loading "foreign" plugins for UI/TUI/Serde (#7347) @gatesn
• Pull out `L2Denorm` from TurboQuant (#7349) @connortsui20
• Buffer Allocator Abstraction (#7337) @gatesn
• feat: iterative execution for most arrays in vortex-array (#7335) @joseph-isaacs
• Dynamic WHT rounds in TurboQuant (#7330) @connortsui20
• feat[gpu]: support mixed-width types in dynamic dispatch (#7331) @0ax1
• + 22 more

⚡ 🚀 Performance

• Add inner product and cosine similarity optimizations (#7364) @connortsui20
• feat: iterative execution patched array (#7345) @joseph-isaacs
• Fix StructArray / StructData after migration (#7305) @gatesn
• alp_rd: speedup (#7064) @joseph-isaacs
• perf: speed up TryFrom<Scalar> for Primitive (#7223) @joseph-isaacs
• feat: Introduce writeBatchFfi in Java JNI to avoid IPC serialization overhead (#7205) @JingsongLi

🐛 🐛 Bug Fixes

• <details>
• <summary>17 changes</summary>
• fix: maturin never logs (#7367) @danking
• Fix patch_chunk index OOB when slicing ALP arrays mid-chunk (#7354) @abnobdoss
• AVX2 take handles indices that are equal to the index type max value (#7359) @robert3005
• Fix TSan data race with FFI tests (#7244) @myrrc
• Don't re-run benchmarks nightly (#7338) @AdamGS
• fix[encodings]: take in progress buffer into account on fsst builder … (#7318) @asubiotto
• + 12 more

📝 📖 Documentation

• docs: add back array children docs moved from slots (#7249) @joseph-isaacs

📦 🧰 Maintenance

• <details>
• <summary>30 changes</summary>
• add env flag for Patched array (#7314) @a10y
• Add a dedicated CI build profile (#7357) @AdamGS
• Remove generic rle (#7350) @connortsui20
• fix lint on develop (#7346) @a10y
• Enable clippy::clone_on_ref_ptr lint and fix all violations (#7262) @joseph-isaacs
• Symlink CLAUDE.md to AGENTS.md to support other coding agents (#7333) @AdamGS
• + 25 more

✨ ✨ Features

• feat: Java Bindings for FixedSizeList (#7188) @JingsongLi
• feat: make dispatch plan dynamically sized (#7186) @0ax1
• Add tsan for vortex-ffi, fix asan instrumentation (#7157) @myrrc
• Upgrade to DataFusion 53 (#6740) @AdamGS

⚡ 🚀 Performance

• Cache python class imports (#7191) @AdamGS
• Specialize export of chunked list arrays (#7180) @AdamGS
• Filter indices without an iterator for Bool (#7176) @AdamGS

🐛 🐛 Bug Fixes

• Make sure all tokio runtime propegate observability in the same way (#7193) @AdamGS
• fix: vortex-cxx test race (#7189) @0ax1
• fix: support total_size=0 in byte_range_to_row_range (#7115) @aalexandrov

📝 📖 Documentation

• feat: teach docs about object store (#7169) @danking

📦 🧰 Maintenance

• <details>
• <summary>12 changes</summary>
• Python benchmarks script and re-organize dev dependencies (#7192) @AdamGS
• move BitPacked slice from kernel -> parent reduce (#7195) @a10y
• Unify benchmark dispatch into a workflow (#7178) @AdamGS
• Run TPC-H SF 100 nightly on Vortex (#7184) @AdamGS
• Array VTables Migration 3 (#7185) @gatesn
• scalar_at take execution_ctx (#7166) @a10y
• + 7 more

💥 ⚠️ Breaks

• Make vortex-scan just the Scan API (#7163) @gatesn

🐛 🐛 Bug Fixes

• Shade jackson in vortex-jni and vortex-spark and avoid scala version conflicts (#7168) @robert3005
• spark: pass format options into reader (#7160) @a10y

📝 📖 Documentation

• Expand execution docs (#7158) @AdamGS

📦 🧰 Maintenance

• <details>
• <summary>4 changes</summary>
• fix path for javadoc CI (#7173) @a10y
• Fix docs deploy after java changes (#7170) @robert3005
• fix: dyn dispatch requires same bit width types (#7164) @0ax1
• Add storybook for component development (#7162) @gatesn
• </details>

💥 ⚠️ Breaks

• Remove BitBuffer::into_mut and require callers to handle failure to acquire unique ownership (#7136) @robert3005
• Update DeltaArray to always pad values to 1024 elements and use bit transpose functions (#7007) @robert3005

✨ ✨ Features

• Add web UI (#7151) @gatesn
• feat[vortex-datafusion]: use PartitionedFile instead of path in create_reader (#7156) @asubiotto
• Part 2 of making Variant a canonical array (#7143) @AdamGS
• feat[array]: use transmute for up/down cast of Arc<Array> (#7140) @joseph-isaacs
• feat[array]: change VTable execution to pass owned arrays (#6867) @joseph-isaacs
• feat(cuda): hybrid GPU dispatch - fuse dyn + standalone kernels (#7127) @0ax1
• default mimalloc for vortex-python wheels (#7096) @onursatici
• feat(cuda): add `SEQUENCE` source op to dyn dispatch (#7078) @0ax1
• + 10 more

⚡ 🚀 Performance

• do not validate already valid arrow varbin arrays (#7089) @onursatici
• perf[fsst]: like pushdown using a dfa (#6935) @joseph-isaacs
• Decimal sum uses smallest physical type that supports output dtype (#7008) @robert3005
• perf: skip allocation for prim cast if possible (#6997) @0ax1
• Faster true count using AVX2 and AVX512 instructions (#6931) @robert3005

🐛 🐛 Bug Fixes

• <details>
• <summary>23 changes</summary>
• fix(python): cap substrait-python before 0.29.0 (#7161) @RealXuChe
• fix(python): cap substrait below 0.85.0 (#7153) @RealXuChe
• fill_forward_nulls resets at 1024 chunk boundary (#7149) @robert3005
• Support reading SparseArray with non zero offset (#7150) @robert3005
• fix: remove nested `block_on` call in `cuda_execute` (#7147) @0ax1
• Revert "fix: disambiguate array ops files" (#7138) @joseph-isaacs
• + 18 more

📝 📖 Documentation

• <details>
• <summary>4 changes</summary>
• Update vtables docs regarding forwarding from erased to typed (#7134) @gatesn
• docs: clarify GPU dispatch flow (#7132) @0ax1
• More detailed docs on `ScalarFnVTable` (#7027) @connortsui20
• Add high-level docs to `vortex-btrblocks` (#6984) @connortsui20
• </details>

📦 🧰 Maintenance

• <details>
• <summary>55 changes</summary>
• chore[array]: iterative execute method p1 (#6860) @joseph-isaacs
• fix fuzz_report script (#7141) @a10y
• Cleanup fastlanes encodings crate (#7118) @robert3005
• Testing FFI from C side (#7126) @myrrc
• chore[release]: add compat-gen-upload to release (#7067) @joseph-isaacs
• chore(deps): lock file maintenance (#7121) @[renovate[bot]](https://github.com/apps/renovate)
• + 50 more

💥 ⚠️ Breaks

• Remove all_valid/all_invalid from Validity (#6926) @gatesn
• Rename Array VTables (#6907) @gatesn

✨ ✨ Features

• Type Coercion (#6960) @gatesn
• feat: aligned chunked pair iter (#6777) @joseph-isaacs
• Add bit transpose operations (#6928) @robert3005
• Add support for nulls and unsigned integers through Arrow Datasets (#6908) @AdamGS

⚡ 🚀 Performance

• CaseWhen uses forward pass with a remaining mask (#6804) @palaska

🐛 🐛 Bug Fixes

• Suppress -Wtype-limits warning in DuckDB C++ compilation (#6957) @joseph-isaacs
• Fix struct cast panic in rule and add tests (#6962) @connortsui20
• Fix `bitwise_unary_op` (#6940) @connortsui20
• fixup fuzzer message (#6936) @joseph-isaacs
• Restore some missing data from benchmarks (#6914) @AdamGS
• Improve URL parsing in VortexMultiFileScan to handle file paths more … (#6842) @iceboundrock
• Fix empty zctl flag (#6891) @connortsui20

📝 📖 Documentation

• docs: mirco-benchmarks best practices (#6922) @0ax1
• More detailed fuzzer report template (#6888) @connortsui20

📦 🧰 Maintenance

• <details>
• <summary>25 changes</summary>
• chore(deps): update docker/setup-qemu-action action to v4 (#6948) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update docker/setup-buildx-action action to v4 (#6947) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update docker/build-push-action action to v7 (#6945) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update docker/login-action action to v4 (#6946) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update github artifact actions to v7 (major) (#6942) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update plugin net.ltgt.errorprone to v5 (#6950) @[renovate[bot]](https://github.com/apps/renovate)
• + 20 more

📋 What's Changed

• Aggregates: accumulate columnar by @gatesn in https://github.com/vortex-data/vortex/pull/6839
• Add a UUID extension type by @connortsui20 in https://github.com/vortex-data/vortex/pull/6832
• move stuff around in `vortex-tensor` by @connortsui20 in https://github.com/vortex-data/vortex/pull/6857
• chore[ci]: enable cuda build+lint by @joseph-isaacs in https://github.com/vortex-data/vortex/pull/6861
• Configure benchmark boxes to try and reduce noise by @AdamGS in https://github.com/vortex-data/vortex/pull/6745
• feat[gpu]: slice support for dyn dispatch by @0ax1 in https://github.com/vortex-data/vortex/pull/6862
• Extension Array Cleanup by @connortsui20 in https://github.com/vortex-data/vortex/pull/6869
• Reducing lock contention around expression partition caches by @AdamGS in https://github.com/vortex-data/vortex/pull/6871
• + 14 more

💥 ⚠️ Breaks

• feat: Iterative Execution (#6824) @joseph-isaacs
• Public array modules (#6834) @gatesn
• Layout readers don't need a mutable context (#6808) @robert3005
• Rename Array -> DynArray (#6760) @gatesn
• Add a `VortexSession` for scalar value deserialization (#6772) @connortsui20

✨ ✨ Features

• Duckdb 1.5-variegata (#6627) @myrrc
• Aggregates: ChunkedArray kernel (#6837) @gatesn
• Vortex Fixed-Shape Tensor (#6812) @connortsui20
• feat[vortex-runend]: handle canonicalizing Ree<Utf8> (#6788) @asubiotto
• Aggregate Fns (#6721) @gatesn
• feat[vortex-array]: support converting arrow REE dtype to vortex (#6784) @asubiotto
• feat[expr]: N-ary CASE WHEN expression (#6786) @lukekim
• gpu scan cli convert subcommand (#6704) @onursatici
• + 1 more

⚡ 🚀 Performance

• gpu: transpose patches faster (#6829) @a10y
• perf: bypass async I/O pipeline for in-memory buffer reads (#6813) @0ax1
• Canonical::empty creates arrays directly instead of going through builders (#6805) @robert3005
• perf: faster sequence array decompress (#6809) @joseph-isaacs
• perf: ensure that `into_array` is used for owned specific arrays (#6795) @joseph-isaacs
• fix[ci]: run slow test in parallel (#6776) @joseph-isaacs
• data parallel patching for FastLanes (#6708) @a10y
• Fast path append_to_builder ConstantArray (#6767) @palaska

🐛 🐛 Bug Fixes

• duckdb: read from local path, correct err on invalid path (#6851) @myrrc
• use full column set on empty projection list in duckdb (#6831) @myrrc
• fix: don't commit to case when binary format (#6806) @joseph-isaacs
• fix[layout]: add required ALLOWED_ENCODINGS to `WriteStrategyBuilder` (#6792) @joseph-isaacs
• fix[layout]: canonical validity in compressor (#6797) @joseph-isaacs
• fix[ci]: duckdb download retry logic (#6791) @joseph-isaacs
• fix: cuda scan requires tokio (#6779) @joseph-isaacs
• fix: listview size & offset ptype mismatch (#6775) @0ax1

📝 📖 Documentation

• add brief docs to the layouts (#6814) @connortsui20

📦 🧰 Maintenance

• <details>
• <summary>24 changes</summary>
• Fix projection pushdown SLT test (#6850) @AdamGS
• Add TPC-H to sqllogictests (#6830) @AdamGS
• chore(deps): update datafusion monorepo to v52.2.0 (#6847) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update taiki-e/cache-cargo-install-action digest to 414f3dc (#6845) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update codspeedhq/action digest to 281164b (#6844) @[renovate[bot]](https://github.com/apps/renovate)
• Use execute where possible (#6835) @gatesn
• + 19 more

💥 ⚠️ Breaks

• break: zip arg (mask, true, false) (#6766) @joseph-isaacs
• break: remove all compute public functions (#6752) @joseph-isaacs
• clean up numeric compute (#6593) @joseph-isaacs
• Remove deprecated to_arrow compute (#6730) @gatesn
• Remove &dyn Array call sites (#6725) @gatesn
• Make ExecutionArgs a dyn trait (#6715) @gatesn
• Inline visitor vtable to ArrayVTable (#6700) @gatesn
• Inline BaseArrayVTable (#6697) @gatesn

✨ ✨ Features

• rwlock instead of mutex around encoding ids (#6720) @danking
• Make the constant metadata a `Scalar` (#6751) @connortsui20
• feat: remove to canonical p2 (#6742) @joseph-isaacs
• Improve benchmark output formatting (#6744) @AdamGS
• vortex read at for pooled cuda reads (#6675) @onursatici

⚡ 🚀 Performance

• unsafe Scalar::struct_unchecked in scalar_at vtable (#6741) @palaska
• Implement BetweenReduceRule for ConstantArray (#6695) @robert3005
• Remove unnecessary to_canonical in DictArray::take (#6686) @palaska
• Eliminate redundant decompressions in VarBinArray canonicalization (#6692) @palaska
• Remove unnecessary to_canonical in ChunkedArray filter_indices (#6688) @palaska
• separate in_memory vs file coalescing config (#6690) @onursatici
• Remove to_canonical from MaskedArray::take (#6689) @palaska

🐛 🐛 Bug Fixes

• Make sequence metadata in-memory + clean up (#6759) @connortsui20
• fix: CUDA bitpacking bench build (#6750) @0ax1
• fix[util]: add balanced reduce iterator (#6737) @joseph-isaacs
• Fix S3 globbing for duckdbfs with httplib (#6709) @myrrc
• Fix old extension scalar logic (#6701) @connortsui20

📝 📖 Documentation

• VTable Unification Docs (#6735) @gatesn

📦 🧰 Maintenance

• <details>
• <summary>22 changes</summary>
• chore: replace Buffer::copy_from with Buffer::from_iter for improved performance (#6749) @SYaoJun
• Remove unused BitView (#6763) @gatesn
• Remove sparse metadata prost wrapper (#6761) @connortsui20
• Parallelize slow CUDA tests (#6746) @joseph-isaacs
• Add explain feature to DF benchmarks (#6683) @AdamGS
• chore: build, test and lint all CUDA crates on CI (#6743) @0ax1
• + 17 more

💥 ⚠️ Breaks

• ScalarFn VTable (#6679) @gatesn
• Extension Scalar VTable (#6680) @connortsui20
• Rename spawn_blocking -> spawn_blocking_io (#6678) @gatesn
• Unify ExtVTable per vtable design doc (#6662) @gatesn
• Add an `extension` module to `vortex` (#6624) @connortsui20
• break: clean up compare (#6591) @joseph-isaacs
• Merge `vortex-dtype` into `vortex-array` (#6582) @connortsui20
• break: clean up compute (#6572) @joseph-isaacs
• + 2 more

✨ ✨ Features

• Support building duckdb from a hash, including CI (#6658) @myrrc
• DuckDB to use the Scan API (#6652) @gatesn
• Use Scan API (#6391) @gatesn
• feat[gpu]: multi-stage dynamic dispatch with plan builder (#6636) @0ax1
• DuckDB lifetimes (#6620) @joseph-isaacs
• Expose table scan progress to DuckDB (#6596) @myrrc
• expose Executor, AbortHandle, AbortHandleRef, and Handle (#6595) @danking

⚡ 🚀 Performance

• Avoid redundant chunk execution in pack_struct_chunks (#6682) @palaska
• read future to skip polled and dropped events for completed requests (#6667) @onursatici
• perf[gpu]: speed up dyn dispatch run-end kernel (#6665) @0ax1
• Route default append_to_builder through full execution loop (#6655) @palaska
• Add take-based rebuild strategy for ListView with small lists (#6492) @palaska
• oncelock shared array for zero cost access after materialisation (#6653) @onursatici
• oncelock shared array future on cuda flat layout (#6585) @onursatici
• cow buffers on array decode (#6590) @onursatici
• + 9 more

🐛 🐛 Bug Fixes

• <details>
• <summary>14 changes</summary>
• cuda ctx bind to thread before unsafe (#6681) @onursatici
• keep DropOnSync alive until after dispatching for cuda slices (#6673) @onursatici
• Convert &mut to & for DuckDB GlobalData (#6634) @gatesn
• resize FSST compressor buffer to be large enough for largest string (#6676) @a10y
• Revert "Support building duckdb from a hash, including CI" (#6670) @myrrc
• Close DuckDB database before re-opening (#6654) @gatesn
• + 9 more

📝 📖 Documentation

• <details>
• <summary>4 changes</summary>
• Language bindings docs (#6666) @gatesn
• Update docs for vtable unification plan (#6661) @gatesn
• docs: Remove unused CLI argument (code always uses temporary path) (#6637) @SYaoJun
• docs: clarify cuda async execution model (#6611) @0ax1
• </details>

📦 🧰 Maintenance

• <details>
• <summary>24 changes</summary>
• DuckDB: test curl impl, don't auto-load extensions (#6677) @myrrc
• chore: exclude cuda gen files from auto-formatting (#6671) @0ax1
• use match_each_canonical macro to reduce some canonical impl boilerplate (#6668) @AdamGS
• chore: trim unused crate deps with `cargo-machete` (#6663) @0ax1
• cudarc use 12.5 instead of 12.8 (#6660) @a10y
• docs: resurrect lost ddb build.rs comment (#6659) @0ax1
• + 19 more

🐛 🐛 Bug Fixes

• Published packages cannot depend on unpublished packages (#6592) @robert3005

🐛 🐛 Bug Fixes

• minor c++ duckdb integration changes (#6566) @myrrc
• Vortex crate doesn't unconditionally depend on vortex-cuda-macros (#6580) @robert3005

📦 🧰 Maintenance

• Add ci check for java publish (#6578) @robert3005

🐛 🐛 Bug Fixes

• fix: CUDA scalar inplace (#6567) @0ax1

📦 🧰 Maintenance

• chore: drop flaky duckdb http e2e test (#6570) @0ax1
• chore: make auto `clang-format` consistent (#6568) @0ax1

📋 Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better `vx-bench` docs in README (#5900) @AdamGS

💥 ⚠️ Breaks

• Remove `vortex-compute` (#6561) @connortsui20
• Introduce `VortexError::Other` and simplify `Display` implementation (#6410) @AdamGS
• Move `vortex-scalar` into `vortex-array` (#6523) @connortsui20
• break[array]: lazy between (#6528) @joseph-isaacs
• break[array]: mask compute to lazy mask execution (#6485) @joseph-isaacs
• break[array]: lazy compare function (#6495) @joseph-isaacs
• Remove `vortex-vector` (#6529) @connortsui20
• break[array]: move like compute to lazy compute (#6497) @joseph-isaacs
• + 23 more

✨ ✨ Features

• Add FileSystem trait and make DuckDB configurable (#6564) @gatesn
• Add scan concurrency config for datafusion source (#5945) @AdamGS
• feat: implement DuckDB filesystem integration for Vortex file handling (#6198) @iceboundrock
• vx convert to support cuda compatible vortex files (#6522) @onursatici
• cuda flat layout (#6521) @onursatici
• feat[xtask]: faster public-api (#6498) @joseph-isaacs
• Improve substrait support to include more types and expressions (#6484) @AdamGS
• feat[array]: refactor `zip` compute kernel to lazy `zip` via ScalarFn Array (#6461) @joseph-isaacs
• + 76 more

⚡ 🚀 Performance

• Faster primitive `take` (#6559) @connortsui20
• perf[vortex-btrblocks]: remove listview expansion ratio (#6530) @asubiotto
• VarBinBuilder: eagerly set IsSorted stat on offsets (#6452) @dimitarvdimitrov
• Implement ReduceNode for Array to not require creating new arc when returning children (#6443) @robert3005
• chore: clean cancelled read requests earlier in driver state transitions (#6382) @caicancai
• shared array to avoid duplicate execute (#6453) @onursatici
• vortex-btrblocks: write listviews when elements expand > 50% (#6393) @asubiotto
• Implement ReduceNode for ScalarFnArray to avoid boxing (#6435) @robert3005
• + 29 more

🐛 🐛 Bug Fixes

• <details>
• <summary>122 changes</summary>
• fix: install and load httpfs in setup phase (#6565) @0ax1
• fix[vortex-array]: support executing into REE (#6562) @asubiotto
• fix is_sorted check failing for large multiplier values (#6560) @a10y
• fix: duckdb `write_all` (#6558) @0ax1
• fix: duckdb object store read coalescing (#6555) @0ax1
• fix[ci]: cuda no nightly (#6505) @joseph-isaacs
• + 117 more

📝 📖 Documentation

• <details>
• <summary>4 changes</summary>
• Add a small notice to polars integration node (#6486) @AdamGS
• Include draft releases in docs site (#6346) @gatesn
• Include draft changelog in docs (#6331) @gatesn
• Vortex Docs Reboot (#6318) @gatesn
• </details>

📦 🧰 Maintenance

• <details>
• <summary>188 changes</summary>
• chore[ci]: speed up the cuda tests with matching compile flags (#6552) @joseph-isaacs
• chore: unify C/C++ formatting (#6563) @0ax1
• Update template nit (#6556) @connortsui20
• less strict PR template (#6551) @connortsui20
• pvalue::cast returns VortexResult (#6532) @myrrc
• chore: fix typos in PR template (#6550) @0ax1
• + 183 more

📋 Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better `vx-bench` docs in README (#5900) @AdamGS

💥 ⚠️ Breaks

• chore[array]: inline take compute function (#6376) @joseph-isaacs
• Add Array::deserialize arguments (#6340) @gatesn
• Update FSST to store components of compressed codes (#6325) @gatesn
• Array Execution (#6307) @gatesn
• Execute parent to return ArrayRef (#6257) @gatesn
• Extension DType vtables (#6081) @gatesn
• break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
• Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
• + 10 more

✨ ✨ Features

• Make `VortexSource::new` public (#6392) @AdamGS
• Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
• feat[expr]: new fill null expr (#6385) @joseph-isaacs
• zstd buffers array (#6362) @onursatici
• gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
• cuda dictionary of vbv decompression (#6361) @0ax1
• CLI flag to enable tracing for benchmarks (#6350) @gatesn
• chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
• + 56 more

⚡ 🚀 Performance

• chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
• constant array metadata for small scalars (#6363) @onursatici
• chore[array]: move over filtering to `{reduce/execute}_parent` (#6312) @joseph-isaacs
• perf[buf]: inline `collect_bool` (#6313) @joseph-isaacs
• perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
• chore: list contains over scalars (#6242) @blaginin
• perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
• performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
• + 17 more

🐛 🐛 Bug Fixes

• <details>
• <summary>89 changes</summary>
• Measure output bytes in cuda bitunpack benchmarks (#6395) @robert3005
• fix[array]: decimal fill_null larger than buffer (#6388) @joseph-isaacs
• fix[array]: dict value nullable values (#6387) @joseph-isaacs
• fix[fuzz]: coverage report (#6375) @joseph-isaacs
• join ops before in format (#6370) @blaginin
• Include config.js in benchmark website docker image (#6379) @palaska
• + 84 more

📝 📖 Documentation

• Include draft releases in docs site (#6346) @gatesn
• Include draft changelog in docs (#6331) @gatesn
• Vortex Docs Reboot (#6318) @gatesn

📦 🧰 Maintenance

• <details>
• <summary>146 changes</summary>
• feat[bench]: add more random access (#6371) @joseph-isaacs
• Add Bitunpack cuda kernel benchmark (#6390) @robert3005
• chore[fuzz]: auto close fuzzer issues (#6378) @joseph-isaacs
• chore[bench-website]: add polarsignals (#6359) @joseph-isaacs
• chore[array]: inline take compute function (#6368) @joseph-isaacs
• bump lance in benchmarks to 2.0.0 (#6366) @robert3005
• + 141 more

✨ ✨ Features

• Feature: Make `vortex-tui` a library (#5779) @connortsui20

⚡ 🚀 Performance

• End DataFusion stream early if file can be pruned away by dynamic expr (#5874) @AdamGS

📦 🧰 Maintenance

• chore: add overflow tests for unsigned types in BigCast (#5866) @caicancai

🐛 🐛 Bug Fixes

• fix: Move from macos-13 runners to macos-15-intel runners (#5875) @robert3005

🐛 🐛 Bug Fixes

• fix: Correctly pin maturin to 1.10.2 (#5873) @robert3005

💥 ⚠️ Breaks

• Fix BoolArray::new to take BitBuffer (#5690) @gatesn
• Lazy array expression evaluation (#5625) @gatesn
• Do not inline vortex_array into vortex:: namespace (#5584) @gatesn
• Move stats into expr (#5582) @gatesn
• Combine Array VTable with Encoding (#5462) @gatesn
• Consolidate into stat_expression function (#5366) @gatesn
• chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs

✨ ✨ Features

• feat: teach TemporalMetadata about its relationship to DTypes (#5864) @danking
• Pluggable expression conversion for DF (#5853) @AdamGS
• Introduce extendable access plan for DF scans (#5855) @AdamGS
• Upgrade DF to 51 (#5189) @AdamGS
• Benchmarks re-org (#5676) @AdamGS
• Push down cast (#5803) @gatesn
• ListView to List (#5802) @gatesn
• feat[array]: stats + validity in display array (#5799) @joseph-isaacs
• + 66 more

⚡ 🚀 Performance

• perf: list contains scalar scalar (#5792) @0ax1
• Upgrade DF to 51 (#5189) @AdamGS
• performance[vortex-array]: don't call is_valid to count bytes in varbinview (#5814) @asubiotto
• perf[array]: extension pushdown filter + constant (#5794) @joseph-isaacs
• perf[expr]: between not fallible (#5798) @joseph-isaacs
• perf[dtp]: push down compare over dtp constant second subsecond (#5793) @joseph-isaacs
• perf[date-time-parts]: compressor handle constant and pushdown rules (#5778) @joseph-isaacs
• perf[duckdb]: dont push down pack & dedup filters for ddb (#5773) @joseph-isaacs
• + 21 more

🐛 🐛 Bug Fixes

• <details>
• <summary>81 changes</summary>
• fix: enable all features for vortex-mask docsrs and enable serde-derive when serde is enabled (#5871) @robert3005
• fix[array]: correct the handling of to_arrow execution and struct casting (#5867) @joseph-isaacs
• fix[ci]: pin maturin due to fix upstream error (#5868) @joseph-isaacs
• fix: int compressor mul overflow bp (#5854) @joseph-isaacs
• Fix: IPC `AsyncMessageReader` does not handle partial reads (#5758) @connortsui20
• Fix: incorrect child vector size in duckdb list array import (#5740) @connortsui20
• + 76 more

📝 📖 Documentation

• docs: ai note for contributions (#5857) @0ax1
• Docs: Fix outdated docs (#5777) @connortsui20
• Feature: makes the decompressor and compressor methods public (#5752) @connortsui20
• docs: fix typo in readme (#5627) @0ax1
• Chore: allow searching subcrates in vortex rustdoc (#5612) @connortsui20
• Describe upcoming API breaks in docs (#5583) @gatesn
• TSC Agenda 2025-11-25 (#5523) @gatesn
• Docs: document `chunk_offsets` in `Patches` (#5429) @connortsui20

📦 🧰 Maintenance

• <details>
• <summary>127 changes</summary>
• CI cache prunning followup (#5859) @AdamGS
• Try and minimize cache size by using cargo sweep (#5858) @AdamGS
• Move BufferHandle into vortex-array (#5856) @gatesn
• chore: clean up ci caching (#5774) @joseph-isaacs
• Update all patch updates (#5831) @[renovate[bot]](https://github.com/apps/renovate)
• chore: Update contribution llm (#5852) @joseph-isaacs
• + 122 more

💥 ⚠️ Breaks

• Do not inline vortex_array into vortex:: namespace (#5584) @gatesn
• Move stats into expr (#5582) @gatesn
• Combine Array VTable with Encoding (#5462) @gatesn
• Consolidate into stat_expression function (#5366) @gatesn
• chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs

✨ ✨ Features

• feat[scalar_fn]: binary ops (#5617) @joseph-isaacs
• Execute ScalarFn arrays (#5611) @gatesn
• FilterArray (#5610) @gatesn
• Add GetItemFn (#5609) @gatesn
• Add MaskFn (#5606) @gatesn
• feat[scalar_fn]: `is_null` (#5601) @joseph-isaacs
• Scalar functions (#5561) @gatesn
• feat[buffer]: use BufferHandle in more places (#5567) @joseph-isaacs
• + 29 more

⚡ 🚀 Performance

• perf[ci]: speed up ci MORE (#5576) @joseph-isaacs
• perf(ci): skip rust-cache on runs-on runners with S3 sccache (#5570) @joseph-isaacs
• chunked array zip (#5530) @onursatici
• varbinview zip kernel (#5526) @onursatici
• chore[vortex-dict]: numeric pushdown & clean up the dict ctor (#5438) @joseph-isaacs
• Handwritten pipeline benchmark and analysis (#5408) @connortsui20
• Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
• perf[dict]: all_values_referenced to allow pushdown optimisation. (#5416) @joseph-isaacs
• + 5 more

🐛 🐛 Bug Fixes

• <details>
• <summary>35 changes</summary>
• fix: avoid ListBuilder (#5636) @a10y
• fix: disable flaky bench for codspeed (#5580) @0ax1
• fix: don't bench `slice_arrow_buffer` on codspeed (#5578) @0ax1
• fix: fixed seed for micro-benchmarks (#5574) @0ax1
• fix: don't clone in micro-benchmarks (#5573) @0ax1
• fix: Decimal sum doesn't panic but returns null on overflow (#5564) @robert3005
• + 30 more

📝 📖 Documentation

• docs: fix typo in readme (#5627) @0ax1
• Chore: allow searching subcrates in vortex rustdoc (#5612) @connortsui20
• Describe upcoming API breaks in docs (#5583) @gatesn
• TSC Agenda 2025-11-25 (#5523) @gatesn
• Docs: document `chunk_offsets` in `Patches` (#5429) @connortsui20

📦 🧰 Maintenance

• <details>
• <summary>87 changes</summary>
• Add advisory semver checks to CI (#5645) @gatesn
• fix(deps): update protobuf monorepo to v4.33.2 (#5644) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update all patch updates (#5643) @[renovate[bot]](https://github.com/apps/renovate)
• fix platform/arch for runs-on config (#5622) @a10y
• ci: add runs-on.yml that points to prebuilt AMI (#5620) @a10y
• ci: add cache-suffix to isolate sanitizer build cache (#5604) @joseph-isaacs
• + 82 more

💥 ⚠️ Breaks

• chore[array/expr]: move `vortex-expr` into `vortex_array::expr` (#5269) @joseph-isaacs

✨ ✨ Features

• Move BitView to vortex-buffer and make generic over N (#5353) @gatesn
• feat: bump duckdb to 1.4.2 (#5337) @0ax1
• arrow -> vortex for dict arrays (#5316) @onursatici
• feat: automated fuzzing issue creation with Claude (#5292) @joseph-isaacs
• Feature: add aligned bitpacked pipeline kernel (#5299) @connortsui20
• feat: impl `expand` for `Buffer` & `BufferMut` (#5270) @0ax1
• feat: use Sparse compression for floats, strings (#5237) @a10y

⚡ 🚀 Performance

• perf: BitBuffer::from(Vec<bool>) use collect_bool (#5325) @joseph-isaacs
• perf: optimize `expand` (#5285) @0ax1
• fix[vortex-dict]: avoid full materialization in min_max (#5268) @asubiotto

🐛 🐛 Bug Fixes

• <details>
• <summary>13 changes</summary>
• fix: add permissions to fuzzer fix automation workflow (#5345) @joseph-isaacs
• fix: `SequenceVTable` bounds check (#5341) @blaginin
• fix: unblock the fuzzers (#5335) @a10y
• Fix: handle validity in bitpack pipeline kernel (#5330) @connortsui20
• Fix zero-copy pipeline input (#5324) @gatesn
• fix: update Claude Code Action to v1 parameter format (#5314) @joseph-isaacs
• + 8 more

📦 🧰 Maintenance

• <details>
• <summary>34 changes</summary>
• chore: All of vortex-gpu requires compiler flag (#5362) @robert3005
• chore(deps): update rust crate bytes to v1.11.0 (#5358) @[renovate[bot]](https://github.com/apps/renovate)
• chore(deps): update all patch updates (#5357) @[renovate[bot]](https://github.com/apps/renovate)
• chore: bug is not a sufficient label (#5351) @danking
• chore: document Expr.__getitem__ (get a field of a struct array) (#5344) @danking
• fix: cfg disable unix-specific operations (#5326) @danking
• + 29 more

📋 Changes

• Fix: remove TPC-H SF 1000 from nightly benchmarks (#5235) @connortsui20

💥 ⚠️ Breaks

• Expression VTable (#5191) @gatesn
• Vortex Session (#5111) @gatesn

✨ ✨ Features

• Feature: add in-place filter for `BufferMut` + benchmarks (#5238) @connortsui20
• Feature: add `into_mut` to vectors (#5221) @connortsui20
• Feature: add `into_mut` methods on mask and buffer (#5214) @connortsui20
• feat: OperatorVTable::bind for VarBin/VarBinView (#5212) @a10y
• Vector scalars (#5213) @gatesn
• pickling support (#5011) @onursatici
• Feature: add `Decimal::bind` (#5210) @connortsui20
• Bind constant array (#5208) @gatesn
• + 38 more

⚡ 🚀 Performance

• Feature: add in-place filter for `BufferMut` + benchmarks (#5238) @connortsui20
• Benchmarks: make random access timed + median instead of set iterations (#5234) @connortsui20
• experiment: eagerly decode dict on read (#5229) @a10y
• perf: fuse alp decode with applying patches (#5065) @0ax1
• Performance: `ListViewArray` zero-copyable to `ListArray` optimizations (#5129) @connortsui20
• perf[btrblocks]: write list (not list view). (#5087) @joseph-isaacs
• performance[vortex-datafusion]: support pushing down filters on struct columns (#5024) @asubiotto

🐛 🐛 Bug Fixes

• <details>
• <summary>39 changes</summary>
• Fix: casting Zstd array to non nullable requires decompression (#5258) @robert3005
• Fix: silence codspeed warning (#5253) @connortsui20
• fix[fuzz]: apply mask to validity in baseline (#5249) @joseph-isaacs
• Fix: make `Filter` compute take `self` instead of `&self` (#5248) @connortsui20
• fix[array]: take propagate `is_constant` if indices are `all_valid` (#5241) @joseph-isaacs
• fix[pco]: cast condition check (#5239) @joseph-isaacs
• + 34 more

📦 🧰 Maintenance

• <details>
• <summary>88 changes</summary>
• [DataFusion] Don't pushdown empty filters (#5264) @AdamGS
• Bump DF to 50.3.0 (#5263) @AdamGS
• chore: Fix fuzzers baseline nullability for compare and take (#5255) @robert3005
• Bump codspeed (action and dependency) (#5261) @AdamGS
• chore: Fix fuzzers baseline nullability for decimal fill_null (#5257) @robert3005
• Update all patch updates (#5259) @[renovate[bot]](https://github.com/apps/renovate)
• + 83 more

💥 ⚠️ Breaks

• chore: remove merge's explicit nullability (#4967) @danking
• chore[btrblocks]: move validity in `deltas` & enabled delta encoding (#4905) @joseph-isaacs

✨ ✨ Features

• feat: teach constant to fill_null (#4985) @danking
• Feature: Make `ListView` the canonical encoding for `List` (#4946) @connortsui20
• feat: GithubArchive benchmark for nested data (#4960) @a10y
• feat: collect patch offsets per chunk during alp encode (#4965) @0ax1
• Support sum stat for decimal arrays (#4963) @AdamGS
• feat: `take_search` constant time index lookups (#4951) @0ax1
• Add BitBuffer to vortex-buffer (#4937) @gatesn
• feat: constant time patches slicing (#4913) @0ax1
• + 25 more

⚡ 🚀 Performance

• perf[fastlanes]: fused for bitpacking decode (#4955) @joseph-isaacs
• Use arrow's bit iterators for BitBuffer (#4952) @gatesn
• chore[btrblocks]: move validity in `deltas` & enabled delta encoding (#4905) @joseph-isaacs
• Perf: Create splits via indices instead of layout reader (#4891) @connortsui20
• perf[gpu]: fused AoT `for` and `bitpacking` kernel (#4872) @joseph-isaacs
• cheaper StructArray::clone (#4877) @onursatici
• field_by_name to use indices lookup (#4868) @onursatici
• faster StructFields::find (#4863) @onursatici
• + 9 more

🐛 🐛 Bug Fixes

• <details>
• <summary>27 changes</summary>
• fix[vortex-io]: register read_at intent in InstrumentedReadAt (#4991) @asubiotto
• Support prunning on the newly supported NullCount (#4990) @AdamGS
• fix[vortex-expr]: implement stat_falsification for IsNullExpr (#4980) @asubiotto
• fix: Continue to prefer List over ListView by default in ToArrow (#4982) @a10y
• fix[vortex-expr]: implement stat_falsification for BetweenExpr (#4977) @asubiotto
• Coerce and cast decimal sum (#4968) @AdamGS
• + 22 more

📦 🧰 Maintenance

• <details>
• <summary>112 changes</summary>
• FIx java bindings packaging (#5008) @AdamGS
• chore: avoid copy when building chunk_offsets in chunked layout reader (#5006) @robert3005
• Split the publish flow into two flows (#5004) @AdamGS
• do not export spans by default on benchmarks (#5003) @onursatici
• Move fastlanes back to released version (#4999) @AdamGS
• Small correction to pre-commit script (#4998) @AdamGS
• + 107 more