Vectorized query execution

In a tuple-at-a-time interface, a small function may be invoked for each output row and repeatedly traverse generic expression machinery. A batch interface passes arrays or column vectors to an operator, allowing one invocation to process many values. The X100 research demonstrates this design direction, but its historical benchmark results are not a speed guarantee for an unrelated database. Use a local model to distinguish per-invocation overhead from per-row work. Batching can reduce the first term while leaving expensive decoding, hashing, or user functions largely unchanged.

Vectorized software is not synonymous with hardware SIMD. A vector loop may use scalar instructions, compiler auto-vectorization, explicit SIMD, or a mixture. Data types, nullability, branches, variable-length strings, and memory dependencies influence what the compiler can do. Likewise, a tuple-oriented engine can compile parts of a query into efficient loops. Compare executed work and measured bottlenecks rather than declaring that one interface name determines all instruction behavior. Preserve overflow, decimal, collation, and NULL semantics when specializing expressions.

For fixed-width columns, multiply rows per batch by bytes per column and the number of simultaneously live columns to estimate value payload. Add validity information, selection indices, offsets, variable-length payloads, and allocator overhead where the representation requires them. A filter may keep a selection vector instead of copying survivors immediately. Later operators must interpret those positions consistently, including NULL masks. A batch with only a few surviving rows can still retain a large input buffer until its last consumer finishes.

Very large batches reduce dispatch frequency but can expand working sets, increase latency before yielding, and retain more temporary data. Very small batches can bring dispatch overhead back. The best size is therefore empirical and can differ across operators or workloads. Keep benchmark output and semantics fixed while varying a batch policy. Measure peak live memory, time to first result, and total throughput together, especially for interactive requests sharing workers with analytical scans. The cost model is useful only while its supposedly constant terms remain plausible.

After dispatch overhead falls, moving data may dominate. Projection, compression, selective filtering, and reuse can then matter more than removing another function call. For the numeric lab here, operational intensity counts floating-point operations per byte transferred between the last-level cache and main memory. A roofline is an upper bound under those definitions, not a prediction for branch-heavy strings or a conversion between database rows and FLOPs. Measure the relevant traffic and operations before using the model to interpret a real query kernel.

Reset the roofline lab, set both intensity I=2 FLOP/byte and bandwidth B=100 GB/s, predict a 200 GFLOP/s ceiling, then Play or Step. Separately Reset, set both I=16 and B=100, predict the fixed 1,000 GFLOP/s compute ceiling, then Play or Step. Playback traces the synthetic roof without changing the chosen kernel. These controls do not vary batch size or run SQL, and their decimal bandwidth units matter. Explain why increasing intensity beyond the ridge cannot raise this fixed compute roof.