Paper: Vectorwise: Beyond Column Stores — Zukowski & Boncz, IEEE Data Engineering Bulletin 2012.

This is a retrospective paper tracing Vectorwise from its academic origins (MonetDB/X100 at CWI, 2003) through commercialisation (Vectorwise BV, 2008) to its acquisition by Actian Corp. The key thesis: Vectorwise’s performance comes from its vectorised execution model, not just columnar storage — and the gap between research prototype and production database is far wider than it appears.

# Origins: MonetDB → X100 → Vectorwise

MonetDB (2002) pioneered full column materialisation: every intermediate result is a complete column stored in RAM. This eliminates Volcano’s per-tuple function call overhead and gives excellent cache locality for column scans, but forces materialisation of all intermediates — expensive memory bandwidth for complex queries.

The X100 project (2003) identified that MonetDB’s full materialisation was itself a bottleneck: writing and re-reading entire intermediate columns wastes RAM bandwidth. The fix: process data in vectors — chunks of ~1000 values that fit in L1/L2 cache. This became the vectorised query processing model, the core technology of Vectorwise.

# The Vectorised Execution Model

Instead of one tuple at a time (Volcano) or one full column at a time (MonetDB), Vectorwise processes vectors: fixed-size arrays of values from a single column, sized to fit in CPU cache (typically 1000 values, ~8KB for 64-bit integers).

This gives three simultaneous benefits:

1. Reduced interpretation overhead: function calls happen once per vector, not once per tuple. For 1000 tuples per vector, interpretation overhead drops by ~1000×.

2. SIMD exploitation: tight loops over arrays of a single type map directly to CPU SIMD instructions (SSE, AVX). The compiler (or the developer) can process 4–16 values per instruction. Predicate evaluation, arithmetic, and comparisons all vectorise naturally.

3. Cache residency: a vector of 1000 × 8-byte integers = 8KB — fits in L1 cache. Operations on a vector touch only L1 cache, not RAM. This is the key advantage over MonetDB’s full-column approach, which forces RAM round-trips for large intermediate results.

The vectorised model sits between Volcano (1 tuple, minimal materialisation) and MonetDB (all tuples, full materialisation) — it materialises just enough to fill the CPU cache, then discards.

# Improvements over the Original X100 Model

Several refinements were made as X100 evolved into Vectorwise:

• Lazy vectorised expression evaluation: don’t evaluate all expressions eagerly; delay evaluation until the value is actually needed, similar to lazy attribute loading in compiled execution.
• Multiple function implementations per operator: choose the implementation based on runtime environment (data type, compression type, selectivity). A selection on a dense integer column uses different code than on a dictionary-compressed string column.
• Pushing selections up: if a selection is highly selective (eliminates most tuples), apply it as early as possible to reduce the work done by subsequent operators — even if this violates the algebraic plan order.
• SIMD predicate evaluation: evaluating predicates on vectors using SIMD allows checking 4–16 values per CPU instruction.
• NULL handling via separate boolean columns: rather than embedding NULL flags in each value (which breaks SIMD since the test introduces unpredictable branches), Vectorwise stores a separate boolean column per nullable attribute. The NULL column can often be ignored entirely during query processing, and when needed it is processed as a normal boolean vector.
• Adaptive row/column layout: strict columnar layout is dropped for operations where row layout is more efficient — e.g., hash table entries are stored in NSM (row) format since hash joins access all columns of a matching tuple together. Vectorwise automatically switches layout based on access pattern.
• Volcano-style exchange operators for multi-core: parallelism is added via exchange operators (as in the Volcano paper ), allowing existing vectorised operators to run on multiple cores without modification.
• Bloom filters for join acceleration: highly efficient Bloom filters reduce the number of tuples that reach expensive hash table probes.

# Data Storage: PAX Layout

Vectorwise does not use pure DSM (one file per column) or pure NSM (row store). It uses PAX (Partition Attributes Across) — a hybrid where data is stored in large blocks, and within each block columns are stored contiguously.

Each table is split into multiple PAX partitions, each covering a group of columns:

• DSM/PAX: one column per PAX partition — pure columnar, best for scans of few columns.

• NSM/PAX: all columns in one PAX partition — pure row store, best for small tables where one disk block per column would waste space.

• Mixed: related columns (e.g., composite primary key columns, nullable value + NULL flag) co-located in the same PAX partition.

The PAX grouping is self-tuned automatically based on query patterns and DDL hints. Block sizes are large (512KB on disk, 32KB on SSD) to amortise seek overhead.

Compression: all data is stored compressed using schemes that decompress at a few cycles per tuple. Data remains compressed in the buffer pool and is decompressed just before processing — effectively increasing the logical buffer pool size. Vectorwise initially avoided compressed execution (operating directly on compressed data), but later added it for high-benefit cases: aggregation over RLE-encoded columns (order-of-magnitude reduction in work) and dictionary-compressed strings (a string comparison becomes an integer comparison).

Clustered index: one index per table, declared in DDL. This determines physical sort order — equivalent to a clustered B-tree. Enables range predicate pushdown and efficient foreign-key joins when both tables are co-clustered on the join key.

MinMax indexes: lightweight metadata storing (min, max) per range of records for every column. Used by the query rewriter to eliminate ranges of tuples from scans without reading the data — the same idea as Parquet/Iceberg file-level statistics, but at finer granularity within a file.

Positional Delta Trees (PDT): a three-level differential update structure for handling transactional writes without slowing reads:

• Per-transaction PDT: very small, private to the current transaction.

• Shared CPU-cache-resident PDT: shared between transactions, kept in L1/L2 cache.

• RAM-resident PDT: larger, holds pending updates not yet merged to the main store.

PDTs store differences by position rather than by key — during a table scan, merging the delta into the scan result has near-zero cost because no key comparisons or key scans are needed. This gives snapshot isolation for read-only queries without any locking overhead.

# The Research-to-Product Gap

A significant portion of the paper documents the hard lessons of commercialisation — instructive for understanding what makes a database system production-ready vs a research prototype:

Stability: exposing X100 to large numbers of real users revealed a “sizable number of stability problems” that didn’t appear in the research setting.

Missing features blocking migration: users couldn’t move from Oracle/SQL Server without full SQL 1999 support, temporary tables, parallel execution, and disk-spilling operations. Adding these took 18 months post-launch (Vectorwise 2.0, November 2011).

Update latency expectations: Vectorwise was designed for batch-loaded data. Once users saw its query speed, they wanted sub-second data loading latency too — requiring incremental load and full ACID support that the original design didn’t prioritise.

PL/SQL migration: application logic stored in stored procedures is the hardest migration problem — not data, not schema, but procedural business logic written in database-specific languages.

Schema complexity: production environments have hundreds of databases, thousands of tables, and tables with thousands of columns — stressing the system in ways a research benchmark never does.

Customer behaviour changes: customers who adopted Vectorwise stopped using techniques they relied on for performance with slower databases:

• Removed indexes (Vectorwise scans are fast enough)
• Normalised previously denormalised tables (joins are cheap)
• Switched to full data reloads instead of incremental ETL
• Ran queries directly on raw data instead of precomputing aggregates

These are not just performance wins — they represent fundamental simplifications to data architecture that compound over time.

# Performance

As of March 2012, Vectorwise held TPC-H leadership on the single-node 100GB–1TB benchmark category.

On the 1TB non-clustered TPC-H benchmark (Figure 2):

• Vectorwise: 436,788 QphH @ $0.88/QphH (32 cores, 1TB RAM)
• Oracle: 140,181 QphH @ $12.15/QphH (64 cores, 0.5TB RAM)
• SybaseIQ: 164,747 QphH @ $8.65/QphH (64 cores, 0.5TB RAM)
• SQL Server: 173,961 QphH @ $1.37/QphH (80 cores, 2TB RAM)

Vectorwise achieved ~2.5× Oracle’s throughput at 1/14th the price/performance ratio.

# Key Takeaways

Vectorwise (now Actian Vector) established that vectorised execution over columnar storage is the right model for analytical databases. DuckDB is the most direct intellectual descendant, implementing the same vectorised model with modern refinements, and is the system you are most likely to encounter this model in practice today.