Vacuumer

​

Overview

• Tracks expired extents (superseded by new extents) and dropped table snapshots
• Performs hole punching via fallocate() to return unused disk blocks to the filesystem
• Cleans up dropped table directories and old snapshot/roots files

​

Key Components

​

Data Flow

  1. EXTENT EXPIRATION (triggered when append-only writes create new extents)
      StorageCache -> expire_extent() -> _extent_map[file][xid].push_back(offset, size)
  2. SNAPSHOT EXPIRATION (triggered by DROP TABLE/INDEX or schema changes)
      DDL operations -> expire_snapshot() -> _snapshot_map[db_id][xid].push_back(table_dir)
  3. COMMIT (on transaction commit)
      commit_expired_extents() -> writes entries to global vacuum file (.global.vcm)
  4. VACUUM RUN (background thread, every 1 second)

​

Implementation Details

• expire_extent() is called via a callback registered with StorageCache (vacuumer.cc:92-95)
• Each expired extent is recorded as a HoleInfo struct containing offset and size within the file, along with the XID at which it was superseded by a new extent
• Entries are held in memory (_extent_map) until committed, then persisted to the global vacuum file
• Memory threshold (_max_entries_in_memory, default 10K) triggers flush to disk if exceeded

• Uses Linux fallocate(fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, offset, len) to deallocate blocks
• Block alignment requirement: Filesystem hole punching only works on block-aligned regions
  • _align_up() / _align_down() align to _hole_punch_block_size (default 4KB)
  • If an extent spans [100, 5000], only [4096, 4096] can be punched; [100, 4096] and [4096, 5000] become partials
• Interval merging: Uses IntervalTree to coalesce adjacent/overlapping expired regions before punching (vacuumer.cc:928-972)
• Partial handling: Unaligned remainders are saved to per-file partial files (_partials.vcm) and merged in subsequent runs

• Cutoff XID = min(min_fdw_xid, last_committed_xid, min_index_xid)
  • min_fdw_xid: Minimum XID in use by foreign data wrappers (active queries from remote)
  • last_committed_xid: Latest committed transaction (protects uncommitted data)
  • min_index_xid: Minimum XID for ongoing index builds/drops
• Only extents with XID < cutoff are vacuumed, ensuring no active transaction can reference the data
• Cutoff XIDs are persisted to Redis per-database for monitoring (_save_last_seen_cutoff_xid)

• Global vacuum schema: (file TEXT, offset UINT64, size UINT64, file_dropped BOOLEAN)
  • file_dropped=true indicates a snapshot/directory deletion rather than hole punch
• Partial file schema: (offset UINT64, size UINT64) — simpler, no file path needed (one file per source)
• Atomic writes via runfiles: write to .vcm.run, then rename() to .vcm

• Dropped tables/indexes are tracked in _snapshot_map
• Uses std::filesystem::remove_all() to recursively delete table directories
• Also cleans up associated partial files via _cleanup_partial_files()

• System tables maintain the roots in the files of the format (roots.{xid})
• Vacuum removes roots files with XID < cutoff, preserving the current symlinked version
• Iterates all system tables defined in sys_tbl::TABLE_IDS

• Background thread wakes every 1 second via condition_variable::wait_until()
• All public methods acquire _mutex before accessing shared state
• Graceful shutdown: _internal_thread_shutdown() signals CV, thread exits loop

• enabled: Enable/disable vacuum service
• hole_punch_block_size: Alignment for hole punching (default 4KB)
• global_file_size_threshold: Minimum global file size to trigger vacuum run (default 20KB)
• max_entries_in_memory: Memory threshold before forced flush (default 10K entries)
• vacuum_dir: Base directory for vacuum metadata files