To manage memory effectively, VoltDB does not immediately release all unused memory. Allocating and deallocating small chunks of memory frequently can be expensive. Instead, VoltDB manages unused memory until larger chunks are available. Similarly, the Java runtime and the operating system perform their own memory pooling techniques.

As a result, RSS is not an exact measurement of actual memory usage. However, VoltDB offers statistics that provide a detailed breakdown of how it is using the memory that it has currently allocated. These statistics provide a more meaningful representation of VoltDB's memory usage than the lump sum allocation reported by the operating system RSS.

VoltDB manages memory for persistent and semi-persistent storage aggressively to ensure unused space is compacted and released when available. In some cases, memory is returned to the operating system, making the RSS more responsive to changes in the database contents. In other cases, where memory is managed as a pool of resources, VoltDB provides detailed statistics on what memory is allocated and what is actually in use.

Persistent storage for database tables (tuples) and indexes is compacted incrementally. A periodic process initiates incremental compaction for each table within a partition once the volume of data and fragmentation for that table reaches a predefined level. This avoids the compaction process thrashing over small amounts of data that would not significantly affect memory usage.

Tuple storage is managed in blocks. Until a full block is released by compaction, no memory is actually regained by the system. Therefore, compaction is not considered until the volume of data for the table exceeds at least three blocks. Similarly, compaction will not start until the allocated but unused space reaches at least 5% of the total storage for a table within a partition. Again, this avoids excessive compaction effort when it is unlikely to free up enough space to recover a block's worth of storage.

Once the persistent storage for a table within a given partition meets the compaction criteria, a transaction is initiated to defragment the unused space. During the transaction, tuples are moved to fill any "holes" in the allocated blocks and — if any blocks are emptied by this process — deallocate the blocks and return memory to the system. Just as the compaction process doesn't start until data volume and fragmentation reach the minimum criteria, the defragmentation is done incrementally, with each transaction moving only up to a predefined maximum number of tuples. If more tuples need to be moved, they will be moved in the next iteration of the compaction process.

By performing compaction of tuple storage incrementally through short single-partitioned transactions, excess unused space can be recovered without impacting the ongoing database workload. At the same time, storage for variable data such as strings and varbinary data greater than 63 bytes in length is being managed as a pool of resources. Free memory in the pool is not immediately returned to the operating system. VoltDB holds and reuses memory that is allocated but unused for these objects.

The consequence of these operations is that when you delete rows, the allocated memory for VoltDB (as shown by RSS) may go up during the delete operation (to allow for the undo buffer), but then it will go down — by differing amounts — based on the type of content that is deleted. Memory for tuples not containing large strings or binary data is returned to the operating system quickly. Memory for large string and binary data is not returned but is held for later reuse.

In other words, the pool size for non-inline string and binary data tends to reach a maximum size (based on the maximum required for your application workload) and then stabilize. Whereas memory for indexes as well as numeric and short string data oscillates as your application needs vary.

To help you understand these changes, the @Statistics system procedure tells you how much memory VoltDB is using and how much unused memory is being held for each type of content. These statistics provide a more accurate view of actual memory usage than the lump sum value of system RSS.