Memtable Pluggability and Alternate Memtables (CEP-11 & CEP-19) – Branimir Lambov & Shylaja Kokoori
Memtable Pluggability and Alternate Memtables (CEP-11 and CEP-19) – Branimir Lambov, DataStax & Shylaja Kokoori, Intel
Memtables have a very profound effect on the performance of Cassandra, as writes to the memtable are on the hot write path, and its large data structures remain on the Java heap for a long time and are a key determinant of garbage collection performance.
CEP-11 provides an interface for plugging in alternative memtable implementations together with a simple sharded memtable as a proof-of-concept. CEP-19 adds a new implementation of memtables based on byte order and a custom in-memory trie built to offer a more compact indexing structure that can be stored off-heap and performs lookups and insertions significantly faster than the legacy solutions. This dramatically increases Cassandra’s maximum peak and sustained write throughput and improves GC efficiency, resulting in reduced latencies at fixed operation rates.
Another option made possible by CEP-11 is the persistent memory memtable, where the data is stored in an Adaptive Radix Tree allocated on persistent memory, and is never flushed to disk, effectively forming a complete storage system replacement. This solution helped translate the persistent memory advantages into some extraordinary speedups in Cassandra performance.
by The Linux Foundation
linux foundation
