An SSTable (Sorted String Table) is an immutable data structure used to store a set of key-value pairs sorted by key. This format was popularized by Google in their seminal paper on Bigtable as a mechanism to achieve high-throughput writes while maintaining efficient read access. In an SSTable, the data is typically organized into blocks, often accompanied by an Index or a Bloom Filter stored in memory to speed up lookups.
The primary advantage of the SSTable format lies in its integration with the Log-Structured Merge-Tree (LSM-Tree). In this architecture, writes are first buffered in a Memtable. Once the Memtable reaches a certain size threshold, it is flushed to disk as a new SSTable. Because the data is already sorted in memory before being written, the resulting file is naturally ordered, allowing for efficient Compaction processes that merge multiple SSTables into larger ones while removing obsolete data or Tombstones.
Modern distributed databases like Apache Cassandra and ScyllaDB rely heavily on SSTables for persistent storage. Similarly, storage engines like LevelDB and RocksDB utilize this format to provide high-performance storage for various applications. For more technical details on the original implementation, refer to the Bigtable research paper published by Google.