MELT: The multidimensional key-value store performance evaluation framework. Melt: memory, energy, latency and throughput

Abstract

1 Abstract Key-value stores have a very large variation in their design and implementation, while still adhering to the key-value abstraction. The available generic benchmarks cannot truly represent the performance a key-value store will have with a specific application, unless your application happens to have the exact same configuration and workloads as the benchmark. Moreover, most benchmarks only measure throughput and latency, ignoring performance metrics like energy efficiency and space efficiency. Introducing MELT: The multidimensional key-value store evaluation framework, which can take any applications usage characteristics of a key-value store and test it on multiple different key-value store implementations with different concurrency and throughputs settings. In addition, it measures four MELT performance metrics, memory, energy, latency and throughput. With this evaluation framework the assumption is that concurrency is better than serial key-value stores in all situations. Here will be shown results that support the claim that for certain applications, with throughput demands less than 10 million operations per second, serial hopscotch implementation outperforms the concurrent Libcuckoo on most of the MELT performance metrics.