Redis and Dragonfly should be worried!
Microsoft Open Sources New Cache Store System Garnet
Recently, Microsoft officially open-sourced the cache store system Garnet. According to Badrish Chandramouli, a senior principal researcher in Microsoft Research's database group, the Garnet project was built from scratch with performance as the core consideration (especially thread scalability in throughput and higher proportions of low latency).
Specifically, Garnet has several major advantages:
- Garnet adopts the popular RESP wire protocol as a starting point, so most users can connect to Garnet directly via Redis clients written in most programming languages without any modifications.
- Garnet provides better scalability and throughput through multiple client connections and small batch forms, helping large applications and services reduce operating costs.
- Garnet shows better client latency at the 99th and 99.9th percentiles, and a higher proportion of stable performance is crucial for real-world scenarios.
- Garnet is based on the latest .NET technology, making it cross-platform, extensible, and modern. It is designed to be easy to develop and adjust without sacrificing performance in common scenarios. It leverages the rich .NET library ecosystem to extend its API and provides open optimization opportunities. By fully utilizing .NET, Garnet achieves top-tier performance on both Linux and Windows platforms.
It is understood that Microsoft Research has been studying modern key-value database architectures since 2016. In 2018, Microsoft open-sourced FASTER, an embedded key-value library, which outperformed existing systems by several orders of magnitude while focusing on a simple single-node in-process key-value model.
Starting in 2021, based on actual use case requirements, Microsoft began building a new remote cache store solution. It includes all necessary features to serve as a viable alternative to existing cache stores. The challenges Microsoft faced at the time included maintaining/enhancing the performance advantages already achieved in earlier work while considering how to better adapt to more realistic general network environments. The result of this work is Garnet.
When asked about suitable deployment scenarios for Garnet, Chandramouli stated that any "existing applications that use Redis, KeyDB, or Dragonfly as a cache store solution are suitable. Garnet can provide higher throughput, lower latency, reduce costs by requiring fewer cache store shards to host, and support spilling data to local disk or SSD to cache datasets larger than memory. Additionally, Garnet is suitable for new applications looking to leverage a high-performance cache layer to boost performance and reduce backend storage server or database costs."
In terms of API functionality, Garnet supports a wide range of APIs, including raw strings, analytics, and object operations. It also provides cluster modes with features like sharding, replication, and dynamic key migration. Garnet supports client RESP transactions and server-side stored procedures written in C#, and allows users to set custom operations on top of raw strings and new object types. All of these can be easily written in C#, lowering the barrier for custom extension development.
In terms of networking, storage, and cluster functionality, Garnet uses a fast and pluggable network layer that supports future extensions, such as integration with kernel-bypass stacks. It supports TLS communication and various basic access controls. Garnet's storage layer, called Tsavorite, is forked from OSS FASTER and provides a range of powerful database features, such as thread scalability, tiered storage support (memory, SSD, cloud storage, etc.), fast non-blocking checkpointing, recovery, persistent op-log logging, multi-key transaction support, and better intrinsic management and reuse. Additionally, Garnet supports cluster operation modes.
In addition to single-node execution, Garnet supports cluster mode, allowing users to create and manage sharded and replicated deployments. Garnet also supports efficient, dynamic key migration schemes to rebalance shards. Users can use standard Redis cluster commands to create and manage Garnet clusters, with nodes performing gossip to share and evolve the cluster state. Overall, Garnet's cluster mode is a large and evolving feature; Microsoft says more details will be shared in subsequent articles.
In an email reply to The Stack, Chandramouli added, "We also look forward to receiving feedback on Garnet's performance in various other real-world applications. Additionally, we have a powerful C#-based stored procedure model that allows users to customize transactions of interest. Finally, we see Garnet as an important innovation tool for the future, including optimizing disk I/O, kernel-bypass networking, and vector database use cases."
What Makes Garnet Stand Out?
The rapid growth of cloud and edge computing has significantly increased the data and coverage of related applications and services. However, it has also raised practical demands for higher efficiency, lower latency, and lower cost in data access, updating, and transformation. These applications and services often require significant operational expenditure on storage interactions, making this one of the most expensive and challenging platform areas today. A cache store software layer in the form of a separately scalable remote process can effectively reduce these costs and improve application performance. This has driven the development of the cache store industry, including many well-known open-source systems such as Redis, Memcached, KeyDB, and Dragonfly.
Unlike traditional remote cache stores that only support simple get/set interfaces, modern caches need to provide rich APIs and feature sets. They support raw strings, analytical data structures like HyperLogLog, and complex data types like sorted sets and hashes. They also must allow users to set checkpoints and recovery for caches, create data shards, maintain replicas, and support transactions and custom extensions.
However, existing systems often struggle to meet such rich functional requirements while maintaining system design simplicity, including failing to fully leverage modern hardware capabilities (e.g., multi-core, tiered storage, fast networking). Additionally, many of these systems were not designed with ease of extension by application developers or good performance across different platforms/operating systems in mind.
According to the introduction, Garnet rethinks the entire cache store stack—from packet acquisition at the network layer, to parsing and processing database operations, to executing storage interactions.
The following diagram shows Garnet's overall architecture. As can be seen, Garnet's network layer inherits the shared-memory design inspired by Microsoft's ShadowFax research. TLS processing and storage interactions are executed on I/O completion threads, avoiding common thread-switching overhead. This approach leverages CPU cache coherence to transfer data to the network, rather than the traditional shuffle design that requires moving data on the server.
Garnet's storage design consists of two Tsavorite key-value stores bound together with a unified operation log. The first store, called "Main Store," is optimized for raw string operations and manages memory to avoid garbage collection. The second is an optional "Object Store," optimized for complex objects and custom data types, including popular data types like sorted sets, sets, hashes, lists, and geospatial data. These are stored on the memory heap (for more efficient updates) and serialized on disk. In the future, Microsoft will explore simplifying Garnet's system maintenance through a unified index and log.
A notable feature of Garnet's design is the use of the Tsavorite storage API. This API provides a larger, richer, and extensible RESP API surface, capable of performing reads, upserts, deletes, and atomic read-modify-write operations, all via Garnet's asynchronous callbacks to insert logic at multiple points during each operation. The storage API model also ensures that Garnet separates problem parsing and query processing from other storage functions like concurrency, storage tiering, and checkpointing.
Additionally, Garnet adds support for multi-key transactions based on two-phase locking. Users can use RESP client transactions (MULTI-EXEC) or server-side transaction stored procedures in C#.
Performance
The Microsoft research team demonstrated key performance indicators comparing Garnet with other leading open-source cache store solutions.
First, the team pre-configured two Azure Standard F72s v2 VMs (72 vCPUs and 144 GiB memory each) running Linux (Ubuntu 20.04) with accelerated TCP enabled. One VM ran various cache store servers, and the other dedicated to issuing workloads. Microsoft used its own benchmark tool, Resp.benchmark, to provide performance test results uniformly.
Microsoft compared Garnet with the latest open-source versions of Redis (v7.2), KeyDB (v6.3.4), and Dragonfly (v6.2.11). In the experiments, uniformly randomly distributed keys were used (Garnet's shared-memory design has better performance optimization for non-randomly distributed keys). Data was pre-loaded onto each server and fit into memory.
Experiment 1: Throughput Comparison with Different Numbers of Client Sessions
Starting with large GET operations (4096 requests per batch) with low payload (8-byte keys and values) to minimize network overhead, and gradually increasing the number of client sessions to compare system performance. As shown in the figure below, Garnet demonstrated better scalability than Redis and KeyDB, while achieving higher throughput than all three baseline systems (y-axis is logarithmic). Note that while Dragonfly's scaling is similar to Garnet's, the former is a purely in-memory system. Additionally, when the database size (i.e., the number of preloaded keys) significantly exceeded the processor's cache size (256 million keys), Garnet still maintained strong throughput compared to other systems.
Throughput (log scale) with different numbers of client sessions for database sizes of (a) 1024 keys and (b) 256 million keys.
Experiment 2: Throughput Comparison with Different Batch Sizes
Next, changing the batch size with a fixed number (64) of client sessions and GET operations. As before, two different database sizes were tested. As shown below, even without batching, Garnet performed better; and with batching, even small batch sizes enhanced Garnet's performance advantage. Payload size was the same as Experiment 1, and the y-axis is logarithmic.
Throughput (log scale) with different batch sizes for database sizes of (a) 1024 keys and (b) 256 million keys.
Experiment 3: Latency Comparison with Different Numbers of Client Sessions
Next, the client latencies of various systems were tested. As shown below, with increasing client sessions, Garnet exhibited lower and more stable latencies (in microseconds) at various percentiles compared to other systems. The operations were a mix of 80% GET and 20% SET without batching.
Latency at (a) median, (b) 99th percentile, and (c) 99.9th percentile for different numbers of client sessions.
Experiment 4: Latency Comparison with Different Batch Sizes
Garnet's latency is optimized for adaptive client batching and query systems. Microsoft increased the batch size from 1 to 64 and plotted latency at different percentiles with 128 active client connections. As shown below, Garnet's latency is generally lower. The same mix of 80% GET and 20% SET was used as in the previous experiment.
Latency at (a) median, (b) 99th percentile, and (c) 99.9th percentile for different batch sizes.
Developers: Redis Needs a Major Performance Optimization!
From the benchmark performance charts, the throughput of GET commands exceeds Dragonfly by more than ten times. Although the 50th percentile latency is slightly higher than Dragonfly, the 99th percentile latency is lower than Dragonfly. Both Garnet and Dragonfly significantly outperform Redis in throughput and latency, leading many developers to believe that Redis may need a major performance optimization.
Developer hipadev23 said, "Garnet is indeed the first alternative that outperforms Redis at both low and high concurrency levels, which is a remarkable achievement. Redis may need a major performance optimization."
Developer mtmk believes that for those who need to run Redis (or a compatible alternative) directly on Microsoft Windows Server without relying on WSL2, Garnet is certainly good news. The memory usage issues caused by the earlier Redis port (now archived) (mainly due to memory-mapped files AFAIK) will no longer exist.
Many developers still firmly choose Redis. Redis is more developer-friendly in some aspects and has been running longer and more stably. For Garnet, concerns generally revolve around licensing, product pricing, and update/maintenance. Developer throwaway38375 said, "Redis should be more stable in terms of licensing or product pricing, and it has been tested in production for billions of hours. Redis is also easier to install and understand." Someone commented, "For a Microsoft Research project like this, my biggest concern is not licensing or pricing, but the lack of updates (feature, maintenance, or even security updates)."
By the way: Garnet is Developed in C#
In community discussions, many developers were surprised that the Garnet project was developed in C#.
Developer west0n said, "What surprised me most is that the Garnet project is developed in C#, while Dragonfly is developed in C++ and Redis in C." Developer whimsicalism bluntly stated, "Unexpected that Garnet, written in a garbage-collected language like C#, beats Redis and Dragonfly."
Other developers offered more balanced opinions. pjmlp noted, "Garbage-collected languages are not all the same. Languages like C# and .NET actually provide all the performance tuning options comparable to C++." He said people should learn properly instead of lumping all garbage-collected languages together and dismissing them. [Editor's note: .NET is a platform, C# is an implementation on .NET; the relationship between C# and .NET is analogous to Java and JDK.]
Furthermore, more specifically, MSIL and .NET are designed to also support C++, and languages like C# and F# have ways to access these features. Even if some features are not exposed in language syntax, developers can directly use MSIL generated by C++/CLI.
What do you think? Feel free to leave your comments below.
Reference links:
https://www.thestack.technology/microsoft-takes-on-redis-with-new-open-source-garnet-cache-store/