Redis benchmark: HotKV vs Valkey and Dragonfly in a three-node cluster

This Redis benchmark runs HotKV, Redis, Valkey and Dragonfly as three-node clusters on AWS. Our earlier benchmarks ran each server as one instance on a whole machine. That layout suits HotKV, which uses every core in one process. Redis and Valkey run commands on one thread per process, and they are commonly scaled across cores with many processes, usually as a cluster. So we measured every product the way a cluster deploys it.

In short. In this Redis benchmark on three AWS servers, HotKV had the highest throughput on 8 of 9 commands: 92.3 million GETs per second against 57.3 million for Redis, 52.9 million for Valkey and 22.5 million for Dragonfly. Redis and Valkey were ahead on APPEND. The load generators were near their limit on some commands, so the figures are lower bounds.

Why we ran this Redis benchmark as a cluster

A single-instance benchmark, like the one in our launch article, asks how much one server process can do with a whole machine. That is a reasonable question for HotKV and Dragonfly, which use all cores in one process. It says less about Redis and Valkey. Their commands run on one thread per process (I/O threads can offload network work), so a 48-vCPU server is normally used by running dozens of processes.

We do not want our numbers to depend on a layout that suits one architecture better than another. A cluster is common ground. All four products can run as one, and for Redis and Valkey it is a usual way to use many cores. We ran this benchmark to see how the comparison looks when each product is deployed that way.

What we tested

Every product ran as a cluster on the same three servers, one product at a time, driven by the same two load generators. The sections below give the machines, each layout and the procedure.

Machines

• Servers. Three c8in.12xlarge instances (Intel Xeon 6975P-C, 48 vCPUs, 75 Gbit/s network) in one AWS cluster placement group in eu-south-2a, running Ubuntu 24.04. Every server process was pinned to CPUs 0 to 39.
• Load generators. One c8in.48xlarge running 16 memtier_benchmark processes of 12 threads each, and one c8in.12xlarge running 4 processes of 12 threads. They were started together and the operations per second of all processes were added up.
• Tuning. All five machines got the same host tuning: the repository's tune_server.sh, receive packet steering over all CPUs, the network card's receive ring at its maximum, a larger softirq budget and a listen backlog of 65,535.

Servers and layout

• HotKV. One process per server with 40 threads, in cluster mode (3 primaries). The build is 0.3.0 plus a change that adds the pipeline-prefetch setting, and it ran with pipeline-prefetch yes. The setting is off by default. It looks ahead in a client's pipeline and asks the CPU for the memory the next commands will read.
• Redis 8.10.3 and Valkey 9.1.3. 40 single-thread processes per server, so 120 primaries, in cluster mode. Each process had a core to itself. Their own batch prefetching was left at its default.
• Dragonfly 2.0.2. One process per server with 40 threads, in cluster mode, built from source.
• Persistence. Off on every server.

Workload and procedure

The data set was 5,000,000 keys with 256-byte values, and pipeline depth was 100. memtier_benchmark ran in cluster mode, so each client talks to the right node. Each command ran for 15 seconds, twice, and we report the median of the two. The data set was loaded again, identically, before every run, so a command never ran on top of the previous command's changes. The scripts and every run are in the benchmark repository.

Results

Millions of operations per second, median of two runs. The highest value in each row is in bold.

Against Dragonfly, HotKV was ahead on all nine commands, from 1.85 times (LPUSH) to 4.09 times (GET).

How to read the ratios

A throughput figure from a load test is a floor when the load generators are the limit. If a load generator is at 100 percent CPU, the server may have been able to take more. The figure then shows only that it took at least this much. A ratio between two floors is not exact, because either side could be higher.

We flag every figure where the main load generator ran above 85 percent CPU. These were GET, SETNX and INCR for HotKV, and GET, SET, SETNX, INCR, SADD, HSET and APPEND for Redis and Valkey. On those figures, read the ratio as a rough size and not a precise one.

On SADD and HSET, HotKV's figure is not flagged but Redis's and Valkey's are. Their true throughput may be higher, and HotKV's lead smaller than shown. The same applies to APPEND, where it would make their lead over HotKV larger than shown.

RPUSH and LPUSH have no flag for any server, so they give the cleanest comparison. HotKV was 1.3 to 1.6 times ahead of Redis and Valkey on them.

What the results show

Three things stand out in the table, and the first is about Redis and Valkey rather than HotKV.

Redis and Valkey scale well as clusters

With 120 single-thread processes, Redis served 57 million GETs per second and 118 million INCRs per second across three servers. That confirms the cluster layout suits them. It is also why HotKV's margin here is smaller than in our single-instance runs.

Those runs compared a design that uses all cores in one process with products that were not run in the layout they are commonly scaled in. This Redis benchmark gives Redis and Valkey that layout. HotKV's lead over them narrows to between 1.0 and 1.7 times on eight commands.

HotKV had the highest throughput on 8 of 9 commands

The lead was widest on GET and RPUSH against Redis (1.6 times) and on GET against Valkey (1.7 times). On RPUSH and LPUSH the load generators had room for every server, so those figures are limited by the servers. On INCR, HotKV and Redis were level within 1 percent, and the load generators, not the servers, set the limit for all three.

Redis and Valkey were ahead on APPEND

HotKV served 35.0 million APPENDs per second against 41.7 million for Redis and 40.9 million for Valkey. We have not found the reason. APPEND grows the dataset by about 10 GB per second across the cluster, so the run is also a test of memory allocation. A 30-second APPEND run ran the servers out of memory.

We profiled HotKV on APPEND and saw page faults, TLB shootdowns and lock contention. We have not shown which of them accounts for the gap.

Limitations

These are HotKV Ltd's measurements of the versions and configuration above. They are not a claim about any other version, configuration or workload of these products.

The test setup

• Load generators near their limit. Many figures are lower bounds, as the section on reading the ratios explains. HotKV's APPEND figure is not one of them, because the load generators were at about 30 percent CPU with it.
• Layouts we chose. We ran Redis and Valkey as one single-thread process per core and did not tune either product beyond that. Other layouts or tuning could change their figures. If you think a setting here disadvantages a product, tell us and we will check it and publish the result.
• Two runs, one region. Each figure is the median of two 15-second runs on one set of machines in one region. We did not measure variation between machines or days.

What was not measured

• Latency. We measured throughput only here. A separate Redis latency benchmark measures latency.
• Persistence, replication and failover. All servers ran with persistence off and no replicas.
• Other setups. Three servers and one workload shape. A larger cluster, other value sizes or other pipeline depths can change the ratios.

Redis, Valkey and Dragonfly are trademarks of their respective owners. They are named only to identify the software we measured, and this implies no sponsorship, endorsement or affiliation.

Reproduce it

The sweep script, the exact command sequence, the host tuning and every raw result are in the benchmark repository. The sweep script is harness/cluster_bench_sweep.sh. A chart of the same figures is on the performance page.

Try it yourself

HotKV Cloud runs HotKV clusters for you. Start on HotKV Cloud, or talk to sales to benchmark a cluster against your own workload.

Frequently asked questions

Why run a Redis benchmark as a cluster?

Redis and Valkey run commands on one thread per process, so a single process cannot use a large machine. They are commonly scaled across cores by running many processes as a cluster. A cluster benchmark measures each product in a layout like that, so the comparison does not rest on a single-instance deployment alone.

Did HotKV have the highest throughput in cluster mode?

On 8 of 9 commands, yes. HotKV led Redis by 1.0 to 1.6 times and Valkey by 1.1 to 1.7 times, and Dragonfly by 1.85 to 4.09 times. Redis and Valkey were ahead on APPEND. On INCR, HotKV and Redis were level within 1 percent, and the load generators set the limit.

Why were Redis and Valkey ahead on APPEND?

We do not know yet. APPEND grows the dataset by about 10 GB per second across the cluster, so the run also tests memory allocation. A profile of HotKV showed page faults, TLB shootdowns and lock contention, but we have not shown which of them accounts for the gap.

What is the pipeline-prefetch setting?

It makes HotKV look ahead in a client's pipeline and ask the CPU for the memory the next commands will read, so those reads overlap. It changes no reply. It is off by default, and HotKV ran with it on in this benchmark.

Can I reproduce this Redis benchmark?

Yes. The sweep script, the exact command sequence, the host tuning and every raw result are in the benchmark repository, and the same figures are charted on the performance page. You need three servers, one or more load generators and the four products installed by the harness.

Does this replace the single-instance results?

No. The two answer different questions. A single-instance run shows what one server process does with a whole machine. A cluster run shows what three machines do when each product is laid out as a cluster. Both are on the performance page.