Hi.
I compared performance of using pipeline in Redis and KeyDB.
Although both Keydb and Memtier_benchmark used multithreads and Redis used singlethread, Contrary to expectations, Redis had a higher throughput.
Could you explain WHY REDIS’s PERFORMANCE IS MORE HIGHER for me?
[Test Environment]
OS: centos 7
Redis 6.0.9
KeyDB : 5.3.3(stable)
benchmark tool : memtier_benchmark
When pipelining commands you will not see as high of a performance gap in general, however with our latest version 6.0.16 the perf should still be higher. v5.3.3 is quite an old version so I would definitely recommend the latest release which has a lot of perf improvements since 5.3.3.
Are you able to share your memtier command, the results of your test, along with your KeyDB and Redis config parameters? If so i can take a look myself
KeyDB conf.
port 6379
bind 0.0.0.0
daemonize yes
protected-mode no
pidfile “/var/run/keydb_6379.pid”
logfile “[log_path]/log”
dir “/var/KeyDB/6379”
databases 64
maxmemory 2gb
maxmemory-policy allkeys-lru
requirepass asdf1234
masterauth asdf1234
loglevel notice
appendonly no
timeout 0
repl-timeout 3600
repl-diskless-sync yes
maxclients 32768
repl-backlog-size 2mb
server-threads 4
active-replica yes
replicaof [another_server] 6379
Redis conf.
port 7000
bind 0.0.0.0
daemonize yes
protected-mode no
pidfile “/var/run/keydb_7000.pid”
logfile “[log_path]/log”
dir “/var/Redis/7000”
databases 64
maxmemory 2gb
maxmemory-policy allkeys-lru
requirepass asdf1234
masterauth asdf1234
loglevel notice
appendonly no
timeout 0
repl-timeout 3600
repl-diskless-sync yes
maxclients 32768
repl-backlog-size 2mb
I ran a few tests on some AWS machines (memtier on m5.8xlarge, redis/keydb on m5.4xlarge) so they can be replicated if needed. I used your configuration, however because the test completes so fast, I ran with more requests and over a series of runs with the following memtier command. Results are similar but more repeatable:
memtier_benchmark -s 172.31.49.176 --pipeline 1000 --hide-histogram --run-count=5 --requests=100000
I found similar relative results to what you were seeing, but much higher ops/sec which could be a result of network/server hardware. It can be noted that v6.0.16 is much faster than 5.3.3 as we have made a lot of perf improvements since then.
| SETS | GETS | Total | |
|---|---|---|---|
| Redis 6.0.9 | 189,322 | 1,893,201 | 2,082,523 |
| KeyDB 5.3.3 | 118,085 | 1,180,842 | 1,298,927 |
| KeyDB 6.0.16 | 176,667 | 1,766,651 | 1,943,318 |
An important thing to understand is the pipeline configuration you are passing. Per memtier:
--pipeline=NUMBER à Number of concurrent pipelined requests (default: 1)`
So when you specify 1000 for the pipeline configuration, the assumption is that every call to the database has 1000 concurrent pipelined requests.
With KeyDB’s multithreading, there is not as much benefit if you are only pipelining in large amounts of requests. You will notice adding more threads to KeyDB will not make much of a difference in this scenario.
Most use cases may pipeline in smaller amounts, have only a percentage of calls pipelined, or maybe not at all. In these scenarios KeyDB is able to generate much higher throughput.
See the chart below where I benchmark on default configs (redis-server, keydb-server –server-threads 8 –server-thread-affinity true), where the memtier command is similar to that above but with different pipeline values passed:
Total Ops/sec
| pipeline=4 | pipeline=10 | pipeline=100 | pipeline=1000 | |
|---|---|---|---|---|
| Redis 6.0.9 | 649,205 | 1,059,855 | 1,981,198 | 2,111,928 |
| KeyDB 6.0.16 | 1,530,779 | 1,618,893 | 2,001,172 | 1,923,228 |
As you can see, when pipelining smaller amounts of requests, the difference in throughput becomes quite noticeable. For most use cases not every request is a batch of 1000 pipelined requests, meaning for most cases KeyDB should be faster.
Take another example where we run two separate memtier instances concurrently, one with only one client pipelining, and the other with default load:
Memtier 1:
memtier_benchmark -s 172.31.49.176 --pipeline 1000 --hide-histogram --requests=10000000 --clients=1
Memtier 2:
memtier_benchmark -s 172.31.49.176 --hide-histogram --requests=100000 --threads=16
We see the following Total Ops/sec:
| Memtier 1 | Memtier 2 | |
|---|---|---|
| Redis 6.0.9 | 337,498 | 152,804 |
| KeyDB 6.0.16 | 847,836 | 409,850 |
Hope this sheds a bit of insight
Can I ask you one more thing?
I wonder difference of throughput when “active-replica” option is enabled or not.
I ran two version of Keydb Instance.
One is single instance, the other one is active-replica option is enabled.
The instance of using active-replica option’s Throughput is more slower than the single instance
Why are they so slow?
If you are not using KeyDB v6.0.16 yet, you should upgrade. We have made some major improvements to performance with replication. That being said, you likely still wont be able to get the exact same perf as a single node because keeping replica nodes synchronized uses additional resources and network.