Active Replication with Modules and Custom Types

The Split-Brain documentation states that…

"Each write is timestamped and when the connection is restored each master will share their new data. The newest write will win. "

But what about cases where all writes are independent, but there’s no way of KeyDB knowing whether they’re dependent or independent?

For example a module command like…

MODULE.COMMAND keyA valueA

In some cases I could want the most recent to overwrite any older commands, but in some cases these could be independent writes to RedisGraph where I want to make sure every single write is replicated.

What are the semantics in these cases and how can I ensure each outcome when desired?

And what about non-split brain scenarios, do all writes just automatically stream to the other masters and execute without any timestamp comparisons?

Also how can it differentiate between modules reads vs writes?

Some context specifically on RedisGraph:

The reduction in memory consumption is relevant for using RedisGraph in Redis 6.0.1 and up. In this case, during RDB save/load (BGSAVE and replication sequence) we split the graph into multiple keys for replication so the additional memory overhead of replicating a graph is significantly reduced. For 5.0.x we still pass the graph object as a whole key.

https://github.com/RedisGraph/RedisGraph/issues/1026

Hi @victorstewart

KeyDB understands when a key is looked up for write or read (e.g. lookupKeyWrite in the code). There is also a type system used to enforce this safety within KeyDB.

For modules this still applies but without the type safety. When you call RM_OpenKey() it is important that the mode is correct, if you intend to write to the key pass in REDISMODULE_WRITE. When you do so the MVCC timestamp will be updated.

In a split brain scenario upon reconnect the databases are merged, the deciding factor when there is a conflict is which has the newest MVCC timestamp. If you have related keys the best practice is to update them in a transaction to ensure they have the same MVCC timestamp and therefore are applied atomically.

1 Like

@jdsully

Does replication replicate write commands (as would get appended to an AOF file), or does it replicate the entire new value of the key (after getting the serialized output from the RDB Save command)?

Being able to just replicate the writes is very important to me. I’m not sure it’s possible for me to design around total value replication in many cases, when allowing the database to be written to across many datacenters globally-- especially under keyspace extinguishing restrictions.