In HarperDB 4.4, we introduced a new built-in replication system, “Plexus.” This system provides substantial performance, security, and reliability improvements. Plexus eliminates the need to go through a message broker and instead implements direct connections between nodes. This facilitates optimizations down to the TCP level, highly secure mTLS connection, and robust consistency tracking. This highly optimized system for delivery of replicated writes has yielded significant performance gains. Although many factors create variability in test results, we are confident that results show an average 50% increase in replicated write performance over HarperDB 4.3.
The tests specifically focused on measuring throughput for replicated writes and do not reflect write performance for single standalone nodes. In all, 12 different tests were performed across 4, 5, 6, and 8 node clusters, with all writes fully replicated to all nodes. Tests were further divided into two groups, one with random primary keys (Random IDs) and the second with sequential keys (Sequential IDs), to understand variability across use cases.
Given the small sample size and the hundreds of potential unseen performance variables at play when using cloud virtual machines, the 6-node Random IDs result was removed after it demonstrated a 121% increase in performance. With outliers removed, the remaining results showed a mean performance improvement of 51% with a standard deviation of 9%.
It is worth noting that sharding was not enabled for these tests. With sharding enabled, higher overall write throughput can be expected across the system, as writes are not committed to each independent node.
Benchmarks utilized a multi-node HarperDB cluster and compared database write performance given the following specifications:
