Clustering Guide — Maree-DB Docs

Cluster Architecture Overview

Maree-DB uses a consensus-based replication protocol where all write operations are committed to a quorum of nodes before acknowledging success. A 3-node cluster tolerates 1 node failure. A 7-node cluster with BFT mode active tolerates 2 Byzantine (malicious or corrupted) node failures.

Cluster Size	Quorum	Crash Faults Tolerated	Byzantine Faults Tolerated	BFT Mode
3 nodes	2	1	0	Not required
5 nodes	3	2	0	Not required
7 nodes	5	3	2	Required
10 nodes	7	4	3	Required

Rule of thumb: For Byzantine fault tolerance, you need at least 3f+1 nodes where f is the number of faulty nodes you want to tolerate. Standard CFT consensus requires only 2f+1 nodes.

Setting Up a 3-Node Cluster

This example uses three servers: node1 (192.168.1.10), node2 (192.168.1.11), node3 (192.168.1.12).

node1

192.168.1.10

Initial Leader

node2

192.168.1.11

Follower

node3

192.168.1.12

Follower

1

Install Maree-DB on all three nodes

Follow the Installation Guide on each server. Do not start the server yet.

2

Configure each node

/etc/mareedb/maree-db.toml (node1)

[cluster]

enabled = true

node_id = "node1"

peers = ["192.168.1.11:7001", "192.168.1.12:7001"]

consensus_port = 7001

replication_factor = 3

/etc/mareedb/maree-db.toml (node2)

[cluster]

enabled = true

node_id = "node2"

peers = ["192.168.1.10:7001", "192.168.1.12:7001"]

consensus_port = 7001

replication_factor = 3

3

Start all nodes

bash (all 3 nodes simultaneously)

# systemctl start mareedb

4

Verify cluster formation

bash (from any node)

$ maree-db-cli cluster status

Cluster: HEALTHY

Leader: node1 (192.168.1.10)

Members: 3/3 online

★ node1 192.168.1.10:7001 leader ok

node2 192.168.1.11:7001 follower ok

node3 192.168.1.12:7001 follower ok

Replication lag: max 2ms

Cluster operational. Quorum: 2/3.

Adding a Node

New nodes can be added to a running cluster without downtime. The new node bootstraps by syncing from the current leader.

bash

# 1. Configure the new node with cluster settings, start it

# systemctl start mareedb # on the new node

# 2. Add the new node to the cluster from any existing node

$ maree-db-cli cluster add-node 192.168.1.13:7001

Node 192.168.1.13 joining cluster...

Syncing data (12.4 GB)...

Node node4 (192.168.1.13) joined. Cluster: 4 nodes.

Removing a Node

bash

# Graceful removal (node is healthy)

$ maree-db-cli cluster remove-node node3

Draining node3... reassigning partitions...

node3 removed. Cluster: 3 nodes.

# Force removal (node is dead/unreachable)

$ maree-db-cli cluster remove-node node3 --force

node3 forcefully removed from cluster membership.

Warning: Forcing removal of a node that may still be running (network partition, not a crash) can cause a split-brain scenario. Only use --force when you are certain the node is permanently unavailable.

Rolling Upgrades

Upgrade the cluster to a new Maree-DB version without downtime by upgrading one node at a time. The cluster remains operational throughout.

bash

# For each follower node (repeat for each):

# maree-db-cli cluster drain node2 # stop new writes to this node

# systemctl stop mareedb # on node2

# cp maree-db-server-1.1.0 /usr/local/bin/maree-db-server

# systemctl start mareedb # on node2

node2 upgraded to 1.1.0 and rejoined cluster.

# Upgrade the leader last (triggers leader election)

# maree-db-cli cluster transfer-leadership # move leader to node2

Leader transferred to node2. node1 is now follower.

# systemctl stop mareedb # on node1 (old leader)

# cp maree-db-server-1.1.0 /usr/local/bin/maree-db-server

# systemctl start mareedb # on node1

All nodes running 1.1.0. Rolling upgrade complete.

Byzantine Fault Tolerant (BFT) Mode

BFT mode protects against nodes that are actively malicious, corrupted, or compromised — not just crashed. This is relevant for high-security deployments, financial infrastructure, and multi-organisation clusters where node trustworthiness cannot be guaranteed.

Requires: minimum 7 nodes (3f+1 where f=2). Enterprise licence. BFT mode cannot be enabled on clusters with fewer than 7 nodes.

maree-db.toml (all 7 nodes)

[cluster]

enabled = true

bft_mode = true

peers = [

"192.168.1.11:7001", "192.168.1.12:7001",

"192.168.1.13:7001", "192.168.1.14:7001",

"192.168.1.15:7001", "192.168.1.16:7001"

]

Verify BFT cluster

$ maree-db-cli cluster status

Cluster: HEALTHY (BFT mode active)

Members: 7/7 online

BFT tolerance: 2 Byzantine faults

All safety properties guaranteed.

Auto-Discovery

On trusted local networks, enable auto_discover = true in [cluster] configuration. Maree-DB nodes broadcast their presence via mDNS and automatically form a cluster without manual peer configuration.

maree-db.toml

[cluster]

enabled = true

auto_discover = true

# peers = [] — not needed when auto_discover is true

Security note: Auto-discovery uses mDNS (multicast DNS) which operates on the local network segment only. Do not enable auto-discovery on untrusted networks. For production, always specify peers explicitly and disable auto_discover.

Cluster Monitoring Dashboard

The built-in monitoring dashboard is accessible via the REST port at http://leader-node:8080/cluster. It provides real-time visibility into:

Node health and replication lag per node
Leader election history
Partition distribution across nodes
Write throughput and latency per node
TamperLock chain consistency across replicas
BFT consensus vote counts (BFT mode only)

CLI cluster monitoring

$ maree-db-cli cluster monitor --interval 2s

┌─────────────────────────────────────────────────────────┐

│ Maree-DB Cluster Monitor — refreshes every 2s │

├──────────┬───────────┬──────────┬───────────┬───────────┤

│ Node │ Role │ Lag │ Writes/s │ Status │

├──────────┼───────────┼──────────┼───────────┼───────────┤

│ node1 ★ │ leader │ — │ 14,203 │ ok │

│ node2 │ follower │ 1ms │ — │ ok │

│ node3 │ follower │ 2ms │ — │ ok │

└──────────┴───────────┴──────────┴───────────┴───────────┘