Docs / Clustering Guide

Clustering Guide

Maree-DB clustering is an Enterprise tier feature. Clusters provide automatic failover, Byzantine fault tolerance, and horizontal scalability.

Requirement: Enterprise tier licence. Minimum 3 nodes recommended (allows 1 node failure tolerance). 5 nodes recommended for Byzantine fault tolerance (f = 1, n = 5, f < n/3).

Step 1: Prepare Each Node

Install Maree-DB on each node. Ensure all nodes can communicate on the cluster port (default 7001).

# On each node: add cluster section to maree-db.toml [cluster] node_id = "node-1" # Unique ID for this node cluster_id = "prod-cluster-1" # Same on all nodes port = 7001 seeds = [ "node-1.internal:7001", "node-2.internal:7001", "node-3.internal:7001" ] byzantine_fault_tolerant = true # Requires 5+ nodes for f=1

Step 2: Start the Bootstrap Node

# On node-1 (bootstrap node) - membership comes from the [cluster] config above maree-db-server

Step 3: Join Remaining Nodes

# On node-2 and node-3 - seeds in the [cluster] config point them at node-1 maree-db-server

Step 4: Verify Cluster

# Check cluster status maree-db cluster status # Output example: # Cluster ID: prod-cluster-1 # Status: HEALTHY # Leader: node-1 # Nodes: 3/3 healthy # BFT Mode: OFF (need 5+ nodes) # List every node and its role/health: maree-db-server cluster nodes

Adding a Node

# 1. Install Maree-DB on the new node with cluster config # 2. Add node-4.internal to seeds in all configs # 3. Start the new node (its [cluster] seeds point at the existing cluster) maree-db-server # 4. Confirm it joined maree-db cluster status

Removing a Node

# Remove a node from the cluster maree-db cluster remove-node node-3

Rolling Upgrades (Zero Downtime)

# Upgrade nodes one at a time. Raft keeps a quorum serving traffic, # so the cluster stays available while each node restarts and rejoins. # Step 1: Confirm the cluster is healthy before you start maree-db cluster status # Step 2: Upgrade the node-1 binary and restart the service curl -sSL https://dist.mareedb.com/install.sh | bash sudo systemctl restart maree-db # Step 3: Confirm node-1 rejoined and is healthy, then move on maree-db cluster status # Repeat for node-2, node-3, etc.

Byzantine Fault Tolerance

In Byzantine fault tolerant mode (5+ nodes), Maree-DB can maintain correctness even if up to f nodes are compromised or behaving maliciously, where f < n/3 (n = number of nodes).

  • 5 nodes: tolerates 1 compromised node
  • 7 nodes: tolerates 2 compromised nodes
  • 10 nodes: tolerates 3 compromised nodes
# Enable BFT mode (requires 5+ nodes) [cluster] byzantine_fault_tolerant = true # Monitor consensus health / partition topology maree-db-server cluster topology

Cluster Monitoring

# Cluster status overview maree-db cluster status # Query replication lag (node_id, role, WAL lag, state) maree-db-server cluster replication # Split-brain / quorum protection is shown in the status overview maree-db-server cluster status