How to determine number of eligible master nodes for Elasticsearch?
elasticsearch
Solution
You should have three dedicated master nodes, and your `discovery.zen.minimum_master_nodes` setting should be 2. Having more than 3 master nodes is superfluous and having less won't allow you to prevent the split brain problem. Only one master node is active at any time, the other two dedicated master nodes are waiting to become the master node if the current master node goes down.
For example, say you have 3 master nodes: node1 (active master node), node2 and node3. If a network partition prevents node1 being able to communicate with node2 and node3, node1 will lose its master status because `discovery.zen.minimum_master_nodes` is set to 2. It will also be blocked, so no operations can occur on the node. Node2 will become the active master node of the cluster as there are 2 possible master nodes available (node2 and node3). When the network partition is fixed, node1 will ping the other master nodes and join the cluster again as a waiting master node.
Problem
Running ES 1.4.3 So I have 4 cluster node. All nodes are configured as default (dual functionning data/master node). So here in default config we can have 4 eligible nodes right? So N/2 + 1 = discovery.zen.minimum_master_nodes: 3 Now lets say I decide to grow my cluster to 20 nodes and get dedicated physical boxes for master nodes... How many boxes should I buy for dedicated master nodes, now that the 20 nodes are strictly data nodes?