Retrieving the optimal number of clusters in R

Question

I have data for which I want to evaluate the optimal number of clusters according to the Gap statistic. I read the page on gap statistic in r which gives the following example: ``` gs.pam.RU <- clusGap(ruspini, FUN = pam1, K.max = 8, B = 500) gs.pam.RU ``` When I call `gs.pam.RU.Tab`, I get ``` Clustering Gap statistic ["clusGap"]. B=500 simulated reference sets, k = 1..8 --> Number of clusters (method 'firstSEmax', SE.factor=1): 4 logW E.logW gap SE.sim [1,] 7.187997 7.135307 -0.05268985 0.03729363 [2,] 6.628498 6.782815 0.15431689 0.04060489 [3,] 6.261660 6.569910 0.30825062 0.04296625 [4,] 5.692736 6.384584 0.69184777 0.04346588 [5,] 5.580999 6.238587 0.65758835 0.04245465 [6,] 5.500583 6.119701 0.61911779 0.04336084 [7,] 5.394195 6.016255 0.62205988 0.04243363 [8,] 5.320052 5.921086 0.60103416 0.04233645 ``` From which I want to retrieve the number of clusters. But, contrary to the pamk function which enables to get this number easily, I couldn't find a way to get this number usin…

Accepted Answer

The answer is in the output:

```
...
 --> Number of clusters (method 'firstSEmax', SE.factor=1): 4
...
```

This is the number of clusters producing the maximum value of `gap` (which is in row 4 of the table).

The arguments to `maxSE(...)` are the `gap` and `SE.sim`, respectively:

```
with(gs.pam.RU,maxSE(Tab[,"gap"],Tab[,"SE.sim"]))
# [1] 4
```

It is sometimes useful to plot `gap`, to see how well differentiated the clustering options are:

```
plot(gs.pam.RU)
gap.range <- range(gs.pam.RU$Tab[,"gap"])
lines(rep(which.max(gs.pam.RU$Tab[,"gap"]),2),gap.range, col="blue", lty=2)
```

Retrieving the optimal number of clusters in R

Solution

Problem