Stretched cluster to multi node vSan cluster 

I assume most of you are aware about stretched cluster and how the component splits between the hosts/witness appliance. In this blog, I am going to walk you through the steps to convert a 2 vSan stretched cluster to a 3 node vSan cluster.

What is a witness appliance in Stretched cluster and how does it help:

In stretched cluster, Instead of using a dedicated physical ESXi host as a witness host, you can deploy the vSAN witness appliance. The appliance is a preconfigured virtual machine that runs ESXi and is distributed as OVA file. Unlike a general purpose ESXi host, the witness appliance does not run any vm's, Its purpose is to serve as vSAN witness i.e., all the witness components are configured in witness appliance disk groups.

The vSAN witness appliance includes two preconfigured network adapters. You must change the configuration of the second adapter so that the appliance can connect to the vSAN network.

Let's proceed now with steps to convert a stretched cluster to multi node:

Info to be collected:

1. Check the current health state of a vSan cluster i.e., resync, any errors, inaccessible objects, drive failure, predictive drive failures, current cluster resource utilisation.
2. If no issues seen, then run the esxcli vsan cluster get to collect the Sub-Cluster Member Count by taking ssh to any of the host or witness appliance in vSan cluster. 
3. Deploy new host, add to cluster and should be able to reach from other vSan hosts in cluster.
4. Do all the basic checks for new host i.e., drive and controller health etc.,
5. Though the host is part of cluster, doesn't mean that it will automatically be part of vSan cluster, find the Sub-Cluster Member Count and Sub-Cluster UUID using command esxcli vsan cluster get, count should be 4 otherwise add the host to vSan cluster manually.
6. In case of 6.2 version or before, then make sure to collect the Agent Group Multicast Address and Master Group Multicast Address from existing hosts of vSan cluster using the command esxcli vsan network list. Run the command in new host to modify the multicast ip using the command: 

esxcli vsan network ipv4 set -i <vsan kernel adapter device name> -d <Agent Group Multicast Address> -u <Master Group Multicast Address>

Ex: esxcli vsan network ipv4 set -i vmk5 -d 228.0.0.169 -u 228.0.0.170

   7. If the count is 1 on new host, then run the command on new host to join vSan cluster manually using the command esxcli vsan cluster join -u <Sub-Cluster UUID>

8. Re use the command esxcli vsan cluster get in new host and Sub-Cluster Member Count should now be 4.

9. vSan stretched cluster looks as shown below:

Below steps show the removal of vSan stretched Cluster. 

1. Disable the stretched configuration of the cluster and click on yes to confirm the same.

2. Now the fault domains (fd 1 and fd 2) gets removed and witness component gets rebuild over to the newly added to host in the cluster, this task generally triggers resync and size depends on the num of vm's in cluster.

3.  Run esxcli vsan cluster get by taking ssh to any of the host in vSan cluster and Sub-Cluster Member Count should show as 3. 

4. After confirming everything is healthy, there is no point go having the witness appliance sitting there just taking the resources without any purpose. So, I would remove the witness appliance from cluster wherever it is added to and delete the appliance.

Thanks for reading and let me know for any questions / suggestions by commenting below.