Introduction
In this blog we go through changing our vSAN service subnet without any downtime. In our case we also want to keep the vSAN vlan we are currently using.
vSAN is a sensitive part of a cluster. With a small cluster (3/4 hosts) or fault domains it could be difficult to change the subnet of a production vSAN environment.
In my environment we have a total of 3 hosts in the cluster. 3 hosts are all three in use because vSAN uses a 2 components and 1 witness structure. This means we can not easily change the ip address or Portgroup of the vSAN network.
Solution
Fortunately we have an solution for this. We can create a second vlan and subnet on our switch and create a temporary portgroup. After the portgroup is created we can create one extra VMkernel on every host.
This VMKernel will be used for vSAN services but for the new vlan/temporary subnet.
With this structure we can move our hosts to our temporary portgroup and change our primary portgroup. After we changed our primary portgroup we will move the hosts back to the primary portgroup and clean up our temporary portgroup.
Preparation
Let’s begin with the preparation of the following parts:
- Create new temporary vlan
- Create new temporary portgroup
- Configure a new VMKernel on all hosts of the cluster
Create new vlan and subnet on Switch
1. Login to your switch and create a new vlan with a new subnet. The size of the subnet does not matter.
2. After creating the vlan tag the vlan on the interfaces where the hosts are connected.
3. Once we have the vlans tagged we can create a portgroup on the DVS switch in vCenter.
Create temporary portgroup
4. Login to vCenter and create a new portgroup.
5. Give the Portgroup a name. Personally I name the portgroup something with “Temp”, this makes the cleanup afterwards easier.
6. Click on “Next”
7. Configure the VLAN part. Use the VLAN configured on the switches in step 1.
8. Click “Next”.
9. Click “Finish”
Create VMKernel port on the hosts in the cluster
10. Navigate to the hosts in the cluster.
11. Choose one host and go to “Configure” tab.
12. Under networking you will see “VMkernel adapters” click on it.
13. Click on “Add Networking”.
14. For the connection type Choose “VMkernel Network Adapter” and click “Next”.
15. For the target device search for the temporary portgroup you just created.
16. Select the temporary portgroup and click “Next”.
17. On the Portgroup properties choose the vSAN service and click “Next”.
18. On the IPv4 settings page choose “Use static IPv4 settings”.
19. Fill in the new subnet IP Address, Subnet mask and Default gateway for the first host and click “Next”.
20. Click on “Finish”.
21. Repeat the steps for the other hosts in the cluster. Every host should have a unique IP address.
22. Once we have all the hosts configured with the new VMKernel we can proceed with the actual move.
Move hosts to temporary VMKernel
Warning! Before starting with the solution, always make sure to put the hosts via vCenter in maintenance mode!
1. Click on the last host in the cluster.
2. Put the host in maintenance mode and choose “Ensure accessibility”.
3. Once the host is in maintenance mode go to “Configure” -> “VMkernel adapters”
4. Now we will disable the vSAN service on our primary vSAN VMKernel adapter.
5. Click the three dots and choose “Edit”.
6. Under “Enabled services” uncheck “vSAN”.
7. Click on “OK”.
8. Take the host out of maintenance mode.
9. Repeat this step for the other host. From last to first host.
Warning! In the process you will see some alarms. You can ignore those.
10. Once we have disabled the primary vSAN service on the primary VMKernel on all hosts we can start reconfiguring our primary VMKernel with the new subnet.
11. Reconfigure the primary subnet and/or vlan on the switches.
Move hosts back to primary VMKernel
Enable primary VMKernel on all hosts
1. Click on the last host in the cluster.
3. Go to “Configure” -> “VMkernel adapters”
4. Now we will enable the vSAN service and reconfigure the IPv4 settings of the primary vSAN VMKernel adapter with our new subnet.
5. Click the three dots and choose “Edit”.
6. Under “Enabled services” check “vSAN”.
7. On the left side click on “IPv4 Settings”.
8. Reconfigure the IPv4 Settings with our new subnet.
9. Click on “OK”.
10. Repeat this step for the other host. From last to first host.
11. now the primary VMKernel has vSAN enabled again we can disable the vSAN service on the temporary VMKernel again.
Disable temporary VMKernel on all hosts
Warning! Before starting with the solution, always make sure to put the hosts via vCenter in maintenance mode!
1. Click on the last host in the cluster.
2. Put the host in maintenance mode and choose “Ensure accessibility”.
3. Once the host is in maintenance mode go to “Configure” -> “VMkernel adapters”
4. Now we will disable the vSAN service on our temporary vSAN VMKernel adapter.
5. Click the three dots and choose “Edit”.
6. Under “Enabled services” uncheck “vSAN”.
7. Click on “OK”.
8. Take the host out of maintenance mode.
9. Repeat this step for the other host. From last to first host.
Warning! In the process you will see some alarms. You can ignore those.
Cleanup
After we have moved all hosts back to the primary vSAN service VMKernel and everything is running without issues we can start the cleanup.
We will clean up the following components:
- Our temporary VMKernel on our hosts.
- Our temporary Portgroup.
- Our temporary vlan on our switches (Including vlan tags on our interfaces).
