Awid Dashtgoli – Page 2

Show vSAN objects and components count on vSAN enabled clusters

Introduction

In this short blog we will show how to get the counts of vSAN objects and components. This can be useful to see which components are causing a issue when you almost hit the limit.

Limits
vSAN OSA has a components limit of 9000 per host.
vSAN ESA has a components limit of 27000 per host.

Limits can be reached for example with large VM’s or VCDA protection replications.

Getting the component count

First we need to start a SSH session to our vCenter.

1. Start a SSH session to the vCenter.
2. Login with “root”.
3. Now we run the following command:

					rvc 127.0.0.1

4. Now fill in the password of “administrator@vsphere.local”
5. Once it is loaded you will see the following:

0 /
1 127.0.0.1/

If you don’t see this use the following command:

ls

6. We want to go to the localhost (127.0.0.1). This can be a different number for you. For me it is number 1, so I will type the following:

					cd 1

7. Once we are in the localhost we can type the following to show the directories:

ls

8. Here we will see all the datacenters. Choose the correct datacenter. For me it is number 0, so I type the following:

					cd 0

9. Again we will show the directories by typing:

ls

10. Here we will see different directories. In our case we need the computers [host] directory. For me it is number 1. So I will type the following:

					cd 1

11. Again we will show the directories by typing:

ls

12. Here we can see all our clusters. Choose the cluster you want to see the objects and components of. For me this is number 8 so I will type the following:

					cd 8

13. We should now be at:

/127.0.0.1/t01pz03-s01-dc01/computers/t01pz03-s01-cl09

14. This is the location we need to be in. Now we run the following command:

					vsan.obj_status_report . -t

15. This command will show us all the objects and the components. The results will look something like:

2024-12-03 12:18:22 +0100: Querying all VMs on vSAN ...
2024-12-03 12:18:24 +0100: Querying DOM_OBJECT in the system from h001.demo.dashcenter.blog ...
2024-12-03 12:18:24 +0100: Querying DOM_OBJECT in the system from h002.demo.dashcenter.blog ...
2024-12-03 12:18:25 +0100: Querying DOM_OBJECT in the system from h003.demo.dashcenter.blog ...
2024-12-03 12:18:25 +0100: Querying DOM_OBJECT in the system from h004.demo.dashcenter.blog ...
2024-12-03 12:18:26 +0100: Querying DOM_OBJECT in the system from h005.demo.dashcenter.blog ...
2024-12-03 12:18:26 +0100: Querying DOM_OBJECT in the system from h006.demo.dashcenter.blog ...
2024-12-03 12:18:26 +0100: Querying DOM_OBJECT in the system from h007.demo.dashcenter.blog ...
2024-12-03 12:18:27 +0100: Querying DOM_OBJECT in the system from h008.demo.dashcenter.blog ...
2024-12-03 12:18:27 +0100: Querying DOM_OBJECT in the system from h009.demo.dashcenter.blog ...
2024-12-03 12:18:28 +0100: Querying all disks in the system from h010.demo.dashcenter.blog ...
2024-12-03 12:18:30 +0100: Querying all object versions in the system ...
/opt/vmware/rvc/lib/rvc/lib/vsanupgrade.rb:1052: warning: calling URI.open via Kernel#open is deprecated, call URI.open directly or use URI#open
2024-12-03 12:18:35 +0100: Got all the info, computing table ...
Histogram of component health for non-orphaned objects
+-------------------------------------+------------------------------+
| Num Healthy Comps / Total Num Comps | Num objects with such status |
+-------------------------------------+------------------------------+
| 4/4 (OK)                            |  203                         |
| 3/3 (OK)                            |  8640                        |
| 8/8 (OK)                            |  111                         |
| 24/24 (OK)                          |  1                           |
| 16/16 (OK)                          |  8                           |
| 5/5 (OK)                            |  104                         |
| 9/9 (OK)                            |  6                           |
| 7/7 (OK)                            |  4                           |
| 10/10 (OK)                          |  18                          |
| 6/6 (OK)                            |  21                          |
| 26/26 (OK)                          |  1                           |
| 32/32 (OK)                          |  1                           |
| 36/36 (OK)                          |  12                          |
| 76/76 (OK)                          |  1                           |
+-------------------------------------+------------------------------+
Total non-orphans: 9131
Histogram of component health for possibly orphaned objects
+-------------------------------------+------------------------------+
| Num Healthy Comps / Total Num Comps | Num objects with such status |
+-------------------------------------+------------------------------+
+-------------------------------------+------------------------------+
Total orphans: 0
Total v19 objects: 22290
+-------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------------------------+
| VM/Object                                                                                                                                       | objects | num healthy / total comps |
+-------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------------------------+
| *********-7CNs                                                                                                                                  | 2       |                           |
|    [*********-vsan01] 53101667-5cb7-0132-e6fb-78ac44b55210/*********-7CNs.vmx                                                                   |         | 3/3                       |
|    [*********-vsan01] 53101667-5cb7-0132-e6fb-78ac44b55210/*********-7CNs.vmdk                                                                  |         | 3/3                       |
| *********-pTVE                                                                                                                                  | 1       |                           |
|    [*********-vsan01] b9741667-1899-cc2b-84fe-78ac44b57670/*********-pTVE.vmx                                                                   |         | 3/3                       |
| *********-6M5Z                                                                                                                                  | 1       |                           |
|    [*********-vsan01] 52981767-f422-12a4-e270-78ac44b55210/*********-6M5Z-28c9bcf8.vswp                                                         |         | 3/3                       |
| *********-UBx2                                                                                                                                  | 2       |                           |
|    [*********-vsan01] 13991767-5c15-63e5-3c9a-78ac44b57670/*********-UBx2.vmdk                                                                  |         | 3/3                       |
|    [*********-vsan01] 13991767-5c15-63e5-3c9a-78ac44b57670/*********-UBx2-c623421f.vswp                                                         |         | 3/3                       |
| *********-Uavb                                                                                                                                  | 2       |                           |
+-------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------------------------+
+-------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------------------------+
| Secondary namespaces                                                                                                                            | 1       |                           |
+-------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------------------------+
| Unassociated objects                                                                                                                            | 8058    |                           |
|    e1e84e67-f4a9-2100-67e9-78ac44b7e060                                                                                                         |         | 4/4                       |
|    72bd4d67-e81f-6300-137e-78ac44b7e060                                                                                                         |         | 8/8                       |
|    0eda4d67-0625-b200-448f-78ac44b7e060                                                                                                         |         | 3/3                       |
|    37be4d67-3483-b200-a52b-78ac44b7e060                                                                                                         |         | 3/3                       |
|    beaf4d67-3eb4-3c01-bdba-78ac44b7e060                                                                                                         |         | 3/3                       |
|    d9e84e67-aa76-5101-cb8e-78ac44b7e060                                                                                                         |         | 3/3                       |
|    d8824e67-ca7a-a301-3052-78ac44b7e060                                                                                                         |         | 3/3                       |
|    f0114e67-1e94-d601-4401-78ac44b54fb0                                                                                                         |         | 3/3                       |
|    7dc94e67-9471-5d02-27a6-78ac44b7e060                                                                                                         |         | 3/3                       |
|    71bd4d67-d6db-7702-e122-78ac44b57670                                                                                                         |         | 3/3                       |
|    b4204e67-36e4-7a02-4c0d-78ac44b7e060                                                                                                         |         | 3/3                       |
|    cd9e4e67-0697-8202-9af3-78ac44b57670                                                                                                         |         | 3/3                       |
|    cdbd4d67-d206-ab02-ab73-78ac44b80710                                                                                                         |         | 3/3                       |
|    d8cb4d67-e8ff-de02-33dc-78ac44b7e060                                                                                                         |         | 3/3                       |
+-------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------------------------+
WARNING: Unassociated does NOT necessarily mean unused/unneeded.
Deleting unassociated objects may cause data loss!!!
You must read the following KB before you delete any unassociated objects!!!
https://kb.vmware.com/s/article/70726

+------------------------------------------------------------------+
| Legend: * = all unhealthy comps were deleted (disks present)     |
|         - = some unhealthy comps deleted, some not or can't tell |
|         no symbol = We cannot conclude any comps were deleted    |
+------------------------------------------------------------------+

I have shorten the output because it was to long, but this should give you a better understanding of the output.

Awid Dashtgoli

Understanding and Configuring Disaster Recovery on Cloud Director Availability

Introduction

In this blog we will go through the Disaster Recovery (DR) of Cloud Director Availability. We will configure Disaster Recovery but also show the possibilities and options.

Our Scenario

I have created a vAPP with 3 virtual machines. We want to create a Disaster Recovery (DR) to our second datacenter. We want to Disaster Recovery to have a RPO of 5 Minutes and a retention policy of 12 instances created every two hours.

This will give us 12 instances over 24 hours. After 24 hours I won’t need Disaster Recovery anymore, because I have backups in place.

Understanding and Configuring Disaster Recovery

Let’s start with configuring our Disaster Recovery.

1. Login to your Cloud Director portal.
2. On the left side under “More” you will see your Cloud Director Availability instances.
3. Click on it.
4. Now you will see Cloud Director Availability.
5. As you can see we don’t have any DR configured.
6. Click on “Outgoing Replications”.

7. Click on “Outgoing Replications”.
8. Select your “Destination site”.
9. Make sure you are on “vApp” and click on the “New Protection” icon.

10. Now you will see the “New Outgoing Protection” screen.
11. Select your “Destination site”.
12. Make sure you are on “vApp” and choose the vApp you want to protect.
13. Click on “Next”.

14. On the next screen you can choose the Destination VDC but also the Storage Tier you want to use.
15. Choose the correct VDC and Tier storage.
16. Click on “Next”.

Information! The “Advanced Datastore Settings” can be used to choose different storage tiers for every VM or even every Hard disk on the VM’s.

17. On this screen we can select a predefined SLA profile provided by your Cloud Provider or manually configure the SLA settings.
18. In our case we will configure our settings manually.

17. Select “Configure settings manually”.
18. Now we go through the “RPO” and “Retention Policy”.

Recovery Point Objective (RPO)
RPO stands for Recovery Point Objective. This will make sure you can go back 5 minutes in case of a Disaster Recovery. Cloud Director Availability will create every 5 minutes a instance and remove the old one.

After it created the instance it will consolidate the previous 5 minute instance. This makes sure it only needs the Delta’s instead of the full copy.

Retention Policy
The Retention Policy provides extra instances instead of the single 5 minutes instance. You can select the amount of instances, distance between every instance and the Unit the distance is configured at.

19. For the RPO we choose 5 minutes, because we want the most close instance in case of a DR.
20. For the Retention Policy we choose 12 instances with a Distance of 2 hours. This will make it possible to return back every 2 hours for a total of 24 hours with 12 instances.
21. Once you have configured the RPO and Retention Policy we can click on “Next”.

Information! Other options can be enabled. These are extra settings that can help you with creating the DR you need. The i icon next to the setting will give you more information about the setting.

22. Now we see a summary of all the settings we have chosen.
23. If everything is correct click on “Finish”.
24. It will start configuring the replication/DR.

25. Once the configuration is completed we can see our “Protection”.
26. Here you can see the configuration of the DR/Protection.

Status
On every vApp and VM you will see a green, blue or red dot.
A green dot means everything is healthy.
A blue dot means there is currently a instance created (5 minutes RPO).
A red dot means there is something wrong with the replication.

Information! Replications won’t work if the VM is “Shut Down”. The VM needs to be running to be able to replicate.

Testing the Protection

We can test our DR/Protection by running a test. Keep in mind this will cause a short outage, because the VM will be migrated to the other Datacenter.

If you run a “Test” Cloud Director Availability will first sync the vApp and VM’s before it performs a test. This is to make sure you won’t lose any data (because of the 5 Minute RPO) while testing.

Awid Dashtgoli

Understanding and Configuring Inter VRF Routing on NSX

Introduction

In this blog we will go through understanding and configuring Inter VRF Routing for specific use cases on NSX.

Inter VRF Routing is used for communication between different VRF Lite on a Edge Cluster. This can be useful for couple of scenarios.

One of the scenarios is having one VRF specifically for Internet routes and subnets. One of the benefits with this scenario is only using BGP connections to our physical environment to our Internet VRF and from there distributing it to other “Costumer VRF’s/Edges” within the Edge Cluster.

In the design below you will see a example of the construction:

In this design we see a BGP connection with a /24 prefix from our physical routers to our T0 Internet VRF. From there we split the /24 in to multiple subnets and give customers via the Inter VRF Routing a /27 and a /28 prefix.

Let’s start the configuration and show you the design within NSX.

Configuration

Create T0 Internet VRF

First we need to create our T0 Internet VRF on NSX.

1. Login to your NSX manager and go to “Networking” -> “Tier-0 Gateways”
2. Click on “Add Gateway” and choose “VRF”

3. Choose a name and connect it to a T0.
4. Click on “Save”.

5. You will see a message “Do you want to continue configuring?”. Choose “Yes”.

Configure Interfaces, Prefixes and BGP on T0 Internet VRF

Now we start with the configuration of the T0 Internet VRF. In this blog we will not go to deep in to BGP connections and how to set them up, but we will prepare the prefixes.

I will have a separate blog about BGP connections.

1. First create the interfaces for the BGP connections.
2. Now we will create the prefixes for the BGP connections to allow the /24 to be advertised from the inside.
3. Under “Routing” click on “IP Prefix Lists”.

4. Click on “Add IP Prefix List” and choose a name.
5. Click on “Set”.

6. I will start with the inbound prefixes. In this case I only add and permit the default route (0.0.0.0/0).
7. Fill in the network and put the Action on “Permit”.

8. Click on “Apply”.

9. Click on “Save”.

10. Next we will create the outbound prefix.
11. Follow the same steps and create a prefix list.
12. In this case we add our /24 public address.

13. Now we have created the prefixes we can create the BGP connections. The BGP connections will have the prefixes we just made for inbound and outbound.
14. Last we add our /24 support to “Route Aggregation. Click on “Route Aggregation”.

15. Click on “Add Prefix” and fill in your /24 public subnet. For “Summary” choose “Yes”.
16. Click on “Add” and after that on “Apply”.

17. Now we can click on “Close Edit”

Create and Configure Customer VRF

For the Customer VRF we follow the same steps as the Internet VRF, but we do not set up the interfaces and BGP connections. In this case only the prefix lists for inbound and outbound.

1. After creating the Customer VRF we will start creating the Prefix lists for inbound and outbound again. Only in this case we do not add the /24 to the outbound prefix but only the /27 (Customer 1) or /28 (Customer 2).

Configure Inter VRF

Now we will start the configuration of the Inter VRF Routing.

Create Route Maps

First we need to create Route Maps on all the VRF’s. In this case Internet VRF, Customer 1 VRF and Customer 2 VRF.

1. Start with the Internet VRF Route Maps.
2. Go the the Internet VRF click on the three dots and click on “Edit”.

3. Click on “Route Maps”

4. Click on “Add Route Map”
5. Choose a name for the inbound Route Map and click on “Set”

6. Click on “Add Match Criteria”
7. Put Action on “Permit” and click on “Set”

8. Search for the prefix you created in the previous steps. In this case the inbound prefix.
9. Select the prefix and click on “Apply”

10. Click on “Add” and after that on “Apply”

11. Click on “Save” to save the Inbound Route Map.
12. Now follow the same steps for the Outbound Route Map and don’t forget to choose the Outbound Prefix in step 8.
13. Now follow the same steps for the Customer VRF’s.

Create Inter VRF Routing

The last part is to create the Inter VRF Routing. This part is pretty easy.

1. Let’s again start with the Internet VRF. Click on “Inter VRF Routing”

2. Click on “Add Inter VRF Routing”
3. Choose the “Connected Gateway” in this case Customer 1.
4. Click on “Set”.

5. Click on the three dots and click on “Edit”.

6. Enable “BGP Route Leaking” and choose the IN and OUT filters.
7. Click on “Add”

8. Verify the settings and click on “Apply”

9. Click on “Save”.
10. Now the Inter VRF Routing is set up on the Internet VRF.
11. Click on “Close”.

12. Do the same for the Customer 2 VRF. Follow the same steps for Customer 2 VRF.

13. Now we have the Internet VRF side ready we need to configure it also on the Customer VRF’s. Follow the same steps only in this case the “Connected Gateway” is the Internet VRF and not the Customer VRF
14. Go to the Customer 1 VRF and click on “Edit”
15. Choose “Inter VRF Routing” and follow the same steps to create the Inter VRF Routing.
16. Repeat the same steps for Customer 2 VRF.

Validation

After all Inter VRF Routing have been configured we can validate everything. These are two steps:

1. First we can check the Inter VRF Routing on the Interfaces of the VRF’s. You will see a extra Interface named: INTERVRF
2. After we have advertised the /27 and /28 from the Customer VRF’s we can check on the Internet VRF if the Routes are also visible in the Routing Table.

Awid Dashtgoli

Fix alarm “Registration of third-party IO filter storage providers fails”

Introduction

In this blog we will show to fix the alarm “Registration of third-party IO filter storage providers fails” on ESXI hosts in VMware vCenter.

Issue

If we go to a cluster within VMware vCenter we will see the following alarm: “Registration/unregistration of third-party IO filter storage provider fails on a host.”.
When we go further in to the alarm we will need to go to “Monitor” and “Triggered Alarms”.

Understanding the Cause

This alarm is caused by the IO Filter on the ESXi host is not correctly registered with the ESXi host certificate.
To get a better understanding we can look in to the logs.

The logs can be found under /var/log/iofiltervpd.log or else /var/run/log/ in the log bundle.

We will look for something like:
iofiltervpd[67088]: run:159:SSL Connection error 30 : SSL_ERROR_SSL error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown

Solution

To solve this issue we need to refresh the IO Filter Registration on the ESXi Host, but before we can do this let’s first verify if the Host Certificate and Certificate chain is OK on the ESXi Host.

Verify Certificates on ESXi Host

To verify if the Certificate and Certificate chain of the ESXi Host is OK we need to follow these steps:

1. Login to vCenter
2. Click on the ESXi host with the alarm
3. Click on “Configure”
4. Scroll down to under “System” you will find “Certificate”. Click on it.
5. Now you will see the “Status”. This should be “Good”

Enable SSH on ESXi Host (Optional)

1. Go to vCenter and login.

2. Choose the host and go to “Configure” -> “Services”. Find “SSH” and click on “START”

Refreshing the IO Filter Registration

After we have validated the Certificate and Certificate Chain on the ESXi Host we can proceed with refreshing the IO Filter Registration.

1. Open a SSH session to your ESXi Host.
2. Run the following command:

					/usr/lib/vmware/iofilter/bin/iofvp-ctrl-app -r

3. After running the command the alarm should be resolved. If this is not the case you can click “Reset to Green”. The alarm should not return anymore.

Awid Dashtgoli

Fix “Host TPM attestation alarm” VMware vCenter

Introduction

In this blog we will show to fix the alarm “Host TPM attestation” on ESXI hosts in VMware vCenter.

Issue

If we go to a cluster within VMware vCenter we will see the following alarm: “Host TPM attestation alarm”.
When we go further in to the alarm we will need to go to “Monitor” and “Triggered Alarms”.

Understanding the Cause

When we go further in to the alarm we will need to go to “Monitor” and “Triggered Alarms”. Here you will see all the current alarms. The one we are looking for is also showing here.

Now we click on the arrow to expand the alarm.
Here we will see information about the alarm, but also the cause of the alarm.

In our case the alarm is caused by not having “Secure Boot” enabled.

Solution

Enable Secure Boot on host

First we need to enable secure boot on our host. This can be done in either ILO/IDRAC or via the BIOS.

For the HPE ProLiant DL Series we will use BIOS.
For the DELL PowerEdge R Series we will use IDRAC.

Warning! Before starting with the solution, always make sure to put the hosts via vCenter in maintenance mode!

HPE ProLiant DL Series (BIOS)

To enter the bios of our HPE server we need to access the console via ILO.

1. browse to the ILO of the host and login to the ILO.

2. Look at the information and make sure “Trusted Platform Module” (TPM) is “Present: Enabled” and the “Module Type” is Present (in my case “TPM 2.0”).

3. Click on the console and choose “HTML5 Console” (I am using ILO 6, your interface may vary)

4. Head back to the vCenter to give the host a reboot.

5. Log a reason for the reboot and click on “OK”.

6. Now go back to the ILO and watch the console.

7. Keep a eye on the Console. Once you see the option for “F9” “System Utilities” press “F9”

8. Once the “System Utilities” has started we choose “System Configuration”.

9. Next we go in to “BIOS/Platform Configuration (RBSU)”.

10. Now we choose “Server Security”.

11. Now we have two options.
If on step 2 the “Trusted Platform Module” (TPM) was not on “Present:Enabled”, but on “Present:Disabled” we can enable this under “Trusted Platform Module Options”.

If on step 2 the “Trusted Platform Module” (TPM) was on “Present:Enabled” select “Secure Boot Settings” and go to step 14.

12. Enable the TPM and click “F10: Save”.

13. Next we will enable “Secure Boot”. Go to “Secure Boot Settings”.

14. Now click on the “Attempt Secure Boot” option and choose “Enabled”.

15. You will get the following message. Click on “OK”

16. Now you will see “Reboot Required”. Click on “Exit”.

17. Choose “F12: Save and Exit”.

18. You will get a message. Click on “OK”.

19. You will get a reboot message. Click on “Reboot”.

20. Go back to vCenter and wait until the server has rebooted.

DELL PowerEdge R Series (IDRAC)

In this part we will change the “Secure Boot” via IDRAC.

1. browse to the IDRAC of the host and login to the IDRAC.

2. Go to “Configuration” and choose “BIOS Settings”.

3. Now go to “System Security”.

4. Make sure “TPM Security” is “On”.

5. Now change “Secure Boot” to “Enabled” and click on “Apply”.

6. Now click on “At Next Reboot”.

7. Now we will go back to vCenter and reboot the host.

8. Log a reason for the reboot and click on “OK”.

9. The host will reboot and you will see the BIOS changes through IDRAC.

Enable Secure Boot on ESXi OS

In this part we will enable “Secure Boot” on the ESXi host. First we need to enable SSH to connect to the host. After that we can enable “Secure Boot”.

Enable SSH on ESXi Host

1. Go to vCenter and login.

2. Choose the host and go to “Configure” -> “Services”. Find “SSH” and click on “START”

Enable Secure Boot through CLI

1. Start a SSH session to the ESXi Host.

2. List the current settings by running:

					esxcli system settings encryption get

3. You will get a result like this:

					   Mode: TPM
   Require Executables Only From Installed VIBs: false
   Require Secure Boot: false

4. If “Mode” appears as “NONE” run the following command:

					esxcli system settings encryption set --mode=TPM

5. Now enable “Secure Boot” by running:

					esxcli system settings encryption set --require-secure-boot=T

6. Verify the change and Confirm that “Required Secure Boot” displays true by running:

					esxcli system settings encryption get

7. You will get a result like this:

					   Mode: TPM
   Require Executables Only From Installed VIBs: false
   Require Secure Boot: true

8. Now we need to save the settings by running:

					/bin/backup.sh 0

9. Reboot the server to make sure settings are applied and in place. Run the command:

					reboot

Awid Dashtgoli

Fix “Index: 0, Size: 0” VMware Cloud Director Error

Introduction

In this blog we will show to fix the error “Index:0, Size: 0” issue in VMware Cloud Director. This issue will prevent users to make changes on the Organization Tenant VM’s.

Issue

If we go to a organization within VMware Cloud Director and we want to make changes (cpu, ram, hard disks or anything else) to a VM we will receive a error “[ f14cfa58-9afa-4730-98ce-0f69347775f9 ] Index 0 out of bounds for length 0”.

Solution

Warning! Before starting with the solution, always make sure to make a snapshot of all the VMware Cloud Director Cells!

Obtaining "dstore_moref" information from impacted VM's

First we need to obtain all the impacted vm’s. We can do this by accessing the Cloud Director database tables. Impacted vm’s will have a empty “dstore_moref” column.

1. Start a SSH session to the Primary Cloud Director Cell.

2. Switch to the “Postgres” user by running:

					su - postgres

3. Start Postgres and open the “vcloud” table by running:

					psql vcloud

4. Obtain all the vm’s that have a empty “dstore_moref” value by running:
You will get a list of all the vm’s with there associated id.

					select vm.id,vm.name,datastore_inv.moref  FROM vm
inner join vapp_vm on vapp_vm.svm_id = vm.id
inner join vm_inv on vm_inv.moref = vm.moref
inner join datastore_inv on datastore_inv.vc_display_name = (substring(vm.location_path,2,(POSITION(']' in vm.location_path))-2))
where vm.dstore_moref is NULL and vm_inv.is_deleted is false;

5. Now we have a complete list of all the impacted vm’s with their id, name and moref datastore.

Here is a output example:

					
                  id                  |                       name                       |      moref
--------------------------------------+--------------------------------------------------+-----------------
 d6259566a-9826-8845-98b9-9a0b445b803c | dash-demovm-m6g4               | datastore-1935

Fixing the issue on a per-VM basis.

In this part we will fix the issue on a per-vm basis. This can be useful in cases where a faulty outcome is not tolerated or if you want to test the results for the first time.

1. Start a SSH session to the Primary Cloud Director Cell.

2. Switch to the “Postgres” user by running:

					su - postgres

3. Start Postgres and open the “vcloud” table by running:

					psql vcloud

4. If your list of impacted vm’s from the previous step is to big to find your vm, you can filter one specific vm by running:
Replace the ‘%dash-demo%’ with your vm name.

					select 'update vm set dstore_moref = ' || '''' || datastore_inv.moref || '''' || ' where id = ' || '''' || vm.id || '''' || ';' from vm
inner join vapp_vm on vapp_vm.svm_id = vm.id
inner join vm_inv on vm_inv.moref = vm.moref
inner join datastore_inv on datastore_inv.vc_display_name = (substring(vm.location_path,2,(POSITION(']' in vm.location_path))-2))
where vm.dstore_moref is NULL and vm_inv.is_deleted is false and vm.name like '%dash-demo%';

5. Now we receive the command to implement the fix.
It should look something like this:

					update vm set dstore_moref = 'datastore-1935' where id = 'd6259566a-9826-8845-98b9-9a0b445b803c';

An alternative method is to run the following command:
This will allow you to create the command manually.
Replace the ‘%dash-demo%’ with your vm name.

					select vm.id,vm.name,datastore_inv.moref  FROM vm
inner join vapp_vm on vapp_vm.svm_id = vm.id
inner join vm_inv on vm_inv.moref = vm.moref
inner join datastore_inv on datastore_inv.vc_display_name = (substring(vm.location_path,2,(POSITION(']' in vm.location_path))-2))
where vm.dstore_moref is NULL and vm_inv.is_deleted is false and vm.name like  '%dash-demo%';

Now we have all the information we can create the fix command:
Copy the datastore name from the previous result and replace the ‘datastore-1935’ below.
Copy the id from the previous result and replace the ‘d6259566a-9826-8845-98b9-9a0b445b803c’ below.

					update vm set dstore_moref = 'datastore-1935' where id = 'd6259566a-9826-8845-98b9-9a0b445b803c';

6. Copy the command and paste it in the command line.

7. Run this command.

8. You will get a response like “UPDATE 1”. This means there is 1 update performed in the database.

9. Go back to Cloud Director Organization and try to change cpu, ram or hard disk settings on the vm.

10. Everything should update fine now without any errors.

Fixing the issue for all VM's at once.

In this part we will fix the issue for all vm’s at once. After you have tested one vm and it is safe to perform this on all the vm’s you can use this guide.

1. Start a SSH session to the Primary Cloud Director Cell.

2. Switch to the “Postgres” user by running:

					su - postgres

3. Start Postgres and open the “vcloud” table by running:

					psql vcloud

4. Get all the commands at once by running:

					select 'update vm set dstore_moref = ' || '''' || datastore_inv.moref || '''' || ' where id = ' || '''' || vm.id || '''' || ';' from vm
inner join vapp_vm on vapp_vm.svm_id = vm.id
inner join vm_inv on vm_inv.moref = vm.moref
inner join datastore_inv on datastore_inv.vc_display_name = (substring(vm.location_path,2,(POSITION(']' in vm.location_path))-2))
where vm.dstore_moref is NULL and vm_inv.is_deleted is false;

5. Now we receive all the commands to implement the fix for all vm’s at once.

					update vm set dstore_moref = 'datastore-1935' where id = 'd6259566a-9826-8845-98b9-9a0b445b803c';
update vm set dstore_moref = 'datastore-1935' where id = 'd6259566a-9826-8845-98b9-9a0b445b803c';
update vm set dstore_moref = 'datastore-1935' where id = 'd6259566a-9826-8845-98b9-9a0b445b803c';

6. Copy the complete output.

7. Run this commands.

8. You will get a response like “UPDATE” with a number behind it. The number represents the number of updates, that have been performed in the database.

9. Go back to Cloud Director Organization and try to change cpu, ram or hard disk settings on one of the vm’s.

10. Everything should update fine now without any errors.

Awid Dashtgoli

Upgrade VMware NSX Advanced Load balancer

Introduction

In this blog we will show how to upgrade VMware NSX Advanced Load Balancer.

Preparation

Before we can proceed to the upgrade we first need to prepare everything.

First we need to download the software from the VMware website. After downloading we can upload the software on VMware NSX Advanced Load Balancer.

Upload the Software

1. Login to the VMware NSX Advanced Load Balancer UI.

2. Access the NSX Advanced Load Balancer interface and go to Administration > Controller > Software. Select “Upload From Computer” to transfer the NSX Advanced Load Balancer software to the Controller.

3. After selecting the file, the software image upload begins. The progress of the upload is shown on the UI.

4. Once the image upload is finished, the software package will be available under the “Software” tab.

Upgrade the NSX Advanced Load Balancer System

1. Login to the VMware NSX Advanced Load Balancer UI.

2. Go to Administration > Controller > System Update, choose the image uploaded in the previous step, and click “UPGRADE” to initiate the upgrade process.

3. Check the “Upgrade All Service Engine Groups” box to update the SE groups along with the Controller upgrade. Click “Continue” to proceed with the software upgrades for the Controller and SE groups.

4. The next screen will appear for final checks before proceeding with the upgrade.

5. When prompted, confirm that a configuration backup has been completed.

6. The update process progress can be viewed on the UI in the “In Progress” section.

7. After the upgrade process is finished, the latest software version will be listed under Administration > Controller > System Update. The current tag will be shown next to the updated software version.

8. After the Controller upgrade is successful, the Service Engine group upgrade will begin, as it was selected in step 2. The following screenshot displays the message regarding the SE group update status.

9. After the SE group update is successful, the upgrade status changes to “successful,” as illustrated below.

Whenever an SE group is upgraded with the “Action to take on SEG update failure” on “Suspend” and an issue is encountered, the upgrade process for that SE group is suspended. After the issue is resolved through manual intervention, use the following command to resume the upgrade:

					resume segroup se_group_refs <se-group-name>

Replace <se-group-name> with your Service Engine Group that is suspended.

Awid Dashtgoli

Adding & Removing DNS server and NTP server on VMware NSX Manager or NSX Edge

Introduction

In this blog we will show how to update the NTP and DNS servers on your NSX Manager or NSX Edge.

DNS server

In this section we will first list our current DNS servers after that we will add our new DNS servers and remove our old DNS servers.

List DNS servers

Before we can add or remove DNS servers we will first need to list the current DNS servers. We can perform this by following these steps:

1. SSH in to your NSX Manager or NSX Edge with the Admin account.

2. Run the following command:

					get name-servers

3. You will get a result like this:

					Sun Jan 14 2024 CEST 21:12:59.787
172.16.2.1
172.16.2.2

Add DNS servers

Now we now the current DNS Servers we can start adding our new DNS servers. We can perform this by following these steps:

1. SSH in to your NSX Manager or NSX Edge with the Admin account.

2. Run the following command:

					set name-servers 172.16.100.1
set name-servers 172.16.100.2

3. Now validate the DNS Servers:

					get name-servers

You will get a result like this:

					Sun Jan 14 2024 CEST 21:17:12.249
172.16.2.1
172.16.2.2
172.16.100.1
172.16.100.2

Remove DNS servers

After we have added our new DNS servers we can remove the old DNS servers. We can perform this by following these steps:

1. SSH in to your NSX Manager or NSX Edge with the Admin account.

2. Run the following command:

					del name-servers 172.16.2.1
del name-servers 172.16.2.2

3. Now validate the DNS Servers:

					get name-servers

You will get a result like this:

					Sun Jan 14 2024 CEST 21:19:36.015
172.16.100.1
172.16.100.2

NTP server

In this section we will first list our current NTP servers after that we will add our new NTP servers and remove our old NTP servers.

List NTP servers

Before we can add or remove NTP servers we will first need to list the current NTP servers. We can perform this by following these steps:

1. SSH in to your NSX Manager or NSX Edge with the Admin account.

2. Run the following command:

					get ntp-servers

3. You will get a result like this:

					Sun Jan 14 2024 CEST 21:22:29.872
172.16.2.1
172.16.2.2

Add NTP servers

Now we now the current NTP Servers we can start adding our new NTP servers. We can perform this by following these steps:

1. SSH in to your NSX Manager or NSX Edge with the Admin account.

2. Run the following command:

					set ntp-servers 172.16.100.1
set ntp-servers 172.16.100.2

3. Now validate the NTP Servers:

					get ntp-servers

You will get a result like this:

					Sun Jan 14 2024 CEST 21:23:05.592
172.16.2.1
172.16.2.2
172.16.100.1
172.16.100.2

Remove NTP servers

After we have added our new NTP servers we can remove the old NTP servers. We can perform this by following these steps:

1. SSH in to your NSX Manager or NSX Edge with the Admin account.

2. Run the following command:

					del ntp-servers 172.16.2.1
del ntp-servers 172.16.2.2

3. Now validate the NTP Servers:

					get ntp-servers

You will get a result like this:

					Sun Jan 14 2024 CEST 21:24:48.847
172.16.100.1
172.16.100.2

Awid Dashtgoli