Add a Workload Cluster to an Oracle Cloud VMware Solution Software-Defined Data Center

Introduction

Oracle Cloud VMware Solution offers multi-cluster Software-Defined Data Center (SDDCs), empowering you to add new workload clusters to your existing SDDC. This significantly enhances resource management, cost efficiency, and flexibility, allowing you to scale your cloud resources as needed.

This tutorial guides you through the entire process of adding a standard shaped workload cluster to your existing SDDC. By following tasks, you will be able to:

• Create workload clusters in the Oracle Cloud Infrastructure (OCI) Console.

• Configure clusters seamlessly within vCenter, NSX Manager and HCX Manager.

Ready to add power and flexibility to your cloud environment. Let’s start building your first workload cluster.

Objectives

• Add a workload cluster to an existing Oracle Cloud VMware Solution SDDC in OCI.

Prerequisites

• Understanding of Oracle Cloud VMware Solution: Get Started with Oracle Cloud VMware Solution document to gain a comprehensive understanding of the platform and its functionalities. This will help you to navigate in this tutorial more efficiently.

• SDDC: To add a workload cluster to an SDDC, it needs to pre-exist with at least one Unified Management Cluster within your OCI Console. This Unified Management Cluster provides essential services for managing the SDDC and your workloads. To create a new SDDC, see Create a Multi-Cluster Oracle Cloud VMware Solution SDDC.

• Access to VMware Management Tools: You must have valid credentials and access to the vCenter, NSX Manager, and HCX Manager associated with your existing SDDC.

• Administrative Privileges: Ensure you hold the necessary permissions within OCI, vCenter, NSX Manager, and HCX Manager to perform all tasks described in this tutorial. This includes managing SDDC, hosts, clusters, networking configurations, and datastores.

Task 1: Create a Cluster in the OCI Console

1. Log in to the OCI Console and navigate to the SDDC Details page.

   

2. Click Add a Workload Cluster to begin the workflow and complete the following sections.

   1. Define Host Details

      • Cluster Name: Provide a unique name (max. 16 characters) for the cluster. (e.g., Cls-3-Std3-DB).

      • Availability Domain (AD): Choose a desired AD (Multi-AD requires Dense shape hosts).

      • Release Name: Select the appropriate VMware Software version from the drop-down list.

      • Number of ESXi Hosts: Enter the desired number of hosts (Refer to minimum requirements for each compute shape).

      • Prefix for ESXi Hosts(Optional): Provide a prefix for the ESXi host names (different from the cluster name).

      • Capacity Type: Choose On-demand or Reserved capacity.

      • Cluster Hardware Type: Select a Standard-shape or Dense-shape as per your requirements by selecting Change Shape.

      • Select the number of OCPU cores: Choose the desired OCPUs per host (if applicable).

      • Shielded Instances: Enable or disable shielded instances.

      • Pricing Interval Commitment: Select the desired commitment type and select the checkbox to confirm.

      • Review all the details and click Next to proceed.

      

   2. Configure Networking

      • VCN: The cluster must reside in the same VCN as the SDDC and is pre-selected.

      • Subnets and VLANs: Choose between creating new ones or using existing ones.

        

      • SDDC Cluster Network: Enter an unused CIDR block for the new cluster.

        

      • Workload Network (Optional): Provide a workload network (if required).

        

      • Review all the details and click Next to proceed.

   3. Datastore Configuration (Standard Shape Only)

      • Click Create block volume to add a datastore. In this tutorial, we added a couple of datastores with varying sizes and performance characteristics to meet our storage needs. More datastores can be added manually after the cluster is created as day 2 operations.

        

      • Review all the details and click Next to proceed.

   4. Set up Notifications

      • Select the desired notification options or disable them. For this tutorial, we will disable notifications.

      • Click Next to proceed.

      

   5. Review and Submit

      • Validate all information in the Review section.

      • Click Edit to make required changes in the respective section.

        

      • Click Submit to begin cluster creation.

3. Monitor the cluster provisioning until successful completion.

   1. View the summary of provisioning resources.

      

   2. Navigate to the Work Requests section of the SDDC details page.

      

   3. Track the Create Cluster operation and its progress status.

   4. Review log messages for detailed task breakdown.

      

   5. The cluster creation process takes around 25-30 minutes. Monitor the progress until the cluster status changes to ACTIVE.

      

   6. Similarly, the SDDC state should also have changed to ACTIVE.

      

4. Gather information for next steps.

   1. Note down the FQDNs of the newly added ESXi hosts.

      

   2. Navigate to Networking in the OCI Console and select your VCN. Within the VCN, access the VLANs section to find the details (VLAN ID, CIDR) of the new cluster’s VLAN. Note these for future configuration steps.

      

   3. Also document the details of vMotion and Replication VLANs of the Unified Management Cluster.

      

Task 2: Add Hosts and Configure a New Cluster in vCenter

Task 2.1: Add Hosts to vCenter Inventory

1. Go to the vCenter Client, right-click the Datacenter and click Add Host.

   

2. In the Add Host window, enter the following details.

   • Name and Location: Provide the FQDN of the first host that was documented in the previous section.

     

   • Connection Settings: Input the username and password of the ESXi host. For adding new hosts to vCenter the username is root and the password is the same as the initial password of your vCenter displayed on the SDDC details page.

     

   • Select Next to verify the host certificate to continue with the process.

     

   • Host Summary: Review all details of the host and click Next.

     

   • Host Lifecycle: Uncheck the Manage host with an image option. This is regarding vLCM managing the host lifecycle. For this document, we will disable this option.

     

   • Assign License: Select the existing license and make sure the validation is successful.

     

   • Lockdown mode: Select the desired lockdown mode (e.g., Normal which is Oracle Cloud VMware Solution default option).

     

   • VM Location: Choose the Datacenter for VM location and click Next.

     

   • Review settings and click Finish to add the host.

     

3. Continue to add all the other hosts into the datacenter inventory.

Task 2.2: Place Hosts in Maintenance Mode

• Before proceeding with further configuration, ensure all newly added hosts are placed in Maintenance Mode. Right-click each host individually, select Maintenance Mode, and choose Enter Maintenance Mode.

  

• Monitor the progress of each host until it successfully enters Maintenance Mode. This ensures smooth completion of subsequent configuration tasks.

Task 2.3: Create a Cluster in vCenter

1. In the hosts and clusters view, right-click on the Datacenter and select New Cluster.

2. Enter the following details in the respective sections.

   • Basic Information

     • Name: Enter a name. It is recommended to use the same name as in OCI i.e., Cls-3-Std3-DB.

     • Location: This should default to the Datacenter.

     • Enable vSphere DRS and vSphere HA.

     • vSAN: We will disable vSAN for this cluster as this is a Virtual Machine File System (VMFS) based cluster utilizing OCI Block storage as the Primary Datastore.

       Note: If you deployed a Dense shaped cluster in OCI, select vSAN and do not enable ESA as this is not a supported feature with Oracle Cloud VMware Solution.

     • Uncheck the vLCM management options, Manage all hosts in the cluster with a single image and Manage configuration at a cluster level. We will describe these in a different document. Select Next to continue.

     

   • Review settings and click Finish to create the cluster.

     

Task 2.4: Create a dedicated Distributed Switch (vDS) for the Cluster

To configure a new cluster, we need to first create a dedicated Distributed Switch and appropriate Distributed Port Groups for the cluster.

1. In vCenter, select the Datacenter within the Networking view.

2. To create a New Distributed Switch, choose Distributed Switch and click New Distributed Switch from the Actions drop-down list.

   

3. Enter New Distributed switch details as described below.

   • Name and Location: Enter a descriptive name (e.g., Cls-3-Std3-DB-DSwitch) and validate that the location is set to the Datacenter.

     

   • Select Version: Match the version used during the initial SDDC deployment (e.g., VMware Software version 8.0).

     

   • Configure Settings: Modify the Number of uplinks to 2 and adjust any other options as shown.

     

   • Review the details and click Finish to create the new vDS.

     

4. Navigate to the vDS properties and edit the MTU value to 9000 bytes.

   

Task 2.5: Configure Distributed Port Groups for the Cluster

Once the vDS is created, we need to configure separate Distributed Port Groups for various network types within the cluster. These Distributed Port Groups will handle different types of traffic and ensure proper isolation and security. We will first create Management Port Group and then create the remaining four port groups.

Create the Management Distributed Port Group

1. Right-click the vDS and select New Distributed Port Group.

   

2. Enter the following details.

   • Name and Location: Enter a descriptive name (e.g., Cls-3-Std3-DB-Management-Network) and ensure the vDS name is correct.

     

   • Configure Settings: Enter the following details.

     • Port binding: Ephemeral - no binding

     • Network Resource Pool: (default)

     • VLAN type: None

     • Select the checkbox to customize the default policies configuration.

     

   • Security: (default)

     

   • Traffic shaping: (default)

     

   • Teaming and failover:

     • Move Uplink 2 to Unused Uplinks.

     • Verify other settings same as below.

     

   • Monitoring: (default)

     

   • Miscellaneous: (default)

     

3. Review all the settings and click Finish to create the Management Distributed Port Group.

   

Create Distributed Port Groups for Other Network Types

We need to create additional Distributed Port Groups for the following network types, using the VLAN IDs documented in Task 1.4.

• Provisioning
• vMotion
• vSAN
• Replication

We will configure a Distributed Port Group for the Provisioning network within the cluster. Enter the following details.

1. Name and Location: Choose a descriptive name based on the network type (e.g., Cls-3-Std3-DB-vds01-provisioning) and ensure the vDS name is correct.

   

2. Configure settings: Enter the following details.

   • Port binding: Ephemeral – no binding

   • Network Resource Pool: (default)

   • VLAN type: VLAN

   • VLAN ID: Enter the corresponding VLAN ID for the network type i.e., Provisioning VLAN as documented earlier .

   • Select the checkbox to customize the default policies.

   

3. Security: (default)

   

4. Traffic Shaping: (default)

   

5. Teaming and failover: Enter the following details.

   • Load balancing: Route based on physical NIC load.

   • Network failure detection: Link status only.

   • Notify switches: Yes.

   • Failover order: Ensure both Uplinks are listed in Active Uplinks.

   

6. Monitoring: (default)

   

7. Miscellaneous: (default)

   

8. Review all settings and click Finish to create the Distributed Port Group for Provisioning VLAN.

   

9. Continue to create Distributed Port Groups for other network types as described earlier. After creating all Distributed Port Groups, your Distributed Switch should resemble the following screenshot.

   

Task 2.6: Add Hosts to the Distributed Switch (vDS)

1. Navigate to the vCenter - Networking view and select the previously created Distributed Switch (vDS).

2. From the Actions drop-down list, select Add and Manage Hosts.

   

3. In the Add and Manage Hosts workflow, enter the following details.

   • Select task: Select Add hosts.

     

   • Select hosts: Select the newly added hosts that were previously put into Maintenance Mode.

     

   • Manage physical adapters: Assign the appropriate Uplinks to the vmnic interfaces.

     • vmnic0: Uplink 1

     • vmnic1: Uplink 2

     

   • Manage VMkernal adapters: Assign each vmk interface to the corresponding Distributed Port Group in the same sequence as shown below.

     

   • Migrate VM networking: (default)

     

   • Review all the details and click Finish to add the selected hosts to the vDS.

     

4. Once completed, the vDS should display the newly added hosts in its list.

   

Task 2.7: Move Hosts into the Cluster

1. In vCenter, select the desired host you want to move.

2. Click Move To from the context menu or drag-and-drop the host directly into the cluster.

   

3. Move Host Confirmation: In the confirmation window, leave the default option Put all of this host’s virtual machines in the cluster’s root resource pool selected and click OK to proceed.

   

4. Monitor Progress: Keep track of the progress to ensure the host is successfully moved into the target cluster.

5. Repeat for Remaining Hosts: Continue moving the remaining relevant hosts into the cluster.

   

6. Log in to the NSX manager and validate that the newly created cluster shows up and the newly added hosts show up as Not Configured.

   

Task 3: Configure a New Cluster in NSX Manager

As part of our cluster provisioning workflow in the OCI Console, a dedicated NSX VTEP VLAN was automatically created. This can be found in Task 1.4, NSX VTEP VLAN as shown below.

For future reference, let’s document the following information about this VLAN. These details will be used throughout the following sections for configuration.

• VLAN Tag or ID: 1886

• CIDR Range: 10.0.16.64/27

Task 3.1: Create a VTEP IP Pool

1. Navigate to the Networking tab in NSX Manager and switch to the Manager View.

2. Select IP Address Pools and click Add to create a new VTEP pool.

   

3. In the Add New IP Pool window, enter the following details.

   • Name: Enter a descriptive name, such as **VTEP-IP-Pool-<_vlan tag_>**.

   • Description: Provide a brief description of the pool’s purpose.

   • Subnets: Click +Add and enter the following information.

     • IP Ranges: Specify the start and end IP addresses available in the pool.

     • Gateway: Enter the default gateway of the NSX VTEP VLAN (e.g., the first usable IP after the network IP).

     • CIDR: Provide the CIDR range documented earlier.

   • Click ADD to create the VTEP IP Pool.

   

4. The newly created VTEP IP Pool should be visible, as shown below.

   

Task 3.2: Create a New Uplink Profile

We will create a new Uplink profile to be used with the Transport Node.

1. Select System from the top menu and navigate to Fabric and Profiles.

2. Click +Add Profiles under Uplink Profiles.

   

3. In the New Uplink Profile window, enter the following details.

   • Name: Choose a clear and distinguishable name, such as Cls-3-Std3-DB-oci-w01hp-consolidated01.

   • Description: Provide a brief description of the profile’s purpose.

   • Teamings: Click +Add and enter the following details.

     • Teaming Policy: Click Edit and select Load Balance Source MAC Address from the drop-down list.

     • Active Uplinks: Click Edit and enter uplink-1,uplink-2.

     • Transport VLAN: Enter the VLAN ID of the NSX VTEP VLAN (documented earlier).

     • MTU: Leave blank (default 1700 bytes).

   • Click Add to create the Uplink Profile.

   

4. Validate the Uplink profile is created as per our requirements.

   

Task 3.3: Create a Transport Node Profile

1. If you are not already in the System section, select System from the top menu.

2. Navigate to Fabric, Hosts and select Transport Node Profile.

3. Click Add Transport Node Profile.

   

4. Enter a descriptive Name for the profile as Cls-3-Std3-DB-nsx-default-transport-node-profile.

5. Under Host Switch, click Set.

   

6. In the Host Switch window, enter the following details.

   • vCenter: Select the vCenter server from the drop-down menu.

   • vDS: Choose the previously created Distributed Switch (vDS) from the drop-down list.

   • Transport Zones: Select both Overlay-TZ and VLAN-TZ from the drop-down menus.

   • Uplink Profile: Choose the previously created Uplink Profile.

   • IP Address Type: Select IPv4.

   • IPv4 Assignment: Choose Use IPv4 Pool from the drop-down list.

   • IPv4 Pool: Select the VTEP IP Pool created earlier.

   • Mode: Select Standard.

   • Teaming Policy Uplink Mapping: Assign the respective vDS Uplinks to each Uplink.

   • Click Add to create the Host Switch.

   

7. Click Apply to set the Host Switch to the TN Profile.

8. Click Save to create the TN Profile.

   

Task 3.4: Apply the Transport Node profile

1. Navigate back to NSX Manager and select System from the top menu.

2. Go to Fabric and Hosts.

3. Under the Clusters tab, locate the newly created cluster Cls-3-Std3-DB, with the moved hosts.

4. Select the cluster and choose Configure NSX.

5. Select the previously created Transport Node Profile from the drop-down list and click Save.

   

6. This process may take several minutes. Monitor the NSX Configuration status, which should eventually show Success.

   

Note: After applying the Transport Node profile, the NSX configuration status may initially show as Unknown. Allow several minutes (4-5 minutes) for the configuration to stabilize. During this time, the status may also temporarily display as Down. This is normal behavior, and you should not initiate any troubleshooting until the status remains consistently Down for an extended period. Once the configuration has stabilized, the NSX status should ultimately show as Up.

Task 4: Remove Cluster Hosts from Maintenance Mode

1. Within vCenter, right-click an ESXi host within the cluster.

2. Select Maintenance Mode and Exit Maintenance Mode.

   

3. Repeat the above steps for all hosts in the cluster, ensuring all are removed from Maintenance Mode.

   

Task 5: Create a Datastore Cluster

We will create a Datastore Cluster dedicated to the newly configured cluster. Datastores in Oracle Cloud VMware Solution are recommended to be local to each cluster, and they can be either Virtual Machine File System (VMFS) and vSAN Datastore. For Standard shapes as we leverage OCI Block Volumes Service as the Primary Storage, the Oracle Cloud VMware Solution automated provisioning workflow could create VMFSs Datastores from the attached OCI Block Volumes.

In this tutorial, we will show how to create a VMFS Datastore Cluster from the attached Datastore(s), if you are using a Dense shape, follow the guidelines in Add ESXi Host Disks to the vSAN Datastore to create a vSAN Datastore.

1. To create a VMFS Datastore Cluster, navigate to vCenter and select the Storage view.

2. Right-click on the Datacenter and navigate to Storage and select New Datastore Cluster.

   

3. Enter the following Details for the Datastore Cluster.

   • Name and Location:

     • Provide a descriptive name.

     • Datastore type: VMFS.

     • Enable Turn ON Storage DRS for optimal data placement and management.

     

   • Storage DRS Automation: Select Fully Automated and leave other settings as default.

     

   • Storage DRS Runtime Settings: Enter the following details as shown below.

     

   • Select Clusters and Hosts: Select the cluster created during this tutorial.

     

   • Select Datastores: Select Show Datastores connected to all hosts from the drop-down list on the top. The Datastores that were already created and attached by the automated provisioning workflow should display. Select them all and click Next.

     

   • Review all settings and click Finish to complete the creation process.

     

4. Monitor the progress of the task, until successful.

   

Task 6: Configure HCX to enable migrations into the new cluster

To streamline the migration of workloads from your on-premises VMware environment to the recently established cluster, leverage HCX, included in the Oracle Cloud VMware Solution SDDC. If you opted for HCX deployment during SDDC creation, the automated provisioning process generates a Compute Profile configured for use with the initially created clusters. However, for clusters added during day 2 operations, the vMotion and Replication networks of the newly created cluster must be manually added to the compute profile.

Here is a recap of the vMotion and Replication VLANs created during the cluster creation.

Let’s summarize the following information about these VLANs as documented in Task 1.4. These details will be used throughout the following section for configuration.

Edit the Compute Profile using the following steps:

1. Log in to your HCX Manager and navigate to Interconnect on the left.

2. Select Compute Profiles and Edit the default profile created during SDDC provisioning. If you have created multiple profiles during day 2 operations, choose the relevant one for adding the new cluster.

   

3. Continue through the sections until you reach section 4, where you can see the vMotion Network Profile.

4. For the vMotion Network Profile.

   • Click Advanced Configurations and choose Add Another Static Route.

   • In the Network section, input the CIDR range of the vMotion VLAN for the newly created cluster (as documented earlier).

   • In Next Hop, enter the default gateway of the vMotion VLAN of the Unified Management cluster (as documented earlier).

   • Click Save to commit the newly added static route.

   • Click Continue.

   

5. In the vSphere Replication Network Profile section.

   • Click Advanced Configurations and choose Add Another Static Route.

   • In the Network section, input the CIDR range of the Replication VLAN for the newly created cluster (as documented earlier).

   • In Next Hop, enter the default gateway of the Replication VLAN of the Unified Management cluster (as documented earlier).

   • Click Save to commit the newly added static route.

   

6. Continue click Continue until the changes to the Compute Profile are saved and take effect.

Using your existing Service Mesh with this Compute Profile, you should now be able to migrate workloads into the newly added cluster.

Conclusion

After completing these tasks, you have effectively established a new cluster in your existing Oracle Cloud VMware Solution SDDC. This entails setting up the cluster, incorporating and configuring hosts in vCenter, NSX Manager, and HCX Manager, as well as creating a dedicated datastore cluster.

Well done! You are now ready to administer and initiate workload migrations from on-premises to the new cluster within this recently established cluster.

Related Links

• Oracle Cloud VMware Solution

• Get Started with Oracle Cloud VMware Solution

• Create a Multi-Cluster Oracle Cloud VMware Solution SDDC

Acknowledgments

• Author - Praveen Kumar Pedda Vakkalam (Principal Solutions Architect)

• Contributor - Adeel Amin (Sr. Principal Solutions Architect)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.