Set Up Your VMware Virtual Machine for a Cloud HA Server Cluster
First configure the VM on the master node, and then configure the VM on each worker node.
Follow these steps to set up and configure your VM on a cluster of servers in a cloud deployment:
-
Verify that each node in your cluster—the master node and two worker nodes—meets the VM requirements.
Resource Minimum Requirements Processor Four (4) virtual CPUs Memory 8 GB RAM Local disk storage 64 GB Network interface speed 1 Gb NIC Hypervisor VMware ESXi™ 6.5 or later (OVA image) for servers running Cumulus Linux, CentOS, Ubuntu, and RedHat operating systems -
Confirm that the required ports are open for communications.
The OPTA must be able to initiate HTTPS connections (destination TCP port 443) to the netq.nvidia.com domain (*.netq.nvidia.com). You must also open the following ports on your NetQ OPTA:
The following ports are used for internal cluster communication and must also be open between servers in your cluster:Port or Protocol Number Protocol Component Access 4 IP Protocol Calico networking (IP-in-IP Protocol) 22 TCP SSH 80 TCP Nginx 179 TCP Calico networking (BGP) 443 TCP Nginx 2379 TCP etcd datastore 4789 UDP Calico networking (VxLAN) 5000 TCP Docker registry 6443 TCP kube-apiserver 31980 TCP NetQ Agent communication 31982 TCP NetQ Agent SSL communication 32708 TCP API Gateway
Port Protocol Component Access 8080 TCP Admin API 5000 TCP Docker registry 6443 TCP Kubernetes API server 10250 TCP kubelet health probe 2379 TCP etcd 2380 TCP etcd 36443 TCP Kubernetes control plane -
Download the NetQ image.
- On the NVIDIA Application Hub, log in to your account.
- Select NVIDIA Licensing Portal.
- Select Software Downloads from the menu.
- Click Product Family and select NetQ.
- Locate the NetQ SW 4.11 VMware Cloud image and select Download.
- If prompted, read the license agreement and proceed with the download.
For enterprise customers, if you do not see a link to the NVIDIA Licensing Portal on the NVIDIA Application Hub, contact NVIDIA support.
For NVIDIA employees, download NetQ directly from the NVIDIA Licensing Portal.
- On the NVIDIA Application Hub, log in to your account.
-
Set up and configure your VM.
VMware Example Configuration
This example shows the VM setup process using an OVA file with VMware ESXi.Enter the address of the hardware in your browser.
Log in to VMware using credentials with root access.
Click Storage in the Navigator to verify you have an SSD installed.
Click Create/Register VM at the top of the right pane.
Select Deploy a virtual machine from an OVF or OVA file, and click Next.
Provide a name for the VM, for example NetQ.
Tip: Make note of the name used during install as this is needed in a later step.
Drag and drop the NetQ Platform image file you downloaded in Step 1 above.
Click Next.
Select the storage type and data store for the image to use, then click Next. In this example, only one is available.
Accept the default deployment options or modify them according to your network needs. Click Next when you are finished.
Review the configuration summary. Click Back to change any of the settings, or click Finish to continue with the creation of the VM.
The progress of the request is shown in the Recent Tasks window at the bottom of the application. This may take some time, so continue with your other work until the upload finishes.
Once completed, view the full details of the VM and hardware.
Log in to the VM and change the password.
Use the default credentials to log in the first time:
- Username: cumulus
- Password: cumulus
$ ssh cumulus@<ipaddr> Warning: Permanently added '<ipaddr>' (ECDSA) to the list of known hosts. Ubuntu 20.04 LTS cumulus@<ipaddr>'s password: You are required to change your password immediately (root enforced) System information as of Thu Dec 3 21:35:42 UTC 2020 System load: 0.09 Processes: 120 Usage of /: 8.1% of 61.86GB Users logged in: 0 Memory usage: 5% IP address for eth0: <ipaddr> Swap usage: 0% WARNING: Your password has expired. You must change your password now and login again! Changing password for cumulus. (current) UNIX password: cumulus Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully Connection to <ipaddr> closed.
Log in again with your new password.
$ ssh cumulus@<ipaddr> Warning: Permanently added '<ipaddr>' (ECDSA) to the list of known hosts. Ubuntu 20.04 LTS cumulus@<ipaddr>'s password: System information as of Thu Dec 3 21:35:59 UTC 2020 System load: 0.07 Processes: 121 Usage of /: 8.1% of 61.86GB Users logged in: 0 Memory usage: 5% IP address for eth0: <ipaddr> Swap usage: 0% Last login: Thu Dec 3 21:35:43 2020 from <local-ipaddr> cumulus@ubuntu:~$
Verify the master node is ready for installation. Fix any errors indicated before installing the NetQ software.
cumulus@hostname:~$ sudo opta-check-cloud
Change the hostname for the VM from the default value.
The default hostname for the NetQ Virtual Machines is ubuntu. Change the hostname to fit your naming conventions while meeting Internet and Kubernetes naming standards.
Kubernetes requires that hostnames are composed of a sequence of labels concatenated with dots. For example, “en.wikipedia.org” is a hostname. Each label must be from 1 to 63 characters long. The entire hostname, including the delimiting dots, has a maximum of 253 ASCII characters.
The Internet standards (RFCs) for protocols specify that labels may contain only the ASCII letters a through z (in lower case), the digits 0 through 9, and the hyphen-minus character ('-').
Use the following command:
cumulus@hostname:~$ sudo hostnamectl set-hostname NEW_HOSTNAME
Add the same NEW_HOSTNAME value to /etc/hosts on your VM for the localhost entry. Example:
127.0.0.1 localhost NEW_HOSTNAME
Verify that your first worker node meets the VM requirements, as described in step 1.
Confirm that the required ports are open for communications, as described in step 2.
Open your hypervisor and set up the VM in the same manner as the master node.
Make a note of the private IP address you assign to the worker node. You will need it at a later point in the installation process.
Verify the worker node is ready for installation. Fix any errors indicated before installing the NetQ software.
cumulus@hostname:~$ sudo opta-check-cloud
Repeat steps 8 through 11 for each additional worker node in your cluster.
Install and activate the NetQ software using the CLI:
Run the following command on your master node to initialize the cluster. Copy the output of the command to use on your worker nodes:
cumulus@<hostname>:~$ netq install cluster master-init
Please run the following command on all worker nodes:
netq install cluster worker-init c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCQVFDM2NjTTZPdVVUWWJ5c2Q3NlJ4SHdseHBsOHQ4N2VMRWVGR05LSWFWVnVNcy94OEE4RFNMQVhKOHVKRjVLUXBnVjdKM2lnMGJpL2hDMVhmSVVjU3l3ZmhvVDVZM3dQN1oySVZVT29ZTi8vR1lOek5nVlNocWZQMDNDRW0xNnNmSzVvUWRQTzQzRFhxQ3NjbndIT3dwZmhRYy9MWTU1a
Run the netq install cluster worker-init <ssh-key> on each of your worker nodes.
Run the following command on your NetQ cloud appliance with the config-key obtained from the email you received from NVIDIA titled NetQ Access Link. You can also obtain the configuration key through the NetQ UI. Use the IP addresses of your worker nodes and the HA cluster virtual IP address (VIP).
The HA cluster virtual IP must be:
- An unused IP address allocated from the same subnet assigned to the default interface for your master and worker nodes. The default interface is the interface used in the
netq installcommand. - A different IP address than the primary IP assigned to the default interface.
cumulus@<hostname>:~$ netq install opta cluster full interface eth0 bundle /mnt/installables/NetQ-4.11.0-opta.tgz config-key <your-config-key> workers <worker-1-ip> <worker-2-ip> cluster-vip <vip-ip> [proxy-host <proxy-hostname> proxy-port <proxy-port>]
NetQ uses the 10.244.0.0/16 (pod-ip-range) and 10.96.0.0/16 (service-ip-range) networks for internal communication by default. If you are using these networks, you must override each range by specifying new subnets for these parameters in the install command:
cumulus@hostname:~$ netq install opta cluster full interface eth0 bundle /mnt/installables/NetQ-4.11.0-opta.tgz config-key <your-config-key> pod-ip-range <pod-ip-range> service-ip-range <service-ip-range>
You can specify the IP address of the server instead of the interface name using the ip-addr <ip-address> argument:
cumulus@hostname:~$ netq install opta cluster full ip-addr <ip-address> bundle /mnt/installables/NetQ-4.11.0-opta.tgz config-key <your-config-key>
If you change the server IP address or hostname after installing NetQ, you must reset the server with the netq bootstrap reset keep-db command and rerun the install command.
If this step fails for any reason, you can run netq bootstrap reset and then try again.
Verify Installation Status
To view the status of the installation, use the netq show status [verbose] command. The following example shows a successful on-premises installation:
State: Active
NetQ Live State: Active
Installation Status: FINISHED
Version: 4.11.0
Installer Version: 4.11.0
Installation Type: Cluster
Activation Key: EhVuZXRxLWVuZHBvaW50LWdhdGV3YXkYsagDIixPSUJCOHBPWUFnWXI2dGlGY2hTRzExR2E5aSt6ZnpjOUvpVVTaDdpZEhFPQ==
Master SSH Public Key: c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCZ1FDNW9iVXB6RkczNkRC
Is Cloud: False
Kubernetes Cluster Nodes Status:
IP Address Hostname Role NodeStatus Virtual IP
------------ ----------- ------ ------------ ------------
10.213.7.52 10.213.7.52 Worker Ready 10.213.7.53
10.213.7.51 10.213.7.51 Worker Ready 10.213.7.53
10.213.7.49 10.213.7.49 Master Ready 10.213.7.53
In Summary, Live state of the NetQ is... ActiveRun the netq show opta-health command to verify all applications are operating properly. Allow 10-15 minutes for all applications to come up and report their status.
cumulus@hostname:~$ netq show opta-health
Application Status Namespace Restarts Timestamp
----------------------------------------------------- -------- ------------- ---------- ------------------------
cassandra-rc-0-w7h4z READY default 0 Fri Apr 10 16:08:38 2020
cp-schema-registry-deploy-6bf5cbc8cc-vwcsx READY default 0 Fri Apr 10 16:08:38 2020
kafka-broker-rc-0-p9r2l READY default 0 Fri Apr 10 16:08:38 2020
kafka-connect-deploy-7799bcb7b4-xdm5l READY default 0 Fri Apr 10 16:08:38 2020
netq-api-gateway-deploy-55996ff7c8-w4hrs READY default 0 Fri Apr 10 16:08:38 2020
netq-app-address-deploy-66776ccc67-phpqk READY default 0 Fri Apr 10 16:08:38 2020
netq-app-admin-oob-mgmt-server READY default 0 Fri Apr 10 16:08:38 2020
netq-app-bgp-deploy-7dd4c9d45b-j9bfr READY default 0 Fri Apr 10 16:08:38 2020
netq-app-clagsession-deploy-69564895b4-qhcpr READY default 0 Fri Apr 10 16:08:38 2020
netq-app-configdiff-deploy-ff54c4cc4-7rz66 READY default 0 Fri Apr 10 16:08:38 2020
...
If any of the applications or services display Status as DOWN after 30 minutes, open a support ticket and attach the output of the opta-support command.
After NetQ is installed, you can log in to NetQ from your browser.