Set Up Your KVM Virtual Machine for a Cloud HA Server Cluster

First configure the VM on the master node, and then configure the VM on each worker node.

Follow these steps to set up and configure your VM on a cluster of servers in a cloud deployment:

Verify that each node in your cluster—the master node and two worker nodes—meets the VM requirements.

Resource	Minimum Requirements
Processor	Four (4) virtual CPUs
Memory	8 GB RAM
Local disk storage	64 GB
Network interface speed	1 Gb NIC
Hypervisor	KVM/QCOW (QEMU Copy on Write) image for servers running CentOS, Ubuntu, and RedHat operating systems

Confirm that the required ports are open for communications. The OPTA must be able to initiate HTTPS connections (destination TCP port 443) to the netq.nvidia.com domain (*.netq.nvidia.com). You must also open the following ports on your NetQ OPTA:

Port or Protocol Number	Protocol	Component Access
4	IP Protocol	Calico networking (IP-in-IP Protocol)
22	TCP	SSH
80	TCP	Nginx
179	TCP	Calico networking (BGP)
443	TCP	Nginx
2379	TCP	etcd datastore
4789	UDP	Calico networking (VxLAN)
5000	TCP	Docker registry
6443	TCP	kube-apiserver
31980	TCP	NetQ Agent communication
31982	TCP	NetQ Agent SSL communication
32708	TCP	API Gateway

The following ports are used for internal cluster communication and must also be open between servers in your cluster:

Port	Protocol	Component Access
8080	TCP	Admin API
5000	TCP	Docker registry
6443	TCP	Kubernetes API server
10250	TCP	kubelet health probe
2379	TCP	etcd
2380	TCP	etcd
36443	TCP	Kubernetes control plane

Download the NetQ image.
1. On the NVIDIA Application Hub, log in to your account.
2. Select NVIDIA Licensing Portal.
3. Select Software Downloads from the menu.
4. Click Product Family and select NetQ.
5. Locate the NetQ SW 4.12 KVM Cloud image and select Download.
6. If prompted, read the license agreement and proceed with the download.
For enterprise customers, if you do not see a link to the NVIDIA Licensing Portal on the NVIDIA Application Hub, contact NVIDIA support.

For NVIDIA employees, download NetQ directly from the NVIDIA Licensing Portal.
Set up and configure your VM.
KVM Example Configuration

This example shows the VM setup process for a system with Libvirt and KVM/QEMU installed.
1. Confirm that the SHA256 checksum matches the one posted on the NVIDIA Application Hub to ensure the image download has not been corrupted.
```
$ sha256sum ./Downloads/netq-4.12.0-ubuntu-20.04-tscloud-qemu.qcow2
$ cd9723339eb44d0acb136f27c1d36653448ae475cc35fd8d5af58dcd7a5d15c9  ./Downloads/netq-4.12.0-ubuntu-20.04-tscloud-qemu.qcow2
```
2. Copy the QCOW2 image to a directory where you want to run it.
  Tip: Copy, instead of moving, the original QCOW2 image that was downloaded to avoid re-downloading it again later should you need to perform this process again.
```
$ sudo mkdir /vms
$ sudo cp ./Downloads/netq-4.12.0-ubuntu-20.04-tscloud-qemu.qcow2 /vms/ts.qcow2
```
3. Create the VM.
  For a Direct VM, where the VM uses a MACVLAN interface to sit on the host interface for its connectivity:
```
$ virt-install --name=netq_ts --vcpus=4 --memory=8192 --os-type=linux --os-variant=generic --disk path=/vms/ts.qcow2,format=qcow2,bus=virtio,cache=none --network=type=direct,source=eth0,model=virtio --import --noautoconsole
```
  Replace the disk path value with the location where the QCOW2 image is to reside. Replace network model value (eth0 in the above example) with the name of the interface where the VM is connected to the external network.
  
  Or, for a Bridged VM, where the VM attaches to a bridge which has already been setup to allow for external access:
```
$ virt-install --name=netq_ts --vcpus=4 --memory=8192 --os-type=linux --os-variant=generic \ --disk path=/vms/ts.qcow2,format=qcow2,bus=virtio,cache=none --network=bridge=br0,model=virtio --import --noautoconsole
```
  Replace network bridge value (br0 in the above example) with the name of the (pre-existing) bridge interface where the VM is connected to the external network.
  
  Make note of the name used during install as this is needed in a later step.
4. Watch the boot process in another terminal window.
```
$ virsh console netq_ts
```

Use the default credentials to log in the first time:

Username: cumulus
Password: cumulus

$ ssh cumulus@<ipaddr>
Warning: Permanently added '<ipaddr>' (ECDSA) to the list of known hosts.
Ubuntu 20.04 LTS
cumulus@<ipaddr>'s password:
You are required to change your password immediately (root enforced)
System information as of Thu Dec  3 21:35:42 UTC 2020
System load:  0.09              Processes:           120
Usage of /:   8.1% of 61.86GB   Users logged in:     0
Memory usage: 5%                IP address for eth0: <ipaddr>
Swap usage:   0%
WARNING: Your password has expired.
You must change your password now and login again!
Changing password for cumulus.
(current) UNIX password: cumulus
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Connection to <ipaddr> closed.

$ ssh cumulus@<ipaddr>
Warning: Permanently added '<ipaddr>' (ECDSA) to the list of known hosts.
Ubuntu 20.04 LTS
cumulus@<ipaddr>'s password:
  System information as of Thu Dec  3 21:35:59 UTC 2020
  System load:  0.07              Processes:           121
  Usage of /:   8.1% of 61.86GB   Users logged in:     0
  Memory usage: 5%                IP address for eth0: <ipaddr>
  Swap usage:   0%
Last login: Thu Dec  3 21:35:43 2020 from <local-ipaddr>
cumulus@ubuntu:~$

Verify the master node is ready for installation. Fix any errors indicated before installing the NetQ software.
```
cumulus@hostname:~$ sudo opta-check-cloud
```
Change the hostname for the VM from the default value.

The default hostname for the NetQ Virtual Machines is ubuntu. Change the hostname to fit your naming conventions while meeting Internet and Kubernetes naming standards.

Kubernetes requires that hostnames are composed of a sequence of labels concatenated with dots. For example, “en.wikipedia.org” is a hostname. Each label must be from 1 to 63 characters long. The entire hostname, including the delimiting dots, has a maximum of 253 ASCII characters.

The Internet standards (RFCs) for protocols specify that labels may contain only the ASCII letters a through z (in lower case), the digits 0 through 9, and the hyphen-minus character ('-').

Use the following command:
```
cumulus@hostname:~$ sudo hostnamectl set-hostname NEW_HOSTNAME
```
Add the same NEW_HOSTNAME value to /etc/hosts on your VM for the localhost entry. Example:
```
127.0.0.1 localhost NEW_HOSTNAME
```
Verify that your first worker node meets the VM requirements, as described in step 1.
Confirm that the required ports are open for communications, as described in step 2.
Open your hypervisor and set up the VM in the same manner as for the master node.

Make a note of the private IP address you assign to the worker node. You need it for later installation steps.
Verify the worker node is ready for installation. Fix any errors indicated before installing the NetQ software.
```
cumulus@hostname:~$ sudo opta-check-cloud
```
Repeat steps 8 through 11 for each additional worker node in your cluster.
Install and activate the NetQ software using the CLI:

Run the following command on your master node to initialize the cluster. Copy the output of the command to use on your worker nodes:

cumulus@<hostname>:~$ netq install cluster master-init
    Please run the following command on all worker nodes:
    netq install cluster worker-init c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCQVFDM2NjTTZPdVVUWWJ5c2Q3NlJ4SHdseHBsOHQ4N2VMRWVGR05LSWFWVnVNcy94OEE4RFNMQVhKOHVKRjVLUXBnVjdKM2lnMGJpL2hDMVhmSVVjU3l3ZmhvVDVZM3dQN1oySVZVT29ZTi8vR1lOek5nVlNocWZQMDNDRW0xNnNmSzVvUWRQTzQzRFhxQ3NjbndIT3dwZmhRYy9MWTU1a

Run the netq install cluster worker-init <ssh-key> on each of your worker nodes.

Run the following command on your NetQ cloud appliance with the config-key obtained from the email you received from NVIDIA titled NetQ Access Link. You can also obtain the configuration key through the NetQ UI. Use the IP addresses of your worker nodes and the HA cluster virtual IP address (VIP).

The HA cluster virtual IP must be:

An unused IP address allocated from the same subnet assigned to the default interface for your master and worker nodes. The default interface is the interface used in the netq install command.
A different IP address than the primary IP assigned to the default interface.

cumulus@<hostname>:~$ netq install opta cluster full interface eth0 bundle /mnt/installables/NetQ-4.12.0-opta.tgz config-key <your-config-key> workers <worker-1-ip> <worker-2-ip> cluster-vip <vip-ip> [proxy-host <proxy-hostname> proxy-port <proxy-port>]

NetQ uses the 10.244.0.0/16 (pod-ip-range) and 10.96.0.0/16 (service-ip-range) networks for internal communication by default. If you are using these networks, you must override each range by specifying new subnets for these parameters in the install command:


cumulus@hostname:~$ netq install opta cluster full interface eth0 bundle /mnt/installables/NetQ-4.12.0-opta.tgz config-key <your-config-key> pod-ip-range <pod-ip-range> service-ip-range <service-ip-range>

You can specify the IP address of the server instead of the interface name using the ip-addr <ip-address> argument:


cumulus@hostname:~$ netq install opta cluster full ip-addr <ip-address> bundle /mnt/installables/NetQ-4.12.0-opta.tgz config-key <your-config-key>

If you change the server IP address or hostname after installing NetQ, you must reset the server with the netq bootstrap reset keep-db command and rerun the install command.

If this step fails for any reason, you can run netq bootstrap reset and then try again.

Verify Installation Status

To view the status of the installation, use the netq show status [verbose] command. The following example shows a successful on-premises installation:

State: Active
    NetQ Live State: Active
    Installation Status: FINISHED
    Version: 4.12.0
    Installer Version: 4.12.0
    Installation Type: Cluster
    Activation Key: EhVuZXRxLWVuZHBvaW50LWdhdGV3YXkYsagDIixPSUJCOHBPWUFnWXI2dGlGY2hTRzExR2E5aSt6ZnpjOUvpVVTaDdpZEhFPQ==
    Master SSH Public Key: c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCZ1FDNW9iVXB6RkczNkRC
    Is Cloud: False
    
    Kubernetes Cluster Nodes Status:
    IP Address    Hostname     Role    NodeStatus    Virtual IP
    ------------  -----------  ------  ------------  ------------
    10.213.7.52   10.213.7.52  Worker  Ready         10.213.7.53
    10.213.7.51   10.213.7.51  Worker  Ready         10.213.7.53
    10.213.7.49   10.213.7.49  Master  Ready         10.213.7.53
    
    In Summary, Live state of the NetQ is... Active

Run the netq show opta-health command to verify all applications are operating properly. Allow 10-15 minutes for all applications to come up and report their status.

cumulus@hostname:~$ netq show opta-health
    Application                                            Status    Namespace      Restarts    Timestamp
    -----------------------------------------------------  --------  -------------  ----------  ------------------------
    cassandra-rc-0-w7h4z                                   READY     default        0           Fri Apr 10 16:08:38 2020
    cp-schema-registry-deploy-6bf5cbc8cc-vwcsx             READY     default        0           Fri Apr 10 16:08:38 2020
    kafka-broker-rc-0-p9r2l                                READY     default        0           Fri Apr 10 16:08:38 2020
    kafka-connect-deploy-7799bcb7b4-xdm5l                  READY     default        0           Fri Apr 10 16:08:38 2020
    netq-api-gateway-deploy-55996ff7c8-w4hrs               READY     default        0           Fri Apr 10 16:08:38 2020
    netq-app-address-deploy-66776ccc67-phpqk               READY     default        0           Fri Apr 10 16:08:38 2020
    netq-app-admin-oob-mgmt-server                         READY     default        0           Fri Apr 10 16:08:38 2020
    netq-app-bgp-deploy-7dd4c9d45b-j9bfr                   READY     default        0           Fri Apr 10 16:08:38 2020
    netq-app-clagsession-deploy-69564895b4-qhcpr           READY     default        0           Fri Apr 10 16:08:38 2020
    netq-app-configdiff-deploy-ff54c4cc4-7rz66             READY     default        0           Fri Apr 10 16:08:38 2020
    ...

If any of the applications or services display Status as DOWN after 30 minutes, open a support ticket and attach the output of the opta-support command.

After NetQ is installed, you can log in to NetQ from your browser.

Set Up Your KVM Virtual Machine for a Cloud HA Server Cluster

Verify Installation Status

Products

Solutions

Learn