NVIDIA User Experience (NVUE) Cheat Sheet

NVUE is an object-oriented, schema-driven model of a complete Cumulus Linux system providing a robust API that allows multiple interfaces to view and configure any element within a system.

You can use NVUE through its CLI or API. Because NVUE is an object model, both CLI and REST API interfaces have equivalent functionality and can work in parallel while keeping all management operations consistent; for example, the CLI show commands reflect any PATCH operation you run through the REST API.

NVUE follows a declarative model, removing context-specific commands and settings. It is structured as a big tree (like a filesystem path) representing the entire system state. At the base of the tree are high-level branches representing objects, such as router and interface. Under each branch, there are additional branches, and as you navigate through the tree, you gain a more specific context of the objects. The leaves of the tree are actual attributes, represented as key-value pairs.

This cheat sheet helps you get up to speed using Cumulus Linux and the NVUE CLI.

NVUE CLI Command Syntax

All NVUE commands begin with nv and fall into one of four syntax categories:

• Configuration (nv set and nv unset)
• Monitoring (nv show)
• Configuration management (nv config)
• Action (nv action)

Like all industry-standard CLIs, the NVUE CLI includes command completion with the TAB key, use of the question mark (?) to display command information, and command abbreviation to speed up CLI interaction. In addition, you can get help with command syntax by using the -h or --help option, and list all commands by running nv list-commands.

Getting Started

After you rack and power on your NVIDIA Spectrum switch with Cumulus Linux, connect a serial console cable so that you can begin configuration. All switches are manufactured with an RJ45 serial port for console connectivity and set to 115200 baud rate. If your switch does not have a pre-installed NOS, you can install a new Cumulus Linux image.

System Management and Services

Use the following commands to configure the management network and system services on the switch.

$ nv set system hostname leaf01

$ nv set interface eth0 ip address 192.168.200.2/24
$ nv set interface eth0 ip gateway 192.168.200.1

$ nv set service ntp default server 4.cumulusnetworks.pool.ntp.org iburst on

$ nv set system timezone US/Eastern

$ nv set service dns mgmt server 198.51.100.31

$ nv set service syslog default server 192.168.0.254 port 514
$ nv set service syslog default server 192.168.0.254 protocol udp

Working with Interfaces

Use the following commands to configure the physical, breakout, loopback, and logical layer 2 and layer 3 interfaces.

Physical Interfaces

$ nv set interface swp1
$ nv set interface swp1,20-32

$ nv set interface swp1 link state down
$ nv set interface swp8-15 link state up

$ nv set interface swp1 link speed 50G

$ nv set interface swp1 link mtu 1500

Breakout Interfaces

To increase the number of ports on the NVIDIA Spectrum switch, you can break out physical interfaces into two or four lower speed ports.
Breakout configurations differ between platforms, refer to the breakout ports section of the switch port attributes documentation for more information.

$ nv set interface swp1 link breakout 4x25G
$ nv set interface swp2 link breakout disabled

$ nv unset interface swp1s0
$ nv unset interface swp1s1
$ nv unset interface swp1s2
$ nv unset interface swp1s3
$ nv unset interface swp1 link breakout

Loopback Interface

Cumulus Linux has a preconfigured loopback interface. When the switch boots up, the loopback interface, called lo, is up and assigned an IP address of 127.0.0.1.

$ nv set interface lo ip address 10.10.10.1

Layer 2 Interfaces

After you enable a physical interface in Cumulus Linux, the interface is routed (layer 3). To set an interface as a switch port (layer 2), you must add it to a bridge.

Bridge and VLANs

Cumulus Linux supports two bridge configuration modes:

• VLAN-aware bridge
• Traditional bridge

The default bridge br_default is a VLAN-aware bridge. Refer to the Ethernet Bridging - VLANs documentation for more information.

$ nv set interface swp1 bridge domain br_default
$ nv set interface swp1-5,7-22 bridge domain br_default

$ nv set interface swp2 bridge domain br_default untagged none

$ nv set bridge domain br_default vlan 10,20

$ nv set interface swp1 bridge domain br_default access 10

$ nv set bridge domain br_default untagged 100

Spanning-Tree Protocol (STP)

The VLAN-aware bridge (br_default) operates only in RSTP mode.

$ nv set bridge domain br_default stp priority 8192

$ nv set interface swp5 bridge domain br_default stp admin-edge on
$ nv set interface swp5 bridge domain br_default stp bpdu-guard on
$ nv set interface swp5 bridge domain br_default stp auto-edge off

Refer to the Spanning Tree and Rapid Spanning Tree - STP documentation for more information.

Layer 3 Interfaces

After you enable a Cumulus Linux interface administratively, it is a routed port (layer 3).

$ nv set interface swp10 ip address 10.1.0.5/24
$ nv set interface swp10 ip address 2001:db8::10/64

$ nv set interface vlan100 ip address 100.1.0.2/24
$ nv set interface vlan100 ip address 2001:db8::1/32

Link Aggregation Interfaces

The Link Aggregation (LAG) interface in Cumulus Linux is called a bond. You can configure a bond as a layer 2 or layer 3 interface.

$ nv set interface bond1 bond members swp1-4
$ nv set interface lag1 bond members swp5-6
$ nv set interface lag1 type bond

$ nv set interface bond1 bond mode static
$ nv set interface bond1 bond mode lacp

$ nv set interface bond1 bond lacp-rate slow
$ nv set interface bond1 bond lacp-rate fast

Working with Network Protocols

This cheat sheet includes some of the basic commands to configure the main data center protocols. To configure other protocols, see additional configuration commands, examples, and more detailed information, refer to the Cumulus Linux User Guide.

Layer 2 Protocols

Multi-Chassis Link Aggregation - MLAG

MLAG provides layer 2 redundancy and greater system throughput. To configure MLAG, you must fulfill these requirements:

• Only two switches can share MLAG configuration. However, you can have multiple (different) MLAG pairs in the network.
• Both MLAG peer switches must be directly connected. This is typically a bond for increased reliability and bandwidth.
• Both switches in the MLAG pair must be of the same Spectrum model and run the same Cumulus Linux version.
• The dual-connected devices (servers or switches) can use LACP or static bond modes. MLAG switches must be set accordingly.

This cheat sheet includes the basic MLAG configuration commands. Refer to the Multi-Chassis Link Aggregation - MLAG documentation for more information.

$ nv set interface bond1 bond members swp1

$ nv set interface bond1 bond mlag id 1

$ nv set interface bond1 bridge domain br_default

$ nv set interface peerlink bond member swp31-32

$ nv set mlag mac-address 44:38:39:BE:EF:AA

$ nv set mlag peer-ip linklocal

nv set mlag backup 10.10.10.2

$ nv set mlag backup 10.10.10.2 vrf mgmt

Virtual Router Redundancy - VRR

VRR enables an active-active gateway for the layer 2 MLAG domain. Both MLAG peers must have an SVI with unique IP addresses for each VLAN. Then, you must set identical VRR instances (one instance per subnet) on both MLAG peers. The VRR instance is configured on the SVIs and holds the virtual IP and MAC addresses. Both peers respond to ARP requests from the host but if one fails, the second still serves as the gateway.

$ nv set interface vlan10 ip vrr address 10.1.10.1/24

$ nv set system global fabric-mac 00:00:5E:00:01:FF
$ nv set system global fabric-id 255

$ nv set interface vlan10 ip vrr mac-address 00:00:5E:00:01:00

Refer to the Virtual Router Redundancy - VRR and VRRP documentation for more information.

Layer 3 Protocols

Virtual Routing and Forwarding - VRF

VRF (also called VRF-Lite) enables you to use multiple independent routing tables that work simultaneously on the same switch. VRFs are useful in multi-tenant environments.

This cheat sheet includes basic VRF commands. For more information about using custom and mgmt VRFs with different protocols, refer to the Virtual Routing and Forwarding - VRF and Management VRF documentation.

$ nv set vrf BLUE table auto
$ nv set vrf RED table 1016

$ nv set interface swp1 ip vrf BLUE

Static Routing

You can use static routing if you do not require the complexity of a dynamic routing protocol (such as BGP or OSPF), if you have routes that do not change frequently and for which the destination is only one or two paths away.

$ nv set vrf default router static 10.10.10.101/32 via 10.0.1.1

Border Gateway Protocol - BGP

BGP is the routing protocol that runs the Internet. BGP manages how packets get routed from network to network by exchanging routing and reachability information.

Cumulus Linux makes BGP configuration in the data center easier with Auto BGP and BGP Unnumbered. NVIDIA recommends using these features to eliminate the need for ASN and point-to-point IP addressing assignments, and to reduce human errors.

This cheat sheet includes basic BGP configuration commands. Refer to the Border Gateway Protocol - BGP documentation for more information.

$ nv set router bgp autonomous-system 65101
$ nv set router bgp autonomous-system leaf

$ nv set router bgp router-id 10.10.10.1
$ nv set vrf RED router bgp router-id 10.10.10.1

$ nv set vrf default router bgp neighbor 10.0.1.1 remote-as internal
$ nv set vrf default router bgp neighbor swp2 remote-as external
$ nv set vrf default router bgp neighbor 2001:db8:0002::0a00:0002 remote-as external

$ nv set vrf default router bgp neighbor 2001:db8:0002::0a00:0002 address-family ipv6-unicast enable on

$ nv set vrf default router bgp address-family ipv4-unicast network 10.1.10.0/24
$ nv set vrf default router bgp address-family ipv6-unicast network 2001:db8::1/128

$ nv set vrf default router bgp address-family ipv4-unicast redistribute static enable on
$ nv set vrf default router bgp address-family ipv6-unicast redistribute connected route-map routemap1

Network Virtualization

VXLAN is a standard overlay protocol for logical virtual networks. It uses a VLAN-like encapsulation technique to encapsulate layer 2 Ethernet segments over layer 3 networks. The encapsulation happens on the VTEP, which establishes an overlay UDP tunnel to the remote VTEP device. Unlike VLANs, VXLAN scales to 16 million segments (a 24-bit VXLAN network identifier (VNI ID) in the VXLAN header) for multi-tenancy. Refer to the Network Virtualization documentation for more information.

VXLAN Devices and Static Tunnels

Cumulus Linux supports single and traditional VXLAN devices. NVUE allows you to work only with a single VXLAN device (single VTEP) in a VLAN-aware bridge. With a single VXLAN device (NVE interface), you specify the VLAN to VNI mapping.

This cheat sheet includes some of the basic configuration commands for static VXLAN tunnels. For more information and for additional configuration, refer to the Network Virtualization documentation.

$ nv set bridge domain br_default vlan 10 vni 10

$ nv set bridge domain br_default vlan 10,20,30,40,50 vni auto

$ nv set bridge domain br_default vlan-vni-offset 10000

$ nv set nve vxlan mac-learning on
$ nv set bridge domain br_default vlan 10 vni 10 mac-learning on

$ nv set nve vxlan arp-nd-suppress on

$ nv set nve vxlan source address 10.10.10.1

$ nv set nve vxlan mlag shared-address 10.0.1.34

$ nv set bridge domain br_default vlan 10 vni 10 flooding head-end-replication 10.10.10.2
$ nv set nve vxlan flooding head-end-replication 10.10.10.2

$ nv set bridge domain br_default vlan 10 vni 10 flooding multicast-group 239.1.1.110
$ nv set nve vxlan flooding multicast-group 224.0.0.10

Ethernet Virtual Private Network - EVPN

EVPN is a standards-based control plane that relies on multi-protocol BGP (MP-BGP) and allows for building and deploying VXLANs at scale. EVPN enables intra-subnet bridging and inter-subnet routing, including multi-tenancy support.

This cheat sheet includes the basic EVPN configuration commands. For more information and additional configuration, refer to the Ethernet Virtual Private Network - EVPN documentation.

$ nv set evpn enable on

$ nv set vrf default router bgp neighbor swp51 address-family l2vpn-evpn enable on

$ nv set evpn route-advertise default-gateway on

$ nv set vrf RED vni 4001

$ nv set vrf RED router bgp address-family ipv4-unicast route-export to-evpn enable on

$ nv set evpn multihoming enable on

$ nv set interface bond2 evpn multihoming segment local-id 2

$ nv set interface bond1 evpn multihoming segment mac-address 44:38:39:BE:EF:AA

$ nv set interface bond1 evpn multihoming segment df-preference 50000

$ nv set interface swp51-54 evpn multihoming uplink on

The Configuration Examples section of the Cumulus Linux user guide provides examples of EVPN layer 2 routing, centralized routing, symmetric routing, and EVPN-MH.

You can also examine and try the EVPN best practices using these pre-built simulations in Air Marketplace:

• EVPN L2 Extension
• EVPN Centralized Routing
• EVPN Symmetric Routing
• EVPN Multihoming

Access Control List - ACL

ACLs in Cumulus Linux are based on Linux iptables and ebtables with the following default behaviors:

• There is no implicit deny. ACLs must end in a match any and action deny rule to drop all unmatched traffic.
• There is no support for wildcard masks. You must list subnets individually.

$ nv set acl DENY_TCP_HTTP type ipv4

$ nv set acl DENY_TCP_HTTP rule 10 match ip protocol tcp
$ nv set acl DENY_TCP_HTTP rule 10 match ip source-ip ANY
$ nv set acl DENY_TCP_HTTP rule 10 match ip source-port ANY
$ nv set acl DENY_TCP_HTTP rule 10 match ip dest-ip 10.0.15.8/32
$ nv set acl DENY_TCP_HTTP rule 10 match ip dest-port 80

$ nv set acl DENY_TCP_HTTP rule 10 action drop

$ nv set interface swp1 acl DENY_TCP_HTTP inbound

$ nv set interface swp1 acl deny_icmp inbound control-plane

Monitoring Commands

The NVUE monitoring commands show how your network is configured. The monitoring commands are divided into categories (objects), which include subcommands. The general command syntax is nv show <category> <subcommand> <subcommand> <...>. You can use TAB completion to navigate through the commands.

NVUE provides additional options for the nv show commands. These options are available using command flags.

Here are some useful show commands:

Action Commands

The NVUE action commands reset counters for interfaces and remove conflicts from protodown MLAG bonds.

$ nv action clear interface swp1 qos roce counters

$ nv action clear interface bond1 bond mlag lacp-conflict

Configuration Management

NVUE leverages the Git engine to manage configuration so that you can treat your configuration as you would code.