layout-3 layout-6 layout-1 layout-top
NVIDIA Docs Hubchevron_right NVIDIA Networkingchevron_right Networking Softwarechevron_right Switch Softwarechevron_right

Cumulus Linux

layout-top layout-6 layout-1

If you are using the current version of Cumulus Linux, the content on this page may not be up to date. The current version of the documentation is available here. If you are redirected to the main page of the user guide, then this page may have been renamed; please search for it there.

Quick Start Guide

This quick start guide provides an end-to-end setup process for installing and running Cumulus Linux.

Prerequisites

This guide assumes you have intermediate-level Linux knowledge. You need to be familiar with basic text editing, Unix file permissions, and process monitoring. A variety of text editors are pre-installed, including vi and nano.

You must have access to a Linux or UNIX shell. If you are running Windows, use a Linux environment like Cygwin as your command line tool for interacting with Cumulus Linux.

You can choose to configure Cumulus Linux either with the NVUE CLI or with Linux commands (you can run vtysh or edit configuration files manually). This guide provides separate procedures for NVUE commands and Linux commands.

Do not run both NVUE configuration commands and Linux commands to configure the switch. NVUE replaces certain configuration in the files, and removes any configuration you add manually or with automation tools like Ansible, Chef, or Puppet.

Get Started

Cumulus Linux is installed on the switch by default. To upgrade to a different Cumulus Linux release or re-install Cumulus Linux, refer to Installation Management. To show the Cumulus Linux release installed on the switch, run the NVUE nv show system command.

When starting Cumulus Linux for the first time, the management port makes a DHCPv4 request. To determine the IP address of the switch, you can cross reference the MAC address of the switch with your DHCP server. The MAC address is typically located on the side of the switch or on the box in which the unit ships.

To get started:

  • Log in to Cumulus Linux on the switch and change the default credentials.
  • Configure Cumulus Linux. This quick start guide provides instructions on changing the hostname of the switch, setting the date and time, and configuring switch ports and a loopback interface.

You can choose to configure Cumulus Linux either with NVUE commands or Linux commands (with vtysh or by manually editing configuration files). Do not run both NVUE configuration commands (such as nv set, nv unset, nv action, and nv config) and Linux commands to configure the switch. NVUE commands replace the configuration in files such as /etc/network/interfaces and /etc/frr/frr.conf, and remove any configuration you add manually or with automation tools like Ansible, Chef, or Puppet.

Login Credentials

The default installation includes two accounts:

  • The system account (root) has full system privileges. Cumulus Linux locks the root account password by default (which prohibits login).

  • The user account (cumulus) has sudo privileges. The cumulus account uses the default password cumulus.

    When you log in for the first time with the cumulus account, Cumulus Linux prompts you to change the default password. After you provide a new password, the SSH session disconnects and you have to reconnect with the new password.

ONIE includes options that allow you to change the default password for the cumulus account automatically when you install a new Cumulus Linux image. Refer to ONIE Installation Options. You can also change the default password using a ZTP script.

In this quick start guide, you use the cumulus account to configure Cumulus Linux.

All accounts except root can use remote SSH login; you can use sudo to grant a non-root account root-level access. Commands that change the system configuration require this elevated level of access.

For more information about sudo, see Using sudo to Delegate Privileges.

Serial Console Management

NVIDIA recommends you perform management and configuration over the network, either in band or out of band. A serial console is fully supported.

Typically, switches ship from the manufacturer with a mating DB9 serial cable. Switches with ONIE are always set to a 115200 baud rate.

Wired Ethernet Management

A Cumulus Linux switch always provides at least one dedicated Ethernet management port called eth0. This interface is specifically for out-of-band management use. The management interface uses DHCPv4 for addressing by default.

To set a static IP address:

cumulus@switch:~$ nv set interface eth0 ip address 192.0.2.42/24
cumulus@switch:~$ nv set interface eth0 ip gateway 192.0.2.1
cumulus@switch:~$ nv config apply

Edit the /etc/network/interfaces file:

cumulus@switch:~$ sudo nano /etc/network/interfaces
# Management interface
auto eth0
iface eth0
    address 192.0.2.42/24
    gateway 192.0.2.1

Configure the Hostname

The hostname identifies the switch; make sure you configure the hostname to be unique and descriptive.

Do not use an underscore (_), apostrophe ('), or non-ASCII characters in the hostname.

To change the hostname:

cumulus@switch:~$ nv set system hostname leaf01
cumulus@switch:~$ nv config apply

  1. Change the hostname with the hostnamectl command; for example:

    cumulus@switch:~$ sudo hostnamectl set-hostname leaf01

  2. In the /etc/hosts file, replace the host for IP address 127.0.1.1 with the new hostname:

    cumulus@switch:~$ sudo nano /etc/hosts
...
127.0.1.1       leaf01

The command prompt in the terminal does not reflect the new hostname until you either log out of the switch or start a new shell.

Configure the Time Zone

The default time zone on the switch is UTC (Coordinated Universal Time). Change the time zone on your switch to be the time zone for your location.

To update the time zone:

Run the nv set system timezone <timezone> command. To see all the available time zones, run nv set system timezone and press the Tab key. The following example sets the time zone to US/Eastern:

cumulus@switch:~$ nv set system timezone US/Eastern
cumulus@switch:~$ nv config apply

  1. In a terminal, run the following command:

    cumulus@switch:~$ sudo dpkg-reconfigure tzdata

  2. Follow the on screen menu options to select the geographic area and region.

Programs that are already running (including log files) and logged in users, do not see time zone changes. To set the time zone for all services and daemons, reboot the switch.

Verify the System Time

Verify that the date and time on the switch are correct with the Linux date command:

cumulus@switch:~$ date
Mon 21 Nov 2022 06:30:37 PM UTC

If the date and time are incorrect, the switch does not synchronize with automation tools, such as Puppet, and returns errors after you restart switchd.

To set the software clock according to the configured time zone, run the Linux sudo date -s command; for example:

cumulus@switch:~$ sudo date -s "Tue Jan 26 00:37:13 2021"

For more information about setting the system time, see Setting the Date and Time.

Configure Breakout Ports with Splitter Cables

If you are using 4x10G DAC or AOC cables, or you want to break out 100G or 40G switch ports, configure the breakout ports. For more details, see Switch Port Attributes.

Test Cable Connectivity

By default, Cumulus Linux disables all data plane ports (every Ethernet port except the management interface, eth0). To test cable connectivity, administratively enable physical ports.

To administratively enable a port:

cumulus@switch:~$ nv set interface swp1
cumulus@switch:~$ nv config apply

To administratively enable all physical ports on a switch that has ports numbered from swp1 to swp52:

cumulus@switch:~$ nv set interface swp1-52
cumulus@switch:~$ nv config apply

To view link status, run the nv show interface command.

To administratively enable a port:

cumulus@switch:~$ sudo ip link set swp1 up

To administratively enable all physical ports, run the following bash script:

cumulus@switch:~$ sudo su -
cumulus@switch:~$ for i in /sys/class/net/*; do iface=`basename $i`; if [[ $iface == swp* ]]; then ip link set $iface up fi done

To view link status, run the ip link show command.

Configure Layer 2 Ports

Cumulus Linux does not put all ports into a bridge by default. To create a bridge and configure one or more front panel ports as members of the bridge:

The following configuration example places the front panel port swp1 into the default bridge called br_default.

cumulus@switch:~$ nv set interface swp1 bridge domain br_default
cumulus@switch:~$ nv config apply

You can add a range of ports in one command. For example, to add swp1 through swp3, swp10, and swp14 through swp20 to the bridge:

cumulus@switch:~$ nv set interface swp1-3,swp6,swp14-20 bridge domain br_default
cumulus@switch:~$ nv config apply

The following configuration example places the front panel port swp1 into the default bridge called br_default:

...
auto br_default
iface br_default
    bridge-ports swp1
...

To put a range of ports into a bridge, use the glob keyword. For example, to add swp1 through swp10, swp12, and swp14 through swp20 to the bridge called br_default:

...
auto br_default
iface br_default
    bridge-ports glob swp1-10 swp12 glob swp14-20
...

To apply the configuration, check for typos:

cumulus@switch:~$ sudo ifquery -a

If there are no errors, run the following command:

cumulus@switch:~$ sudo ifup -a

For more information about Ethernet bridges, see Ethernet Bridging - VLANs.

Configure Layer 3 Ports

You can configure a front panel port or bridge interface as a layer 3 port.

The following configuration example configures the front panel port swp1 as a layer 3 access port:

cumulus@switch:~$ nv set interface swp1 ip address 10.0.0.0/31
cumulus@switch:~$ nv config apply

To add an IP address to a bridge interface, you must put it into a VLAN interface. If you want to use a VLAN other than the native one, set the bridge PVID:

cumulus@switch:~$ nv set interface swp1-2 bridge domain br_default
cumulus@switch:~$ nv set bridge domain br_default vlan 10
cumulus@switch:~$ nv set interface vlan10 ip address 10.1.10.2/24
cumulus@switch:~$ nv set bridge domain br_default untagged 1
cumulus@switch:~$ nv config apply

The following configuration example configures the front panel port swp1 as a layer 3 access port:

auto swp1
iface swp1
  address 10.0.0.0/31

To add an IP address to a bridge interface, include the address under the iface stanza in the /etc/network/interfaces file. If you want to use a VLAN other than the native one, set the bridge PVID:

auto br_default
iface br_default
    address 10.1.10.2/24
    bridge-ports swp1 swp2
    bridge-pvid 1

To apply the configuration, check for typos:

cumulus@switch:~$ sudo ifquery -a

If there are no errors, run the following command:

cumulus@switch:~$ sudo ifup -a

Configure a Loopback Interface

Cumulus Linux has a preconfigured loopback interface. When the switch boots up, the loopback interface, called lo, is up and assigned an IP address of 127.0.0.1.

The loopback interface lo must always exist on the switch and must always be up. To check the status of the loopback interface, run the NVUE nv show interface lo command or the Linux ip addr show lo command.

To add an IP address to a loopback interface, configure the lo interface:

cumulus@switch:~$ nv set interface lo ip address 10.10.10.1/32
cumulus@switch:~$ nv config apply

Add the IP address directly under the iface lo inet loopback definition in the /etc network/interfaces file:

auto lo
iface lo inet loopback
    address 10.10.10.1

If you configure an IP address without a subnet mask, it becomes a /32 IP address. For example, 10.10.10.1 is 10.10.10.1/32.

You can add multiple loopback addresses. For more information, see Interface Configuration and Management.

If you run NVUE Commands to configure the switch, run the nv config save command before you reboot. The command saves the applied configuration to the startup configuration so that the changes persist after the reboot.

cumulus@switch:~$ nv config save

Show Platform and System Settings

  • To show the hostname of the switch, the time zone, and the version of Cumulus Linux running on the switch, run the NVUE nv show system command.
  • To show switch platform information, such as the platform model, RAM, serial number, and system MAC address, run the NVUE nv show platform hardware command.

Next Steps

You are now ready to configure the switch according to your needs. This guide provides separate sections that describe how to configure system, layer 1, layer 2, layer 3, and network virtualization settings. Each section includes example configurations and pre-built demos.

For a deep dive into the NVUE object model that provides a CLI to simplify configuration, see NVUE.

    Copyright © 2024  NVIDIA Corporation