Upgrading Cumulus Linux

This topic describes how to upgrade Cumulus Linux on your switch.

Consider deploying, provisioning, configuring, and upgrading switches using automation, even with small networks or test labs. During the upgrade process, you can quickly upgrade dozens of devices in a repeatable manner. Using tools like Ansible, Chef, or Puppet for configuration management greatly increases the speed and accuracy of the next major upgrade; these tools also enable the quick swap of failed switch hardware.

Before You Upgrade

Back up Configuration Files

Understanding the location of configuration data is required for successful upgrades, migrations, and backup. As with other Linux distributions, the /etc directory is the primary location for all configuration data in Cumulus Linux. The following list is a likely set of files that you need to back up and migrate to a new release. Make sure you examine any file that has been changed. Make the following files and directories part of a backup strategy.

Create a cl-support File

Before and after you upgrade the switch, run the cl-support script to create a cl-support archive file. The file is a compressed archive of useful information for troubleshooting. If you experience any issues during upgrade, you can send this archive file to the Cumulus Linux support team to investigate.

1. Create the cl-support archive file with the cl-support command:

   cumulus@switch:~$ sudo cl-support
   

2. Copy the cl-support file off the switch to a different location.

3. After upgrade is complete, run the cl-support command again to create a new archive file:

   cumulus@switch:~$ sudo cl-support
   

Upgrade Cumulus Linux

You can upgrade Cumulus Linux in one of two ways:

• Install a Cumulus Linux image of the new release, using ONIE.
• Upgrade only the changed packages using the sudo -E apt-get update and sudo -E apt-get upgrade command.

Cumulus Linux also provides the Smart System Manager that enables you to upgrade an active switch with minimal disruption to the network. See Smart System Manager.

Should I Install a Cumulus Linux Image or Upgrade Packages?

The decision to upgrade Cumulus Linux by either installing a Cumulus Linux image or upgrading packages depends on your environment and your preferences. Here are some recommendations for each upgrade method.

Installing a Cumulus Linux image is recommended if you are performing a rolling upgrade in a production environment and if you are using up-to-date and comprehensive automation scripts. This upgrade method enables you to choose the exact release to which you want to upgrade and is the only method available to upgrade your switch to a new release train (for example, from 3.7.12 to 4.1.0).

Be aware of the following when installing the Cumulus Linux image:

• Installing a Cumulus Linux image is destructive; any configuration files on the switch are not saved; copy them to a different server before you start the Cumulus Linux image install.
• You must move configuration data to the new OS using ZTP or automation while the OS is first booted, or soon afterwards using out-of-band management.
• Moving a configuration file might cause issues;
• Identifying all the locations of configuration data is not always an easy task. See Before You Upgrade Cumulus Linux above.
• Merge conflicts with configuration file changes in the new release might go undetected.
• If configuration files are not restored correctly, you might be unable to ssh to the switch from in-band management. Out-of-band connectivity (eth0 or console) is recommended.
• You must reinstall and reconfigure third-party applications after upgrade.

Package upgrade is recommended if you are upgrading from Cumulus Linux 4.0, or if you use third-party applications (package upgrade does not replace or remove third-party applications, unlike the Cumulus Linux image install).

Be aware of the following when upgrading packages:

• You cannot upgrade the switch to a new release train. For example, you cannot upgrade the switch from 3.7.x to 4.1.0.
• The sudo -E apt-get upgrade command might result in services being restarted or stopped as part of the upgrade process.
• The sudo -E apt-get upgrade command might disrupt core services by changing core service dependency packages.
• After you upgrade, account UIDs and GIDs created by packages might be different on different switches, depending on the configuration and package installation history.

Cumulus Linux Image Install (ONIE)

ONIE is an open source project (equivalent to PXE on servers) that enables the installation of network operating systems (NOS) on a bare metal switch.

To upgrade the switch:

1. Back up the configurations off the switch.

2. Download the Cumulus Linux image.

3. Install the Cumulus Linux image with the onie-install -a -i <image-location> command, which boots the switch into ONIE. The following example command installs the image from a web server, then reboots the switch. There are additional ways to install the Cumulus Linux image, such as using FTP, TFTP, a local file, or a USB drive. For more information, see Installing a New Cumulus Linux Image.

   cumulus@switch:~$ sudo onie-install -a -i http://10.0.1.251/cumulus-linux-4.1.0-mlx-amd64.bin && sudo reboot
   

4. Restore the configuration files to the new release - ideally with automation.

5. Verify correct operation with the old configurations on the new release.

6. Reinstall third party applications and associated configurations.

Package Upgrade

Cumulus Linux completely embraces the Linux and Debian upgrade workflow, where you use an installer to install a base image, then perform any upgrades within that release train with sudo -E apt-get update and sudo -E apt-get upgrade commands. Any packages that have been changed since the base install get upgraded in place from the repository. All switch configuration files remain untouched, or in rare cases merged (using the Debian merge function) during the package upgrade.

When you use package upgrade to upgrade your switch, configuration data stays in place while the packages are upgraded. If the new release updates a configuration file that you changed previously, you are prompted for the version you want to use or if you want to evaluate the differences.

Cumulus Linux 4.3.1 is supported on Broadcom switches only and requires a different upgrade procedure.

Upgrade Notes

Package upgrade always updates to the latest available release available for the switch ASIC in the Cumulus Linux repository. For example, if you are currently running Cumulus Linux 4.0.0 and run the sudo -E apt-get upgrade command on that switch, the packages are upgraded to the latest releases contained in the latest 4.y.z release.

Because Cumulus Linux is a collection of different Debian Linux packages, be aware of the following:

• The /etc/os-release and /etc/lsb-release files are updated to the currently installed Cumulus Linux release when you upgrade the switch using either package upgrade or Cumulus Linux image install. For example, if you run sudo -E apt-get upgrade and the latest Cumulus Linux release on the repository is 4.1.0, these two files display the release as 4.1.0 after the upgrade.
• The /etc/image-release file is updated only when you run a Cumulus Linux image install. Therefore, if you run a Cumulus Linux image install of Cumulus Linux 4.0.0, followed by a package upgrade to 4.1.0 using sudo -E apt-get upgrade, the /etc/image-release file continues to display Cumulus Linux 4.0.0, which is the originally installed base image.

Upgrade Switches in an MLAG Pair

If you are using MLAG to dual connect two switches in your environment, follow the steps below to upgrade the switches.

1. Verify the switch is in the secondary role:

   cumulus@switch:~$ clagctl status
   

2. Shut down the core uplink layer 3 interfaces:

   cumulus@switch:~$ sudo ip link set swpX down
   

3. Shut down the peer link:

   cumulus@switch:~$ sudo ip link set peerlink down
   

4. To boot the switch into ONIE, run the onie-install -a -i <image-location> command. The following example command installs the image from a web server. There are additional ways to install the Cumulus Linux image, such as using FTP, a local file, or a USB drive. For more information, see Installing a New Cumulus Linux Image.

   cumulus@switch:~$ sudo onie-install -a -i http://10.0.1.251/downloads/cumulus-linux-4.1.0-mlx-amd64.bin
   

   To upgrade the switch with package upgrade instead of booting into ONIE, run the sudo -E apt-get update and sudo -E apt-get upgrade commands; see Package Upgrade.

5. Reboot the switch:

   cumulus@switch:~$ sudo reboot
   

6. If you installed a new image on the switch, restore the configuration files to the new release.

7. Verify STP convergence across both switches:

   cumulus@switch:~$ mstpctl showall
   

8. Verify core uplinks and peer links are UP:

   cumulus@switch:~$ net show interface
   

9. Verify MLAG convergence:

   cumulus@switch:~$ clagctl status
   

10. Make this secondary switch the primary:

    cumulus@switch:~$ clagctl priority 2048
    

11. Verify the other switch is now in the secondary role.

12. Repeat steps 2-9 on the new secondary switch.

13. Remove the priority 2048 and restore the priority back to 32768 on the current primary switch:

    cumulus@switch:~$ clagctl priority 32768
    

Roll Back a Cumulus Linux Installation

Even the most well planned and tested upgrades can result in unforeseen problems; sometimes the best solution is to roll back to the previous state. There are three main strategies; all require detailed planning and execution:

• Flatten and rebuild: If the OS becomes unusable, you can use orchestration tools to reinstall the previous OS release from scratch and then rebuild the configuration automatically.
• Backup and restore: Another common strategy is to restore to a previous state using a backup captured before the upgrade. See Back up and Restore.

The method you employ is specific to your deployment strategy, so providing detailed steps for each scenario is outside the scope of this document.

Third Party Packages

Third party packages in the Linux host world often use the same package system as the distribution into which it is to be installed (for example, Debian uses apt-get). Or, the package might be compiled and installed by the system administrator. Configuration and executable files generally follow the same filesystem hierarchy standards as other applications.

If you install any third party applications on a Cumulus Linux switch, configuration data is typically installed into the /etc directory, but it is not guaranteed. It is your responsibility to understand the behavior and configuration file information of any third party packages installed on the switch.

After you upgrade using a full Cumulus Linux image install, you need to reinstall any third party packages or any Cumulus Linux add-on packages.

Related Information

• Upgrades: Network Device Worldview and Linux Host Worldview Comparison
• Automation Solutions
• ONIE Design Specification
• Multi-Chassis Link Aggregation - MLAG
• Zero Touch Provisioning - ZTP