Smart System Manager
Use Smart System Manager, also known as ISSU, to upgrade and troubleshoot an active switch with minimal disruption to the network.
Smart System Manager includes the following modes:
- Restart
- Upgrade
- Maintenance
- The Smart System Manager NCLU commands do not require a
net commit
.
Restart Mode
You can restart the switch in one of the following modes.
-
cold restarts the system and resets all the hardware devices on the switch (including the switching ASIC).
-
fast restarts the system more efficiently with minimal impact to traffic by reloading the kernel and software stack without a hard reset of the hardware. During a fast restart, the system decouples from the network to the extent possible using existing protocol extensions before recovering to the operational mode of the system. The restart process maintain the forwarding entries of the switching ASIC and the data plane is not affected. However, if you upgrade the SDK,
switchd
resets and reconfigures the ASIC, which interrupts the data plane. Traffic outage is much lower in this mode. -
warm restarts the system with minimal impact to traffic and without affecting the data plane. Warm mode diverts traffic from itself and restarts the system without a hardware reset of the switch ASIC. While this process does not affect the data plane, the control plane is absent during restart and is unable to process routing updates. However, if no alternate paths exist, the switch continues forwarding with the existing entries with no interruptions.
When you restart the switch in warm mode, BGP performs a graceful restart if the BGP Graceful Restart option is on. To enable BGP Graceful Restart, refer to Optional BGP Configuration.
A warm boot disrupts bonds, VXLAN traffic, and IP multicast traffic until reboot completes.
The following command restarts the system in cold mode:
cumulus@switch:~$ net system maintenance restart cold
cumulus@switch:~$ sudo csmgrctl -c
The following command restarts the system in fast mode:
cumulus@switch:~$ net system maintenance restart fast
cumulus@switch:~$ sudo csmgrctl -f
The following command restarts the system in warm mode:
Warm boot resets any manually configured FEC settings.
cumulus@switch:~$ net system maintenance restart warm
cumulus@switch:~$ sudo csmgrctl -w
Upgrade Mode
Upgrade mode updates all the components and services on the switch to the latest Cumulus Linux release without traffic loss. After upgrade is complete, you must restart the switch with either a cold or fast restart.
Upgrade mode includes the following options:
- all runs
apt-get upgrade
to upgrade all the system components to the latest release without affecting traffic flow. You must restart the system after the upgrade completes with one of the restart modes. - dry-run provides information on the components you want to upgrade.
The following command upgrades all the system components:
cumulus@switch:~$ net system maintenance upgrade all
cumulus@switch:~$ sudo csmgrctl -u
The following command provides information on the components you want to upgrade:
cumulus@switch:~$ net system maintenance upgrade dry-run
cumulus@switch:~$ sudo csmgrctl -d
Maintenance Mode
Maintenance mode isolates the system from the rest of the network so that you can perform intrusive troubleshooting tasks and data collection or perform system changes, such as break out ports and replace optics or cables with minimal disruption.
Depending on your configuration and network topology, complete isolation is not possible.
Enable Maintenance Mode
Run the following command to enable maintenance mode. When maintenance mode is on, Smart System Manager performs a graceful BGP shutdown, redirects traffic over the peerlink and brings down the MLAG port link. switchd
maintains full capability.
cumulus@switch:~$ net system maintenance mode enable
cumulus@switch:~$ sudo csmgrctl -m1
You can run additional commands to bring all the ports down, then up to restore the port admin state.
cumulus@switch:~$ net system maintenance ports down
cumulus@switch:~$ net system maintenance ports up
cumulus@switch:~$ sudo csmgrctl -p0
cumulus@switch:~$ sudo csmgrctl -p1
Before you disable maintenance mode, be sure to bring the ports back up.
Disable Maintenance Mode
Run the following command to disable maintenance mode and restore normal operation. When maintenance mode is off, Smart System Manager performs a soft restart, runs a BGP graceful restart, and brings the MLAG port link back up. switchd
maintains full capability.
cumulus@switch:~$ net system maintenance mode disable
cumulus@switch:~$ sudo csmgrctl -m0
Show Maintenance Mode Status
To see the status of maintenance mode, run the NCLU net system maintenance show status
command or the Linux sudo csmgrctl -s
command. For example:
cumulus@switch:~$ net system maintenance show status
Current System Mode: Maintenance since Tue Jan 5 00:13:37 2021 (Duration: 00:00:31)
Boot Mode: reboot_cold
2 registered modules
frr : Maintenance, down
switchd : Maintenance, down