layout-3 layout-6 layout-1 layout-top
NVIDIA Docs Hubchevron_right NVIDIA Networkingchevron_right Networking Softwarechevron_right Management Softwarechevron_right

NVIDIA NetQ

layout-top layout-6 layout-1

If you are using the current version of Cumulus NetQ, the content on this page may not be up to date. The current version of the documentation is available here. If you are redirected to the main page of the user guide, then this page may have been renamed; please search for it there.

System Event Messages Reference

The following table lists all system (including threshold-based) event messages organized by type. These messages can be viewed through third-party notification applications. For details about configuring notifications, refer to Configure Notifications.

For a list of What Just Happened events supported, refer to WJH Supported Events.

Agent Events

Type Trigger Severity Message Format Example
agent NetQ Agent state changed to Rotten (not heard from within two minutes) Critical Agent state changed to rotten Agent state changed to rotten
agent NetQ Agent state changed to Dead (user has decomissioned the agent using NetQ CLI) Critical Agent state changed to rotten Agent state changed to rotten
agent NetQ Agent rebooted Critical Netq-agent rebooted at (@last_boot) Netq-agent rebooted at 1573166417
agent Node running NetQ Agent rebooted Critical Switch rebooted at (@sys_uptime) Switch rebooted at 1573166131
agent NetQ Agent state changed to Fresh Info Agent state changed to fresh Agent state changed to fresh
agent NetQ Agent state was reset Info Agent state was paused and resumed at (@last_reinit) Agent state was paused and resumed at 1573166125
agent Version of NetQ Agent has changed Info Agent version has been changed old_version:@old_version and new_version:@new_version. Agent reset at @sys_uptime Agent version has been changed old_version:2.1.2 and new_version:2.3.1. Agent reset at 1573079725

BGP Events

Type Trigger Severity Message Format Example
bgp BGP Session state changed Critical BGP session with peer @peer @neighbor vrf @vrf state changed from @old_state to @new_state BGP session with peer leaf03 leaf04 vrf mgmt state changed from Established to Failed
bgp BGP Session state changed from Failed to Established Info BGP session with peer @peer @peerhost @neighbor vrf @vrf session state changed from Failed to Established BGP session with peer swp5 spine02 spine03 vrf default session state changed from Failed to Established
bgp BGP Session state changed from Established to Failed Info BGP session with peer @peer @neighbor vrf @vrf state changed from established to failed BGP session with peer leaf03 leaf04 vrf mgmt state changed from down to up
bgp The reset time for a BGP session changed Info BGP session with peer @peer @neighbor vrf @vrf reset time changed from @old_last_reset_time to @new_last_reset_time BGP session with peer spine03 swp9 vrf vrf2 reset time changed from 1559427694 to 1559837484

BTRFS Events

Type Trigger Severity Message Format Example
btrfsinfo Disk space available after BTRFS allocation is less than 80% of partition size or only 2 GB remain. Critical @info : @details high btrfs allocation space : greater than 80% of partition size, 61708420
btrfsinfo Indicates if space would be freed by a rebalance operation on the disk Critical @info : @details data storage efficiency : space left after allocation greater than chunk size 6170849.2","

Cable Events

Type Trigger Severity Message Format Example
cable Link speed is not the same on both ends of the link Critical @ifname speed @speed, mismatched with peer @peer @peer_if speed @peer_speed swp2 speed 10, mismatched with peer server02 swp8 speed 40
cable The speed setting for a given port changed Info @ifname speed changed from @old_speed to @new_speed swp9 speed changed from 10 to 40
cable The transceiver status for a given port changed Info @ifname transceiver changed from @old_transceiver to @new_transceiver swp4 transceiver changed from disabled to enabled
cable The vendor of a given transceiver changed Info @ifname vendor name changed from @old_vendor_name to @new_vendor_name swp23 vendor name changed from Broadcom to Mellanox
cable The part number of a given transceiver changed Info @ifname part number changed from @old_part_number to @new_part_number swp7 part number changed from FP1ZZ5654002A to MSN2700-CS2F0
cable The serial number of a given transceiver changed Info @ifname serial number changed from @old_serial_number to @new_serial_number swp4 serial number changed from 571254X1507020 to MT1552X12041
cable The status of forward error correction (FEC) support for a given port changed Info @ifname supported fec changed from @old_supported_fec to @new_supported_fec swp12 supported fec changed from supported to unsupported

swp12 supported fec changed from unsupported to supported
cable The advertised support for FEC for a given port changed Info @ifname supported fec changed from @old_advertised_fec to @new_advertised_fec swp24 supported FEC changed from advertised to not advertised
cable The FEC status for a given port changed Info @ifname fec changed from @old_fec to @new_fec swp15 fec changed from disabled to enabled

CLAG/MLAG Events

-->
Type Trigger Severity Message Format Example
clag CLAG remote peer state changed from up to down Critical Peer state changed to down Peer state changed to down
clag Local CLAG host MTU does not match its remote peer MTU Critical SVI @svi1 on vlan @vlan mtu @mtu1 mismatched with peer mtu @mtu2 SVI svi7 on vlan 4 mtu 1592 mistmatched with peer mtu 1680
clag CLAG SVI on VLAN is missing from remote peer state Warning SVI on vlan @vlan is missing from peer SVI on vlan vlan4 is missing from peer
clag CLAG peerlink is not opperating at full capacity. At least one link is down. Warning

 Clag peerlink not at full redundancy, member link @slave is down Clag peerlink not at full redundancy, member link swp40 is down
clag CLAG remote peer state changed from down to up Info Peer state changed to up Peer state changed to up
clag Local CLAG host state changed from down to up Info Clag state changed from down to up Clag state changed from down to up
clag CLAG bond in Conflicted state was updated with new bonds Info Clag conflicted bond changed from @old_conflicted_bonds to @new_conflicted_bonds Clag conflicted bond changed from swp7 swp8 to @swp9 swp10
clag CLAG bond changed state from protodown to up state Info Clag conflicted bond changed from @old_state_protodownbond to @new_state_protodownbond Clag conflicted bond changed from protodown to up

CL Support Evemts

Type Trigger Severity Message Format Example
clsupport A new CL Support file has been created for the given node Critical HostName @hostname has new CL SUPPORT file HostName leaf01 has new CL SUPPORT file

Config Diff Events

Type Trigger Severity Message Format Example
configdiff Configuration file deleted on a device Critical @hostname config file @type was deleted spine03 config file /etc/frr/frr.conf was deleted
configdiff Configuration file has been created Info @hostname config file @type was created leaf12 config file /etc/lldp.d/README.conf was created
configdiff Configuration file has been modified Info @hostname config file @type was modified spine03 config file /etc/frr/frr.conf was modified

EVPN Events

Type Trigger Severity Message Format Example
evpn A VNI was configured and moved from the up state to the down state Critical VNI @vni state changed from up to down VNI 36 state changed from up to down
evpn A VNI was configured and moved from the down state to the up state Info VNI @vni state changed from down to up VNI 36 state changed from down to up
evpn The kernel state changed on a VNI Info VNI @vni kernel state changed from @old_in_kernel_state to @new_in_kernel_state VNI 3 kernel state changed from down to up
evpn A VNI state changed from not advertising all VNIs to advertising all VNIs Info VNI @vni vni state changed from @old_adv_all_vni_state to @new_adv_all_vni_state VNI 11 vni state changed from false to true

Lifecycle Management Events

Type Trigger Severity Message Format Example
lcm Cumulus Linux backup started for a switch or host Info CL configuration backup started for hostname @hostname CL configuration backup started for hostname spine01
lcm Cumulus Linux backup completed for a switch or host Info CL configuration backup completed for hostname @hostname CL configuration backup completed for hostname spine01
lcm Cumulus Linux backup failed for a switch or host Critical CL configuration backup failed for hostname @hostname CL configuration backup failed for hostname spine01
lcm Cumulus Linux upgrade from one version to a newer version has started for a switch or host Critical CL Image upgrade from version @old_cl_version to version @new_cl_version started for hostname @hostname CL Image upgrade from version 4.1.0 to version 4.2.1 started for hostname server01
lcm Cumulus Linux upgrade from one version to a newer version has completed successfully for a switch or host Info CL Image upgrade from version @old_cl_version to version @new_cl_version completed for hostname @hostname CL Image upgrade from version 4.1.0 to version 4.2.1 completed for hostname server01
lcm Cumulus Linux upgrade from one version to a newer version has failed for a switch or host Critical CL Image upgrade from version @old_cl_version to version @new_cl_version failed for hostname @hostname CL Image upgrade from version 4.1.0 to version 4.2.1 failed for hostname server01
lcm Restoration of a Cumulus Linux configuration started for a switch or host Info CL configuration restore started for hostname @hostname CL configuration restore started for hostname leaf01
lcm Restoration of a Cumulus Linux configuration completed successfully for a switch or host Info CL configuration restore completed for hostname @hostname CL configuration restore completed for hostname leaf01
lcm Restoration of a Cumulus Linux configuration failed for a switch or host Critical CL configuration restore failed for hostname @hostname CL configuration restore failed for hostname leaf01
lcm Rollback of a Cumulus Linux image has started for a switch or host Critical CL Image rollback from version @old_cl_version to version @new_cl_version started for hostname @hostname CL Image rollback from version 4.2.1 to version 4.1.0 started for hostname leaf01
lcm Rollback of a Cumulus Linux image has completed successfully for a switch or host Info CL Image rollback from version @old_cl_version to version @new_cl_version completed for hostname @hostname CL Image rollback from version 4.2.1 to version 4.1.0 completed for hostname leaf01
lcm Rollback of a Cumulus Linux image has failed for a switch or host Critical CL Image rollback from version @old_cl_version to version @new_cl_version failed for hostname @hostname CL Image rollback from version 4.2.1 to version 4.1.0 failed for hostname leaf01
lcm Installation of a Cumulus NetQ image has started for a switch or host Info NetQ Image version @netq_version installation started for hostname @hostname NetQ Image version 3.2.0 installation started for hostname spine02
lcm Installation of a Cumulus NetQ image has completed successfully for a switch or host Info NetQ Image version @netq_version installation completed for hostname @hostname NetQ Image version 3.2.0 installation completed for hostname spine02
lcm Installation of a Cumulus NetQ image has failed for a switch or host Critical NetQ Image version @netq_version installation failed for hostname @hostname NetQ Image version 3.2.0 installation failed for hostname spine02
lcm Upgrade of a Cumulus NetQ image has started for a switch or host Info NetQ Image upgrade from version @old_netq_version to version @netq_version started for hostname @hostname NetQ Image upgrade from version 3.1.0 to version 3.2.0 started for hostname spine02
lcm Upgrade of a Cumulus NetQ image has completed successfully for a switch or host Info NetQ Image upgrade from version @old_netq_version to version @netq_version completed for hostname @hostname NetQ Image upgrade from version 3.1.0 to version 3.2.0 completed for hostname spine02
lcm Upgrade of a Cumulus NetQ image has failed for a switch or host Critical NetQ Image upgrade from version @old_netq_version to version @netq_version failed for hostname @hostname NetQ Image upgrade from version 3.1.0 to version 3.2.0 failed for hostname spine02

Cumulus Linux License Events

Type Trigger Severity Message Format Example
license License state is missing or invalid Critical License check failed, name @lic_name state @state License check failed, name agent.lic state invalid
license License state is missing or invalid on a particular device Critical License check failed on @hostname License check failed on leaf03
Type Trigger Severity Message Format Example
link Link operational state changed from up to down Critical HostName @hostname changed state from @old_state to @new_state Interface:@ifname HostName leaf01 changed state from up to down Interface:swp34
link Link operational state changed from down to up Info HostName @hostname changed state from @old_state to @new_state Interface:@ifname HostName leaf04 changed state from down to up Interface:swp11

LLDP Events

Type Trigger Severity Message Format Example
lldp Local LLDP host has new neighbor information Info LLDP Session with host @hostname and @ifname modified fields @changed_fields LLDP Session with host leaf02 swp6 modified fields leaf06 swp21
lldp Local LLDP host has new peer interface name Info LLDP Session with host @hostname and @ifname @old_peer_ifname changed to @new_peer_ifname LLDP Session with host spine01 and swp5 swp12 changed to port12
lldp Local LLDP host has new peer hostname Info LLDP Session with host @hostname and @ifname @old_peer_hostname changed to @new_peer_hostname LLDP Session with host leaf03 and swp2 leaf07 changed to exit01

MTU Events

Type Trigger Severity Message Format Example
mtu VLAN interface link MTU is smaller than that of its parent MTU Warning vlan interface @link mtu @mtu is smaller than parent @parent mtu @parent_mtu vlan interface swp3 mtu 1500 is smaller than parent peerlink-1 mtu 1690
mtu Bridge interface MTU is smaller than the member interface with the smallest MTU Warning bridge @link mtu @mtu is smaller than least of member interface mtu @min bridge swp0 mtu 1280 is smaller than least of member interface mtu 1500

NTP Events

Type Trigger Severity Message Format Example
ntp NTP sync state changed from in sync to not in sync Critical Sync state changed from @old_state to @new_state for @hostname Sync state changed from in sync to not sync for leaf06
ntp NTP sync state changed from not in sync to in sync Info Sync state changed from @old_state to @new_state for @hostname Sync state changed from not sync to in sync for leaf06

OSPF Events

Type Trigger Severity Message Format Example
ospf OSPF session state on a given interface changed from Full to a down state Critical OSPF session @ifname with @peer_address changed from Full to @down_state

OSPF session swp7 with 27.0.0.18 state changed from Full to Fail

OSPF session swp7 with 27.0.0.18 state changed from Full to ExStart
ospf OSPF session state on a given interface changed from a down state to full Info OSPF session @ifname with @peer_address changed from @down_state to Full

OSPF session swp7 with 27.0.0.18 state changed from Down to Full

OSPF session swp7 with 27.0.0.18 state changed from Init to Full

OSPF session swp7 with 27.0.0.18 state changed from Fail to Full

Package Information Events

Type Trigger Severity Message Format Example
packageinfo Package version on device does not match the version identified in the existing manifest Critical @package_name manifest version mismatch netq-apps manifest version mismatch

PTM Events

Type Trigger Severity Message Format Example
ptm Physical interface cabling does not match configuration specified in topology.dot file Critical PTM cable status failed PTM cable status failed
ptm Physical interface cabling matches configuration specified in topology.dot file Critical PTM cable status passed PTM cable status passed

Resource Events

Type Trigger Severity Message Format Example
resource A physical resource has been deleted from a device Critical Resource Utils deleted for @hostname Resource Utils deleted for spine02
resource Root file system access on a device has changed from Read/Write to Read Only Critical @hostname root file system access mode set to Read Only server03 root file system access mode set to Read Only
resource Root file system access on a device has changed from Read Only to Read/Write Info @hostname root file system access mode set to Read/Write leaf11 root file system access mode set to Read/Write
resource A physical resource has been added to a device Info Resource Utils added for @hostname Resource Utils added for spine04

Running Config Diff Events

Type Trigger Severity Message Format Example
runningconfigdiff Running configuration file has been modified Info @commandname config result was modified @commandname config result was modified

Sensor Events

Type Trigger Severity Message Format Example
sensor A fan or power supply unit sensor has changed state Critical Sensor @sensor state changed from @old_s_state to @new_s_state Sensor fan state changed from up to down
sensor A temperature sensor has crossed the maximum threshold for that sensor Critical Sensor @sensor max value @new_s_max exceeds threshold @new_s_crit Sensor temp max value 110 exceeds the threshold 95
sensor A temperature sensor has crossed the minimum threshold for that sensor Critical Sensor @sensor min value @new_s_lcrit fall behind threshold @new_s_min Sensor psu min value 10 fell below threshold 25
sensor A temperature, fan, or power supply sensor state changed Info Sensor @sensor state changed from @old_state to @new_state

Sensor temperature state changed from critical to ok

Sensor fan state changed from absent to ok

Sensor psu state changed from bad to ok
sensor A fan or power supply sensor state changed Info Sensor @sensor state changed from @old_s_state to @new_s_state

Sensor fan state changed from down to up

Sensor psu state changed from down to up
sensor A fan or power supply unit sensor is in a new state Critical Sensor @sensor state is @new_s_state Sensor psu state is bad

Services Events

Type Trigger Severity Message Format Example
services A service status changed from down to up Critical Service @name status changed from @old_status to @new_status Service bgp status changed from down to up
services A service status changed from up to down Critical Service @name status changed from @old_status to @new_status Service lldp status changed from up to down
services A service changed state from inactive to active Info Service @name changed state from inactive to active

Service bgp changed state from inactive to active

Service lldp changed state from inactive to active

SSD Utilization Events

Type Trigger Severity Message Format Example
ssdutil 3ME3 disk health has dropped below 10% Critical @info: @details low health : 5.0%
ssdutil A dip in 3ME3 disk health of more than 2% has occurred within the last 24 hours Critical @info: @details significant health drop : 3.0%

Threshold-based Events

Type Trigger Severity Message Format Example
tca Percentage of CPU utilization exceeded user-defined maximum threshold on a switch Critical CPU Utilization for host @hostname exceed configured mark @cpu_utilization CPU Utilization for host leaf11 exceed configured mark 85
tca Percentage of disk utilization exceeded user-defined maximum threshold on a switch Critical Disk Utilization for host @hostname exceed configured mark @disk_utilization Disk Utilization for host leaf11 exceed configured mark 90
tca Percentage of memory utilization exceeded user-defined maximum threshold on a switch Critical Memory Utilization for host @hostname exceed configured mark @mem_utilization Memory Utilization for host leaf11 exceed configured mark 95
tca Number of transmit bytes exceeded user-defined maximum threshold on a switch interface Critical TX bytes upper threshold breached for host @hostname ifname:@ifname value: @tx_bytes TX bytes upper threshold breached for host spine02 ifname:swp4 value: 20000
tca Number of broadcast transmit bytes exceeded user-defined maximum threshold on a switch interface Critical TX broadcast upper threshold breached for host @hostname ifname:@ifname value: @rx_broadcast TX broadcast upper threshold breached for host leaf04 ifname:swp45 value: 40200
tca Number of multicast transmit bytes exceeded user-defined maximum threshold on a switch interface Critical TX multicast upper threshold breached for host @hostname ifname:@ifname value: @rx_broadcast TX multicast upper threshold breached for host leaf04 ifname:swp45 value: 30000
tca Number of receive bytes exceeded user-defined maximum threshold on a switch interface Critical RX bytes upper threshold breached for host @hostname ifname:@ifname value: @tx_bytes RX bytes upper threshold breached for host spine02 ifname:swp4 value: 20000
tca Number of broadcast receive bytes exceeded user-defined maximum threshold on a switch interface Critical RX broadcast upper threshold breached for host @hostname ifname:@ifname value: @rx_broadcast RX broadcast upper threshold breached for host leaf04 ifname:swp45 value: 40200
tca Number of multicast receive bytes exceeded user-defined maximum threshold on a switch interface Critical RX multicast upper threshold breached for host @hostname ifname:@ifname value: @rx_broadcast RX multicast upper threshold breached for host leaf04 ifname:swp45 value: 30000
tca Fan speed exceeded user-defined maximum threshold on a switch Critical Sensor for @hostname exceeded threshold fan speed @s_input for sensor @s_name Sensor for spine03 exceeded threshold fan speed 700 for sensor fan2
tca Power supply output exceeded user-defined maximum threshold on a switch Critical Sensor for @hostname exceeded threshold power @s_input watts for sensor @s_name Sensor for leaf14 exceeded threshold power 120 watts for sensor psu1
tca Temperature (° C) exceeded user-defined maximum threshold on a switch Critical Sensor for @hostname exceeded threshold temperature @s_input for sensor @s_name Sensor for leaf14 exceeded threshold temperature 90 for sensor temp1
tca Power supply voltage exceeded user-defined maximum threshold on a switch Critical Sensor for @hostname exceeded threshold voltage @s_input volts for sensor @s_name Sensor for leaf14 exceeded threshold voltage 12 volts for sensor psu2

Version Events

Type Trigger Severity Message Format Example
version An unknown version of the operating system was detected Critical unexpected os version @my_ver unexpected os version cl3.2
version Desired version of the operating system is not available Critical os version @ver os version cl3.7.9
version An unknown version of a software package was detected Critical expected release version @ver expected release version cl3.6.2
version Desired version of a software package is not available Critical different from version @ver different from version cl4.0

VXLAN Events

Type Trigger Severity Message Format Example
vxlan Replication list is contains an inconsistent set of nodes<> Critical<> VNI @vni replication list inconsistent with @conflicts diff:@diff<> VNI 14 replication list inconsistent with ["leaf03","leaf04"] diff:+:["leaf03","leaf04"] -:["leaf07","leaf08"]

    Copyright © 2024  NVIDIA Corporation