Layer 3 Extensions

Layer 3 extensions use EVPN as a control plane and are similar to layer 3 VPN with VXLAN tunnels. The leaf switches set up full mesh VXLAN tunnels within and across PODs, and the routing exchange between pods occurs with EVPN type-5 routes. Within the pod, there are type-1 and type-4 routes for EVPN multihoming, type-2 for MAC addresses, IP addresses and MAC routes, and type-3 for BUM HER routes.

Configuration

DCI 1 DCI 2
VRFRED
Layer 2 VNI10
Layer 3 VNI4001
VRFRED
Layer 2 VNI1010
Layer 3 VNI5001
VRFGREEN
Layer 2 VNI20
Layer 3 VNI4002
VRFGREEN
Layer 2 VNI2020
Layer 3 VNI5002

The following configuration example connects VRF RED in DC1 with VRF RED in DC2, using downstream VNI and symmetrical routing. The route-target import statements connect two RED VRFs at layer 3 (for prefix exchange). This configuration provides IP connectivity between server01 and server03 within VRF RED, and server02 and server04 within VRF GREEN, but the RED and GREEN VRFs cannot communicate with each other. All servers are in different IP subnets; there is no layer 2 adjacency between them. A server communicates with its peer in the other DC through its default gateway, which is the local VRR MAC address in the ARP cache.

The example shows a layer 3 interconnect configuration, where the border leafs filter EVPN prefixes (except type-5) to distribute across DCI links. This configuration ensures that DCI only exchanges type-5 prefixes, and that the remote DC does not receive and process unwanted prefix types. The ESI and MAC addresses are visible for each local POD, but not across PODs.

ubuntu@server01:~$ cat /etc/netplan/config.yaml 
################################################################ 
# IMPORTANT: When using NVIDIA Air services,                   # 
#  your Internet-facing interface must include the following:  # 
#   dhcp6: false                                               # 
#   accept-ra: true                                            # 
################################################################ 
network: 
  version: 2 
  renderer: networkd 
  ethernets: 
    eth0: 
      dhcp4: true 
    eth1: 
      dhcp4: false 
    eth2: 
      dhcp4: false 
  bonds: 
      bond0: 
          interfaces: [eth1, eth2] 
          addresses: [192.168.1.10/24] 
          gateway4: 192.168.1.1 
          routes: 
          - to: 192.168.0.0/16 
            via: 192.168.1.1 
          parameters: 
              mode: 802.3ad 
  version: 2 
ubuntu@server01:~$ ip addr 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 
    inet 127.0.0.1/8 scope host lo 
       valid_lft forever preferred_lft forever 
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 
    link/ether 44:38:39:22:aa:01 brd ff:ff:ff:ff:ff:ff 
    inet 192.168.200.20/24 brd 192.168.200.255 scope global dynamic eth0 
       valid_lft 163305sec preferred_lft 163305sec 
    inet6 fe80::4638:39ff:fe22:aa01/64 scope link 
       valid_lft forever preferred_lft forever 
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000 
    link/ether 42:20:47:91:95:a7 brd ff:ff:ff:ff:ff:ff 
4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000 
    link/ether 42:20:47:91:95:a7 brd ff:ff:ff:ff:ff:ff 
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 
    link/ether 42:20:47:91:95:a7 brd ff:ff:ff:ff:ff:ff 
    inet 192.168.1.10/24 brd 192.168.1.255 scope global bond0 
       valid_lft forever preferred_lft forever 
    inet6 fe80::4020:47ff:fe91:95a7/64 scope link 
       valid_lft forever preferred_lft forever 
ubuntu@server03:~$ cat /etc/netplan/config.yaml 
################################################################ 
# IMPORTANT: When using NVIDIA Air services,                   # 
#  your Internet-facing interface must include the following:  # 
#   dhcp6: false                                               # 
#   accept-ra: true                                            # 
################################################################ 
network: 
  version: 2 
  renderer: networkd 
  ethernets: 
    eth0: 
      dhcp4: true 
    eth1: 
      dhcp4: false 
    eth2: 
      dhcp4: false 
  bonds: 
      bond0: 
          interfaces: [eth1, eth2] 
          addresses: [192.168.10.110/24] 
          gateway4: 192.168.10.100 
          routes: 
          - to: 192.168.0.0/16 
            via: 192.168.10.100 
          parameters: 
              mode: 802.3ad 
  version: 2 
ubuntu@server03:~$ ip addr 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 
    inet 127.0.0.1/8 scope host lo 
       valid_lft forever preferred_lft forever 
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 
    link/ether 44:38:39:22:aa:03 brd ff:ff:ff:ff:ff:ff 
    inet 192.168.200.22/24 brd 192.168.200.255 scope global dynamic eth0 
       valid_lft 163264sec preferred_lft 163264sec 
    inet6 fe80::4638:39ff:fe22:aa03/64 scope link 
       valid_lft forever preferred_lft forever 
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000 
    link/ether b6:4b:0f:ea:f2:02 brd ff:ff:ff:ff:ff:ff 
4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000 
    link/ether b6:4b:0f:ea:f2:02 brd ff:ff:ff:ff:ff:ff 
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 
    link/ether b6:4b:0f:ea:f2:02 brd ff:ff:ff:ff:ff:ff 
    inet 192.168.10.110/24 brd 192.168.10.255 scope global bond0 
       valid_lft forever preferred_lft forever 
    inet6 fe80::b44b:fff:feea:f202/64 scope link 
       valid_lft forever preferred_lft forever 
cumulus@leaf01:mgmt:~$ nv set bridge domain br_default vlan 10 vni 10 
cumulus@leaf01:mgmt:~$ nv set bridge domain br_default vlan 20 vni 20 
cumulus@leaf01:mgmt:~$ nv set evpn enable on 
cumulus@leaf01:mgmt:~$ nv set evpn multihoming enable on 
cumulus@leaf01:mgmt:~$ nv set interface bond1 bond member swp3 
cumulus@leaf01:mgmt:~$ nv set interface bond1 bridge domain br_default access 10 
cumulus@leaf01:mgmt:~$ nv set interface bond1 evpn multihoming segment local-id 1 
cumulus@leaf01:mgmt:~$ nv set interface bond1-2 bond lacp-bypass on 
cumulus@leaf01:mgmt:~$ nv set interface bond1-2 evpn multihoming segment df-preference 50000 
cumulus@leaf01:mgmt:~$ nv set interface bond1-2 evpn multihoming segment enable on 
cumulus@leaf01:mgmt:~$ nv set interface bond1-2 evpn multihoming segment mac-address 00:00:00:00:00:AA 
cumulus@leaf01:mgmt:~$ nv set interface bond1-2 type bond 
cumulus@leaf01:mgmt:~$ nv set interface bond2 bond member swp4 
cumulus@leaf01:mgmt:~$ nv set interface bond2 bridge domain br_default access 20 
cumulus@leaf01:mgmt:~$ nv set interface bond2 evpn multihoming segment local-id 2 
cumulus@leaf01:mgmt:~$ nv set interface eth0 ip vrf mgmt 
cumulus@leaf01:mgmt:~$ nv set interface eth0 type eth 
cumulus@leaf01:mgmt:~$ nv set interface lo ip address 10.10.10.1/32 
cumulus@leaf01:mgmt:~$ nv set interface lo type loopback 
cumulus@leaf01:mgmt:~$ nv set interface swp1-2 evpn multihoming uplink on 
cumulus@leaf01:mgmt:~$ nv set interface swp1-2 type swp 
cumulus@leaf01:mgmt:~$ nv set interface vlan10 ip address 192.168.1.2/24 
cumulus@leaf01:mgmt:~$ nv set interface vlan10 ip vrf RED 
cumulus@leaf01:mgmt:~$ nv set interface vlan10 ip vrr address 192.168.1.1/24 
cumulus@leaf01:mgmt:~$ nv set interface vlan10 vlan 10 
cumulus@leaf01:mgmt:~$ nv set interface vlan10,20 ip vrr enable on 
cumulus@leaf01:mgmt:~$ nv set interface vlan10,20 ip vrr state up 
cumulus@leaf01:mgmt:~$ nv set interface vlan10,20 type svi 
cumulus@leaf01:mgmt:~$ nv set interface vlan20 ip address 192.168.2.2/24 
cumulus@leaf01:mgmt:~$ nv set interface vlan20 ip vrf GREEN 
cumulus@leaf01:mgmt:~$ nv set interface vlan20 ip vrr address 192.168.2.1/24 
cumulus@leaf01:mgmt:~$ nv set interface vlan20 vlan 20 
cumulus@leaf01:mgmt:~$ nv set nve vxlan arp-nd-suppress on 
cumulus@leaf01:mgmt:~$ nv set nve vxlan enable on 
cumulus@leaf01:mgmt:~$ nv set nve vxlan source address 10.10.10.1 
cumulus@leaf01:mgmt:~$ nv set router bgp autonomous-system 65101 
cumulus@leaf01:mgmt:~$ nv set router bgp enable on 
cumulus@leaf01:mgmt:~$ nv set router bgp router-id 10.10.10.1 
cumulus@leaf01:mgmt:~$ nv set router vrr enable on 
cumulus@leaf01:mgmt:~$ nv set service lldp 
cumulus@leaf01:mgmt:~$ nv set system config auto-save enable on 
cumulus@leaf01:mgmt:~$ nv set system global anycast-id 10 
cumulus@leaf01:mgmt:~$ nv set system global fabric-id 10 
cumulus@leaf01:mgmt:~$ nv set system hostname leaf01 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN evpn enable on 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN evpn vni 4002 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast enable on 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp autonomous-system 65101 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp enable on 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:4002 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:5002 
cumulus@leaf01:mgmt:~$ nv set vrf GREEN router bgp router-id 10.10.10.1 
cumulus@leaf01:mgmt:~$ nv set vrf RED evpn enable on 
cumulus@leaf01:mgmt:~$ nv set vrf RED evpn vni 4001 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast enable on 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp autonomous-system 65101 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp enable on 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:4001 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:5001 
cumulus@leaf01:mgmt:~$ nv set vrf RED router bgp router-id 10.10.10.1 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast enable on 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.10.10.1/32 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp enable on 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp neighbor swp1 peer-group underlay 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp neighbor swp1 type unnumbered 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp neighbor swp2 peer-group underlay 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp neighbor swp2 type unnumbered 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp peer-group underlay address-family l2vpn-evpn enable on 
cumulus@leaf01:mgmt:~$ nv set vrf default router bgp peer-group underlay remote-as external 
cumulus@spine01:mgmt:~$ nv set interface eth0 ip vrf mgmt 
cumulus@spine01:mgmt:~$ nv set interface eth0 type eth 
cumulus@spine01:mgmt:~$ nv set interface lo ip address 10.10.10.101/32 
cumulus@spine01:mgmt:~$ nv set interface lo type loopback 
cumulus@spine01:mgmt:~$ nv set interface swp1-4 type swp 
cumulus@spine01:mgmt:~$ nv set router bgp autonomous-system 65199 
cumulus@spine01:mgmt:~$ nv set router bgp enable on 
cumulus@spine01:mgmt:~$ nv set router bgp router-id 10.10.10.101 
cumulus@spine01:mgmt:~$ nv set system config auto-save enable on 
cumulus@spine01:mgmt:~$ nv set system hostname spine01 
cumulus@spine01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast enable on 
cumulus@spine01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.10.10.101/32 
cumulus@spine01:mgmt:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on 
cumulus@spine01:mgmt:~$ nv set vrf default router bgp enable on 
cumulus@spine01:mgmt:~$ nv set vrf default router bgp neighbor swp1-4 address-family l2vpn-evpn enable on 
cumulus@spine01:mgmt:~$ nv set vrf default router bgp neighbor swp1-4 remote-as external 
cumulus@spine01:mgmt:~$ nv set vrf default router bgp neighbor swp1-4 type unnumbered 
cumulus@borderleaf01:mgmt:~$ nv set evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set interface eth0 ip vrf mgmt 
cumulus@borderleaf01:mgmt:~$ nv set interface eth0 type eth 
cumulus@borderleaf01:mgmt:~$ nv set interface lo ip address 10.10.10.10/32 
cumulus@borderleaf01:mgmt:~$ nv set interface lo type loopback 
cumulus@borderleaf01:mgmt:~$ nv set interface swp1-3 type swp 
cumulus@borderleaf01:mgmt:~$ nv set nve vxlan enable on 
cumulus@borderleaf01:mgmt:~$ nv set router bgp autonomous-system 65110 
cumulus@borderleaf01:mgmt:~$ nv set router bgp enable on 
cumulus@borderleaf01:mgmt:~$ nv set router bgp router-id 10.10.10.10 
cumulus@borderleaf01:mgmt:~$ nv set router policy route-map control_t5 rule 1 action permit 
cumulus@borderleaf01:mgmt:~$ nv set router policy route-map control_t5 rule 1 match evpn-route-type ip-prefix 
cumulus@borderleaf01:mgmt:~$ nv set router policy route-map control_t5 rule 2 action deny 
cumulus@borderleaf01:mgmt:~$ nv set service lldp 
cumulus@borderleaf01:mgmt:~$ nv set system config auto-save enable on 
cumulus@borderleaf01:mgmt:~$ nv set system global anycast-id 10 
cumulus@borderleaf01:mgmt:~$ nv set system global fabric-id 10 
cumulus@borderleaf01:mgmt:~$ nv set system hostname borderleaf01 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN evpn vni 4002 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp autonomous-system 65110 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp peer-group underlay address-family l2vpn-evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:4002 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:5002 
cumulus@borderleaf01:mgmt:~$ nv set vrf GREEN router bgp router-id 10.10.10.10 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED evpn vni 4001 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp autonomous-system 65110 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp peer-group underlay address-family l2vpn-evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:4001 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:5001 
cumulus@borderleaf01:mgmt:~$ nv set vrf RED router bgp router-id 10.10.10.10 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.10.10.10/32 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp neighbor swp1 peer-group underlay 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp neighbor swp1 type unnumbered 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp neighbor swp2 peer-group underlay 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp neighbor swp2 type unnumbered 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp neighbor swp3 peer-group dci_group1 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp neighbor swp3 type unnumbered 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp peer-group dci_group1 address-family l2vpn-evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp peer-group dci_group1 address-family l2vpn-evpn policy outbound route-map control_t5 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp peer-group dci_group1 remote-as external 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp peer-group underlay address-family l2vpn-evpn enable on 
cumulus@borderleaf01:mgmt:~$ nv set vrf default router bgp peer-group underlay remote-as external 
cumulus@leaf03:mgmt:~$ nv set bridge domain br_default vlan 1010 vni 1010 
cumulus@leaf03:mgmt:~$ nv set bridge domain br_default vlan 2020 vni 2020 
cumulus@leaf03:mgmt:~$ nv set evpn enable on 
cumulus@leaf03:mgmt:~$ nv set evpn multihoming enable on 
cumulus@leaf03:mgmt:~$ nv set interface bond1 bond member swp3 
cumulus@leaf03:mgmt:~$ nv set interface bond1 bridge domain br_default access 1010 
cumulus@leaf03:mgmt:~$ nv set interface bond1 evpn multihoming segment local-id 1 
cumulus@leaf03:mgmt:~$ nv set interface bond1-2 bond lacp-bypass on 
cumulus@leaf03:mgmt:~$ nv set interface bond1-2 evpn multihoming segment df-preference 50000 
cumulus@leaf03:mgmt:~$ nv set interface bond1-2 evpn multihoming segment enable on 
cumulus@leaf03:mgmt:~$ nv set interface bond1-2 evpn multihoming segment mac-address 00:00:00:00:00:BB 
cumulus@leaf03:mgmt:~$ nv set interface bond1-2 type bond 
cumulus@leaf03:mgmt:~$ nv set interface bond2 bond member swp4 
cumulus@leaf03:mgmt:~$ nv set interface bond2 bridge domain br_default access 2020 
cumulus@leaf03:mgmt:~$ nv set interface bond2 evpn multihoming segment local-id 2 
cumulus@leaf03:mgmt:~$ nv set interface eth0 ip vrf mgmt 
cumulus@leaf03:mgmt:~$ nv set interface eth0 type eth 
cumulus@leaf03:mgmt:~$ nv set interface lo ip address 10.10.20.1/32 
cumulus@leaf03:mgmt:~$ nv set interface lo type loopback 
cumulus@leaf03:mgmt:~$ nv set interface swp1-2 evpn multihoming uplink on 
cumulus@leaf03:mgmt:~$ nv set interface swp1-2 type swp 
cumulus@leaf03:mgmt:~$ nv set interface vlan1010 ip address 192.168.10.101/24 
cumulus@leaf03:mgmt:~$ nv set interface vlan1010 ip vrf RED 
cumulus@leaf03:mgmt:~$ nv set interface vlan1010 ip vrr address 192.168.10.100/24 
cumulus@leaf03:mgmt:~$ nv set interface vlan1010 vlan 1010 
cumulus@leaf03:mgmt:~$ nv set interface vlan1010,2020 ip vrr enable on 
cumulus@leaf03:mgmt:~$ nv set interface vlan1010,2020 ip vrr state up 
cumulus@leaf03:mgmt:~$ nv set interface vlan1010,2020 type svi 
cumulus@leaf03:mgmt:~$ nv set interface vlan2020 ip address 192.168.20.101/24 
cumulus@leaf03:mgmt:~$ nv set interface vlan2020 ip vrf GREEN 
cumulus@leaf03:mgmt:~$ nv set interface vlan2020 ip vrr address 192.168.20.100/24 
cumulus@leaf03:mgmt:~$ nv set interface vlan2020 vlan 2020 
cumulus@leaf03:mgmt:~$ nv set nve vxlan arp-nd-suppress on 
cumulus@leaf03:mgmt:~$ nv set nve vxlan enable on 
cumulus@leaf03:mgmt:~$ nv set nve vxlan source address 10.10.20.1 
cumulus@leaf03:mgmt:~$ nv set router bgp autonomous-system 65201 
cumulus@leaf03:mgmt:~$ nv set router bgp enable on 
cumulus@leaf03:mgmt:~$ nv set router bgp router-id 10.10.20.1 
cumulus@leaf03:mgmt:~$ nv set router vrr enable on 
cumulus@leaf03:mgmt:~$ nv set service lldp 
cumulus@leaf03:mgmt:~$ nv set system config auto-save enable on 
cumulus@leaf03:mgmt:~$ nv set system global anycast-id 20 
cumulus@leaf03:mgmt:~$ nv set system global fabric-id 20 
cumulus@leaf03:mgmt:~$ nv set system hostname leaf03 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN evpn enable on 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN evpn vni 5002 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast enable on 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp autonomous-system 65201 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp enable on 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:4002 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:5002 
cumulus@leaf03:mgmt:~$ nv set vrf GREEN router bgp router-id 10.10.20.1 
cumulus@leaf03:mgmt:~$ nv set vrf RED evpn enable on 
cumulus@leaf03:mgmt:~$ nv set vrf RED evpn vni 5001 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast enable on 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp autonomous-system 65201 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp enable on 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:4001 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:5001 
cumulus@leaf03:mgmt:~$ nv set vrf RED router bgp router-id 10.10.20.1 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast enable on 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.10.20.1/32 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp enable on 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp neighbor swp1 address-family l2vpn-evpn enable on 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp neighbor swp1 remote-as external 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp neighbor swp1 type unnumbered 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp neighbor swp2 address-family l2vpn-evpn enable on 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp neighbor swp2 remote-as external 
cumulus@leaf03:mgmt:~$ nv set vrf default router bgp neighbor swp2 type unnumbered 
cumulus@spine03:mgmt:~$ nv set interface eth0 ip vrf mgmt 
cumulus@spine03:mgmt:~$ nv set interface eth0 type eth 
cumulus@spine03:mgmt:~$ nv set interface lo ip address 10.10.20.103/32 
cumulus@spine03:mgmt:~$ nv set interface lo type loopback 
cumulus@spine03:mgmt:~$ nv set interface swp1-4 type swp 
cumulus@spine03:mgmt:~$ nv set router bgp autonomous-system 65299 
cumulus@spine03:mgmt:~$ nv set router bgp enable on 
cumulus@spine03:mgmt:~$ nv set router bgp router-id 10.10.20.103 
cumulus@spine03:mgmt:~$ nv set system config auto-save enable on 
cumulus@spine03:mgmt:~$ nv set system global anycast-id 20 
cumulus@spine03:mgmt:~$ nv set system global fabric-id 20 
cumulus@spine03:mgmt:~$ nv set system hostname spine03 
cumulus@spine03:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast enable on 
cumulus@spine03:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.10.20.103/32 
cumulus@spine03:mgmt:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on 
cumulus@spine03:mgmt:~$ nv set vrf default router bgp enable on 
cumulus@spine03:mgmt:~$ nv set vrf default router bgp neighbor swp1-4 address-family l2vpn-evpn enable on 
cumulus@spine03:mgmt:~$ nv set vrf default router bgp neighbor swp1-4 remote-as external 
cumulus@spine03:mgmt:~$ nv set vrf default router bgp neighbor swp1-4 type unnumbered 
cumulus@borderleaf04:mgmt:~$ nv set evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set interface eth0 ip vrf mgmt 
cumulus@borderleaf04:mgmt:~$ nv set interface eth0 type eth 
cumulus@borderleaf04:mgmt:~$ nv set interface lo ip address 10.10.20.11/32 
cumulus@borderleaf04:mgmt:~$ nv set interface lo type loopback 
cumulus@borderleaf04:mgmt:~$ nv set interface swp1-3 type swp 
cumulus@borderleaf04:mgmt:~$ nv set nve vxlan enable on 
cumulus@borderleaf04:mgmt:~$ nv set router bgp autonomous-system 65210 
cumulus@borderleaf04:mgmt:~$ nv set router bgp enable on 
cumulus@borderleaf04:mgmt:~$ nv set router bgp router-id 10.10.20.11 
cumulus@borderleaf04:mgmt:~$ nv set router policy route-map control_t5 rule 1 action permit 
cumulus@borderleaf04:mgmt:~$ nv set router policy route-map control_t5 rule 1 match evpn-route-type ip-prefix 
cumulus@borderleaf04:mgmt:~$ nv set router policy route-map control_t5 rule 2 action deny 
cumulus@borderleaf04:mgmt:~$ nv set service lldp 
cumulus@borderleaf04:mgmt:~$ nv set system config auto-save enable on 
cumulus@borderleaf04:mgmt:~$ nv set system global anycast-id 20 
cumulus@borderleaf04:mgmt:~$ nv set system global fabric-id 20 
cumulus@borderleaf04:mgmt:~$ nv set system hostname borderleaf04 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN evpn vni 5002 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp autonomous-system 65210 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp peer-group underlay address-family l2vpn-evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:4002 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp route-import from-evpn route-target ANY:5002 
cumulus@borderleaf04:mgmt:~$ nv set vrf GREEN router bgp router-id 10.10.20.11 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED evpn vni 5001 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp address-family ipv4-unicast route-export to-evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp autonomous-system 65210 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp peer-group underlay address-family l2vpn-evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:4001 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp route-import from-evpn route-target ANY:5001 
cumulus@borderleaf04:mgmt:~$ nv set vrf RED router bgp router-id 10.10.20.11 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.10.20.11/32 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp neighbor swp1 peer-group underlay 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp neighbor swp1 type unnumbered 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp neighbor swp2 peer-group underlay 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp neighbor swp2 type unnumbered 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp neighbor swp3 peer-group dci_group1 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp neighbor swp3 type unnumbered 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp peer-group dci_group1 address-family l2vpn-evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp peer-group dci_group1 address-family l2vpn-evpn policy outbound route-map control_t5 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp peer-group dci_group1 remote-as external 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp peer-group underlay address-family l2vpn-evpn enable on 
cumulus@borderleaf04:mgmt:~$ nv set vrf default router bgp peer-group underlay remote-as external 

Diagnostic Commands

The following examples show troubleshooting commands.

cumulus@leaf01:mgmt:~$ net show bgp sum 
show bgp ipv4 unicast summary 
============================= 
BGP router identifier 10.10.10.1, local AS number 65101 vrf-id 0 
BGP table version 216 
RIB entries 23, using 4600 bytes of memory 
Peers 2, using 46 KiB of memory 
Peer groups 1, using 64 bytes of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine01(swp1)   4      65199      2924      2922        0    0    0 02:18:51           10       12 
spine02(swp2)   4      65199      2924      2922        0    0    0 02:18:51           10       12 
 
Total number of neighbors 2 
 
show bgp ipv6 unicast summary 
============================= 
% No BGP neighbors found 
 
show bgp l2vpn evpn summary 
=========================== 
BGP router identifier 10.10.10.1, local AS number 65101 vrf-id 0 
BGP table version 0 
RIB entries 75, using 15 KiB of memory 
Peers 2, using 46 KiB of memory 
Peer groups 1, using 64 bytes of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine01(swp1)   4      65199      2924      2922        0    0    0 02:18:52           26       46 
spine02(swp2)   4      65199      2924      2922        0    0    0 02:18:52           26       46 
 
Total number of neighbors 2
cumulus@leaf01:mgmt:~$ net show evpn es 
Type: B bypass, L local, R remote, N non-DF 
ESI                            Type ES-IF                 VTEPs 
03:00:00:00:00:00:aa:00:00:01  LR   bond1                 10.10.10.2 
03:00:00:00:00:00:aa:00:00:02  LR   bond2                 10.10.10.2 
cumulus@leaf01:mgmt:~$ net show bgp l2vpn evpn es 
ES Flags: B - bypass, L local, R remote, I inconsistent 
VTEP Flags: E ESR/Type-4, A active nexthop 
ESI                            Flags RD                    #VNIs    VTEPs 
03:00:00:00:00:00:aa:00:00:01  LR    10.10.10.1:3          1        10.10.10.2(EA) 
03:00:00:00:00:00:aa:00:00:02  LR    10.10.10.1:4          1        10.10.10.2(EA) 
cumulus@leaf01:mgmt:~$ net show bgp l2vpn evpn es-evi 
Flags: L local, R remote, I inconsistent 
VTEP-Flags: E EAD-per-ES, V EAD-per-EVI 
VNI      ESI                            Flags VTEPs 
20       03:00:00:00:00:00:aa:00:00:02  LR    10.10.10.2(EV) 
10       03:00:00:00:00:00:aa:00:00:01  LR    10.10.10.2(EV) 
cumulus@leaf01:mgmt:~$ net show bgp l2vpn evpn es-vrf 
ES-VRF Flags: A Active 
ESI                            VRF             Flags IPv4-NHG IPv6-NHG Ref 
03:00:00:00:00:00:aa:00:00:01  VRF RED         A     72580645 72580646 1 
03:00:00:00:00:00:aa:00:00:02  VRF GREEN       A     72580647 72580648 1 
cumulus@leaf01:mgmt:~$ net show int bond1 
    Name   MAC                Speed  MTU   Mode 
--  -----  -----------------  -----  ----  ------- 
UP  bond1  48:b0:2d:3d:e9:84  1G     9216  802.3ad 
 
Bond Details 
------------------  -------- 
Bond Mode:          802.3ad 
Load Balancing:     layer3+4 
Minimum Links:      1 
LACP Sys Priority: 
LACP Rate:          1 
LACP Bypass:        Active 
 
All VLANs on L2 Port 
-------------------- 
10 
 
Untagged 
-------- 
10 
 
cl-netstat counters 
------------------- 
 RX_OK  RX_ERR  RX_DRP  RX_OVR   TX_OK  TX_ERR  TX_DRP  TX_OVR 
------  ------  ------  ------  ------  ------  ------  ------ 
257688       0       1       0  163218       0       0       0 
 
LLDP Details 
------------ 
LocalPort  RemotePort(RemoteHost) 
---------  --------------------------- 
swp3       48:b0:2d:33:24:a3(server01) 
 
Routing 
------- 
  Interface bond1 is up, line protocol is up 
  Link ups:       4    last: 2023/04/14 15:51:23.59 
  Link downs:     0    last: (never) 
  PTM status: disabled 
  vrf: default 
  index 10 metric 0 mtu 9216 speed 4294967295 
  flags: <UP,BROADCAST,RUNNING,MULTICAST> 
  Type: Ethernet 
  HWaddr: 48:b0:2d:3d:e9:84 
  Interface Type bond 
  Master interface: br_default PVID: 10 
  EVPN-MH: ES id 1 ES sysmac 00:00:00:00:00:aa 
  protodown: off (n/a) 
  ARP-ND redirect enabled: ARP 1713 ND 2791 
cumulus@leaf01:mgmt:~$ net show bgp vrf RED 
show bgp vrf RED ipv4 unicast 
============================= 
BGP table version is 57, local router ID is 10.10.10.1, vrf id 19 
Default local pref 100, local AS 65101 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.1.0/24   10.10.10.2<                            0 65199 65102 ? 
*                   10.10.10.2<                            0 65199 65102 ? 
*>                  0.0.0.0                  0         32768 ? 
*  192.168.10.0/24  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*=                  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*>                  10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
*                   10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
 
Displayed  2 routes and 7 total paths 

show bgp vrf RED ipv6 unicast 
============================= 
No BGP prefixes displayed, 0 exist
cumulus@leaf01:mgmt:~$ net show route vrf RED 
show ip route vrf RED 
====================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 2d22h09m 
C * 192.168.1.0/24 [0/1024] is directly connected, vlan10-v0, 2d19h15m 
C>* 192.168.1.0/24 is directly connected, vlan10, 2d22h09m 
B>* 192.168.10.0/24 [20/0] via 10.10.20.1, vxlan99 (vrf default) onlink, label 5001, weight 1, 00:02:56 
  *                        via 10.10.20.2, vxlan99 (vrf default) onlink, label 5001, weight 1, 00:02:56 
  
show ipv6 route vrf RED 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 2d22h09m 
C * fe80::/64 is directly connected, vlan10-v0, 2d19h15m 
C * fe80::/64 is directly connected, vlan220_l3, 2d22h08m 
C>* fe80::/64 is directly connected, vlan10, 2d22h09m 
cumulus@leaf01:mgmt:~$ net show bgp vrf GREEN 
show bgp vrf GREEN ipv4 unicast 
=============================== 
BGP table version is 67, local router ID is 10.10.10.1, vrf id 13 
Default local pref 100, local AS 65101 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.2.0/24   10.10.10.2<                            0 65199 65102 ? 
*                   10.10.10.2<                            0 65199 65102 ? 
*>                  0.0.0.0                  0         32768 ? 
*  192.168.20.0/24  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*=                  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*>                  10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
*                   10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
 
Displayed  2 routes and 7 total paths 
 
show bgp vrf GREEN ipv6 unicast 
=============================== 
No BGP prefixes displayed, 0 exist 
cumulus@leaf01:mgmt:~$ net show route vrf GREEN 
show ip route vrf GREEN 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 2d22h09m 
C * 192.168.2.0/24 [0/1024] is directly connected, vlan20-v0, 2d19h16m 
C>* 192.168.2.0/24 is directly connected, vlan20, 2d22h09m 
B>* 192.168.20.0/24 [20/0] via 10.10.20.1, vxlan99 (vrf default) onlink, label 5002, weight 1, 00:03:43 
  *                        via 10.10.20.2, vxlan99 (vrf default) onlink, label 5002, weight 1, 00:03:43 
  
show ipv6 route vrf GREEN 
========================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 2d22h09m 
C * fe80::/64 is directly connected, vlan20-v0, 2d19h16m 
C * fe80::/64 is directly connected, vlan20, 2d22h09m 
C>* fe80::/64 is directly connected, vlan370_l3, 2d22h09m 
cumulus@leaf01:mgmt:~$ net show bgp vrf RED 192.168.1.0/24 
BGP routing table entry for 192.168.1.0/24 
Paths: (3 available, best #3, vrf RED) 
  Not advertised to any peer 
  Imported from 10.10.10.2:4:[5]:[0]:[24]:[192.168.1.0], VNI 4001 
  65199 65102 
    10.10.10.2 from spine01(swp1) (10.10.10.101) announce-nh-self 
      Origin incomplete, valid, external, bestpath-from-AS 65199 
      Extended Community: RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
      Last update: Mon Apr 17 08:46:49 2023 
  Imported from 10.10.10.2:4:[5]:[0]:[24]:[192.168.1.0], VNI 4001 
  65199 65102 
    10.10.10.2 from spine02(swp2) (10.10.10.102) announce-nh-self 
      Origin incomplete, valid, external 
      Extended Community: RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
      Last update: Mon Apr 17 08:46:49 2023 
  Local 
    0.0.0.0 from 0.0.0.0 (10.10.10.1) 
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (Weight) 
      Last update: Mon Apr 17 08:46:47 2023 
cumulus@leaf01:mgmt:~$ net show bgp vrf RED 192.168.10.0/24 
BGP routing table entry for 192.168.10.0/24 
Paths: (4 available, best #3, vrf RED) 
  Not advertised to any peer 
  Imported from 10.10.20.2:6:[5]:[0]:[24]:[192.168.10.0], VNI 5001 
  65199 65110 65210 65299 65202 
    10.10.20.2 from spine01(swp1) (10.10.10.101) announce-nh-self 
      Origin incomplete, valid, external 
      Extended Community: RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
      Last update: Mon Apr 17 11:04:25 2023 
  Imported from 10.10.20.2:6:[5]:[0]:[24]:[192.168.10.0], VNI 5001 
  65199 65110 65210 65299 65202 
    10.10.20.2 from spine02(swp2) (10.10.10.102) announce-nh-self 
      Origin incomplete, valid, external, multipath 
      Extended Community: RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
      Last update: Mon Apr 17 11:04:25 2023 
  Imported from 10.10.20.1:8:[5]:[0]:[24]:[192.168.10.0], VNI 5001 
  65199 65110 65210 65299 65201 
    10.10.20.1 from spine01(swp1) (10.10.10.101) announce-nh-self 
      Origin incomplete, valid, external, multipath, bestpath-from-AS 65199, best (Router ID) 
      Extended Community: RT:65201:5001 ET:8 Rmac:44:38:39:22:bb:08 
      Last update: Mon Apr 17 11:04:25 2023 
  Imported from 10.10.20.1:8:[5]:[0]:[24]:[192.168.10.0], VNI 5001 
  65199 65110 65210 65299 65201 
    10.10.20.1 from spine02(swp2) (10.10.10.102) announce-nh-self 
      Origin incomplete, valid, external 
      Extended Community: RT:65201:5001 ET:8 Rmac:44:38:39:22:bb:08 
      Last update: Mon Apr 17 11:04:25 2023 
cumulus@leaf01:mgmt:~$  net show bgp evpn vni 
Advertise Gateway Macip: Disabled 
Advertise SVI Macip: Disabled 
Advertise All VNI flag: Enabled 
BUM flooding: Head-end replication 
VXLAN flooding: Enabled 
Number of L2 VNIs: 2 
Number of L3 VNIs: 2 
Flags: * - Kernel 
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF 
* 20         L2   10.10.10.1:2          65101:20                  65101:20                 GREEN 
* 10         L2   10.10.10.1:7          65101:10                  65101:10                 RED 
* 4002       L3   10.10.10.1:5          0:4002, ...               65101:4002               GREEN 
* 4001       L3   10.10.10.1:6          0:4001, ...               65101:4001               RED 
cumulus@leaf01:mgmt:~$ net show bgp evpn vni 4001 
VNI: 4001 (known to the kernel) 
  Type: L3 
  Tenant VRF: RED 
  RD: 10.10.10.1:6 
  Originator IP: 10.10.10.1 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.10.1 
  System-MAC: 44:38:39:22:bb:06 
  Router-MAC: 44:38:39:22:bb:06 
  Import Route Target: 
    0:4001 
    0:5001 
  Export Route Target: 
    65101:4001 
cumulus@leaf01:mgmt:~$ net show bgp evpn vni 4002 
VNI: 4002 (known to the kernel) 
  Type: L3 
  Tenant VRF: GREEN 
  RD: 10.10.10.1:5 
  Originator IP: 10.10.10.1 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.10.1 
  System-MAC: 44:38:39:22:bb:06 
  Router-MAC: 44:38:39:22:bb:06 
  Import Route Target: 
    0:4002 
    0:5002 
  Export Route Target: 
    65101:4002
cumulus@leaf01:mgmt:~$ net show evpn mac vni all 
VNI 10 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
44:38:39:22:bb:06 local        vlan10                               0/0 
48:b0:2d:33:24:a3 local  X     bond1                          10    0/0 
44:38:39:22:bb:07 remote       10.10.10.2                           0/0 
42:20:47:91:95:a7 local  NP    bond1                          10    1/0 
48:b0:2d:bc:7e:83 remote       10.10.10.2                           0/0 
 
VNI 20 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
48:b0:2d:9d:31:ae local  X     bond2                          20    0/0 
44:38:39:22:bb:06 local        vlan20                               0/0 
48:b0:2d:7f:a9:bd remote       10.10.10.2                           0/0 
44:38:39:22:bb:07 remote       10.10.10.2                           0/0 
a6:e0:55:25:f3:b2 local  NP    bond2                          20    1/0 
cumulus@leaf02:mgmt:~$ net show bgp sum 
show bgp ipv4 unicast summary 
============================= 
BGP router identifier 10.10.10.2, local AS number 65102 vrf-id 0 
BGP table version 205 
RIB entries 23, using 4600 bytes of memory 
Peers 2, using 46 KiB of memory 
Peer groups 1, using 64 bytes of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine01(swp1)   4      65199      3015      3013        0    0    0 02:23:25           10       12 
spine02(swp2)   4      65199      3015      3013        0    0    0 02:23:25           10       12 
 
Total number of neighbors 2 
 
show bgp ipv6 unicast summary 
============================= 
% No BGP neighbors found 
 
show bgp l2vpn evpn summary 
=========================== 
BGP router identifier 10.10.10.2, local AS number 65102 vrf-id 0 
BGP table version 0 
RIB entries 75, using 15 KiB of memory 
Peers 2, using 46 KiB of memory 
Peer groups 1, using 64 bytes of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine01(swp1)   4      65199      3015      3013        0    0    0 02:23:26           24       46 
spine02(swp2)   4      65199      3015      3013        0    0    0 02:23:26           24       46 
 
Total number of neighbors 2 
cumulus@leaf02:mgmt:~$ net show bgp vrf RED 
show bgp vrf RED ipv4 unicast 
============================= 
BGP table version is 53, local router ID is 10.10.10.2, vrf id 19 
Default local pref 100, local AS 65102 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.1.0/24   10.10.10.1<                            0 65199 65101 ? 
*                   10.10.10.1<                            0 65199 65101 ? 
*>                  0.0.0.0                  0         32768 ? 
*  192.168.10.0/24  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*=                  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*                   10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
*>                  10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
 
Displayed  2 routes and 7 total paths 
 
show bgp vrf RED ipv6 unicast 
============================= 
No BGP prefixes displayed, 0 exist
cumulus@leaf02:mgmt:~$ net show bgp vrf GREEN 
show bgp vrf GREEN ipv4 unicast 
=============================== 
BGP table version is 66, local router ID is 10.10.10.2, vrf id 13 
Default local pref 100, local AS 65102 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.2.0/24   10.10.10.1<                            0 65199 65101 ? 
*                   10.10.10.1<                            0 65199 65101 ? 
*>                  0.0.0.0                  0         32768 ? 
*  192.168.20.0/24  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*=                  10.10.20.2<                            0 65199 65110 65210 65299 65202 ? 
*                   10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
*>                  10.10.20.1<                            0 65199 65110 65210 65299 65201 ? 
 
Displayed  2 routes and 7 total paths 
 
show bgp vrf GREEN ipv6 unicast 
=============================== 
No BGP prefixes displayed, 0 exist
cumulus@leaf02:mgmt:~$ net show route vrf RED 
show ip route vrf RED 
====================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 2d22h13m 
C * 192.168.1.0/24 [0/1024] is directly connected, vlan10-v0, 2d19h20m 
C>* 192.168.1.0/24 is directly connected, vlan10, 2d22h13m 
B>* 192.168.10.0/24 [20/0] via 10.10.20.1, vxlan99 (vrf default) onlink, label 5001, weight 1, 00:07:19 
  *                        via 10.10.20.2, vxlan99 (vrf default) onlink, label 5001, weight 1, 00:07:19 
  
show ipv6 route vrf RED 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 2d22h13m 
C * fe80::/64 is directly connected, vlan10-v0, 2d19h20m 
C * fe80::/64 is directly connected, vlan10, 2d22h13m 
C>* fe80::/64 is directly connected, vlan220_l3, 2d22h13m 
cumulus@leaf02:mgmt:~$ net show route vrf GREEN 
show ip route vrf GREEN 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 2d22h13m 
C * 192.168.2.0/24 [0/1024] is directly connected, vlan20-v0, 2d19h20m 
C>* 192.168.2.0/24 is directly connected, vlan20, 2d22h13m 
B>* 192.168.20.0/24 [20/0] via 10.10.20.1, vxlan99 (vrf default) onlink, label 5002, weight 1, 00:07:23 
  *                        via 10.10.20.2, vxlan99 (vrf default) onlink, label 5002, weight 1, 00:07:23 
  
show ipv6 route vrf GREEN 
========================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 2d22h13m 
C * fe80::/64 is directly connected, vlan20-v0, 2d19h20m 
C * fe80::/64 is directly connected, vlan20, 2d22h13m 
C>* fe80::/64 is directly connected, vlan370_l3, 2d22h13m 
cumulus@leaf02:mgmt:~$ net show evpn es 
Type: B bypass, L local, R remote, N non-DF 
ESI                            Type ES-IF                 VTEPs 
03:00:00:00:00:00:aa:00:00:01  LRN  bond1                 10.10.10.1 
03:00:00:00:00:00:aa:00:00:02  LRN  bond2                 10.10.10.1 
cumulus@leaf02:mgmt:~$ net show bgp l2vpn evpn es 
ES Flags: B - bypass, L local, R remote, I inconsistent 
VTEP Flags: E ESR/Type-4, A active nexthop 
ESI                            Flags RD                    #VNIs    VTEPs 
03:00:00:00:00:00:aa:00:00:01  LR    10.10.10.2:6          1        10.10.10.1(EA) 
03:00:00:00:00:00:aa:00:00:02  LR    10.10.10.2:7          1        10.10.10.1(EA) 
cumulus@leaf02:mgmt:~$ net show bgp l2vpn evpn es-evi 
Flags: L local, R remote, I inconsistent 
VTEP-Flags: E EAD-per-ES, V EAD-per-EVI 
VNI      ESI                            Flags VTEPs 
20       03:00:00:00:00:00:aa:00:00:02  LR    10.10.10.1(EV) 
10       03:00:00:00:00:00:aa:00:00:01  LR    10.10.10.1(EV) 
cumulus@leaf02:mgmt:~$ net show bgp l2vpn evpn es-vrf 
ES-VRF Flags: A Active 
ESI                            VRF             Flags IPv4-NHG IPv6-NHG Ref 
03:00:00:00:00:00:aa:00:00:01  VRF RED         A     72580647 72580648 1 
03:00:00:00:00:00:aa:00:00:02  VRF GREEN       A     72580645 72580646 1 
cumulus@leaf02:mgmt:~$  net show bgp evpn vni 
Advertise Gateway Macip: Disabled 
Advertise SVI Macip: Disabled 
Advertise All VNI flag: Enabled 
BUM flooding: Head-end replication 
VXLAN flooding: Enabled 
Number of L2 VNIs: 2 
Number of L3 VNIs: 2 
Flags: * - Kernel 
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF 
* 20         L2   10.10.10.2:2          65102:20                  65102:20                 GREEN 
* 10         L2   10.10.10.2:3          65102:10                  65102:10                 RED 
* 4001       L3   10.10.10.2:4          0:4001, ...               65102:4001               RED 
* 4002       L3   10.10.10.2:5          0:4002, ...               65102:4002               GREEN 
cumulus@leaf02:mgmt:~$ net show int bond1 
    Name   MAC                Speed  MTU   Mode 
--  -----  -----------------  -----  ----  ------- 
UP  bond1  48:b0:2d:f0:64:6e  1G     9216  802.3ad 
 
Bond Details 
------------------  -------- 
Bond Mode:          802.3ad 
Load Balancing:     layer3+4 
Minimum Links:      1 
LACP Sys Priority: 
LACP Rate:          1 
LACP Bypass:        Active 
 
All VLANs on L2 Port 
-------------------- 
10 
 
Untagged 
-------- 
10 
 
cl-netstat counters 
------------------- 
 RX_OK  RX_ERR  RX_DRP  RX_OVR   TX_OK  TX_ERR  TX_DRP  TX_OVR 
------  ------  ------  ------  ------  ------  ------  ------ 
255593       0       0       0  165821       0       0       0 
 
LLDP Details 
------------ 
LocalPort  RemotePort(RemoteHost) 
---------  --------------------------- 
swp3       48:b0:2d:bc:7e:83(server01) 
 
Routing 
------- 
  Interface bond1 is up, line protocol is up 
  Link ups:       6    last: 2023/04/14 15:51:19.73 
  Link downs:     0    last: (never) 
  PTM status: disabled 
  vrf: default 
  index 9 metric 0 mtu 9216 speed 1000 
  flags: <UP,BROADCAST,RUNNING,MULTICAST> 
  Type: Ethernet 
  HWaddr: 48:b0:2d:f0:64:6e 
  Interface Type bond 
  Master interface: br_default PVID: 10 
  EVPN-MH: ES id 1 ES sysmac 00:00:00:00:00:aa 
  protodown: off (n/a) 
  ARP-ND redirect enabled: ARP 906 ND 3030 
cumulus@leaf02:mgmt:~$ net show bgp evpn vni 4001 
VNI: 4001 (known to the kernel) 
  Type: L3 
  Tenant VRF: RED 
  RD: 10.10.10.2:4 
  Originator IP: 10.10.10.2 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.10.2 
  System-MAC: 44:38:39:22:bb:07 
  Router-MAC: 44:38:39:22:bb:07 
  Import Route Target: 
    0:4001 
    0:5001 
  Export Route Target: 
    65102:4001 
cumulus@leaf02:mgmt:~$ net show bgp evpn vni 4002 
VNI: 4002 (known to the kernel) 
  Type: L3 
  Tenant VRF: GREEN 
  RD: 10.10.10.2:5 
  Originator IP: 10.10.10.2 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.10.2 
  System-MAC: 44:38:39:22:bb:07 
  Router-MAC: 44:38:39:22:bb:07 
  Import Route Target: 
    0:4002 
    0:5002 
  Export Route Target: 
    65102:4002
cumulus@leaf02:mgmt:~$ net show evpn mac vni all 
VNI 10 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
44:38:39:22:bb:06 remote       10.10.10.1                           0/0 
48:b0:2d:33:24:a3 local  PI    bond1                          10    0/0 
44:38:39:22:bb:07 local        vlan10                               0/0 
42:20:47:91:95:a7 local  NP    bond1                          10    1/0 
48:b0:2d:bc:7e:83 local        bond1                          10    0/0 
 
VNI 20 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
48:b0:2d:9d:31:ae local  PI    bond2                          20    0/0 
44:38:39:22:bb:06 remote       10.10.10.1                           0/0 
48:b0:2d:7f:a9:bd local        bond2                          20    0/0 
44:38:39:22:bb:07 local        vlan20                               0/0 
a6:e0:55:25:f3:b2 local  NP    bond2                          20    1/0 

Verify that the bridge br_default is learning MAC entries:

cumulus@leaf01:mgmt:~$ nv show bridge domain br_default mac-table 
    age     bridge-domain  entry-type  interface   last-update  MAC address        src-vni  vlan  vni   Summary 
--  ------  -------------  ----------  ----------  -----------  -----------------  -------  ----  ----  ---------------------- 
0   3       br_default     static      bond2       9770         48:b0:2d:9d:31:ae           20 
1   58      br_default     static      bond2       237427       a6:e0:55:25:f3:b2           20 
2   253880  br_default     permanent   bond2       253880       48:b0:2d:ce:f9:6f 
3   9770    br_default                 vxlan48     9770         48:b0:2d:bc:7e:83           10    None  remote-dst: 10.10.10.2 
4   9770    br_default                 vxlan48     9770         44:38:39:22:bb:07           10    None  remote-dst: 10.10.10.2 
5   9770    br_default                 vxlan48     9770         48:b0:2d:7f:a9:bd           20    None  remote-dst: 10.10.10.2 
6   253880  br_default     permanent   vxlan48     253880       96:8b:4a:de:a2:71                 None 
7   9770                   permanent   vxlan48     4            00:00:00:00:00:00  10             None  remote-dst: 10.10.10.2 
8   26      br_default     static      bond1       9770         48:b0:2d:33:24:a3           10 
9   53      br_default     static      bond1       235497       42:20:47:91:95:a7           10 
10  253880  br_default     permanent   bond1       253880       48:b0:2d:3d:e9:84 
11                         permanent   br_default               00:00:5e:00:01:0a 
12  243496  br_default     permanent   br_default  243496       44:38:39:22:bb:06           10 
cumulus@leaf02:mgmt:~$ nv show bridge domain br_default mac-table 
    age     bridge-domain  entry-type  interface   last-update  MAC address        src-vni  vlan  vni   Summary 
--  ------  -------------  ----------  ----------  -----------  -----------------  -------  ----  ----  ---------------------- 
0   250398  br_default     static      bond2       250398       48:b0:2d:9d:31:ae           20 
1   65      br_default     static      bond2       250384       a6:e0:55:25:f3:b2           20 
2   2       br_default                 bond2       253825       48:b0:2d:7f:a9:bd           20 
3   253891  br_default     permanent   bond2       253891       48:b0:2d:aa:08:bd 
4   250398  br_default     static      bond1       250398       48:b0:2d:33:24:a3           10 
5   105     br_default     static      bond1       235506       42:20:47:91:95:a7           10 
6   2       br_default                 bond1       253825       48:b0:2d:bc:7e:83           10 
7   253891  br_default     permanent   bond1       253891       48:b0:2d:f0:64:6e 
8   9775    br_default                 vxlan48     65           44:38:39:22:bb:06           20    None  remote-dst: 10.10.10.1 
9   253891  br_default     permanent   vxlan48     253891       1a:64:66:e8:14:39           1     None 
10  9775                   permanent   vxlan48     50           00:00:00:00:00:00  10             None  remote-dst: 10.10.10.1 
11                         permanent   br_default               00:00:5e:00:01:0a 
12  243505  br_default     permanent   br_default  243505       44:38:39:22:bb:07           10 

From the table above, locate the layer 3 VLAN interface MAC address and the VRR MAC address:

cumulus@leaf01:mgmt:~$  nv show int vlan10 ip vrr 
             operational        applied 
-----------  -----------------  -------------- 
enable                          on 
mac-address  00:00:5e:00:01:0a  auto 
mac-id                          none 
[address]    192.168.1.1/24     192.168.1.1/24 
state        up                 up 
cumulus@leaf01:mgmt:~$ nv show int vlan10 | grep mac 
    mac-address                              auto 
    mac-id                                   none 
  mac                     44:38:39:22:bb:06 
cumulus@leaf02:mgmt:~$ nv show int vlan10 | grep mac 
    mac-address                              auto 
    mac-id                                   none 
  mac                     44:38:39:22:bb:07 

Verify EVPN type-5 routes at the ingress PE (leaf01) for the end host 192.168.10.110, which connects to leaf03 and leaf04:

cumulus@leaf01:mgmt:~$ net show bgp evpn route type 5 | grep 192.168.10 -A 4 -B 1 
Route Distinguisher: 10.10.20.1:8 
*  [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.1:8 
                    10.10.20.1 (spine02) 
                                                           0 65199 65110 65210 65299 65201 ? 
                    RT:65201:5001 ET:8 Rmac:44:38:39:22:bb:08 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.1:8 
                    10.10.20.1 (spine01) 
                                                           0 65199 65110 65210 65299 65201 ? 
                    RT:65201:5001 ET:8 Rmac:44:38:39:22:bb:08 
Route Distinguisher: 10.10.20.2:6 
*  [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.2:6 
                    10.10.20.2 (spine02) 
                                                           0 65199 65110 65210 65299 65202 ? 
                    RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.2:6 
                    10.10.20.2 (spine01) 
                                                           0 65199 65110 65210 65299 65202 ? 
                    RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
Route Distinguisher: 10.10.20.2:7 

Verify EVPN type-5 routes at the egress PE (leaf03) for the end host 192.168.10.110, which connects to leaf03 and leaf04:

cumulus@leaf03:mgmt:~$ net show bgp evpn route type 5 | grep 192.168.10 -A 4 -B 1 
Route Distinguisher: 10.10.20.1:8 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.1:8 
                    10.10.20.1 (leaf03) 
                                             0         32768 ? 
                    ET:8 RT:65201:5001 Rmac:44:38:39:22:bb:08 
Route Distinguisher: 10.10.20.2:6 
*  [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.2:6 
                    10.10.20.2 (spine03) 
                                                           0 65299 65202 ? 
                    RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.2:6 
                    10.10.20.2 (spine04) 
                                                           0 65299 65202 ? 
                    RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
Route Distinguisher: 10.10.20.2:7 

To verify routing on the border leaf:

cumulus@borderleaf01:mgmt:~$ net show bgp sum 
show bgp ipv4 unicast summary 
============================= 
BGP router identifier 10.10.10.10, local AS number 65110 vrf-id 0 
BGP table version 12 
RIB entries 19, using 3800 bytes of memory 
Peers 3, using 68 KiB of memory 
Peer groups 2, using 128 bytes of memory 
 
Neighbor           V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
borderleaf04(swp3) 4      65210       735       734        0    0    0 00:35:39            5       10 
spine01(swp1)      4      65199       778       776        0    0    0 00:36:20            3       10 
spine02(swp2)      4      65199       777       775        0    0    0 00:36:16            3       10 
 
Total number of neighbors 3 
 
show bgp ipv6 unicast summary 
============================= 
% No BGP neighbors found 
 
show bgp l2vpn evpn summary 
=========================== 
BGP router identifier 10.10.10.10, local AS number 65110 vrf-id 0 
BGP table version 0 
RIB entries 31, using 6200 bytes of memory 
Peers 3, using 68 KiB of memory 
Peer groups 2, using 128 bytes of memory 
 
Neighbor           V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
borderleaf04(swp3) 4      65210       735       734        0    0    0 00:35:39            4        8 
spine01(swp1)      4      65199       778       776        0    0    0 00:36:20           42       46 
spine02(swp2)      4      65199       777       775        0    0    0 00:36:16           42       46 
 
Total number of neighbors 3 
cumulus@borderleaf01:mgmt:~$ net show bgp vrf RED 
show bgp vrf RED ipv4 unicast 
============================= 
BGP table version is 3, local router ID is 10.10.10.10, vrf id 13 
Default local pref 100, local AS 65110 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.1.0/24   10.10.10.2<                            0 65199 65102 ? 
*                   10.10.10.1<                            0 65199 65101 ? 
*=                  10.10.10.2<                            0 65199 65102 ? 
*>                  10.10.10.1<                            0 65199 65101 ? 
*  192.168.1.10/32  10.10.10.2<                            0 65199 65102 i 
*                   10.10.10.1<                            0 65199 65101 i 
*=                  10.10.10.2<                            0 65199 65102 i 
*>                  10.10.10.1<                            0 65199 65101 i 
*= 192.168.10.0/24  10.10.20.2<                            0 65210 65299 65202 ? 
*>                  10.10.20.1<                            0 65210 65299 65201 ? 
 
Displayed  3 routes and 10 total paths 

show bgp vrf RED ipv6 unicast 
============================= 
No BGP prefixes displayed, 0 exist
cumulus@borderleaf01:mgmt:~$ net show bgp evpn route type 5 | grep 192.168.1 -A 3 -B 1 
Route Distinguisher: 10.10.10.1:6 
*  [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.1:6 
                    10.10.10.1 (spine02) 
                                                           0 65199 65101 ? 
                    RT:65101:4001 ET:8 Rmac:44:38:39:22:bb:06 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.1:6 
                    10.10.10.1 (spine01) 
                                                           0 65199 65101 ? 
                    RT:65101:4001 ET:8 Rmac:44:38:39:22:bb:06 
Route Distinguisher: 10.10.10.2:4 
*  [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.2:4 
                    10.10.10.2 (spine02) 
                                                           0 65199 65102 ? 
                    RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.2:4 
                    10.10.10.2 (spine01) 
                                                           0 65199 65102 ? 
                    RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
-- 
Route Distinguisher: 10.10.20.1:8 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.1:8 
                    10.10.20.1 (borderleaf04) 
                                                           0 65210 65299 65201 ? 
                    RT:65201:5001 ET:8 Rmac:44:38:39:22:bb:08 
Route Distinguisher: 10.10.20.2:6 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.2:6 
                    10.10.20.2 (borderleaf04) 
                                                           0 65210 65299 65202 ? 
                    RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
cumulus@leaf03:mgmt:~$ net show bgp sum 
show bgp ipv4 unicast summary 
============================= 
BGP router identifier 10.10.20.1, local AS number 65201 vrf-id 0 
BGP table version 57 
RIB entries 23, using 4600 bytes of memory 
Peers 2, using 46 KiB of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine03(swp1)   4      65299     79699     79701        0    0    0 2d18h00m           10       12 
spine04(swp2)   4      65299     79714     79714        0    0    0 2d18h00m           10       12 
 
Total number of neighbors 2 
 
show bgp ipv6 unicast summary 
============================= 
% No BGP neighbors found 
 
show bgp l2vpn evpn summary 
=========================== 
BGP router identifier 10.10.20.1, local AS number 65201 vrf-id 0 
BGP table version 0 
RIB entries 71, using 14 KiB of memory 
Peers 2, using 46 KiB of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine03(swp1)   4      65299     79699     79702        0    0    0 2d18h00m           25       46 
spine04(swp2)   4      65299     79714     79715        0    0    0 2d18h00m           25       46 
 
Total number of neighbors 2
cumulus@leaf03:mgmt:~$ net show evpn es 
Type: B bypass, L local, R remote, N non-DF 
ESI                            Type ES-IF                 VTEPs 
03:00:00:00:00:00:bb:00:00:01  LR   bond1                 10.10.20.2 
03:00:00:00:00:00:bb:00:00:02  LR   bond2                 10.10.20.2 
cumulus@leaf03:mgmt:~$ net show bgp l2vpn evpn es 
ES Flags: B - bypass, L local, R remote, I inconsistent 
VTEP Flags: E ESR/Type-4, A active nexthop 
ESI                            Flags RD                    #VNIs    VTEPs 
03:00:00:00:00:00:bb:00:00:01  LR    10.10.20.1:3          1        10.10.20.2(EA) 
03:00:00:00:00:00:bb:00:00:02  LR    10.10.20.1:4          1        10.10.20.2(EA) 
cumulus@leaf03:mgmt:~$ net show bgp l2vpn evpn es-evi 
Flags: L local, R remote, I inconsistent 
VTEP-Flags: E EAD-per-ES, V EAD-per-EVI 
VNI      ESI                            Flags VTEPs 
2020     03:00:00:00:00:00:bb:00:00:02  LR    10.10.20.2(EV) 
1010     03:00:00:00:00:00:bb:00:00:01  LR    10.10.20.2(EV) 
cumulus@leaf03:mgmt:~$ net show bgp l2vpn evpn es-vrf 
ES-VRF Flags: A Active 
ESI                            VRF             Flags IPv4-NHG IPv6-NHG Ref 
03:00:00:00:00:00:bb:00:00:01  VRF RED         A     72580647 72580648 1 
03:00:00:00:00:00:bb:00:00:02  VRF GREEN       A     72580645 72580646 1
cumulus@leaf03:mgmt:~$ net show int bond1 
    Name   MAC                Speed  MTU   Mode 
--  -----  -----------------  -----  ----  ------- 
UP  bond1  48:b0:2d:7d:9c:74  1G     9216  802.3ad 
 
Bond Details 
------------------  -------- 
Bond Mode:          802.3ad 
Load Balancing:     layer3+4 
Minimum Links:      1 
LACP Sys Priority: 
LACP Rate:          1 
LACP Bypass:        Active 
 
All VLANs on L2 Port 
-------------------- 
1010 
 
Untagged 
-------- 
1010 
 
cl-netstat counters 
------------------- 
 RX_OK  RX_ERR  RX_DRP  RX_OVR   TX_OK  TX_ERR  TX_DRP  TX_OVR 
------  ------  ------  ------  ------  ------  ------  ------ 
241819       0       0       0  157114       0       0       0 
 
LLDP Details 
------------ 
LocalPort  RemotePort(RemoteHost) 
---------  --------------------------- 
swp3       48:b0:2d:72:28:ff(server03) 
 
Routing 
------- 
  Interface bond1 is up, line protocol is up 
  Link ups:       2    last: 2023/04/17 08:44:08.84 
  Link downs:     0    last: (never) 
  PTM status: disabled 
  vrf: default 
  index 9 metric 0 mtu 9216 speed 1000 
  flags: <UP,BROADCAST,RUNNING,MULTICAST> 
  Type: Ethernet 
  HWaddr: 48:b0:2d:7d:9c:74 
  Interface Type bond 
  Master interface: br_default PVID: 1010 
  EVPN-MH: ES id 1 ES sysmac 00:00:00:00:00:bb 
  protodown: off (n/a) 
  ARP-ND redirect enabled: ARP 1653 ND 2667 
cumulus@leaf03:mgmt:~$ net show bgp vrf RED 
show bgp vrf RED ipv4 unicast 
============================= 
BGP table version is 11, local router ID is 10.10.20.1, vrf id 19 
Default local pref 100, local AS 65201 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.1.0/24   10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*                   10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*=                  10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*>                  10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*  192.168.10.0/24  10.10.20.2<                            0 65299 65202 ? 
*                   10.10.20.2<                            0 65299 65202 ? 
*>                  0.0.0.0                  0         32768 ? 
 
Displayed  2 routes and 7 total paths 
 
show bgp vrf RED ipv6 unicast 
============================= 
No BGP prefixes displayed, 0 exist
cumulus@leaf03:mgmt:~$ net show bgp vrf GREEN 
show bgp vrf GREEN ipv4 unicast 
=============================== 
BGP table version is 10, local router ID is 10.10.20.1, vrf id 13 
Default local pref 100, local AS 65201 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.2.0/24   10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*                   10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*=                  10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*>                  10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*  192.168.20.0/24  10.10.20.2<                            0 65299 65202 ? 
*                   10.10.20.2<                            0 65299 65202 ? 
*>                  0.0.0.0                  0         32768 ? 
 
Displayed  2 routes and 7 total paths 
 
show bgp vrf GREEN ipv6 unicast 
=============================== 
No BGP prefixes displayed, 0 exist 
cumulus@leaf03:mgmt:~$ net show route vrf RED 
show ip route vrf RED 
====================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 2d18h01m 
B>* 192.168.1.0/24 [20/0] via 10.10.10.1, vxlan99 (vrf default) onlink, label 4001, weight 1, 00:38:33 
  *                       via 10.10.10.2, vxlan99 (vrf default) onlink, label 4001, weight 1, 00:38:33 
C * 192.168.10.0/24 [0/1024] is directly connected, vlan1010-v0, 02:56:10 
C>* 192.168.10.0/24 is directly connected, vlan1010, 02:56:10 
  
show ipv6 route vrf RED 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 2d18h01m 
C * fe80::/64 is directly connected, vlan1010, 02:56:08 
C * fe80::/64 is directly connected, vlan1010-v0, 02:56:10 
C>* fe80::/64 is directly connected, vlan220_l3, 2d18h01m 
cumulus@leaf03:mgmt:~$ net show route vrf GREEN 
show ip route vrf GREEN 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 2d18h01m 
B>* 192.168.2.0/24 [20/0] via 10.10.10.1, vxlan99 (vrf default) onlink, label 4002, weight 1, 00:38:36 
  *                       via 10.10.10.2, vxlan99 (vrf default) onlink, label 4002, weight 1, 00:38:36 
C * 192.168.20.0/24 [0/1024] is directly connected, vlan2020-v0, 02:56:13 
C>* 192.168.20.0/24 is directly connected, vlan2020, 02:56:13 
  
show ipv6 route vrf GREEN 
========================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 2d18h01m 
C * fe80::/64 is directly connected, vlan2020, 02:56:12 
C * fe80::/64 is directly connected, vlan2020-v0, 02:56:14 
C>* fe80::/64 is directly connected, vlan370_l3, 2d18h01m 
cumulus@leaf03:mgmt:~$ net show bgp vrf RED 192.168.10.0/24 
BGP routing table entry for 192.168.10.0/24 
Paths: (3 available, best #3, vrf RED) 
  Not advertised to any peer 
  Imported from 10.10.20.2:6:[5]:[0]:[24]:[192.168.10.0], VNI 5001 
  65299 65202 
    10.10.20.2 from spine03(swp1) (10.10.20.103) announce-nh-self 
      Origin incomplete, valid, external, bestpath-from-AS 65299 
      Extended Community: RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
      Last update: Mon Apr 17 08:46:49 2023 
  Imported from 10.10.20.2:6:[5]:[0]:[24]:[192.168.10.0], VNI 5001 
  65299 65202 
    10.10.20.2 from spine04(swp2) (10.10.20.104) announce-nh-self 
      Origin incomplete, valid, external 
      Extended Community: RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
      Last update: Mon Apr 17 08:46:49 2023 
  Local 
    0.0.0.0 from 0.0.0.0 (10.10.20.1) 
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (Weight) 
      Last update: Mon Apr 17 08:46:48 2023 
cumulus@leaf03:mgmt:~$ net show bgp vrf RED 192.168.1.0/24 
BGP routing table entry for 192.168.1.0/24 
Paths: (4 available, best #4, vrf RED) 
  Not advertised to any peer 
  Imported from 10.10.10.2:4:[5]:[0]:[24]:[192.168.1.0], VNI 4001 
  65299 65210 65110 65199 65102 
    10.10.10.2 from spine03(swp1) (10.10.20.103) announce-nh-self 
      Origin incomplete, valid, external 
      Extended Community: RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
      Last update: Mon Apr 17 11:04:26 2023 
  Imported from 10.10.10.1:6:[5]:[0]:[24]:[192.168.1.0], VNI 4001 
  65299 65210 65110 65199 65101 
    10.10.10.1 from spine03(swp1) (10.10.20.103) announce-nh-self 
      Origin incomplete, valid, external 
      Extended Community: RT:65101:4001 ET:8 Rmac:44:38:39:22:bb:06 
      Last update: Mon Apr 17 11:04:26 2023 
  Imported from 10.10.10.2:4:[5]:[0]:[24]:[192.168.1.0], VNI 4001 
  65299 65210 65110 65199 65102 
    10.10.10.2 from spine04(swp2) (10.10.20.104) announce-nh-self 
      Origin incomplete, valid, external, multipath 
      Extended Community: RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
      Last update: Mon Apr 17 11:04:24 2023 
  Imported from 10.10.10.1:6:[5]:[0]:[24]:[192.168.1.0], VNI 4001 
  65299 65210 65110 65199 65101 
    10.10.10.1 from spine04(swp2) (10.10.20.104) announce-nh-self 
      Origin incomplete, valid, external, multipath, bestpath-from-AS 65299, best (Older Path) 
      Extended Community: RT:65101:4001 ET:8 Rmac:44:38:39:22:bb:06 
      Last update: Mon Apr 17 11:04:24 2023 
cumulus@leaf03:mgmt:~$ net show bgp evpn vni 
Advertise Gateway Macip: Disabled 
Advertise SVI Macip: Disabled 
Advertise All VNI flag: Enabled 
BUM flooding: Head-end replication 
VXLAN flooding: Enabled 
Number of L2 VNIs: 2 
Number of L3 VNIs: 2 
Flags: * - Kernel 
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF 
* 2020       L2   10.10.20.1:2          65201:2020                65201:2020               GREEN 
* 1010       L2   10.10.20.1:9          65201:1010                65201:1010               RED 
* 5002       L3   10.10.20.1:7          0:4002, ...               65201:5002               GREEN 
* 5001       L3   10.10.20.1:8          0:4001, ...               65201:5001               RED 
cumulus@leaf03:mgmt:~$ net show bgp evpn vni 5001 
VNI: 5001 (known to the kernel) 
  Type: L3 
  Tenant VRF: RED 
  RD: 10.10.20.1:8 
  Originator IP: 10.10.20.1 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.20.1 
  System-MAC: 44:38:39:22:bb:08 
  Router-MAC: 44:38:39:22:bb:08 
  Import Route Target: 
    0:4001 
    0:5001 
  Export Route Target: 
    65201:5001 
cumulus@leaf03:mgmt:~$ net show bgp evpn vni 5002 
VNI: 5002 (known to the kernel) 
  Type: L3 
  Tenant VRF: GREEN 
  RD: 10.10.20.1:7 
  Originator IP: 10.10.20.1 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.20.1 
  System-MAC: 44:38:39:22:bb:08 
  Router-MAC: 44:38:39:22:bb:08 
  Import Route Target: 
    0:4002 
    0:5002 
  Export Route Target: 
    65201:5002 
cumulus@leaf03:mgmt:~$ net show evpn mac vni all 
VNI 1010 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
48:b0:2d:72:28:ff local  P     bond1                          1010  0/0 
48:b0:2d:82:05:04 local  P     bond1                          1010  0/0 
b6:4b:0f:ea:f2:02 local  NP    bond1                          1010  0/0 
44:38:39:22:bb:09 remote       10.10.20.2                           0/0 
44:38:39:22:bb:08 local        vlan1010                             0/0 
 
VNI 2020 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
ee:54:69:be:3a:3f local  NP    bond2                          2020  0/0 
48:b0:2d:75:8e:7a local  P     bond2                          2020  0/0 
44:38:39:22:bb:09 remote       10.10.20.2                           0/0 
48:b0:2d:a7:e2:6e local  P     bond2                          2020  0/0 
44:38:39:22:bb:08 local        vlan2020                             0/0 
cumulus@leaf04:mgmt:~$ net show bgp sum 
show bgp ipv4 unicast summary 
============================= 
BGP router identifier 10.10.20.2, local AS number 65202 vrf-id 0 
BGP table version 21 
RIB entries 23, using 4600 bytes of memory 
Peers 2, using 46 KiB of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine03(swp1)   4      65299       229       236        0    0    0 00:08:42           10       12 
spine04(swp2)   4      65299       224       231        0    0    0 00:08:34           10       12 
 
Total number of neighbors 2 
 
show bgp ipv6 unicast summary 
============================= 
% No BGP neighbors found 
 
show bgp l2vpn evpn summary 
=========================== 
BGP router identifier 10.10.20.2, local AS number 65202 vrf-id 0 
BGP table version 0 
RIB entries 31, using 6200 bytes of memory 
Peers 2, using 46 KiB of memory 
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
spine03(swp1)   4      65299       229       236        0    0    0 00:08:42           25       46 
spine04(swp2)   4      65299       224       231        0    0    0 00:08:34           25       46 
 
Total number of neighbors 2
cumulus@leaf04:mgmt:~$ net show bgp vrf RED 
show bgp vrf RED ipv4 unicast 
============================= 
BGP table version is 2, local router ID is 10.10.20.2, vrf id 19 
Default local pref 100, local AS 65202 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.1.0/24   10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*                   10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*=                  10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*>                  10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*  192.168.10.0/24  10.10.20.1<                            0 65299 65201 ? 
*                   10.10.20.1<                            0 65299 65201 ? 
*>                  0.0.0.0                  0         32768 ? 
 
Displayed  2 routes and 7 total paths 
 
show bgp vrf RED ipv6 unicast 
============================= 
No BGP prefixes displayed, 0 exist 
cumulus@leaf04:mgmt:~$ net show bgp vrf GREEN 
show bgp vrf GREEN ipv4 unicast 
=============================== 
BGP table version is 2, local router ID is 10.10.20.2, vrf id 13 
Default local pref 100, local AS 65202 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*  192.168.2.0/24   10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*                   10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*=                  10.10.10.2<                            0 65299 65210 65110 65199 65102 ? 
*>                  10.10.10.1<                            0 65299 65210 65110 65199 65101 ? 
*  192.168.20.0/24  10.10.20.1<                            0 65299 65201 ? 
*                   10.10.20.1<                            0 65299 65201 ? 
*>                  0.0.0.0                  0         32768 ? 
 
Displayed  2 routes and 7 total paths 
 
show bgp vrf GREEN ipv6 unicast 
=============================== 
No BGP prefixes displayed, 0 exist 
cumulus@leaf04:mgmt:~$ net show route vrf RED 
show ip route vrf RED 
====================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:09:06 
B>* 192.168.1.0/24 [20/0] via 10.10.10.1, vxlan99 (vrf default) onlink, label 4001, weight 1, 00:09:03 
  *                       via 10.10.10.2, vxlan99 (vrf default) onlink, label 4001, weight 1, 00:09:03 
C * 192.168.10.0/24 [0/1024] is directly connected, vlan1010-v0, 00:09:06 
C>* 192.168.10.0/24 is directly connected, vlan1010, 00:09:06 
  
show ipv6 route vrf RED 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF RED: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 00:09:06 
C * fe80::/64 is directly connected, vlan220_l3, 00:09:05 
C * fe80::/64 is directly connected, vlan1010, 00:09:06 
C>* fe80::/64 is directly connected, vlan1010-v0, 00:09:06 
cumulus@leaf04:mgmt:~$ net show route vrf GREEN 
show ip route vrf GREEN 
======================== 
Codes: K - kernel route, C - connected, S - static, R - RIP, 
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, 
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, 
       F - PBR, f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:09:13 
B>* 192.168.2.0/24 [20/0] via 10.10.10.1, vxlan99 (vrf default) onlink, label 4002, weight 1, 00:09:10 
  *                       via 10.10.10.2, vxlan99 (vrf default) onlink, label 4002, weight 1, 00:09:10 
C * 192.168.20.0/24 [0/1024] is directly connected, vlan2020-v0, 00:09:13 
C>* 192.168.20.0/24 is directly connected, vlan2020, 00:09:13 
  
show ipv6 route vrf GREEN 
========================== 
Codes: K - kernel route, C - connected, S - static, R - RIPng, 
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, 
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, 
       f - OpenFabric, Z - FRR, 
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup 
       t - trapped, o - offload failure 
 
VRF GREEN: 
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 00:09:13 
C * fe80::/64 is directly connected, vlan370_l3, 00:09:13 
C * fe80::/64 is directly connected, vlan2020-v0, 00:09:13 
C>* fe80::/64 is directly connected, vlan2020, 00:09:13 
cumulus@leaf04:mgmt:~$ net show evpn es 
Type: B bypass, L local, R remote, N non-DF 
ESI                            Type ES-IF                 VTEPs 
03:00:00:00:00:00:bb:00:00:01  LRN  bond1                 10.10.20.1 
03:00:00:00:00:00:bb:00:00:02  LRN  bond2                 10.10.20.1 
cumulus@leaf04:mgmt:~$ net show bgp l2vpn evpn es 
ES Flags: B - bypass, L local, R remote, I inconsistent 
VTEP Flags: E ESR/Type-4, A active nexthop 
ESI                            Flags RD                    #VNIs    VTEPs 
03:00:00:00:00:00:bb:00:00:01  LR    10.10.20.2:3          1        10.10.20.1(EA) 
03:00:00:00:00:00:bb:00:00:02  LR    10.10.20.2:4          1        10.10.20.1(EA) 
cumulus@leaf04:mgmt:~$ net show bgp l2vpn evpn es-evi 
Flags: L local, R remote, I inconsistent 
VTEP-Flags: E EAD-per-ES, V EAD-per-EVI 
VNI      ESI                            Flags VTEPs 
2020     03:00:00:00:00:00:bb:00:00:02  LR    10.10.20.1(EV) 
1010     03:00:00:00:00:00:bb:00:00:01  LR    10.10.20.1(EV) 
cumulus@leaf04:mgmt:~$ net show bgp l2vpn evpn es-vrf 
ES-VRF Flags: A Active 
ESI                            VRF             Flags IPv4-NHG IPv6-NHG Ref 
03:00:00:00:00:00:bb:00:00:01  VRF RED         A     72580645 72580646 1 
03:00:00:00:00:00:bb:00:00:02  VRF GREEN       A     72580647 72580648 1 
cumulus@leaf04:mgmt:~$ net show bgp evpn vni 
Advertise Gateway Macip: Disabled 
Advertise SVI Macip: Disabled 
Advertise All VNI flag: Enabled 
BUM flooding: Head-end replication 
VXLAN flooding: Enabled 
Number of L2 VNIs: 2 
Number of L3 VNIs: 2 
Flags: * - Kernel 
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF 
* 2020       L2   10.10.20.2:2          65202:2020                65202:2020               GREEN 
* 1010       L2   10.10.20.2:9          65202:1010                65202:1010               RED 
* 5002       L3   10.10.20.2:7          0:4002, ...               65202:5002               GREEN 
* 5001       L3   10.10.20.2:8          0:4001, ...               65202:5001               RED 
cumulus@leaf04:mgmt:~$ net show int bond1 
    Name   MAC                Speed  MTU   Mode 
--  -----  -----------------  -----  ----  ------- 
UP  bond1  48:b0:2d:84:e8:0f  1G     9216  802.3ad 
 
Bond Details 
------------------  -------- 
Bond Mode:          802.3ad 
Load Balancing:     layer3+4 
Minimum Links:      1 
LACP Sys Priority: 
LACP Rate:          1 
LACP Bypass:        Active 
 
All VLANs on L2 Port 
-------------------- 
1010 
 
Untagged 
-------- 
1010 
 
cl-netstat counters 
------------------- 
RX_OK  RX_ERR  RX_DRP  RX_OVR  TX_OK  TX_ERR  TX_DRP  TX_OVR 
-----  ------  ------  ------  -----  ------  ------  ------ 
  481       0       0       0    257       0       0       0 
 
LLDP Details 
------------ 
LocalPort  RemotePort(RemoteHost) 
---------  --------------------------- 
swp3       48:b0:2d:82:05:04(server03) 
 
Routing 
------- 
  Interface bond1 is up, line protocol is up 
  Link ups:       1    last: 2023/04/17 11:51:10.75 
  Link downs:     0    last: (never) 
  PTM status: disabled 
  vrf: default 
  index 10 metric 0 mtu 9216 speed 1000 
  flags: <UP,BROADCAST,RUNNING,MULTICAST> 
  Type: Ethernet 
  HWaddr: 48:b0:2d:84:e8:0f 
  Interface Type bond 
  Master interface: br_default PVID: 1010 
  EVPN-MH: ES id 1 ES sysmac 00:00:00:00:00:bb 
  protodown: off (n/a) 
  ARP-ND redirect enabled: ARP 1 ND 8 
cumulus@leaf04:mgmt:~$ net show bgp evpn vni 5001 
VNI: 5001 (known to the kernel) 
  Type: L3 
  Tenant VRF: RED 
  RD: 10.10.20.2:8 
  Originator IP: 10.10.20.2 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.20.2 
  System-MAC: 44:38:39:22:bb:09 
  Router-MAC: 44:38:39:22:bb:09 
  Import Route Target: 
    0:4001 
    0:5001 
  Export Route Target: 
    65202:5001 
cumulus@leaf04:mgmt:~$ net show bgp evpn vni 5002 
VNI: 5002 (known to the kernel) 
  Type: L3 
  Tenant VRF: GREEN 
  RD: 10.10.20.2:7 
  Originator IP: 10.10.20.2 
  Advertise-gw-macip : n/a 
  Advertise-svi-macip : n/a 
  Advertise-pip: Yes 
  System-IP: 10.10.20.2 
  System-MAC: 44:38:39:22:bb:09 
  Router-MAC: 44:38:39:22:bb:09 
  Import Route Target: 
    0:4002 
    0:5002 
  Export Route Target: 
    65202:5002 
cumulus@leaf04:mgmt:~$ net show evpn mac vni all 
VNI 1010 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
48:b0:2d:72:28:ff local  P     bond1                          1010  0/0 
48:b0:2d:82:05:04 local  P     bond1                          1010  0/0 
b6:4b:0f:ea:f2:02 local  NP    bond1                          1010  0/0 
44:38:39:22:bb:09 local        vlan1010                             0/0 
44:38:39:22:bb:08 remote       10.10.20.1                           0/0 
 
VNI 2020 #MACs (local and remote) 5 
 
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy 
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s 
ee:54:69:be:3a:3f local  NP    bond2                          2020  0/0 
48:b0:2d:75:8e:7a local  P     bond2                          2020  0/0 
44:38:39:22:bb:09 local        vlan2020                             0/0 
48:b0:2d:a7:e2:6e local  P     bond2                          2020  0/0 
44:38:39:22:bb:08 remote       10.10.20.1                           0/0 

To verify that the bridge br_default is learning MAC address entries:

cumulus@leaf03:mgmt:~$ nv show bridge domain br_default mac-table 
    age  bridge-domain  entry-type  interface   last-update  MAC address        src-vni  vlan  vni   Summary 
--  ---  -------------  ----------  ----------  -----------  -----------------  -------  ----  ----  ---------------------- 
0   27   br_default     static      bond2       725          48:b0:2d:75:8e:7a           2020 
1   867  br_default     static      bond2       867          48:b0:2d:a7:e2:6e           2020 
2   63   br_default     static      bond2       867          ee:54:69:be:3a:3f           2020 
3   874  br_default     permanent   bond2       874          48:b0:2d:d5:d5:91 
4   27   br_default     static      bond1       725          48:b0:2d:72:28:ff           1010 
5   867  br_default     static      bond1       867          48:b0:2d:82:05:04           1010 
6   189  br_default     static      bond1       867          b6:4b:0f:ea:f2:02           1010 
7   874  br_default     permanent   bond1       874          48:b0:2d:7d:9c:74 
8   725  br_default                 vxlan48     189          44:38:39:22:bb:09           1010  None  remote-dst: 10.10.20.2 
9   874  br_default     permanent   vxlan48     874          ca:e9:d5:f3:4b:b9                 None 
10  725                 permanent   vxlan48     2            00:00:00:00:00:00  2020           None  remote-dst: 10.10.20.2 
11                      permanent   br_default               00:00:5e:00:01:14 
12  874  br_default     permanent   br_default  874          44:38:39:22:bb:08           1010 
cumulus@leaf04:mgmt:~$ nv show bridge domain br_default mac-table 
    age  bridge-domain  entry-type  interface   last-update  MAC address        src-vni  vlan  vni   Summary 
--  ---  -------------  ----------  ----------  -----------  -----------------  -------  ----  ----  ---------------------- 
0   725  br_default     static      bond2       725          48:b0:2d:75:8e:7a           2020 
1   2    br_default     static      bond2       725          48:b0:2d:a7:e2:6e           2020 
2   180  br_default     static      bond2       725          ee:54:69:be:3a:3f           2020 
3   729  br_default     permanent   bond2       729          48:b0:2d:95:3f:b0 
4   725  br_default                 vxlan48     725          44:38:39:22:bb:08           2020  None  remote-dst: 10.10.20.1 
5   729  br_default     permanent   vxlan48     729          52:d0:7e:cf:6e:ac                 None 
6   725                 permanent   vxlan48     43           00:00:00:00:00:00  1010           None  remote-dst: 10.10.20.1 
7   725  br_default     static      bond1       725          48:b0:2d:72:28:ff           1010 
8   59   br_default     static      bond1       725          b6:4b:0f:ea:f2:02           1010 
9   2    br_default     static      bond1       725          48:b0:2d:82:05:04           1010 
10  729  br_default     permanent   bond1       729          48:b0:2d:84:e8:0f 
11                      permanent   br_default               00:00:5e:00:01:14 
12  729  br_default     permanent   br_default  729          44:38:39:22:bb:09           1010 

From the table above, locate the layer 3 VLAN interface MAC address and the VRR MAC address:

cumulus@leaf03:mgmt:~$ nv show int vlan1010 | grep mac 
    mac-address                              auto 
    mac-id                                   none 
  mac                     44:38:39:22:bb:08 
cumulus@leaf03:mgmt:~$ nv show int vlan1010 ip vrr 
             operational        applied 
-----------  -----------------  ----------------- 
enable                          on 
mac-address  00:00:5e:00:01:14  auto 
mac-id                          none 
[address]    192.168.10.100/24  192.168.10.100/24 
state        up                 up 
cumulus@leaf04:mgmt:~$ nv show int vlan1010 | grep mac 
    mac-address                              auto 
    mac-id                                   none 
  mac                     44:38:39:22:bb:09 
cumulus@leaf04:mgmt:~$ nv show int vlan1010 ip vrr 
             operational        applied 
-----------  -----------------  ----------------- 
enable                          on 
mac-address  00:00:5e:00:01:14  auto 
mac-id                          none 
[address]    192.168.10.100/24  192.168.10.100/24 
state        up                 up 

To verify EVPN type-5 routes at the ingress PE (leaf03) for the end host 192.168.1.10 connected to leaf01 and leaf02:

cumulus@leaf03:mgmt:~$ net show bgp evpn route type 5 | grep 192.168.1\. -A 4 -B 1 
Route Distinguisher: 10.10.10.1:6 
*  [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.1:6 
                    10.10.10.1 (spine03) 
                                                           0 65299 65210 65110 65199 65101 ? 
                    RT:65101:4001 ET:8 Rmac:44:38:39:22:bb:06 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.1:6 
                    10.10.10.1 (spine04) 
                                                           0 65299 65210 65110 65199 65101 ? 
                    RT:65101:4001 ET:8 Rmac:44:38:39:22:bb:06 
Route Distinguisher: 10.10.10.2:4 
*  [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.2:4 
                    10.10.10.2 (spine03) 
                                                           0 65299 65210 65110 65199 65102 ? 
                    RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.2:4 
                    10.10.10.2 (spine04) 
                                                           0 65299 65210 65110 65199 65102 ? 
                    RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
Route Distinguisher: 10.10.10.2:5 

Verify EVPN type-5 routes at the egress PE (leaf01) for the end host 192.168.1.10 connected to leaf01 and leaf02:

cumulus@leaf01:mgmt:~$ net show bgp evpn route type 5 | grep 192.168.1\. -A 4 -B 1 
Route Distinguisher: 10.10.10.1:6 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.1:6 
                    10.10.10.1 (leaf01) 
                                             0         32768 ? 
                    ET:8 RT:65101:4001 Rmac:44:38:39:22:bb:06 
Route Distinguisher: 10.10.10.2:4 
*  [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.2:4 
                    10.10.10.2 (spine02) 
                                                           0 65199 65102 ? 
                    RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.2:4 
                    10.10.10.2 (spine01) 
                                                           0 65199 65102 ? 
                    RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
Route Distinguisher: 10.10.10.2:5 
-- 

To view routing from the border leaf:

cumulus@borderleaf04:mgmt:~$ net show bgp sum 
show bgp ipv4 unicast summary 
============================= 
BGP router identifier 10.10.20.11, local AS number 65210 vrf-id 0 
BGP table version 18 
RIB entries 19, using 3800 bytes of memory 
Peers 3, using 68 KiB of memory 
Peer groups 2, using 128 bytes of memory 
 
Neighbor           V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
borderleaf01(swp3) 4      65110      1260      1263        0    0    0 01:01:26            5       10 
spine03(swp1)      4      65299      1341      1345        0    0    0 01:01:24            4       10 
spine04(swp2)      4      65299      1340      1343        0    0    0 01:01:16            4       10 
 
Total number of neighbors 3 
 
show bgp ipv6 unicast summary 
============================= 
% No BGP neighbors found 
 
show bgp l2vpn evpn summary 
=========================== 
BGP router identifier 10.10.20.11, local AS number 65210 vrf-id 0 
BGP table version 0 
RIB entries 35, using 7000 bytes of memory 
Peers 3, using 68 KiB of memory 
Peer groups 2, using 128 bytes of memory 
 
Neighbor           V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt 
borderleaf01(swp3) 4      65110      1260      1263        0    0    0 01:01:26            4        8 
spine03(swp1)      4      65299      1342      1346        0    0    0 01:01:24           42       46 
spine04(swp2)      4      65299      1340      1343        0    0    0 01:01:16           42       46 
 
Total number of neighbors 3
cumulus@borderleaf04:mgmt:~$ net show bgp vrf RED 
show bgp vrf RED ipv4 unicast 
============================= 
BGP table version is 11, local router ID is 10.10.20.11, vrf id 13 
Default local pref 100, local AS 65210 
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath, 
               i internal, r RIB-failure, S Stale, R Removed 
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self 
Origin codes:  i - IGP, e - EGP, ? - incomplete 
 
   Network          Next Hop            Metric LocPrf Weight Path 
*= 192.168.1.0/24   10.10.10.2<                            0 65110 65199 65102 ? 
*>                  10.10.10.1<                            0 65110 65199 65101 ? 
*= 192.168.10.0/24  10.10.20.2<                            0 65299 65202 ? 
*                   10.10.20.2<                            0 65299 65202 ? 
*>                  10.10.20.1<                            0 65299 65201 ? 
*                   10.10.20.1<                            0 65299 65201 ? 
*= 192.168.10.110/32 
                    10.10.20.2<                            0 65299 65202 i 
*                   10.10.20.2<                            0 65299 65202 i 
*>                  10.10.20.1<                            0 65299 65201 i 
*                   10.10.20.1<                            0 65299 65201 i 
 
Displayed  3 routes and 10 total paths 

show bgp vrf RED ipv6 unicast 
============================= 
No BGP prefixes displayed, 0 exist
cumulus@borderleaf04:mgmt:~$ net show bgp evpn route type 5 | grep 192.168.1\. -A 3 -B 1 
Route Distinguisher: 10.10.10.1:6 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.1:6 
                    10.10.10.1 (borderleaf01) 
                                                           0 65110 65199 65101 ? 
                    RT:65101:4001 ET:8 Rmac:44:38:39:22:bb:06 
Route Distinguisher: 10.10.10.2:4 
*> [5]:[0]:[24]:[192.168.1.0] RD 10.10.10.2:4 
                    10.10.10.2 (borderleaf01) 
                                                           0 65110 65199 65102 ? 
                    RT:65102:4001 ET:8 Rmac:44:38:39:22:bb:07 
-- 
Route Distinguisher: 10.10.20.1:8 
*  [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.1:8 
                    10.10.20.1 (spine03) 
                                                           0 65299 65201 ? 
                    RT:65201:5001 ET:8 Rmac:44:38:39:22:bb:08 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.1:8 
                    10.10.20.1 (spine04) 
                                                           0 65299 65201 ? 
                    RT:65201:5001 ET:8 Rmac:44:38:39:22:bb:08 
-- 
Route Distinguisher: 10.10.20.2:8 
*  [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.2:8 
                    10.10.20.2 (spine04) 
                                                           0 65299 65202 ? 
                    RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09 
*> [5]:[0]:[24]:[192.168.10.0] RD 10.10.20.2:8 
                    10.10.20.2 (spine03) 
                                                           0 65299 65202 ? 
                    RT:65202:5001 ET:8 Rmac:44:38:39:22:bb:09