Arista VXLAN Indirect Routing Cheat Sheet (DRAFT) by sh-arista
Details on VXLAN Indirect Routing Model
This is a draft cheat sheet. It is a work in progress and is not finished yet.
VXLAN Indirect Routing
To route between VLANs which are crossing the VXLAN overlay (also known as "stretched" or "extended" VLANs), Switched Virtual Interfaces (SVIs) need to be added to the VTEPs to perform the routing. With the indirect routing model, the dest host is not directly attached to the VTEP(s) where the SVIs are configured. As packets might need to go through multiple hops in the overlay to reach the final destination, this model is called "indirect". This scales better than direct routing as each VTEP is only responsible for routing a subset of the overlay subnets (fewer ARP entries, fewer routes.)
VXLAN and NAT cannot coexist on a switch.
show interface vxlan 1
should be "up"; correctly reflect configured VLAN-to-VNI mappings; confirm control plane (multicast, HER, CVX, EVPN)
show mac address-table
The L2 forwarding table should show that mac addresses are either learned locally or from across the VXLAN overlay - "Vx1"; if we are not learning MACs from another VTEP confirm flood list and L3 reachability between VTEPs
show vxlan address-table
Shows the VXLAN MAC info, including the Host MAC, remote VTEP IP, and MAC moves.
show ip route
All VTEP IPs (primary and secondary) should have L3 reachability (ping to confirm)
show vxlan counters software
See "Arista VXLAN Bridging" for more info.
show platform fap arp
Confirm ARP in hardware.
show platform fap ip route
Confirms the prefix points to an L3FloodFap0 interface used for recirculation. These ports are 100G.
VXLAN encapsulated protocol packets (BGP/ISIS/OSPF) use the CoppSystemL2Bcast queue.
VTEP1Configure a loopback as the L3 source interface for the VXLAN tunnel/VTEP.!interface loopback 1 ip address 184.108.40.206/32Configure a second IP, the virtual VTEP IP, on routing VTEPs.ip address 100.100.100.100/32 secondary!Add DGs for routing - aka "anycast gateway"; can use VARP as seen here or "ip address virtual" to save IP space.!interface vlan 10ip address 10.0.0.1/24ip virtual-router address 10.0.0.254!Configure the VXLAN tunnel interface.!interface vxlan 1Add the source interface configured earlier.vxlan source-interface loopback1Configure the UDP destination port; be consistent across VTEPs.vxlan udp-port 4789Configure the VLAN to VNI bindings for any extended VLANs.vxlan vlan 10 vni 10010Configure the static flood-set if using HER for control-plane; include all VTEP IPs that need to receive BUM traffic.vxlan flood vtep 220.127.116.11 18.104.22.168!Configure the virtual MAC to be used with all virtual IPs.!ip virtual-router mac-address 00:00:11:22:33:44!Ensure routing is enabled.ip routing!VTEP2!interface loopback 1ip address 22.214.171.124/32Ensure the virtual VTEP IP is identical across all routing VTEPs.ip address 100.100.100.100/32 secondary!interface vlan 10Here using "ip address virtual" as the anycast gateway IP.ip address virtual 10.0.0.254/24!interface vlan 20ip address virtual 126.96.36.199/24!interface vxlan 1vxlan source-interface loopback1vxlan udp-port 4789vxlan vlan 10 vni 10010vxlan vlan 20 vni 10020vxlan flood vtep 188.8.131.52 184.108.40.206!ip virtual-router mac-address 00:00:11:22:33:44!ip routing!VTEP 3!interface loopback 1ip address 220.127.116.11/32 ip address 100.100.100.100/32 secondary!interface vxlan 1vxlan source-interface loopback1vxlan udp-port 4789vxlan vlan 30 vni 10030vxlan flood vtep 18.104.22.168 22.214.171.124!ip routing
All routing VTEPs have secondary IPs, called Virtual VTEP IPs, configured that are identical.
In order to learn routes to remote destinations, routing protocols such as OSPF/BGP/ISIS can be configured in the overlay.
There is no "indirect" specific configuration necessary.