VXLAN Configuration Guide

evpn evpn introduction introduction vxlan as a network virtualization technology helps us to achieve scenarios such as multi tenant expansion and vm migration however, traditional vxlans are established through manual configuration, which is complex to deploy and has no control surface, and address learning is done through multicast based flooding, which generates a large number of flood packets and is not suitable for large scale network evpn (ethernet virtual private network), using a mechanism similar to bgp/mpls ip vpn, automatically establishes vxlan tunnels and automatically synchronizes mac and ip addresses through mp bgp (multi protocol extensions for border gateway protocol), which is a good solution to these problems evpn, as a layer 2 vpn technology, the control plane uses mp bgp to announce evpn routing information, and the data plane supports vxlan encapsulation for forwarding packets basic concepts basic concepts bgp neighbors bgp neighbors bgp neighbors are divided into ibgp and ebgp ibgp neighbor ibgp neighbor ibgp means that the neighbor is located in the same as as the local router when ibgp is deployed, route reflector (rr) can be introduced to simplify full connection configuration the rr discovers and receives bgp connections initiated by vteps and forms a client list, reflecting the routes received from a vtep to all other vteps the rr can be deployed in a spine, a leaf or a standalone device ebgp neighbor ebgp neighbor ebgp means that the neighbor and local router are located in different as when ebgp is deployed, rr is not required and bgp automatically sends evpn packets received from ebgp neighbors to other ebgp and ibgp neighbors, and spine is equivalent to the rr function in general, the asn differs from device to device, so ebgp is mostly used in sonic if redundant leaf is used, the asn is the same for the pair of leafs symmetric irb symmetric irb in evpn networks, vtep can do both layer 2 bridge forwarding and layer 3 router routing functions, hence called integrated bridging and routing (irb) in a distributed gateway, irb forwarding can be divided into two types symmetric irb and asymmetric irb symmetric irb refers to the fact that at the ingress gateway and the egress gateway, both do only the l3 routing function (or only the bridging function if they are on the same network segment) in this mode, each distributed gateway only needs to configure the vni where the virtual machine hanging below it is located, and it does not need to maintain arps for all hosts or virtual machines within this tenant, but only a small number of arps corresponding to other distributed gateways asymmetric irb refers to the fact that at the ingress gateway do both the l2 bridge and l3 routing function, and at the egress gateway do only the l2 bridge functions the following 2 concepts, l3vni and routermac, are introduced in symmetric irb l3vni means that when traffic is forwarded between distributed gateways via vxlan tunnels, traffic belonging to the same tenant (vrf) is identified by l3vni, which is uniquely associated with a vrf (vpn instance) and ensures service isolation between different tenants the routermac address of a gateway is a unique local mac address owned by each distributed gateway that is used to identify the local machine this mac is used to forward layer 3 traffic between gateways through vxlan tunnels when packets are forwarded between gateways, the inner mac address of the packet is the routermac address of the egress gateway the symmetric irb optimizes the arp and mac address tables in each vtep to provide better scalability based on the overall number of vnis that can be supported in a vxlan overlay network the sonic implements a symmetric irb forwarding model vrf vrf vrf, virtual routing forwarding vrf is used to resolve local routing conflicts each vrf can be seen as a virtual router, consisting of the following elements an independent routing table/forwarding table, a set of interfaces belonging to this vrf, and a set of routing protocols exclusively used for this vrf packet type packet type legacy bgp 4 can only manage ipv4 unicast routing information, which is limited for applications using other network layer protocols (e g ipv6, multicast, etc ) when propagating across as mp bgp is an extension of bgp 4 to provide support for multiple network layer protocols, and the update packet carries three pieces of information related to ipv4 in the packets used by bgp the nlri (network layer reachability information) field, the next hop attribute, and the aggregator attribute mp bgp reflects the network layer protocol information in the nlri and next hop fields, and introduces two new optional non transition path attributes, as shown in the following table table 1 optional non transition path attributes table 1 optional non transition path attributes properties description mp reach nlri multiprotocol reachable nlri, used to publish reachable routes and next hop information mp unreach nlri multiprotocol unreachable nlri, used to undo unreachable routes evpn defines the following types of bgp evpn route types in mp reach nlri and mp unreach nlri by extending the nlri of the bgp protocol for learning and publishing host information between different sites in a layer 2 network mac/ip advertisement route (type 2 route) mac/ip advertisement route (type 2 route) as mentioned earlier, in the absence of a control plane, mac learning for hosts in vxlan network is done by flooding to solve this problem, evpn has defined type 2 route, i e mac/ip route, with the packet format shown below type 2 route achieves the following main things host mac address advertise it can carry host mac information and is used to inform each other of the mac information of local downstream hosts between evpn peers, enabling layer 2 intercommunication with hosts on the same subnet host arp advertise can carry both host mac+ip, i e host arp, for passing arp table entries of local downstream hosts between evpn peers, for arp broadcast suppression and vm migration host ip route advertise the ip address length and ip address fields are the destination address of the host ip route, while the mpls label 2 field must carry a layer 3 vni, at which point the mac/ip route, also known as an irb type route, can mutually notify each other of the local downstream host ip route that has been obtained, realizing layer 3 intercommunication with hosts across subnets in a distributed gateway inclusive multicast ethernet tag route (type 3 route) inclusive multicast ethernet tag route (type 3 route) evpn defines type 3 route, i e inclusive multicast routes, which are mainly used in the vxlan control plane for vtep auto discovery and dynamic establishment of vxlan tunnels the packet format is shown in the figure below as a vtep in a bgp evpn peer relationship, it exchanges layer 2 vni and vtep ip address information with peers via type 3 routes where the originating router's ip address field is the local vtep ip address and the mpls label field is the layer 2 vni if the peer vtep ip address is reachable by a layer 3 route and the peer's vni is the same as the local one, a vxlan tunnel to the peer is established and a mapping is created and a header replication table is created that for subsequent bum packet forwarding ip prefix advertisement route (type 5 route) ip prefix advertisement route (type 5 route) in addition, evpn defines type 5 route, i e ip prefix routes, declaring their routing packet format as shown in the following figure the ip prefix length and ip prefix fields can carry either the host ip address or the network segment address when carrying the host ip address, it is the same as the host ip route announcement function of type 2 route; when carrying the network segment address, it is mainly used to implement hosts in vxlan networks to access external networks type 5 route is used to structure tunnel route and implement arp proxy, which are not currently used in evpn scenarios, but are used in picfa scenarios working principle working principle vxlan tunnel establishment vxlan tunnel establishment enable evpn on leaf, configure local vtep, after configuring the mapping, it will announce evpn type 3 route, carrying the main information of local vtep + vni; the remote leaf receives the announcement to see if the same vni is configured locally, if so, then establish l2vxlan tunnel; similarly, the remote leaf announces it, and after the local receives it, then establish the l2vxlan tunnel locally after the vm comes online, it sends a free arp, which leaf learns and updates its own fdb, arp table and at the same time announces an evpn type 2 route carrying l3vni the remote leaf receives the announcement and first establishes the l3vxlan tunnel, then establishes the arp cache for the remote vm (remote vm real ip + real mac) and the next hop information for the tunnel (remote vtep ip + corresponding routermac) mac/ip route announcements and learning mac/ip route announcements and learning as shown above, vm a1 comes online, vtep 1 synchronizes the learned mac and host route of vm a1 to rr via type 2 route; rr receives it and synchronizes the route to all neighbors vtep 2 receives it and sends down the mac of vm a1 to the fdb table, and the ip of vm a1 forms a 32 bit host route to the routing table of its vrf traffic forward traffic forward interworking between vms on the same leaf is the most basic layer 2 and 3 forwarding, and is not related to vxlan tunnels, so it will not be described here cross device vm interworking is carried out through vxlan tunnels the following will describe in detail the flow of cross device layer 2 forwarding and cross device layer 3 forwarding in evpn scenarios, using vm a1 to vm a2 communication and vm b1 to vm a2 communication as examples background vm a2 is online, evpn interaction learning on each leaf has been completed, tunnels have been established and each forwarding table has been generated layer 2 forward layer 2 forward vm a1 communicates with vm a2 vm a1 first sends arp requests for the mac address of vm a2 upon receipt of this request, leaf1 looks up the table based on the layer 3 port (vlan100) and destination ip (ip a2) of the received packet and replies to the result (mac a2) to vm a1 vm a1 sends an icmp packet to vm a2, where dip is ip a2 and dmac is mac a2 leaf1 receives it, queries the fdb table based on the packet vlan and dmac, the query hits, and encapsulates it to go through the tunnel the outer dmac of the encapsulated packet is the corresponding routermac (mac spine), and the outer dip is the opposite end vtep ip, vni is 100 the encapsulated packet is forwarded to leaf2 via spine leaf2 parses the packet and finds that the outer dmac is the local mac, the outer dip is the local vtep ip, and the vxlan mapping (vni100 vlan100) exists, so it is unencapsulated and then forwards it to vm a2 by querying the fdb table based on the inner dmac (mac a2) layer 3 forward layer 3 forward vm b1 communicates with vm a2 vm b1 first sends an arp request for the gateway mac, and leaf1 receives the request and replies with the gateway mac to vm b1 vm b1 sends an icmp packet to vm a2, where dip is ip a2 and dmac is the gateway mac (mac leaf1) dmac is found to be the local mac upon receipt by leaf1, so layer 3 forwarding is performed, the routing table is queried based on the vrf (vrf123) and dip where the packet vlan is located, the query hits, the next hop is the peer vtep ip, encapsulated go through the tunnel, the outer dmac of the encapsulated packet is the corresponding routermac (mac spine), the outer dip is the vtep ip of the opposite end, vni is 1000, and the inner dmac is the mac of the vtep of the opposite end the encapsulated packet is forwarded by spine to leaf2, which parses the packet and finds that the outer dmac is the local mac, the outer dip is the local vtep ip, and the vxlan mapping (vni1000 vrf456) exists, so it is unencapsulated; in turn, the inner dmac is also the local mac, which is forwarded at layer 3, based on vrf456 and dip query routing table, query hits, modify dmac to mac a2, and then forward to vm a2 enhancements enhancements in addition to the basic functions, the sonic also offers several enhancements arp suppression arp suppression when the scale of virtual machines in the network expands, arp broadcast will consume the network bandwidth and there will be the hidden danger of broadcast storms to reduce the impact caused by arp broadcast, we can suppress arp flooding by means of arp proxy turn off arp flooding and enable arp proxy on the leaf device when leaf receives an arp request from the local host, it will query the database and reply with the real mac to the local host vm migration vm migration in an evpn scenario, when a host migrates from one leaf node to another, the host sends free arp packets, and the migrated leaf refreshes the local host route/host arp information based on the free arp packet and advertises an evpn type 2 route to the remote leaf, directing traffic to the migrated leaf the difference between this route and the mac/ip route advertised before the migration is that the bgp update packet carries a new extension group mac mobility, where the seqnum field is incremented by 1 each time the migration takes place when the remote leaf receives packets with seqnum larger than the local arp, it updates its own mac/ip routing information with the next hop pointing to the vtep or gateway that advertised this route after the migration the original vtep, upon receiving this route update, revokes the previously advertised route border border border refers to a network edge device that is used for hosts in the overlay network to access the external network configuring routes in the vrf on a device configured as border will synchronize these routes to the vxlan network via evpn type 5 route; different routes are configured in different vrfs, thus controlling access to the external network by hosts of different users in the vxlan network evpn configuration evpn configuration table 2 overview of evpn configuration tasks table 2 overview of evpn configuration tasks configuration tasks description refer to pre configuration configure vlan required docid\ skptsp8urkzykud rlmus configure vrf required docid\ skptsp8urkzykud rlmus evpn related configuration configure vtep ip required docid\ skptsp8urkzykud rlmus configure underlay bgp required docid\ skptsp8urkzykud rlmus configure overlay bgp required docid\ skptsp8urkzykud rlmus configure vxlan map required docid\ skptsp8urkzykud rlmus configure arp proxy optional docid\ skptsp8urkzykud rlmus configure vlan configure vlan it is required that the gateways (vlan) of the vms under the same l2 vni must be the same, i e , the ips and macs of the vlans are the same, and the vlan ids can be different (because the vlan ids only work locally) table 3 configure vlan table 3 configure vlan purpose commands description enter global configuration view configure terminal enter vlan configuration view and create vlan vlan vlan id vlan id range 1 4094 exit vlan configuration view exit enter vlanif configuration view interface vlan vlan id set the ip for vlanif ip address a b c d/m configure mac for vlanif mac address hh\ hh\ hh\ hh\ hh \ hh mac address is not case sensitive exit vlanif configuration view exit enter interface configuration view interface { ethernet interface name | link aggregation lag id } add vlan member ports switchport { trunk | access } vlan vlan id configure vrf configure vrf in the evpn mc lag scenario, it is required that the mac of vrfs corresponding to the same l3 vnis are the same on the master and standby devices deploying mc lag table 4 configure vrf table 4 configure vrf purpose commands description enter global configuration view configure terminal enter the vrf configuration view and create vrf vrf vrf name set the mac of vrf (optional) mac hh\ hh\ hh\ hh\ hh \ hh exit vrf configuration view exit enter vlanif configuration view interface vlan vlan id bind the vlan to the vrf vrf vrf name configure vtep ip configure vtep ip it is recommended to configure a loopback1 ip as the local vtep ip note in evpn mc lag scenarios, the local vtep ips of the two peers are required to be same in evpn multihoming scenarios, the local vtep ips of the two peers are required to be different table 5 configure vtep ip table 5 configure vtep ip purpose commands description enter global configuration view configure terminal enter loopback 0 interface view interface loopback 0 configure the ip address of loopback 0 ip address a b c d/m since the ip address of loopback0 will be used as the router id when bgp neighbors are established, the ip address of loopback0 is required to be unique exit loopback 0 interface view exit enter loopback 1 interface view interface loopback 1 configure the ip address of loopback 1 ip address a b c d/m the ip address of loopback1 will be used as the vtep ip for vxlan exit loopback 1 interface view exit enter vxlan view interface vxlan vxlan id vxlan id vxlan id, range 0 9 cx308p 48y n v2 and cx532p n v2 devices support this configurations in the range of 0 9, other devices can only be configured as 0 configure the local address of vtep source ip address loopback1 ip is normally configured as vtep ip configure underlay bgp configure underlay bgp create bgp neighbors and proactively advertise routes for vtep ips to be reachable by vtep ips note in evpn mc lag scenarios, it is required to assign same bgp as number for peers in evpn multihoming scenarios, it is recommended to assign different bgp as number for peers table 6 configure underlay bgp table 6 configure underlay bgp purpose commands description enter global configuration view configure terminal enter bgp view router bgp asn asn local as number (optional) configure router identifier bgp router id a b c d disable the ebgp requires policy no bgp ebgp requires policy add bgp neighbors neighbor neighbor ip remote as asn enter address family view for ipv4 unicast address family ipv4 unicast advertise loopback0 and loopback1 ip route network { a b c d/m | a b c d } configure overlay bgp configure overlay bgp configure overlay bgp and advertise all vnis table 7 configure overlay bgp table 7 configure overlay bgp purpose commands description enter global configuration view configure terminal enter bgp view router bgp asn asn local as number add bgp neighbors neighbor neighbor ip remote as asn neighbor ip neighbor's loopback0 ip, same below configure ebgp multihop neighbor neighbor ip ebgp multihop max hop max hop maximum hop count for bgp messages, its range is \[1,255] configure source ip for bgp messages neighbor neighbor ip update source loopback0 ip loopback0 ip local loopback0 ip enter address family view of ipv4 unicast address family ipv4 unicast disable ipv4 for the bgp neighbor no neighbor neighbor ip activate exit address family view exit address family enter address family view of l2vpn evpn address family l2vpn evpn enable evpn neighbor neighbor ip activate advertise all vni advertise all vni configure vxlan map configure vxlan map configure layer 2 and layer 3 vxlan map table 8 configure vxlan map table 8 configure vxlan map purpose commands description enter global configuration view configure terminal enter vlan configuration view vlan vlan id configure layer 2 vxlan mappings vni vni id exit current view exit enter vrf configuration view vrf vrf name configure layer 3 vxlan mapping vni vni id vxlan vxlan id (optional) configure arp proxy (optional) configure arp proxy table 9 configure the arp proxy table 9 configure the arp proxy purpose commands description enter global configuration view configure terminal disable arp broadcast arp broadcast disable enter vlanif configuration view interface vlan vlan id configure the arp proxy arp proxy mode evpn note there are two arp proxy modes, namely default and evpn the default mode is the default arp proxy mode, which replies to the gateway mac uniformly; and the evpn mode is used to enable the route port as overlay interface to open arp proxy after enabling evpn display and maintenance display and maintenance table 10 evpn display and maintenance table 10 evpn display and maintenance purpose commands description show vxlan mapping table show vxlan map check established vxlan tunnels show vxlan tunnel display the remote mac entries synchronized via vxlan show vxlan remotemac { all | a b c d } typical configuration example typical configuration example distributed gateway symmetric irb distributed gateway symmetric irb networking requirements a large number of vms are deployed in the data center of an enterprise, currently there are two subnets with a total of four vms distributed on two leaf switches, vm a1, vm a2 and vm b1 on leaf1 and vm a3 on leaf2 it is assumed that layer 3 interworking has been implemented between each leaf switch it is now required to interwork between the vms by means of bgp evpn the asn of each device is given in the topology topology equipment interface ip address spine eth 0/0 10 93 0 1/30 eth 0/8 10 93 0 9/30 loopback0 172 16 1 165/32 leaf1 eth 0/48 10 93 0 2/30 vlan10 10 10 0 1/24 vlan20 10 20 0 1/24 loopback0 172 16 1 179/32 loopback1 172 16 2 179/32 leaf2 eth 0/48 10 93 0 10/30 vlan10 10 10 0 1/24 loopback0 172 16 1 170/32 loopback1 172 16 2 170/32 configuration roadmap (1)check that the link status of the ports used on the switch is normal, and that all ports are stable in the up state (2)configure the switch interface ip addresses and the loopback0 ip address (3)configure the vtep ip address on the leaf devices (4)configure the underlay bgp (5)configure the overlay bgp (6)configure vlans and add vlan member ports, create vrf instances, and configure vlanif interfaces (7)configure vxlan vni mapping 4 procedure spine \#configure the switch interface ip addresses interface ethernet 0/0 ip address 10 93 0 1/30 exit ! interface ethernet 0/8 ip address 10 93 0 9/30 exit ! interface loopback 0 ip address 172 16 1 165/32 exit \#configure underlay bgp router bgp 65165 bgp router id 172 16 1 165 no bgp ebgp requires policy bgp bestpath as path multipath relax neighbor peer to leaf peer group neighbor peer to leaf remote as external neighbor 10 93 0 2 peer group peer to leaf neighbor 10 93 0 10 peer group peer to leaf address family ipv4 unicast network 172 16 1 165/32 \#configure overlay bgp router bgp 65165 neighbor peer to leaf evpn peer group neighbor peer to leaf evpn remote as external neighbor peer to leaf evpn ebgp multihop 5 neighbor peer to leaf evpn update source 172 16 1 165 neighbor 172 16 1 179 peer group peer to leaf evpn neighbor 172 16 1 170 peer group peer to leaf evpn address family ipv4 unicast no neighbor peer to leaf evpn activate exit address family ! address family l2vpn evpn neighbor peer to leaf evpn activate advertise all vni exit address family exit leaf1 \#configure the switch interface ip addresses interface ethernet 0/48 ip address 10 93 0 2/30 exit ! interface loopback 0 ip address 172 16 1 179/32 exit ! interface loopback 1 ip address 172 16 2 179/32 exit \#configure vtep ip interface vxlan 0 source 172 16 2 179 exit \#configure underlay bgp router bgp 65100 bgp router id 172 16 1 179 no bgp ebgp requires policy bgp bestpath as path multipath relax neighbor 10 93 0 1 remote as 65165 address family ipv4 unicast network 172 16 1 179/32 network 172 16 2 179/32 \#configure overlay bgp router bgp 65100 neighbor 172 16 1 165 remote as 65165 neighbor 172 16 1 165 ebgp multihop 5 neighbor 172 16 1 165 update source 172 16 1 179 address family ipv4 unicast no neighbor 172 16 1 165 activate address family l2vpn evpn neighbor 172 16 1 165 activate advertise all vni exit address family exit \#configure vlans and add vlan member ports, create vrf instances vlan 10 exit ! vlan 20 exit ! interface ethernet 0/0 switchport access vlan 10 exit ! interface ethernet 0/1 switchport access vlan 10 exit ! interface ethernet 0/2 switchport access vlan 20 exit ! vrf 10123 mac 00 00 00 01 23 00 exit ! arp broadcast disable ! interface vlan 10 mac address 00 00 00 10 00 00 vrf 10123 ip address 10 10 0 1/24 arp proxy mode evpn ! interface vlan 20 mac address 00 00 00 20 00 00 vrf 10123 ip address 10 20 0 1/24 arp proxy mode evpn \#configure vxlan vni mapping vlan 10 vni 10 ! vlan 20 vni 20 ! vrf 10123 vni 10000 exit vrf ! leaf2 \#configure the switch interface ip addresses interface ethernet 0/48 ip address 10 93 0 10/30 exit ! interface loopback 0 ip address 172 16 1 170/32 exit ! interface loopback 1 ip address 172 16 2 170/32 exit \#configure vtep ip interface vxlan 0 source 172 16 2 170 exit \#configure underlay bgp router bgp 65102 bgp router id 172 16 1 170 no bgp ebgp requires policy bgp bestpath as path multipath relax neighbor 10 93 0 9 remote as 65165 address family ipv4 unicast network 172 16 1 170/32 network 172 16 2 170/32 \#configure overlay bgp router bgp 65102 neighbor 172 16 1 165 remote as 65165 neighbor 172 16 1 165 ebgp multihop 5 neighbor 172 16 1 165 update source 172 16 1 170 address family ipv4 unicast no neighbor 172 16 1 165 activate address family l2vpn evpn neighbor 172 16 1 165 activate advertise all vni exit address family exit \#configure vlans and add vlan member ports, create vrf instances vlan 10 exit ! interface ethernet 0/0 switchport access vlan 10 exit vrf 10123 mac 00 00 00 01 23 01 exit ! arp broadcast disable ! interface vlan 10 mac address 00 00 00 10 00 00 vrf 10123 ip address 10 10 0 1/24 arp proxy mode evpn \#configure vxlan vni mapping vlan 10 vni 10 ! vrf 10123 vni 10000 exit vrf ! configuration verification spine \#display underlay bgp neighbors sonic# show ip bgp summary ipv4 unicast summary (vrf default) bgp router identifier 172 16 1 165, local as number 65165 vrf id 0 bgp table version 9 rib entries 13, using 2392 bytes of memory peers 2, using 1447 kib of memory peer groups 1, using 64 bytes of memory neighbor v as msgrcvd msgsent tblver inq outq up/down state/pfxrcd pfxsnt desc 10 93 0 2 4 65100 1334 1334 0 0 0 01 58 22 7 7 n/a 10 93 0 10 4 65102 133 133 0 0 0 01 58 22 7 7 n/a total number of neighbors 2 \#display overlay bgp neighbors sonic# show bgp l2vpn evpn summary bgp router identifier 172 16 1 165, local as number 65165 vrf id 0 bgp table version 0 rib entries 31, using 5704 bytes of memory peers 2, using 1447 kib of memory peer groups 1, using 64 bytes of memory neighbor v as msgrcvd msgsent tblver inq outq up/down state/pfxrcd pfxsnt desc 172 16 1 179 4 65100 196 328 0 0 0 00 05 52 4 10 n/a 172 16 1 170 4 65102 1037 1081 0 0 0 02 42 12 4 10 n/a total number of neighbors 2 leaf1 \#display underlay bgp neighbors sonic# show ip bgp summary ipv4 unicast summary (vrf default) bgp router identifier 172 16 1 179, local as number 65100 vrf id 0 bgp table version 9 rib entries 13, using 2392 bytes of memory peers 1, using 724 kib of memory peer groups 0, using 0 bytes of memory neighbor v as msgrcvd msgsent tblver inq outq up/down state/pfxrcd pfxsnt desc 10 93 0 1 4 65165 133 133 0 0 0 01 58 22 4 7 n/a total number of neighbors 1 \#display overlay bgp neighbors sonic# show bgp l2vpn evpn summary bgp router identifier 172 16 1 179, local as number 65100 vrf id 0 bgp table version 0 rib entries 151, using 27 kib of memory peers 1, using 1447 kib of memory peer groups 0, using 0 bytes of memory neighbor v as msgrcvd msgsent tblver inq outq up/down state/pfxrcd pfxsnt desc 172 16 1 165 4 65165 196 328 0 0 0 00 05 52 6 10 n/a total number of neighbors 1 \#display vxlan tunnels sonic # show vxlan tunnel + + + + + \| remotevtep | vni | vlan | vrf | +==============+=======+========+=======+ \| 172 16 2 170 | 10 | 10 | | + + + + + \| 172 16 2 170 | 10000 | | 10123 | + + + + + leaf2 \#display underlay bgp neighbors sonic# show ip bgp summary ipv4 unicast summary (vrf default) bgp router identifier 172 16 1 170, local as number 65102 vrf id 0 bgp table version 4 rib entries 15, using 2392 bytes of memory peers 1, using 724 kib of memory peer groups 0, using 0 bytes of memory neighbor v as msgrcvd msgsent tblver inq outq up/down state/pfxrcd pfxsnt desc 10 93 0 9 4 65165 1231 1250 0 0 0 01 15 12 4 7 n/a total number of neighbors 1 \#display overlay bgp neighbors sonic# show bgp l2vpn evpn summary bgp router identifier 172 16 1 179, local as number 65100 vrf id 0 bgp table version 0 rib entries 151, using 27 kib of memory peers 1, using 1447 kib of memory peer groups 0, using 0 bytes of memory neighbor v as msgrcvd msgsent tblver inq outq up/down state/pfxrcd pfxsnt desc 172 16 1 165 4 65165 32 28 0 0 0 00 08 44 6 10 n/a total number of neighbors 1 \#display vxlan tunnels sonic # show vxlan tunnel + + + + + \| remotevtep | vni | vlan | vrf | +==============+=======+========+=======+ \| 172 16 2 179 | 10 | 10 | | + + + + + \| 172 16 2 179 | 10000 | | 10123 | + + + + + the vms under each leaf ping each other and can ping through static vxlan static vxlan example of layer 2 vxlan scenario configuration example of layer 2 vxlan scenario configuration networking requirements a company needs to implement layer 2 interconnection between vms under different leaf devices through layer 2 vxlan topology configuration roadmap (1)check that the link status of the ports used on the switch is normal, and that all ports are stable in the up state (2)configure the switch interface ip addresses and the loopback0 ip address (3)configure static routes to ensure reachability between loopback ip addresses (4)configure the vtep ip address on the leaf devices (5)configure vlans and add vlan member ports (6)configure vxlan vni mapping and static mac entries for the host procedure spine configure the switch interface ip addresses and the loopback0 ip address interface ethernet 0/0 ip address 10 93 0 1/30 exit ! interface ethernet 0/8 ip address 10 93 0 9/30 exit ! interface loopback 0 ip address 172 16 1 165/32 exit configure static routes to ensure reachability between loopback ip addresses ! ip route 172 16 1 179/32 10 93 0 2 ! ip route 172 16 2 179/32 10 93 0 2 ! ip route 172 16 1 170/32 10 93 0 10 ! ip route 172 16 2 170/32 10 93 0 10 leaf1 configure the switch interface ip addresses and the loopback0 ip address interface ethernet 0/48 ip address 10 93 0 2/30 exit ! interface loopback 0 ip address 172 16 1 179/32 exit ! interface loopback 1 ip address 172 16 2 179/32 exit configure static routes to ensure reachability between loopback ip addresses ! ip route 172 16 1 170/32 10 93 0 1 ! ip route 172 16 2 170/32 10 93 0 1 configure the vtep ip address on the leaf devices interface vxlan 0 source 172 16 2 179 exit configure vlans and add vlan member ports vlan 10 exit ! vlan 20 exit ! interface ethernet 0/0 switchport access vlan 10 exit ! interface ethernet 0/1 switchport access vlan 10 exit ! interface ethernet 0/2 switchport access vlan 20 exit ! configure vxlan vni mapping and static mac entries for the hosts there are macs of vm vm a1 mac： 00 10 94 05 00 01 vm a2 mac： 00 10 94 05 00 02 vm a3 mac： 00 10 94 05 00 03 vm b1 mac： 00 20 94 05 00 01 ! vlan 10 vni 10 exit ! vlan 20 vni 20 exit ! interface vxlan 0 vni 10 peer 172 16 2 170 vni 20 peer 172 16 2 170 exit ! mac address static 00 10 94 05 00 03 vlan 10 vxlan vni 10 peer 172 16 2 170 leaf2 configure the switch interface ip addresses and the loopback0 ip address interface ethernet 0/48 ip address 10 93 0 10/30 exit ! interface loopback 0 ip address 172 16 1 170/32 exit ! interface loopback 1 ip address 172 16 2 170/32 exit configure static routes to ensure reachability between loopback ip addresses ! ip route 172 16 1 179/32 10 93 0 9 ! ip route 172 16 2 179/32 10 93 0 9 configure the vtep ip address on the leaf devices interface vxlan 0 source 172 16 2 170 exit configure vlans and add vlan member ports vlan 10 exit ! vlan 20 exit ! interface ethernet 0/0 switchport access vlan 10 exit configure vxlan vni mapping and static mac entries for the hosts there are macs of vm vm a1的mac： 00 10 94 05 00 01 vm a2的mac： 00 10 94 05 00 02 vm a3的mac： 00 10 94 05 00 03 vm b1的mac： 00 20 94 05 00 01 ! vlan 10 vni 10 exit ! vlan 20 vni 20 exit ! interface vxlan 0 vni 10 peer 172 16 2 179 vni 20 peer 172 16 2 179 exit ! mac address static 00 10 94 05 00 01 vlan 10 vxlan vni 10 peer 172 16 2 179 mac address static 00 10 94 05 00 02 vlan 10 vxlan vni 10 peer 172 16 2 179 mac address static 00 20 94 05 00 01 vlan 20 vxlan vni 20 peer 172 16 2 179 example of layer 3 vxlan scenario configuration example of layer 3 vxlan scenario configuration networking requirements a company has a large number of vms deployed on two servers, where vm b1 and vm a2 have different services and need to achieve layer 3 interoperability it is assumed that layer 3 interworking is already in place between the leaf switches (in this example, two leafs are directly connected) topology configuration roadmap (1)check that the link status of the ports used on the switch is normal, and that all ports are stable in the up state (2)configure the switch interface ip addresses and the loopback0 ip address (3)configure static routes to ensure reachability between loopback ip addresses (4)configure the vtep ip address on the leaf devices (5)configure vlan and add vlan member ports, create a vlan layer 3 interface, and create a vrf instance (6)configure vxlan vni mapping and static route entries for the host procedure spine configure the switch interface ip addresses and the loopback0 ip address interface ethernet 0/0 ip address 10 93 0 1/30 exit ! interface ethernet 0/8 ip address 10 93 0 9/30 exit ! interface loopback 0 ip address 172 16 1 165/32 exit configure static routes to ensure reachability between loopback ip addresses ! ip route 172 16 1 179/32 10 93 0 2 ! ip route 172 16 2 179/32 10 93 0 2 ! ip route 172 16 1 170/32 10 93 0 10 ! ip route 172 16 2 170/32 10 93 0 10 leaf1 configure the switch interface ip addresses and the loopback0 ip address interface ethernet 0/48 ip address 10 93 0 2/30 exit ! interface loopback 0 ip address 172 16 1 179/32 exit ! interface loopback 1 ip address 172 16 2 179/32 exit configure static routes to ensure reachability between loopback ip addresses ! ip route 172 16 1 170/32 10 93 0 1 ! ip route 172 16 2 170/32 10 93 0 1 configure the vtep ip address on the leaf devices interface vxlan 0 source 172 16 2 179 exit configure vlan and add vlan member ports, create a vlan layer 3 interface, and create a vrf instance ! arp broadcast disable ! vlan 10 exit ! vlan 20 exit ! interface ethernet 0/0 switchport access vlan 10 exit ! interface ethernet 0/1 switchport access vlan 10 exit ! interface ethernet 0/2 switchport access vlan 20 exit ! vrf 10123 mac 00 00 00 01 23 00 exit ! interface vlan 10 mac address 00 00 00 10 00 00 vrf 10123 ip address 10 10 0 1/24 arp proxy mode default ! interface vlan 20 mac address 00 00 00 20 00 00 vrf 10123 ip address 10 20 0 1/24 arp proxy mode default ! configure vxlan vni mapping and static route entries for the host there are ip addresses for hosts vm a1 10 10 0 2 vm a2 10 10 0 3 vm a3 10 10 0 4 vm b1 10 20 0 2 vrf 10123 vni 10000 exit vrf ! interface vxlan 0 vni 10000 peer 172 16 2 170 rmac 00 00 00 01 23 01 ! vrf 10123 ip route 10 10 0 4/32 172 16 2 170 vxlan vni 10000 onlink leaf2 configure the switch interface ip addresses and the loopback0 ip address interface ethernet 0/48 ip address 10 93 0 10/30 exit ! interface loopback 0 ip address 172 16 1 170/32 exit ! interface loopback 1 ip address 172 16 2 170/32 exit configure static routes to ensure reachability between loopback ip addresses ! ip route 172 16 1 179/32 10 93 0 9 ! ip route 172 16 2 179/32 10 93 0 9 configure the vtep ip address on the leaf devices interface vxlan 0 source 172 16 2 170 exit configure vlan and add vlan member ports, create a vlan layer 3 interface, and create a vrf instance ! arp broadcast disable ! vlan 10 exit ! vlan 20 exit ! interface ethernet 0/0 switchport access vlan 10 exit ! vrf 10123 mac 00 00 00 01 23 01 exit ! interface vlan 10 mac address 00 00 00 10 00 00 vrf 10123 ip address 10 10 0 1/24 arp proxy mode default ! interface vlan 20 mac address 00 00 00 20 00 00 vrf 10123 ip address 10 20 0 1/24 arp proxy mode default ! configure vxlan vni mapping and static route entries for the host there are ip addresses for hosts vm a1 10 10 0 2 vm a2 10 10 0 3 vm a3 10 10 0 4 vm b1 10 20 0 2 vrf 10123 vni 10000 exit vrf ! interface vxlan 0 vni 10000 peer 172 16 2 179 rmac 00 00 00 01 23 00 ! vrf 10123 ip route 10 10 0 2/32 172 16 2 179 vxlan vni 10000 onlink ip route 10 10 0 3/32 172 16 2 179 vxlan vni 10000 onlink ip route 10 20 0 2/32 172 16 2 179 vxlan vni 10000 onlink