MC-LAG Configuration Guide

introduction introduction mc lag (multi chassis link aggregation group) is a mechanism for achieving link aggregation across devices it involves aggregating links between one device and two other devices, combining the benefits of regular link aggregation while providing device level redundancy mc lag introduces a form of horizontal virtualization, where two physical devices are virtualized as a single logical device this virtual "single device" is used to perform "one to one" link aggregation with the connected upstream or downstream devices the following information is synchronized between mc lag peers system information ensures synchronization of mac addresses for mc lag member ports, enabling the "system id" field in lacp (link aggregation control protocol) messages sent to the server to be the same, achieving cross device link aggregation mc lag member port configuration information records details like names of local and remote mc lag member ports for consistency checks mc lag member port status information keeps track of the status of local and remote mc lag member ports to ensure isolation between the peer link and mc lag member ports in fault free scenarios, and to release isolation between the peer link and the same named port on the remote side in case of a failure in the remote mc lag member port arp (address resolution protocol) information arp entries related to mc lag member ports are synchronized between mc lag peers fdb (forwarding database) information fdb entries associated with mc lag member ports are synchronized between mc lag peers explanation of principles explanation of principles as shown in the diagram, on two separate devices, a cross device link aggregation group is established and connected to ordinary link aggregation ports on the user side once the mc lag is established, entries can be synchronized between devices the direct link between device1 and device2 serves as the peer link interface, used for protocol messages and forwarding traffic during failures the links connecting the devices to users, acting as member interfaces of the mc lag, handle incoming traffic, distribute loads, and provide backup protection for the links in the context of mc lag, sonic employs a lightweight inter chassis communication protocol (iccp) on the control plane, conducting only a limited amount of consistency checks and information synchronization while ensuring the functionality iccp, defined in rfc7275, serves as the standard protocol for inter chassis communication in the mc lag implementation, the iccp protocol establishes connections between mc lag peer devices using tcp port 8888 this streamlined iccp protocol primarily focuses on configuring consistency checks and synchronizing arp and mac table entries between the two mc lag peer devices, the local ip and peer ip addresses are used as the source and destination addresses for establishing the tcp connection to form the iccp neighbor relationship once the iccp connection is successfully established, the system sends heartbeat messages to the peer every 1 second if no heartbeat messages are received for 15 consecutive intervals, the connection is deemed timed out, leading to the termination of the iccp connection mc lag loop prevention mechanism mc lag loop prevention mechanism mc lag itself comes with a loop prevention mechanism, as shown in the diagram when a device receives a broadcast packet from the mc lag side, this broadcast packet is transmitted through the peer link link to the opposite device due to the flow isolation between the peer link link and the mc lag member interfaces, any traffic coming in from the peer link port will not be forwarded out through the mc lag member interfaces this effectively prevents the formation of loops mc lag fault handling mc lag fault handling as illustrated in the diagram, in the event of a link failure on the mc lag side, the device will proactively update the interface information for the corresponding terminal's mac table entries and arp table entries to reflect the peer link interface consequently, downstream traffic will be forwarded through the peer link interface to the other device, ensuring seamless fault handling for the user once the fault is rectified, and the mc lag interface is back up, traffic will resume its normal forwarding behavior mc lag configuration mc lag configuration operation command description enter the system configuration view configure terminal enter the link aggregation configuration view and create a link aggregation group interface link aggregation lag id only supported single domain the range of 1 to 4095 create a mc lag domain mclag domain domain id delete a mc lag domain no mclag domain domain id configure peerlink interface peer link {ethernet|link aggregation} name configure the peer ip address for the mc lag control link peer address a b c d set the local ip address for the mc lag control link local address a b c d add a member lag port to the mc lag group member lag lag id remove a member lag port from the mc lag group no member lag lag id enter the configuration view for a vlan interface interface vlan vlan id modify the mac address of a vlan interface mac address hh\ hh\ hh\ hh\ hh\ hh configure dad to detect errdown delay time dad detection delay time time delay time, range 0 512 configure dad detection and recovery delay time dad recovery delay {mlag|non mlag} time time delay time, range 0 512 configure the vrf where the dad link is located dad vrf {default|string} string\ specify vrf name configure the management port for dad detection dad mgmt interface interface name interface name：the port name, such as ethernet1, can be configured to prevent the backup device port errdown from simultaneously shutting down the ip port of the configuration management port, resulting in the device being unable to connect configuration example configuration example network requirements a server is dual homed to an ethernet network through m lag the customer requires high service reliability link aggregation between the server and devices only achieves link level reliability, and a fault on a device may cause service interruption m lag can be configured when devices work properly, links load balance traffic and a fault of any device does not affect services high service reliability is therefore ensured procedure 1 create an aggregation port and add the physical interfaces connected to server and device c to the aggregation port group \# device a sonic(config)# interface link aggregation 1 sonic(config)# interface ethernet 1 sonic(config if 1)# link aggregation group 1 \# device b sonic(config)# interface link aggregation 1 sonic(config)# interface ethernet 1 sonic(config if 1)# link aggregation group 1 2 create an aggregation port and add the interconnection port between device a and device b to the aggregation group as the peerlink interface \# device a sonic(config)# interface link aggregation 2 sonic(config)# interface ethernet 2 sonic(config if 2)# link aggregation group 2 sonic(config)# interface ethernet 3 sonic(config if 3)# link aggregation group 2 \# device b sonic(config)# interface link aggregation 2 sonic(config)# interface ethernet 2 sonic(config if 2)# link aggregation group 2 sonic(config)# interface ethernet 3 sonic(config if 3)# link aggregation group 2 3 create vlan100 to add all aggregation ports to vlan \# device a sonic(config)# vlan 100 sonic(config)# interface link aggregation 1 sonic(config lagif 1)# switchport access vlan 100 sonic(config)# interface link aggregation 2 sonic(config lagif 2)# switchport access vlan 100 sonic(config)# interface vlan 100 sonic(config vlanif 100)# ip address 101 1 1 1/24 sonic(config vlanif 100)# mac address 00 00 00 11 22 10 \# device b sonic(config)# vlan 100 sonic(config)# interface link aggregation 1 sonic(config lagif 1)# switchport access vlan 100 sonic(config)# interface link aggregation 2 sonic(config lagif 2)# switchport access vlan 100 sonic(config)# interface vlan 100 sonic(config vlanif 100)# ip address 101 1 1 1/24 sonic(config vlanif 100)# mac address 00 00 00 11 22 10 4 create vlan200 and configure the ip address to add the peerlink interface to the vlan \# device a sonic(config)# vlan 200 sonic(config)# interface link aggregation 2 sonic(config lagif 2)# switchport trunk vlan 200 sonic(config lagif 2)# switchport trunk vlan 100 sonic(config)# interface vlan 200 sonic(config vlanif 200)# ip address 10 1 1 24/24 \# device b sonic(config)# vlan 200 sonic(config)# interface link aggregation 2 sonic(config lagif 3)# switchport trunk vlan 200 sonic(config lagif 2)# switchport trunk vlan 100 sonic(config)# interface vlan 200 sonic(config vlanif 200)# ip address 10 1 1 48/24 5 create an mc lag and specify the member interface, peerlink interface and peerlink port ip address \# device a sonic(config)# mclag domain 1 sonic(mclag domain)# local address 10 1 1 24 sonic(mclag domain)# peer address 10 1 1 48 sonic(mclag domain)# peer link lag 2 sonic(mclag domain)# member lag 1 \# device b sonic(config)# mclag domain 1 sonic(mclag domain)# local address 10 1 1 48 sonic(mclag domain)# peer address 10 1 1 24 sonic(mclag domain)# peer link lag 2 sonic(mclag domain)# member lag 1 verify configuration 1 check mc lag status sonic# show mclag state the mclag's keepalive is ok mclag info sync is completed domain id 1 mclag session channel primary channel vrf name default consistency check action idle local ip 10 1 1 48 peer ip 10 1 1 24 peer link interface portchannel0003 keepalive time 1 sesssion timeout 15 peer link mac 00 11 24 1a 1b 61 admin role none role standby mclag interface portchannel0001 loglevel debug 2 the server can ping the gateway and check the mac address of the device \# use the server ping gateway address testuser\@78 $ping 101 1 1 1 ping 101 1 1 1 (101 1 1 1) 56(84) bytes of data 64 bytes from 101 1 1 1 icmp seq=1 ttl=64 time=3 38 ms 64 bytes from 101 1 1 1 icmp seq=2 ttl=64 time=1 33 ms 64 bytes from 101 1 1 1 icmp seq=3 ttl=64 time=1 32 ms 64 bytes from 101 1 1 1 icmp seq=4 ttl=64 time=1 29 ms ^c \ 101 1 1 1 ping statistics 4 packets transmitted, 4 received, 0% packet loss, time 3005ms rtt min/avg/max/mdev = 1 291/1 835/3 389/0 897 ms \# display the mac address on device b device sonic# show mac address no vlan macaddress port type \ 1 101 00 51 82 11 0f 78 portchannel0001 dynamic 3 cut off the link between deviceb and devicec, use the server ping gateway again, and still be able to ping, and check the mac address of the device at this time \# use the server ping gateway address testuser\@78 $ping 101 1 1 1 ping 101 1 1 1 (101 1 1 1) 56(84) bytes of data 64 bytes from 101 1 1 1 icmp seq=1 ttl=64 time=3 38 ms 64 bytes from 101 1 1 1 icmp seq=2 ttl=64 time=1 33 ms 64 bytes from 101 1 1 1 icmp seq=3 ttl=64 time=1 32 ms 64 bytes from 101 1 1 1 icmp seq=4 ttl=64 time=1 29 ms ^c \ 101 1 1 1 ping statistics 4 packets transmitted, 4 received, 0% packet loss, time 3005ms rtt min/avg/max/mdev = 1 291/1 835/3 389/0 897 ms \# display the mac address on your device b device sonic# show mac address no vlan macaddress port type \ 1 101 00 51 82 11 0f 78 portchannel0002 dynamic example of mclag network migration scenario example of mclag network migration scenario network requirements the server adopts a dual homing access method to connect to the standard ethernet network, utilizing cross device link aggregation technology to ensure device level service high availability during normal operation, the links perform load sharing, and any single device failure will not impact services based on the above network configuration, when the enterprise needs to migrate one of the mclag devices, it is necessary to configure and save the startup delay time for the peerlink port this prevents potential loops from occurring when the device rejoins the mclag network before the mclag is fully established, thereby ensuring network stability procedure \# device a sonic(config)# interface link aggregation 1 sonic(config)# interface ethernet 1 sonic(config if 1)# link aggregation group 1 sonic(config)# interface link aggregation 2 sonic(config)# interface ethernet 2 sonic(config if 2)# link aggregation group 2 sonic(config)# interface ethernet 3 sonic(config if 3)# link aggregation group 2 sonic(config)# vlan 100 sonic(config)# interface link aggregation 1 sonic(config lagif 1)# switchport access vlan 100 sonic(config)# interface link aggregation 2 sonic(config lagif 2)# switchport access vlan 100 sonic(config)# interface vlan 100 sonic(config vlanif 100)# ip address 101 1 1 1/24 sonic(config vlanif 100)# mac address 00 00 00 11 22 10 sonic(config)# vlan 200 sonic(config)# interface link aggregation 2 sonic(config lagif 2)# switchport trunk vlan 200 sonic(config lagif 2)# switchport trunk vlan 100 sonic(config)# interface vlan 200 sonic(config vlanif 200)# ip address 10 1 1 24/24 sonic(config)# mclag domain 1 sonic(mclag domain)# local address 10 1 1 24 sonic(mclag domain)# peer address 10 1 1 48 sonic(mclag domain)# peer link lag 2 sonic(mclag domain)# member lag 1 sonic(config)# interface ethernet 2 sonic(config if 2)# startup delay 120 sonic(config if 2)# interface ethernet 3 sonic(config if 3)# startup delay 120 sonic# write save running config existing files will be overwritten, continue? \[y/n] y running command /usr/local/bin/sonic cfggen d print data > /etc/sonic/config db json \# device b sonic(config)# interface link aggregation 1 sonic(config)# interface ethernet 1 sonic(config if 1)# link aggregation group 1 sonic(config)# interface link aggregation 2 sonic(config)# interface ethernet 2 sonic(config if 2)# link aggregation group 2 sonic(config)# interface ethernet 3 sonic(config if 3)# link aggregation group 2 sonic(config)# vlan 100 sonic(config)# interface link aggregation 1 sonic(config lagif 1)# switchport access vlan 100 sonic(config)# interface link aggregation 2 sonic(config lagif 2)# switchport access vlan 100 sonic(config)# interface vlan 100 sonic(config vlanif 100)# ip address 101 1 1 1/24 sonic(config vlanif 100)# mac address 00 00 00 11 22 10 sonic(config)# vlan 200 sonic(config)# interface link aggregation 2 sonic(config lagif 2)# switchport trunk vlan 200 sonic(config lagif 2)# switchport trunk vlan 100 sonic(config)# interface vlan 200 sonic(config vlanif 200)# ip address 10 1 1 48/24 sonic(config)# mclag domain 1 sonic(mclag domain)# local address 10 1 1 48 sonic(mclag domain)# peer address 10 1 1 24 sonic(mclag domain)# peer link lag 2 sonic(mclag domain)# member lag 1 sonic(config)# interface ethernet 2 sonic(config if 2)# startup delay 120 sonic(config if 2)# interface ethernet 3 sonic(config if 3)# startup delay 120 sonic# write save running config existing files will be overwritten, continue? \[y/n] y running command /usr/local/bin/sonic cfggen d print data > /etc/sonic/config db json this configuration ensures that when devices in the network undergo migration or encounter failures, mclag can still be properly established after reboot, thereby preventing loop storms