MLAG: An Example of Complexity That Should Not Be

In Monday’s blog post, Derick explained the network engineering cycle, traversal in the referential space and the need to provide solutions that enable the network engineer to do his or her job better, more accurate, easier, simpler, more complete. We cannot automate or encapsulate a network engineer’s job and we should not try. We must however encapsulate and automate specific tasks and workflows.

Multichassis Link Aggregation (MLAG) is one of those features that should be so straightforward, but isn’t. MLAG allows a single device to be connected to 2 ethernet switches using a single Link Aggregation Group (LAG). The device is configured with a single LAG with ports that are connected to two switches, rather than a single switch. The two switches coordinate between each other and make it appear to the device as if they are single device.

This part is actually straightforward, it really comes down to using a single LACP system-id across both links from both switches. The end device is blind to the fact there are different switches at the end of each link.

MLAG is Complicated

The hardest part of MLAG is the packet forwarding coordination and behavior between the two switches. For instance, if the end device sends a broadcast packet onto one of the links of the LAG towards switch 1 of the MLAG, the solution must ensure that switch 2 does not send that same broadcast packet back to the end device. Because the two switches together create a LAG, the basic rule that a packet received on a LAG can never be send back out that same LAG must be observed. Sounds simple, but if that broadcast packet gets to switch 2, how does it know it came from the device at the other end of the LAG to begin with? Sounds trivial, just look at the source MAC address, but ethernet forwarding usually does not do anything with a source MAC address.

If a broadcast packet comes in the rest of the network and arrives at switch 1 and 2, who will forward this packet? Only one of them can, again to avoid duplication of packets. Similarly for multicast. In multicast rich environments, would you always pick the same switch to forward this onto the LAG, or would you share that responsibility. And if you share, how do you inform the rest of the network that it is this switch for this specific group that is responsible for distribution?

When one of then links in the LAG fails, what does that switch do with packets towards the end device? How does it get that packet to its MLAG peer so that it is delivered? If you have configured MLAG on any popular platform out there, you have now discovered the reason for the private interconnect between two MLAG peers.

They’re all the same, but different

Whether you call it MLAG, SMLT, VSS, vPC, vLAG or anything else, they all implement the same concept. And there is no question that MLAG is a rather complicated feature to implement and get all the data forwarding possibilities right. There are many failure scenarios to consider to ensure that traffic is not lost, looped, or duplicated.

There is however no reason to expose any of this complexity to you as the user. Why do I have to create a port group between two switches, then explain to each that they are MLAG peers on a special VLAN, then stick IP addresses on this VLAN, create an MLAG peering session, verify it is up and running, then create actual MLAG ports that are mapped to some unique identifier I need to track that needs to match up with the one used on the peer? I counted 24 individual configuration steps just to get the MLAG peering configured.

Why so Complicated?

This is a perfect example of exposing the gory details of the scaffolding required for something that as a user really should be as simple as “I want this port on this and that port on that switch to be part of the same LAG”. Because really that is what you want. Sure, for debugging purposes you may need to understand what is communicated between switches and who has taken responsibility for what, but why did you the user have to manually create all this plumbing between the two systems? That should be encapsulated by us, the vendor, so that you can focus on automating the actual provisioning of ports in an MLAG.

And I completely understand the implementation reasons for having 2 switches matched up to become MLAG peers and MLAGs can only exist between those two peers. But it’s one of those limitations imposed on you that should not be, there is absolutely no reason you could not have 3 MLAG peers. Or 4. Or any combination of 2 switches, different for each MLAG. For us there is no difference between a LAG and an MLAG. That is, there most certainly is a difference, but as far as provisioning one goes, they are identical. You simply configure a LAG. And you have a choice to add ports from other switches to that LAG. And that’s it. All that took was a desire to remove these constraints to make your job easier and more accurate.

Focusing on the user experience of the network takes time, it takes determination, it requires a completely different view on delivering capabilities. Read Derick’s blog post from this past Monday and you will get a sense of our beliefs and approach.