Elevate Your Data Center with EVPN Multihoming on Enterprise SONiC: A Fun, Comprehensive Guide

Table of Contents

1. Introduction
2. Traditional Data Center Approaches
   1. MC-LAG: The Old Workhorse
3. EVPN Multihoming 101: The Basics
4. Under the Hood: Ethernet Segment Identifiers and Route Types
5. Comparing EVPN MH with MC-LAG and Stacking
6. Diagram: High-Level EVPN MH Topology
7. Example Configuration Snippet
8. Extra Features: Uplink Tracking & Startup Delay
9. FAQ: Common Questions
10. Where to Test EVPN MH for Free
11. Wrapping Up: Why EVPN MH in Enterprise SONiC Rocks

1. Introduction

Welcome to a fun, comprehensive tour of EVPN Multihoming (MH) in Enterprise SONiC! If you’re tired of half your links being blocked by spanning tree, or you’re seeking a more modern, open-source path for data center networking, then EVPN MH might just make your day.

We’ll explore how EVPN MH works, how it compares to older approaches like MC-LAG, and where you can try it out for free—all while keeping things clear and entertaining. Let’s jump in!

2. The traditional approach: MC-LAG – The old workhorse

MC-LAG (Multi-Chassis Link Aggregation Group) has been a staple for providing redundancy to servers or hosts. It typically involves two Top-of-Rack (ToR) switches sharing a peer link, letting them appear as one logical device to the server. In the VXLAN fabric, both of the switches also appear as one logical VTEP. Sounds great—until the peer link fails, or you realize you can’t easily attach your server to more than two switches.

Figure 1 – MC-LAG vs. EVPN.

This diagram contrasts a classic MC-LAG setup (left) with EVPN MH (right). Notice how MC-LAG switches share a peer link, while EVPN MH uses independent leaf switches coordinated by the BGP EVPN control plane—no specialized peer link.

3. EVPN Multihoming 101: The Basics

EVPN stands for Ethernet VPN, an overlay control plane technology (often using VXLAN as the data plane) for stretching Layer 2 networks, and even complete layer 3 neworks, across multiple racks or data centers. EVPN Multihoming (MH) is an extension that allows a single server (or device) to connect to multiple leaf switches without needing a peer link between those switches.

• No Peer Link: Each leaf is autonomous and uses BGP EVPN to coordinate.
• Connect Up to Four Switches: More robust redundancy than MC-LAG’s limit of two.
• All Links Active: Full bandwidth usage; no need for spanning tree.

4. Under the Hood: Ethernet Segment Identifiers and Route Types

Two special EVPN route types make Multihoming possible:

1. Type-1 (Ethernet Auto-Discovery): Signals multi-homed segments across the fabric. They are used both for fast failover in case a leaf switch loses the connection to the server and to advertise all available links to facilitate load sharing of traffic.
2. Type-4 (Ethernet Segment Route): Advertises the Ethernet Segment Identifier (ESI), a unique 10-byte value that identifies each multi-homed connection. These are used to elect a Designated Forwarder.

In Enterprise SONiC, you can configure an ESI manually or generate it from system MAC addresses. The Designated Forwarder (DF) that is chosen among all leaf switches connecting to that ESI with the help of the Type 4 routes is responsible to send BUM traffic (broadcasts, unknown unicasts and multicasts) to the downstream server, avoiding duplicates and loops.

5. Comparing EVPN MH with MC-LAG and Stacking

• MC-LAG: Two switches, special peer link, complex configuration
• EVPN MH: Up to four leaf switches, no peer links, open-standard BGP EVPN, easy configuration

(If you want to reinforce the differences visually, you can point readers back to Figures 1 and 2.)

6. Diagram: High-Level EVPN MH Topology

Before we show you the configuration, here’s a big-picture look at an EVPN MH deployment.

Figure 3 – High-Level EVPN MH Topology.

This example topology shows how two servers (H2, H3) can each connect to multiple leaf switches. Every leaf runs VXLAN, and the entire fabric communicates through BGP EVPN — no single switch or proprietary link required.

7. Example Configuration Snippet

Here’s a simplified EVPN MH setup on Leaf-1 in Enterprise SONiC:

(Leaf-2 would look nearly identical, except with its own IP addresses and the same ESI parameters.)

8. Extra Features: Uplink Tracking & Startup Delay

8.1 Uplink Tracking

If your leaf loses all uplinks (to spines or the upstream core), you don’t want it still pulling traffic from servers. Uplink Tracking disables the server-facing PortChannel in that scenario:

8.2 Startup Delay

When a leaf reboots, BGP adjacency might take time. startup-delay 300 in your evpn esi-multihoming config keeps the PortChannel to the server down until the leaf is ready, preventing blackholing.

9. FAQ: Common Questions

1. Is EVPN MH more complex than MC-LAG?
   • It’s different, but once you understand BGP EVPN fundamentals, it’s quite straightforward. No special peer link or stacking cables—just standard BGP-based control – and it is much easier to configure
2. Do I need spanning tree for my multi-homed links?
   • No. EVPN’s split-horizon and DF election avoid loops. Spanning tree BPDUs will not even be allowed on the downstream links.
3. How many switches can I connect to one server?
   • Up to four, as long as your server supports enough NICs or ports.

10. Where to Test EVPN MH for Free

You don’t need a physical lab. Here are two excellent (and free) options:

1. Enterprise SONiC Linux GNS3 VM
   • Download, import into VMware ESXi or Proxmox, and launch GNS3.
   • Spin up multiple virtual leaf switches and practice building EVPN MH topologies.
2. Hands-On Labs at route2open.com
   • Book a session in a ready-to-go environment.
   • Follow guided tasks for MC-LAG, VRRP, VLAN configuration, and more—perfect if you prefer a structured approach.

11. Wrapping Up: Why EVPN MH in Enterprise SONiC Rocks

EVPN Multihoming offers a fresh, flexible, and open-standard alternative to legacy MC-LAG. By removing the requirement for specialized peer links or stack cables, it lets you fully utilize your links without fear of looping, bridging, or half-blocked bandwidth.

• Scalable: Connect up to four leaf switches to a single server.
• Simplicity: Eliminate special hardware links, rely on BGP EVPN routes.
• Open Networking: Built on RFC-based standards for vendor neutrality.
• Easy to Learn & Test: Use a free GNS3 VM image or remote labs—no big upfront costs.

Give EVPN MH a spin and watch your data center become more efficient and agile than ever before!

Krzysztof Prałat

Infuses our technical support with the narrative depth of a book lover and the strategic acumen of a seasoned gamer. His passion for photography, capturing nature’s tranquil moments, mirrors his meticulous approach to resolving technical complexities.