VXLAN Mastery · Lesson 01 / 08

Why does VXLAN exist?

Start from zero. In nine short chapters you will go from “what is a VLAN?” to following a real VXLAN packet hop by hop, and spotting the mistakes that break overlays in production. Every idea has a picture, a plain-English explanation and something to click.

🌱 No prior VXLAN needed⏱ ~35 minutes🖱 9 interactive demos🧠 Quiz and checklist
0 of 9 chapters completeProgress is saved in this browser
1Starter⏱ 4 min

Networking in 5 minutes

VXLAN builds on three old ideas. If you know them already, skim this and move on. If not, this is all you need.

📦 Ethernet frame

Data on a LAN travels in frames. A frame has a destination MAC, a source MAC and the data. A MAC address is a hardware name like 00:AA:AA:AA:AA:01. Think of it as the name on an envelope.

🔀 Switch

A switch learns which MAC lives behind which port by watching the source of every frame. Next time, it sends frames for that MAC out of just that port. It stores this in a MAC table.

🏷 VLAN

A VLAN splits one physical switch into separate virtual LANs. Hosts in VLAN 10 cannot hear hosts in VLAN 20, even on the same switch. A VLAN is also called a broadcast domain.

Try it: who hears a broadcast?

Click any host

A broadcast is a frame addressed to everyone (FF:FF:FF:FF:FF:FF). Click a host to make it shout, and see who is allowed to hear it.

Click a host above. Blue VLAN 10 hosts and amber VLAN 20 hosts are on the same switch.
Plain English: a VLAN is a “private group chat” on a switch. Only members of the group see the messages.
2Starter⏱ 4 min

Why VLAN-only designs can become difficult at scale

VLANs are still extremely useful. The challenge appears when many separate tenant segments must be extended across racks while the physical network needs flexible Layer 3 routing.

1 · Too few IDs

4,094

A VLAN ID is only 12 bits. After the reserved values 0 and 4095, only 4,094 usable VLANs remain. A very large environment may need more isolated segments than that single VLAN ID space provides.

2 · Big flat Layer 2 is fragile

To stretch a VLAN across many racks, classic networks used Layer 2 everywhere, and Spanning Tree may need to block redundant links to prevent loops in spanning-tree-based designs. Poorly controlled broadcasts or loops can affect a large domain.

3 · Routed networks are better

Layer 3 (IP) fabrics can use multiple equal-cost paths (ECMP), support scalable routing and offer useful failure isolation. Data centers prefer to build the physical network as routed IP.

4 · …but apps still want Layer 2

Virtual machines want to move between racks and keep their IP and MAC. Some workloads need to remain in the same Layer 2 segment across racks, although many modern applications can use Layer 3 mobility instead.

The big question: how do we keep a routed physical network and still give servers a Layer 2 segment that stretches across it? VXLAN is the answer.

Try it: how many segments do you need?

Move the slider
VLAN IDs used: VNI values used:

This compares number spaces only. It is not a hardware limit. Real devices support far fewer segments than 16 million.

Plain English: imagine every office floor numbered 1–4094. A VLAN number is a local label for a Layer 2 segment. VXLAN carries a separately numbered overlay segment between buildings, while the roads in between are routed IP links.
3Starter⏱ 4 min

The big idea: put the frame in an envelope

VXLAN does not change the frame. It wraps it inside a normal IP packet, sends it across the routed network, and unwraps it at the other side. This is called encapsulation.

The postal analogy

Step 1 of 4
Outer envelope: addressed to Branch B
✉ Original letterTo: Bob (desk 12) · From: Alice (desk 7)

Post officeVXLAN network
Original letter (Alice → Bob)Ethernet frame from Host A to Host B
Outer envelope addressed to the branch officeOuter IP/UDP/VXLAN headers addressed to the remote switch (VTEP)
Post office reads only the envelopeUnderlay routers read only the outer IP address
Branch office opens the envelopeRemote VTEP decapsulates and delivers the original frame
Alice and Bob never see the envelopeHosts never know VXLAN is used
Common mistake: thinking the host sends VXLAN. It does not. Only the leaf switch adds and removes the wrapper. Hosts send ordinary Ethernet.
4Starter⏱ 3 min

Four words you must know

Everything in VXLAN uses these four terms. Learn them once and the rest is easy.

🛣 Underlay

The physical routed IP network that carries the wrapped packets. It only needs to deliver IP packets between switches, and knows nothing about tenants or VLANs.

🌐 Overlay

The virtual Layer 2 network that hosts believe they are on. It exists only as a logical layer on top of the underlay.

🚪 VTEP

VXLAN Tunnel EndPoint. The device that wraps and unwraps frames, usually a leaf switch. It owns an IP address (often on a loopback) used as the tunnel endpoint.

🔢 VNI

VXLAN Network Identifier. A 24-bit number in the VXLAN header naming the overlay segment, like a VLAN ID with 16.7 million possible values.

Example used in this lab: VLAN 100 on the access port maps to VNI 10100 in the tunnel. The numbers do not have to match: VLAN 100 is a local access-side identifier, while VNI 10100 identifies the overlay segment between VTEPs.
Common mistake: a VNI is not a bigger 802.1Q tag. It lives in the VXLAN header, not on the host-facing Ethernet frame, and only VTEPs use it.
5Intermediate⏱ 5 min

Build a VXLAN packet, layer by layer

Start with the original frame and add the four wrappers a VTEP adds. Click Add next layer, then click any layer to see its fields.

Overhead added: 0 bytes

Outer UDP destination port
4789
VNI field size
24 bits
Total overhead
50 bytes
Why UDP? Routers spread traffic across equal paths by hashing IP/UDP fields. The VTEP sets the outer UDP source port from a hash of the inner flow, so different conversations take different paths while each one stays in order.
Pro detail: where does 50 come from?

On the wire, a VXLAN-encapsulated Ethernet frame adds 50 bytes (outer Ethernet 14 + outer IPv4 20 + UDP 8 + VXLAN 8), excluding FCS and any outer VLAN tag. When checking the underlay IP MTU for an inner IP packet, count 14 bytes of inner Ethernet + 20 outer IPv4 + 8 UDP + 8 VXLAN = 50 bytes; the outer Ethernet header is not included in IP MTU. IPv6 outer IP headers add 20 bytes to either calculation.

6Intermediate⏱ 6 min

Follow a frame across the fabric

Host A sends a frame to Host B. Watch the packet travel step by step. Notice what changes at every hop, and what never changes. Follow the moving green frame and watch the outer headers appear at Leaf 1. Assumption: Leaf 1 already knows where Host B lives (chapter 7 explains how it learns).

Known unicast · VNI 10100

VLAN 100 ↔ VNI 10100
Routed IP underlay (Layer 3) logical VXLAN tunnel (VNI 10100) Host A00:AA…:01 Leaf 1VTEP 10.0.0.1 Spine 1routes by IP Spine 2(other path) Leaf 2VTEP 10.0.0.2 Host B00:BB…:02 outer: Eth+IP+UDP+VXLANA → B frame
Watch for three things: (1) inner MAC addresses, (2) outer VTEP IP addresses, and (3) outer Ethernet MAC addresses at each hop. Use Next step to reveal them one by one.

The packet right now

Amber = changed at this step · Teal = what the current device reads to decide where to send it.

Key forwarding decision: the spine routes on outer destination IP 10.0.0.2 only. It never needs Host B’s MAC for ordinary IP forwarding. The outer Ethernet MAC addresses are rewritten at each routed hop; the inner host frame is preserved in this known-unicast example.
7Intermediate⏱ 5 min

But how did Leaf 1 learn where Host B lives?

In chapter 6 Leaf 1 already knew “Host B is behind VTEP 10.0.0.2”. In real life it starts with an empty table. With plain VXLAN, switches flood and learn, just like an ordinary switch.

Flood and learn · VNI 10100

New word: BUM traffic = Broadcast, Unknown unicast and Multicast. BUM traffic generally needs controlled replication to participating VTEPs in that VNI, using ingress replication or multicast in flood-and-learn designs. Replication consumes bandwidth and must be scoped correctly.
Looking ahead: EVPN uses MP-BGP to advertise MAC/IP reachability between VTEPs. It can reduce unknown-unicast flooding and, with appropriate ARP/ND suppression, some broadcast traffic. BUM handling is still required. That is the later lessons.
8Mastery⏱ 4 min

The MTU trap: the number-one real-world VXLAN bug

The VXLAN wrapper makes packets larger. If the resulting outer IP packet exceeds the path MTU, some packets may be dropped or fragmented, depending on platform settings and fragmentation behavior. This can make small tests succeed while larger traffic fails. Try it.

Will it fit?

Change the numbers

Rule of thumb: For this IPv4 example, outer IP packet length = inner IP packet length + 14 (inner Ethernet) + 8 (VXLAN) + 8 (UDP) + 20 (outer IPv4). Thus a 1,500-byte inner IP packet needs an underlay IP MTU of at least 1,550. The physical Ethernet frame also has a 14-byte outer Ethernet header (and usually FCS), which an IP MTU does not count. Check the vendor-specific L2 MTU definition and every underlay link.
Pro detail: why can’t the VTEP fragment?

Fragmentation behavior is platform- and configuration-dependent. Outer IPv4 DF handling and Path MTU Discovery determine whether oversized encapsulated packets are dropped or fragmented. Do not assume every oversized packet silently disappears; check underlay MTU, ICMP Packet Too Big / fragmentation-needed messages, tunnel counters, and platform behavior.

9Mastery⏱ 3 min

Optional: connect the lesson to real switch commands

This is an advanced preview. Beginners can skip it and take the quiz. Actual vendor syntax depends on NOS release, platform, and whether flood-and-learn or EVPN is used. The three concepts from this lesson appear in configuration as: a loopback IP (the VTEP address), a VLAN-to-VNI mapping, and a tunnel interface toward other VTEPs. Syntax differs by vendor, but the ideas are the same.

    Same idea, different words

    ConceptCisco NX-OSJuniper JunosArista EOSCumulus Linux
    VTEP addressinterface loopback0lo0.0 + vtep-source-interfaceLoopback0lo address / nve vxlan source
    VLAN → VNIvn-segmentvxlan vnivxlan vlan … vnivlan … vni
    Tunnel interfaceinterface nve1(implicit)interface Vxlan1nve vxlan
    Remote VTEP (static)peer-ipEVPN (later)vxlan flood vtephead-end-replication
    Illustrative only: this is a minimal static-VXLAN sketch to show the building blocks. Production designs add an EVPN control plane, routing for the underlay and many more settings, covered in the later lessons.
    ✓Check yourself⏱ 5 min

    Test what you learned

    Eight questions. Every answer comes with an explanation. This quiz runs in your browser.

    Mastery checklist

    Tick each one when you can explain it in your own words.

    Cheat sheet

    • VXLAN = Ethernet frame wrapped in IP/UDP
    • VTEP wraps/unwraps; hosts are unaware
    • VNI = 24-bit segment ID (16,777,216 values)
    • UDP destination port 4789
    • Overhead 50 bytes (14+20+8+8)
    • Underlay routes on the outer IP only

    What VXLAN alone does not do

    • Advertise remote MAC reachability through a control plane (EVPN can provide this)
    • Remove BUM flooding by itself
    • Route between tenants (needs a gateway design)
    • Fix MTU problems automatically
    • Build the underlay routing for you
    Next lesson: VTEP discovery and BUM replication in detail, then EVPN MP-BGP route types.
    Source

    Learn from the standard

    RFC 7348 — Virtual eXtensible Local Area Network (VXLAN). This lab models a simple IPv4 underlay with the standard header layout.