LSP ping and traceroute

Theoretical foundations and architecture

ICMP ping and traceroute verify only IP routing: a packet can reach its destination over the IP path even though label forwarding is broken, so they do not prove that an LSP is healthy (and P routers show up in traceroute only when propagate-ttl is enabled). MPLS Echo Request/Reply was standardized to test LSPs: LSP Ping and LSP Traceroute (RFC 8029, which replaced RFC 4379). 1. LSP Ping: the echo request is sent inside the LSP under test, with the same label stack customer traffic would use, as a UDP packet to port 3503; the destination address comes from 127.0.0.0/8 (127.0.0.1 by default), the IP TTL is 1 and the Router Alert option is set. Transit P routers forward the packet by label and never look at the IP header. The egress pops the label, hands the packet to the MPLS OAM process and that process checks the Target FEC Stack against its own bindings; if the FEC belongs to it, it returns Return Code 3, otherwise an error code. The reply travels over the normal IP path. 2. LSP Traceroute: requests with MPLS TTL 1, 2, 3 and so on; every node answers Return Code 8 with a Downstream Mapping TLV (next node, interface, MRU, outgoing labels) and the last node answers code 3. Both tests check one LSP for one FEC: they do not check the VPN label or the VRF table (that needs a VPN IPv4 prefix FEC or ping vrf).

Why LSPs need their own ping

An ordinary ICMP ping tests the IP path, while an MPLS network can be broken in a way that leaves the IGP and LDP looking healthy: the LDP session is Oper, the prefix is in the RIB, but a transit P router has no outgoing label in its LFIB or its hardware table was programmed incorrectly. The control plane says "all is well" while customer packets disappear. LSP ping and LSP traceroute (RFC 4379, replaced by RFC 8029) solve exactly this problem: they send a probe inside the LSP under test, with the same label stack that customer traffic would receive, and compare the result with the tables of the egress node. The test answers the question "does a packet carrying the label of this FEC reach its egress node, and does the egress agree that the FEC belongs to it".

MPLS Echo Request format

The echo request is a UDP packet with destination port 3503. The destination IP address is taken from 127.0.0.0/8 (127.0.0.1 by default), the IP TTL is 1, and the Router Alert option is set. The 127/8 address is a safeguard: if a label is lost and the packet leaves the LSP at an ordinary IP node, it is not routed any further (loopback addresses are not forwarded) but ends up in a local process or is dropped. This is also why transit nodes ignore the IP header: the decision is made on the label. The 127/8 range gives a practical trick as well: with ECMP the hash depends on the destination address, so iterating 127.0.0.1-127.0.0.N makes the echo requests take different equal-cost paths.

The fixed echo header is 32 bytes: Version (1), Global Flags (the V bit means "validate FEC stack"), Message Type (1 - Echo Request, 2 - Echo Reply), Reply Mode, Return Code, Return Subcode, Sender's Handle (copied into the reply), Sequence Number, TimeStamp Sent and TimeStamp Received (used to compute RTT). TLVs follow. Reply Mode: 1 - do not reply, 2 - reply with a UDP packet over IP (the default), 3 - reply with a UDP packet with Router Alert (processed hop by hop), 4 - reply over an application control channel (VCCV for pseudowires). The Echo Reply goes to the UDP port the request came from and follows ordinary IP routing, so a broken return path produces "no reply" even when the forward LSP is fine.

Target FEC Stack and the egress reply

The main TLV (type 1) is the Target FEC Stack: a list of FECs, one per label of the stack, starting from the top. Examples of sub-types: LDP IPv4 prefix (1), RSVP IPv4 LSP session (3), VPN IPv4 prefix (6), FEC 128 pseudowire (10), FEC 129 pseudowire (11), BGP labeled IPv4 prefix (12), Nil FEC (16). The egress receives the packet, pops the labels, sees the address 127.0.0.1 and hands the request to the OAM process. The process validates the FEC: for LDP IPv4 prefix 10.255.0.2/32 the egress must itself be the "end point" of that prefix (the address is on its loopback). If the check passes, the reply carries Return Code 3 ("Replying router is an egress for the FEC at stack-depth"). So the ping verifies not only delivery but also the match between label and FEC: a packet delivered by mistake to another node on a wrong label gets Return Code 4 or 10, not 3.

Return Code: how to read the reply

The return code is the main diagnostic result. RFC 8029 values: 0 - no return code; 1 - malformed request; 2 - TLVs not understood; 3 - the node is an egress for the FEC; 4 - the node has no mapping for this FEC at this stack depth; 5 - Downstream Mapping mismatch; 8 - the label is switched at this stack depth (a transit node, the normal traceroute answer); 9 - the label exists but there is no MPLS forwarding for it; 10 - the mapping for this FEC is not the given label; 11 - no entry for the label; 12 - protocol not enabled on the interface. Codes 4, 9, 10, 11 mean "the LSP is broken here". Note that code 10 is often wrongly described as "label switched but no MPLS forwarding" - that is code 9.

LSP traceroute: Downstream Mapping and TTL

Traceroute sends echo requests with MPLS TTL 1, 2, 3 and so on. At a transit node the label TTL expires, the packet goes to the OAM process, and the node answers with Return Code 8 together with a Downstream Mapping TLV (DSMAP, type 2; modern implementations use DDMAP, type 20, RFC 6424): the address of the next node, the interface, the MRU (the maximum size of a packet the downstream accepts) and the outgoing label stack. The ingress copies this information into the next request, so the next node can check that it receives the packet with exactly the label the previous node promised. A discrepancy yields Return Code 5 (Downstream Mapping Mismatch). The last node answers with Return Code 3. With PHP the penultimate node reports the outgoing label implicit-null (3): this is normal, not a fault.

One important property: LSP traceroute does not depend on propagate-ttl. In an ordinary IP traceroute no mpls ip propagate-ttl hides the P routers (the IP TTL is not copied into the label), whereas LSP traceroute sets the label TTL itself and sees every hop. So test with it first, not with IP traceroute.