TTL propagation and PE hardening

Theoretical foundations and architecture

Lab: comprehensive audit and hardening of PE edge routers and the MPLS core following NSA and Cisco SAFE practices. 1. MPLS TTL propagation control (RFC 3443): in uniform mode (the Cisco default) the IP TTL is copied into the label and a customer traceroute sees every transit P router. no mpls ip propagate-ttl enables pipe mode: the label TTL is initialized to 255, the IP TTL is not decremented in the core and the customer sees the core as a single hop between PEs; the forwarded option hides the core only for customer transit traffic and leaves the engineer a traceroute from the PE itself. This merely hampers reconnaissance and is not a protection of the P routers: real protection means the customer has no route to infrastructure addresses (VRF) and CoPP and ACLs do not answer it. 2. Management port filtering: disable Telnet and HTTP, SSHv2 with keys and timeouts, VTY ACLs, management through an OOB interface or a dedicated MGMT VRF. 3. BGP session protection: TCP MD5 or TCP-AO (RFC 5925) authentication and GTSM (Generalized TTL Security Mechanism, RFC 5082): both sides send TTL 255 and for a directly connected peer ttl-security hops 1 accepts only TTL 254-255. GTSM protects against attacks from afar but not against an attacker on the same segment, it is incompatible with ebgp-multihop and must be enabled on both sides at once. uRPF (RFC 3704) on customer interfaces complements them.

TTL in MPLS: uniform and pipe (RFC 3443)

When a label is imposed, the ingress PE chooses how to tie the IP header TTL to the label TTL. In uniform mode the IP TTL is copied into the label, each LSR decrements the label TTL, and on exit the value is copied back (taking the minimum into account): a customer traceroute sees every P router. In pipe (and short pipe) mode the label TTL is initialized to a fixed value (255), the IP TTL inside the packet is not decremented in the core, so the customer sees the core as a single hop between the PEs. Cisco works in uniform mode by default; no mpls ip propagate-ttl enables pipe for everything, forwarded limits the mode to transit customer traffic, and local keeps propagation for traffic originated by the PE itself. The practical choice is forwarded: the customer does not see the core, while an engineer on the PE can still trace through it. Junos uses no-propagate-ttl at the protocols mpls level; in VRP the mode is controlled by the ttl propagate commands in the mpls view.

Side effects: ICMP errors generated inside the core (for example, for PMTUD) cannot be sent by a P router directly to the customer, so they are tunnelled along the LSP to the egress PE (RFC 3032, section 2.3.2), and core diagnostics move to ping mpls/traceroute mpls. Hiding the P addresses is only a hindrance to reconnaissance, not a defence: the real defence is that the customer has no route to infrastructure addresses (the VRF), and that CoPP and ACLs do not answer it.

GTSM (RFC 5082): TTL-based protection of eBGP

A packet generator that is not directly connected cannot send a packet with TTL 255 so that it arrives with TTL 255: each router decrements the TTL. GTSM uses this: both neighbours send BGP segments with TTL 255, and the receiver accepts only packets with a TTL at or above a threshold. For a directly connected peer the correct value is 255; Cisco ttl-security hops N accepts a TTL from 255 - N, so hops 1 lets 254-255 through (a margin of one transit router). The mechanism is incompatible with ebgp-multihop and protects only against attacks from afar: an attacker in the same segment bypasses it, so GTSM is combined with authentication. GTSM has to be enabled on both sides: the side without GTSM sends TTL 1, the side with GTSM drops such segments, and the session does not come up.

Session authentication: MD5 and TCP-AO

The TCP MD5 option (RFC 2385) is an MD5 digest over the segment and a shared secret: a weak hash function, no key change without a session reset, no algorithm choice. TCP-AO (RFC 5925; the algorithms HMAC-SHA-1-96 and AES-128-CMAC-96 per RFC 5926) uses option 29, key identifiers (KeyID and RNextKeyID) for smooth rollover, traffic keys derived from the master key for each connection, and protects the TCP options. Neither mechanism encrypts data. Support depends on the platform: IOS-XE and Junos have TCP-AO in modern releases, RouterOS only supports MD5, and Huawei offers its own variants (including a keychain). The secret must be random and long, and identical on both sides; on a mismatch segments are silently dropped, and on Cisco %TCP-6-BADAUTH appears in the log (which is also a useful sign of guessing).

uRPF (RFC 3704, BCP 84)

Unicast Reverse Path Forwarding checks that the source of a packet is reachable through the interface on which the packet arrived. Strict mode (Cisco ip verify unicast source reachable-via rx, Junos rpf-check, Huawei ip urpf strict, RouterOS rp-filter=strict) suits a single PE-CE link of a customer. Loose mode checks only that a route exists and suits peering and multihoming; it does not protect against spoofing of addresses that exist in the table. Asymmetric traffic breaks strict mode, so with multihoming feasible mode or loose is used. The check is done against the VRF FIB, so for customer A a spoof of the address 10.255.0.2 from its VRF will be dropped if there is no route to it through this interface.

Switching off what is unnecessary

The attack surface is reduced by disabling unused services: HTTP/HTTPS where not needed, Telnet, CDP/LLDP on customer interfaces, proxy-ARP, ICMP redirects, IP source-routing, small servers, and ip unreachables on external interfaces. Management is moved into a separate VRF or an out-of-band network, SSH with keys and timeouts, login block-for against brute force, time is synchronized through authenticated NTP and a log is kept. Customer interfaces have no MPLS at all (see the VPN isolation lesson).

Attacks and checks

Safe rollout

Hardening breaks sessions if a mistake is made on one of the sides. So changes are made with a rollback: Cisco reload in 10, Junos commit confirmed 5, RouterOS Safe Mode; GTSM and the password are changed simultaneously on both sides in the maintenance window, and no mpls ip propagate-ttl is verified with a traceroute from the customer and from the PE itself. Every measure is verified with an attack and not just with show running-config: if a countermeasure leaves no counter or log, it most likely does not work.

Architecture Diagram & Topology

+-----------+        Customer Traceroute Packet (TTL=1)         +-----------+
|   CE-1    | ------------------------------------------------> |   PE-1    |
| (Customer |   Hidden Core: mpls ip propagate-ttl = disabled   | (Ingress  |
|  Network) | <================================================ |  Hardening|
+-----------+        Core P-routers invisible in traceroute     +-----------+

Traffic Flow & Label Stack Scheme

MPLS TTL Propagation vs Pipe Hardening:
1. Uniform model (Cisco default): the IP TTL is copied into the MPLS TTL
   and every core hop decrements it.
   Result: a customer traceroute shows all internal P router addresses
   (reconnaissance help for an attacker).
2. Pipe model ('no mpls ip propagate-ttl'; 'forwarded' limits it to
   customer transit traffic):
   - The MPLS TTL is initialized to 255 at the ingress PE.
   - Core P routers decrement only the MPLS TTL; the IP TTL is not
     decremented in the core.
   - A customer traceroute shows the ingress and egress PE only, so the core
     addresses are hidden.
3. This only makes reconnaissance harder. Real protection: no customer
   route to infrastructure addresses (VRF), CoPP and ACLs that do not
   answer, uRPF on customer links.
4. Verify with traceroute from the CE and from the PE itself.

State & Parameter Comparison Table