Theoretical foundations and architecture

Carrier Supporting Carrier (CsC, RFC 4364, section 9) is a hierarchy in which a backbone carrier sells an MPLS VPN to another operator (the customer carrier). The customer carrier builds its own L3VPN or Internet services on top, without renting dedicated lines between its sites. For the backbone the customer carrier is a CE in a VRF; the VRF holds only its internal routes (PE/RR loopbacks and link addresses), not the routes of end subscribers. Therefore the backbone PE cannot switch by an IP lookup: labeled packets travel on the PE-CE link, and labels for the internal prefixes are exchanged through eBGP with labels (IPv4+label, RFC 8277/3107) or through LDP in the VRF. The stack in the backbone core is: the backbone transport label (LDP/SR), the backbone VPN label for the customer carrier's loopback, and the customer carrier's inner VPN label, which travels unchanged. The backbone core knows nothing about end-subscriber routes.

Who is who in CsC

Carrier Supporting Carrier (RFC 4364, section 9) is an MPLS VPN service whose customer is another operator. The backbone carrier (AS 65000: PE1 10.255.0.1, PE2 10.255.0.2, RR 10.255.0.100) sells the customer carrier (AS 65001) connectivity between its sites. The border routers of the customer carrier (CE1 172.16.10.2 and CE2 172.16.20.2 - for the backbone they are CEs, for the customer carrier they are its own P/PE/ASBR) are attached to the VRF CSC-CUST on PE1 and PE2. The customer carrier has its own subscribers and VPNs: the backbone operator need not know either their addresses or their number.

Why it is needed and what it saves

If a customer carrier simply rents an L3VPN from the backbone, it would have to hand the backbone PE all the prefixes of its customers (hundreds of thousands of routes, overlapping addresses, the full Internet). In CsC the backbone sees only the internal routes of the customer carrier: the loopbacks of its PEs/RRs (tens of /32s) and link addresses. The table in the VRF CSC-CUST is small, and the customer VPNs travel as "nested" MPLS inside the backbone tunnel. This is a hierarchy: the customer carrier builds its own LSPs and VPNs on top of the backbone LSPs.

Why labels are needed on the PE-CE link

A packet of the customer carrier travels with a label: [label of the customer carrier's LSP | its VPN label | IP]. PE1 does not know the subscribers' addresses and cannot perform an IP lookup in the VRF: packets must be switched by label. Therefore PE1 and CE1 exchange labels for the prefixes inside the VRF:

  • eBGP with labels (IPv4 + label, RFC 8277/3107): neighbor ... send-label in address-family ipv4 vrf (Cisco), peer ... label-route-capability (Huawei), family inet labeled-unicast in a group inside the routing-instance (Junos). Labeled forwarding is enabled on the interface: mpls bgp forwarding (Cisco), mpls (Huawei), family mpls (Junos).
  • IGP/LDP in the VRF: an IGP (OSPF/RIP/static) plus LDP runs on the PE-CE link; the PE redistributes the IGP routes of the customer carrier into MP-BGP. LDP on a VRF interface requires the LDP router-id to be reachable from the VRF.

eBGP is usually chosen: one protocol carries both the prefix and the label, scales and is filtered by policies.

The label stack along the path

Suppose a subscriber behind PE-cust1 (10.200.0.1/32 behind CE1) sends traffic to a subscriber behind PE-cust2 (10.200.0.2/32 behind CE2):
| Segment | Label stack (top to bottom) | Issued by |
|---|---|---|
| CE1 -> PE1 | the PE1 label for 10.200.0.2/32, the customer carrier's VPN label | PE1 (eBGP-LU), PE-cust2 |
| PE1 -> P1 -> PE2 | LDP transport to PE2, the backbone VPN label for 10.200.0.2/32, the customer carrier's VPN label | backbone LDP, PE2 (MP-BGP), PE-cust2 |
| PE2 -> CE2 | the CE2 label for 10.200.0.2/32 (unless implicit-null), the customer carrier's VPN label | CE2 (eBGP-LU), PE-cust2 |
In the backbone core there are three labels (12 bytes), on the PE-CE link two. The P routers see only the top label and know nothing about CsC or the customers of the customer carrier. The inner VPN label of the customer carrier travels the whole path unchanged: only its own PE pushes and pops it.

Control plane step by step

  1. CE1 advertises over eBGP (AS 65001) the loopback 10.200.0.1/32 and a label; PE1 places the prefix in the VRF CSC-CUST.
  2. PE1 exports the prefix into VPNv4 with RD 65000:50, RT 65000:500 and the backbone VPN label; PE2 imports it into its VRF CSC-CUST.
  3. PE2 advertises the prefix to CE2 over eBGP-LU with a new label; the same AS 65001 on both sites requires as-override on PE2 (Huawei substitute-as) or allowas-in on the CE.
  4. The customer carrier sets up its own iBGP/MP-BGP between PE-cust1 and PE-cust2 by loopbacks; its transport is the LSP created in steps 1-3.

Scale and limitations

  • On the backbone PE the VRF CSC-CUST holds only the customer carrier's infrastructure: hundreds of prefixes instead of millions. Limit acceptance with maximum-prefix (Huawei route-limit, Junos prefix-limit) and a filter for the allowed /32s.
  • One VRF serves one customer carrier; all its VPNs pass through this VRF transparently.
  • Labels on the PE-CE link are allocated per prefix: too many prefixes in the VRF exhaust the label space of the platform.
  • MTU: CsC adds labels; for customer packets of 1500 bytes the backbone MTU must be at least 1500 + 12 (three labels) + L2.
  • QoS: the EXP/TC of the customer label is copied to the upper labels only if a policy is configured; by default the backbone classifies by the top label.