SP-B SR-MPLS: Predictable Labels, TI-LFA vs Classic LFA, and fragmentation problem

The previous stage gave SP-A an LDP-based MPLS core where labels are locally assigned and distributed hop-by-hop. This stage uses a different technology and different approach. When Segment Routing is fully up on SP-B, every router in the network has the same label for B-PE1’s loopback – 16500 – without exchanging a single LDP message. That label appears in the LSDB the moment B-PE1’s LSP is flooded, and every other router will get its LFIB entry from the same globally distributed information.

B-P1’s label table after it is fully up. All labels are received from LSDB:

RP/0/RP0/CPU0:B-P1#show isis segment-routing label table
Sat May  2 18:08:49.065 UTC

IS-IS 1 IS Label Table
Label         Prefix                   Interface
----------    ----------------         ---------
16100         10.2.0.1/32              Loopback0
16200         10.2.0.2/32
16300         10.2.0.3/32
16400         10.2.0.4/32
16500         10.2.0.5/32
16600         10.2.0.6/32
16700         10.2.0.7/32
16800         10.2.0.8/32

Label 16100 is B-P1’s own loopback – the only entry flagged Loopback0. The other seven arrived as LSP floods across the IS-IS domain. 16500 maps to B-PE1’s loopback on every router in SP-B. That is what “globally significant within the SRGB” means.

Topology file: topology.clab.yml · Addressing: ipam.md · Stage configs, all eight SP-B nodes: stage_configs/lab01-s2b-spb-sr/

The SRGB and Prefix-SID Index Scheme

The Segment Routing Global Block (SRGB) is the range of labels a node reserves for prefix-SIDs. SP-B pins the XR nodes to 16000-16999. The CSRs keep the IOS-XE default, and B-ASBR’s LSP reports it as SRGB Base: 16000 Range: 8000.

A prefix-SID is advertised as an index. Each router runs its own calculation: the next-hop’s SRGB base, plus the index. B-PE1 advertises index 500, its neighbours all start at 16000, and 16500 turns up in every label table in SP-B.

The index has to fit the narrowest SRGB in the domain, which here is the XR block of 1000. Index 1000 would resolve to 17000, which will be out of range for the XR routers. In this lab, I’m running indexes 100 to 800.

Node Loopback Index Label
B-P1 10.2.0.1 100 16100
B-P2 10.2.0.2 200 16200
B-P3 10.2.0.3 300 16300
B-P4 10.2.0.4 400 16400
B-PE1 10.2.0.5 500 16500
B-PE2 10.2.0.6 600 16600
B-RR 10.2.0.7 700 16700
B-ASBR 10.2.0.8 800 16800

Index is N×100 throughout, where N is the last octet of the router-ID.

The index travels as the Prefix-SID sub-TLV, Type 3, inside IS-IS TLV 135 Extended IP Reachability, so no extra protocol needed to transfer labels.

Below is the example from B-ASBR’s database for the sub-TLV, where we can see all eight indexes. Each has a tag Algorithm:SPF:

B-ASBR#show isis database level-2 verbose | i Prefix-SID
    Prefix-SID Index: 100, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0
    Prefix-SID Index: 200, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0
    Prefix-SID Index: 300, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0
    Prefix-SID Index: 400, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0
    Prefix-SID Index: 500, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0
    Prefix-SID Index: 600, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0
    Prefix-SID Index: 700, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0
    Prefix-SID Index: 800, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0

B-ASBR’s own LSP carries the capability block that makes it an SR participant. In this output I specifically cut SR sub-TLVs section :

! Output trimmed  
B-ASBR#show isis database verbose B-ASBR.00-00       

IS-IS Level-2 LSP B-ASBR.00-00
LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime/Rcvd      ATT/P/OL
B-ASBR.00-00        * 0x000002C3   0xF70F                1126/*         0/0/0
  Router CAP:   10.2.0.8, D:0, S:0
    Segment Routing: I:1 V:0, SRGB Base: 16000 Range: 8000
    Segment Routing Local Block: SRLB Base: 15000 Range: 1000
    Segment Routing Algorithms: SPF, Strict-SPF
    Node-MSD
      MSD: 16
  Metric: 0          IP 10.2.0.8/32
    Prefix-attr: X:0 R:0 N:1
    Prefix-SID Index: 800, Algorithm:SPF, R:0 N:1 P:0 E:0 V:0 L:0

This is IS-IS TLV 242, Router Capability, in this TLV router explains itself to the neighbors. The SRGB is in the SR-Capabilities sub-TLV, Type 2. The SRLB, the algorithm list and the Node Maximum SID Depth come as their own sub-TLVs. MSD is the label-stack depth this node can impose – 16 on the XE B-ASBR, against 10 on the XRv9k nodes. The Prefix-SID is in TLV 135. SR requires both TLVs present.

IOS-XR and IOS-XE syntax difference

IOS-XR – prefix-SID configured inside the IS-IS process, under Loopback0:

segment-routing
 global-block 16000 16999
!
router isis 1
 address-family ipv4 unicast
  segment-routing mpls
 !
 interface Loopback0
  address-family ipv4 unicast
   prefix-sid index 100

The global segment-routing / global-block configuration sets the SRGB. The segment-routing mpls line under the IS-IS IPv4 address family turns the SR extensions on for that process. And the prefix-SID is configured under the loopback interface. The block above is trimmed, the full file is B-P1.ios.

IOS-XE – prefix-SID configured in a global segment-routing block, outside IS-IS:

segment-routing mpls
 connected-prefix-sid-map
  address-family ipv4
   10.2.0.2/32 index 200 range 1
  exit-address-family
!
router isis
 segment-routing mpls

The connected-prefix-sid-map maps a locally connected prefix to a SID index. The router isis / segment-routing mpls line enables IS-IS to advertise and process SR labels, so the SID assignment is configured outside the IS-IS process.

IOS-XR TI-LFA vs IOS-XE Classic LFA

IOS-XR (B-P1, B-PE1, B-PE2, B-RR) – TI-LFA:

router isis 1
 interface GigabitEthernet0/0/0/0
  address-family ipv4 unicast
   fast-reroute per-prefix
   fast-reroute per-prefix ti-lfa

TI-LFA precomputes a backup path for every prefix and installs it in the LFIB next to the primary. The backup is the post-convergence path: the router runs SPF again with the protected link taken out, and the repair follows the route the network will settle on after reconvergence anyway. To keep the packet on that path, the router pushes a repair list on it – a stack of prefix-SIDs and adj-SIDs that names the nodes and links the packet must cross. The SIDs carry the packet through whichever nodes the post-convergence path needs, so a backup exists wherever the topology has a second path.

TI-LFA repair list

In the diagram, S protects its link to E for destination D. P-space is every node S still reaches without that link; Q-space is every node that reaches D without it. N is the last P-space node on the post-convergence path and M is the first Q-space node, so the repair list is the prefix-SID of N followed by the adj-SID of the N-M link. N on its own would forward the packet back through S, since its shortest path to D still runs that way until it reconverges; the adj-SID overrides that for one hop, and from M the packet follows the plain shortest path.

IOS-XE (B-P2, B-P3, B-P4, B-ASBR) – classic IS-IS LFA:

router isis
 fast-reroute per-prefix level-2 all

One line at the process level, whereas XR requires configuration per interface per address family. level-2 scopes it to the Level-2 database. all is the prefix selector, and it has only one alternative: fast-reroute per-prefix level-2 route-map <name>, which computes repair paths only for prefixes the route-map matches.

TI-LFA is not an option on my CSR version. IS-IS TI-LFA in IOS-XE is supported on 17.3.x.

Classic LFA has no stack to push. It can only hand the packet to a neighbour that already has a loop-free route to the destination (the RFC 5286 condition), and where no neighbour qualifies, the prefix stays unprotected. On the same topology, A is the only other neighbour of S, and A’s shortest path to D runs back through S at cost 3, against 12 the long way round. The condition fails, and S installs no backup for D.

Classic LFA loop-free condition fails

With N-M at 1 and S-E at 2, A’s shortest path to D becomes A-N-M-D at cost 3, under the 4 the condition allows, and S installs A as the backup next-hop for D. The packet leaves S carrying only its own label; A forwards it by its own routing table, which is why the neighbour had to be loop-free in the first place.

Classic LFA loop-free condition holds

The coverage gap is visible in the summary outputs. B-P1 (XR, TI-LFA):

RP/0/RP0/CPU0:B-P1#show isis ipv4 fast-reroute summary
Sat May  2 18:09:10.708 UTC

IS-IS 1 IPv4 Unicast FRR summary

                          Critical   High       Medium     Low        Total
                          Priority   Priority   Priority   Priority
Prefixes reachable in L2
  All paths protected     0          0          6          0          6
  Some paths protected    0          0          0          0          0
  Unprotected             0          0          1          0          1
  Protection coverage     0.00%      0.00%      85.71%     0.00%      85.71%

The summary counts prefixes, and in SP-B the Level-2 prefixes are the eight loopbacks, so B-P1 is reporting on the seven that are not its own. Six are protected. The seventh is B-RR’s 10.2.0.7/32, and B-RR is connected with a single link. There is no alternate next-hop for TI-LFA to encode a path to, so 85.71% is the topology’s limit.

B-PE1 and B-PE2 (XR, TI-LFA, both with dual uplinks):

RP/0/RP0/CPU0:B-PE1#show isis ipv4 fast-reroute summary
Sat May  2 18:16:43.511 UTC

IS-IS 1 IPv4 Unicast FRR summary

                          Critical   High       Medium     Low        Total
                          Priority   Priority   Priority   Priority
Prefixes reachable in L2
  All paths protected     0          0          7          0          7
  Some paths protected    0          0          0          0          0
  Unprotected             0          0          0          0          0
  Protection coverage     0.00%      0.00%      100.00%    0.00%      100.00%

100%, because B-PE1 has two upstream paths – Gi0/0/0/0 to B-P1 and Gi0/0/0/1 to B-P2. The LFIB shows two different mechanisms delivering that number:

! prefix-SID rows; the 24000-block adj-SID rows are cut
RP/0/RP0/CPU0:B-PE1#show mpls forwarding | i SR      
Sat May  2 18:16:48.434 UTC
16100  Pop         SR Pfx (idx 100)   Gi0/0/0/0    10.2.1.21       0
       16100       SR Pfx (idx 100)   Gi0/0/0/1    10.2.1.25       0            (!)
16200  Pop         SR Pfx (idx 200)   Gi0/0/0/1    10.2.1.25       0
       16200       SR Pfx (idx 200)   Gi0/0/0/0    10.2.1.21       0            (!)
16300  16300       SR Pfx (idx 300)   Gi0/0/0/0    10.2.1.21       0
       16300       SR Pfx (idx 300)   Gi0/0/0/1    10.2.1.25       0
16400  16400       SR Pfx (idx 400)   Gi0/0/0/0    10.2.1.21       0
       16400       SR Pfx (idx 400)   Gi0/0/0/1    10.2.1.25       0            (!)
16600  16600       SR Pfx (idx 600)   Gi0/0/0/0    10.2.1.21       0
       16600       SR Pfx (idx 600)   Gi0/0/0/1    10.2.1.25       0            (!)
16700  16700       SR Pfx (idx 700)   Gi0/0/0/0    10.2.1.21       0
       16700       SR Pfx (idx 700)   Gi0/0/0/1    10.2.1.25       0            (!)
16800  16800       SR Pfx (idx 800)   Gi0/0/0/0    10.2.1.21       0
       16800       SR Pfx (idx 800)   Gi0/0/0/1    10.2.1.25       0            (!)

(!) is a pre-programmed backup. Six of the seven prefixes have one, and when the primary next-hop drops, the dataplane moves to that entry without waiting for IS-IS to reconverge – the sub-50ms behaviour. 16300 for B-P3 has two rows without the flag. B-P3 sits at equal cost from B-PE1, so ECMP covers it, and TI-LFA has nothing left to compute. The summary counts it as protected, which is why the summary alone will not tell you which mechanism is carrying a given prefix.

The XE nodes (B-P2, B-P3, B-P4, B-ASBR) report 100% coverage in their summary output:

B-ASBR#show isis fast-reroute summary

Microloop Avoidance State: Disabled
Segment-Routing Microloop Avoidance State: Disabled
IPv4 Fast-Reroute Protection Summary:

 Prefix Counts:          Total      Protected    Coverage
   High priority:        0          0            0%
   Normal priority:      7          7            100%
   Total:                7          7            100%

100% on B-ASBR, which also has two uplinks, to B-P1 and B-P4. Classic LFA finds an alternate for all seven remote loopbacks because the loop-free condition holds from a dual-homed position. B-P2 and B-P3, also well-connected, show the same. The difference between TI-LFA and classic LFA is most visible on nodes with fewer alternates or in topologies where the LFA condition fails.

The lsp-mtu 128 Issue: Silent SR-CAP Withdrawal

Earlier, during IS-IS configuration, I set lsp-mtu 128 on B-P1 and B-PE1 as a fragmentation exercise. When SR was enabled, both nodes began intermittently advertising SR-Cap: N and missing Prefix-SID sub-TLVs in show isis segment-routing global-block, as observed from all peers. Restarting IS-IS resolved this problem for a few minutes before the symptom recurred.

The failing state itself was never captured. I changed the configuration part and forgot about that, but when I started reviewing my notes, I decided to mention it here. The archive holds the converged state on the other side of the fix, where every node reports SR-Cap: Y:

B-ASBR#show isis segment-routing global-block


IS-IS Level-2 Segment-routing Global Blocks:

System ID             SRGB Base  SRGB Range SID Index  Strict-SID SR-Cap Strict
B-P1                  16000      1000       100        None       Y       N
B-P2                  16000      8000       200        None       Y       N
B-P3                  16000      8000       300        None       Y       N
B-P4                  16000      8000       400        None       Y       N
B-PE1                 16000      1000       500        None       Y       N
B-PE2                 16000      1000       600        None       Y       N
B-RR                  16000      1000       700        None       Y       N
B-ASBR              * 16000      8000       800        None       Y       N

The XR nodes advertise SRGB Range 1000 and the XE nodes 8000. B-PE2 never failed because it didn’t have lsp-mtu 128 applied to it. Two XRv9k nodes were affected. That differential identified the root cause in a short troubleshooting session.

With 128-byte LSPs, each adj-SID reallocation re-originates the adjacency-carrying fragments. The Router-CAP TLV (carrying SRGB, SRLB, Node-MSD, and SR Algorithm sub-TLVs) plus the Prefix-SID sub-TLV cannot reliably coexist with the identification TLVs (~50 bytes) inside a single 128-byte fragment. During the window between a fragment’s expiry and the next periodic refresh, neighbours temporarily lose the SR capabilities of those nodes. IS-IS statistics stay clean throughout. The only indicator is SR-Cap: N in the segment-routing display.

The fix was to take lsp-mtu 128 off both nodes and let the LSP MTU fall back to its default, and the capability withdrawal stopped. B-P1’s LSP turned into a single fragment – B-P1.00-00, with no .00-01 behind it. SR extensions want more LSP headroom than plain IS-IS does.

Full Verification Matrix

All eight SP-B nodes verified and I collected the following outputs:

LSDB Prefix-SID visibility. Every node resolves all eight host-to-index bindings. B-ASBR’s view, with * marking its own:

B-ASBR#show isis segment-routing connected-sid

IS-IS Level-2 connected prefix-sids:
Host                   Prefix               SID Index    Range        Flags
B-ASBR               * 10.2.0.8/32          800          1
B-P1                   10.2.0.1/32          100          1
B-P2                   10.2.0.2/32          200          1
B-PE1                  10.2.0.5/32          500          1
B-P4                   10.2.0.4/32          400          1
B-P3                   10.2.0.3/32          300          1
B-RR                   10.2.0.7/32          700          1
B-PE2                  10.2.0.6/32          600          1

LFIB on IOS-XE (B-ASBR):

B-ASBR#show mpls forwarding-table
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
Label      Label      or Tunnel Id     Switched      interface
16         Pop Label  10.2.1.41-A      0             Gi2        10.2.1.41
17         Pop Label  10.2.1.45-A      0             Gi3        10.2.1.45
18         Pop Label  10.2.1.41-A      0             Gi2        10.2.1.41
19         Pop Label  10.2.1.45-A      0             Gi3        10.2.1.45
16100      Pop Label  10.2.0.1/32      0             Gi2        10.2.1.41
16200      16200      10.2.0.2/32      0             Gi2        10.2.1.41
16300      16300      10.2.0.3/32      0             Gi2        10.2.1.41
           16300      10.2.0.3/32      0             Gi3        10.2.1.45
16400      Pop Label  10.2.0.4/32      0             Gi3        10.2.1.45
16500      16500      10.2.0.5/32      0             Gi2        10.2.1.41
16600      16600      10.2.0.6/32      0             Gi2        10.2.1.41
           16600      10.2.0.6/32      0             Gi3        10.2.1.45
16700      16700      10.2.0.7/32      0             Gi2        10.2.1.41

A  - Adjacency SID

Four adj-SIDs sit at the bottom of the table, in the platform’s dynamic range: 16 and 18 toward B-P1, 17 and 19 toward B-P4. Two per adjacency, because IS-IS advertises each one twice: the B-flag in the Adj-SID sub-TLV says whether that SID is eligible for fast-reroute protection, and B-ASBR floods both a B:0 and a B:1 copy for each neighbour – the pair visible in its own LSP earlier. Prefix-SIDs occupy the 16xxx range above them. 10.2.0.3/32 and 10.2.0.6/32 each carry two entries, one via B-P1 and one via B-P4. That is ECMP; classic LFA’s backups do not get their own line in this table. 10.2.0.1/32 and 10.2.0.4/32 show Pop Label, the penultimate-hop pop for a directly connected neighbour.

End-to-end SR LSP – B-PE1 to B-PE2:

RP/0/RP0/CPU0:B-PE1#ping sr-mpls 10.2.0.6/32 fec-type generic source 10.2.0.5        ← return-code legend cut
Sat May  2 18:16:58.923 UTC

Sending 5, 100-byte MPLS Echos to 10.2.0.6/32,
      timeout is 2 seconds, send interval is 0 msec:

!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/7/11 ms
RP/0/RP0/CPU0:B-PE2#ping sr-mpls 10.2.0.5/32 fec-type generic source 10.2.0.6        ← return-code legend cut
Sat May  2 18:18:12.166 UTC

Sending 5, 100-byte MPLS Echos to 10.2.0.5/32,
      timeout is 2 seconds, send interval is 0 msec:

!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/8/22 ms

Both directions at 100%. B-PE1’s LFIB puts 16600 out Gi0/0/0/0 toward B-P1, so the LSP runs B-PE1 → B-P1 → B-PE2, and B-P1 holds pre-computed TI-LFA backups for six of the seven prefix-SIDs it forwards.

B-P1 forwarding entries for B-PE1 and B-PE2 (showing primary + backup):

RP/0/RP0/CPU0:B-P1#show mpls forwarding | i 16500
Sat May  2 18:09:25.351 UTC
16500  Pop         SR Pfx (idx 500)   Gi0/0/0/3    10.2.1.22       0
       16500       SR Pfx (idx 500)   Gi0/0/0/0    10.2.1.2        0            (!)
       16500       SR Adj (idx 1)     Gi0/0/0/0    10.2.1.2        0            (!)
RP/0/RP0/CPU0:B-P1#show mpls forwarding | i 16600
Sat May  2 18:09:32.634 UTC
16600  Pop         SR Pfx (idx 600)   Gi0/0/0/6    10.2.1.50       0
       16600       SR Pfx (idx 600)   Gi0/0/0/2    10.2.1.14       0            (!)
       16600       SR Adj (idx 1)     Gi0/0/0/2    10.2.1.14       0            (!)

B-P1 pops both 16500 and 16600 (PHP – both PE nodes are directly connected). Backup paths are pre-programmed: if B-PE1’s direct link fails, B-P1 reroutes 16500 via B-P2 (10.2.1.2); if B-PE2’s direct link fails, 16600 reroutes via B-P4 (10.2.1.14).

Summary Table: Platform Differences

Behavior IOS-XR (B-P1, B-PE1, B-PE2, B-RR) IOS-XE (B-P2, B-P3, B-P4, B-ASBR)
Prefix-SID assignment router isis 1 / interface Loopback0 / address-family ipv4 unicast / prefix-sid index N segment-routing mpls / connected-prefix-sid-map / address-family ipv4 / <prefix> index N
IS-IS SR enable router isis 1 / address-family ipv4 unicast / segment-routing mpls router isis / segment-routing mpls
Fast-reroute mechanism TI-LFA (per-interface, per-AF) Classic LFA (process-level, level-2 all)
Fast-reroute config location Per interface under address-family Single process-level statement
SRGB range Set explicitly: segment-routing / global-block 16000 16999 Default 16000-23999, left as-is
PHP behavior Pops label for directly connected prefix-SIDs Same
adj-SID numbering Dynamic (24000+) Dynamic (16 upward)

What’s Next

Both providers now have MPLS transport. SP-A runs LDP over OSPF with session protection and OSPF-LDP sync. SP-B runs SR-MPLS over IS-IS, with prefix-SIDs and TI-LFA configured on every XR node.

Next stage is the BGP control plane: iBGP with Route Reflectors on both providers, VPNv4 and VPNv6 address families. The LSPs we built become the transport tunnels underneath it, and MPLS starts carrying customer packets.

Leave a comment