In
Luc de Ghein's « MPLS fundamentals », at page 49, we learn that
« If multiple equal-cost paths exist for an IPv4 prefix, the Cisco IOS can load-balance labeled packets »
And at page 50, we learn that
« If a prefix is reachable via a mix of labeled and unlabeled (IP) paths, Cisco IOS does not consider the unlabeled paths for load-balancing labeled packets. ».
We wanted to verify these assertions.
Software used in this scenario :
- Hypervisor : VMware ESXi 6.0.0 build-2494585
- Routers : IOS XE version 03.14.01.S, IOS version 15.5(1)S1
- Servers : Windows 10 Education
- Sniffer : Wireshark 2.0.1
- Traffic simulator : Ostinato 0.7.1
Routers configuration (without admin interfaces) :
===================================
R1 =================================== interface Loopback0 ip address 1.1.1.1 255.255.255.255 ! !snip ADMIN INTERFACE snip ! !R1 <=> R2 VLAN 10 interface GigabitEthernet2 ip address 192.168.10.1 255.255.255.0 negotiation auto mpls ip ! !R1 <=> R2 VLAN 20 interface GigabitEthernet3 ip address 192.168.20.1 255.255.255.0 negotiation auto ! !SRV1 <=> R1 VLAN 100 interface GigabitEthernet4 ip address 192.168.100.254 255.255.255.0 negotiation auto ! router ospf 1 passive-interface GigabitEthernet4 network 1.1.1.1 0.0.0.0 area 0 network 192.168.10.0 0.0.0.255 area 0 network 192.168.20.0 0.0.0.255 area 0 network 192.168.100.0 0.0.0.255 area 0 =================================== R2 =================================== interface Loopback0 ip address 2.2.2.2 255.255.255.255 ! !snip ADMIN INTERFACE snip ! !R1 <=> R2 VLAN 10 interface GigabitEthernet2 ip address 192.168.10.2 255.255.255.0 negotiation auto mpls ip ! !R1 <=> R2 VLAN 20 interface GigabitEthernet3 ip address 192.168.20.2 255.255.255.0 negotiation auto ! !R2 <=> R3 VLAN 30 interface GigabitEthernet4 ip address 192.168.30.2 255.255.255.0 negotiation auto mpls ip ! router ospf 1 network 2.2.2.2 0.0.0.0 area 0 network 192.168.10.0 0.0.0.255 area 0 network 192.168.20.0 0.0.0.255 area 0 network 192.168.30.0 0.0.0.255 area 0 =================================== R3 =================================== interface Loopback0 ip address 3.3.3.3 255.255.255.255 ! !snip ADMIN INTERFACE snip ! !R2 <=> R3 interface GigabitEthernet2 ip address 192.168.30.3 255.255.255.0 negotiation auto mpls ip ! !R3 <=> SRV2 interface GigabitEthernet3 ip address 192.168.200.254 255.255.255.0 negotiation auto ! router ospf 1 passive-interface GigabitEthernet3 network 3.3.3.3 0.0.0.0 area 0 network 192.168.30.0 0.0.0.255 area 0 network 192.168.200.0 0.0.0.255 area 0 |
Load-balancing between labeled paths
First we try to highlight load-balancing over VLAN 10 and VLAN 20 with MPLS activated on both links between R1 and R2. We send 100 TCP SYN packets from SRV1 to SRV2 on port 3389.
R1(config)#interface gigabitEthernet 3
R1(config-if)#mpls ip R1(config-if)#end R2(config)#interface gigabitEthernet 3 R2(config-if)#mpls ip R2(config-if)#end R1#show mpls interfaces Interface IP Tunnel BGP Static Operational GigabitEthernet2 Yes (ldp) No No No Yes GigabitEthernet3 Yes (ldp) No No No Yes R2#show mpls interfaces Interface IP Tunnel BGP Static Operational GigabitEthernet2 Yes (ldp) No No No Yes GigabitEthernet3 Yes (ldp) No No No Yes GigabitEthernet4 Yes (ldp) No No No Yes ! Two paths learned from OSPF R1#show ip route 192.168.200.1 Routing entry for 192.168.200.0/24 Known via "ospf 1", distance 110, metric 3, type intra area Last update from 192.168.20.2 on GigabitEthernet3, 00:00:36 ago Routing Descriptor Blocks:
192.168.20.2, from 3.3.3.3, 00:00:36 ago, via GigabitEthernet3 Route metric is 3, traffic share count is 1 *
192.168.10.2, from 3.3.3.3, 00:00:36 ago, via GigabitEthernet2 Route metric is 3, traffic share count is 1 ! Two labeled paths in LFIB R1#show mpls forwarding-table 192.168.200.1 detail Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 19
19 192.168.200.0/24 0
Gi2 192.168.10.2 MAC/Encaps=14/18, MRU=1500, Label Stack{19} 000C29610C7A000C2952B1488847 00013000 No output feature configured Per-destination load-sharing, slots: 0 2 4 6 8 10 12 14 19 192.168.200.0/24 0
Gi3 192.168.20.2 MAC/Encaps=14/18, MRU=1500, Label Stack{19} 000C29610C84000C2952B1528847 00013000 No output feature configured Per-destination load-sharing, slots: 1 3 5 7 9 11 13 15 |
Ostinato configuration (do not forget to fill source MAC and source IP otherwise packets are not forwarded by R1) :
First observation, packets are not loadbalanced between Gi2 and Gi3 on R1 !
VLAN 10 capture on the left, VLAN 20 on the right
After a few searches, we understand that MPLS forwarding is related to CEF and we learn that there are two CEF load-sharing modes (per-packet and per-destination). On R1 « per-destination » is configured for prefix 192.168.200.0/24 :
R1#show ip cef 192.168.200.1 detail
192.168.200.0/24, epoch 2,
per-destination sharing local label info: global/19 nexthop 192.168.10.2 GigabitEthernet2 label 19 nexthop 192.168.20.2 GigabitEthernet3 |
With « per-destination » algorithm, source IP and destination IP of each IP packet are hashed. Packets with same computed hash are forwared through the same interface.
But we learn also that CEF load-sharing algorithm can be tuned :
R1(config)#ip cef load-sharing algorithm ? include-ports Algorithm that includes layer 4 ports original Original algorithm tunnel Algorithm for use in tunnel only environments universal Algorithm for use in most environments R1(config)#ip cef load-sharing algorithm include-ports ? destination Use destination port in hash function source Use source port in hash function R1(config)#ip cef load-sharing algorithm include-ports source R1(config)#end |
According to this modification, we also change Ostinato configuration to send 100 TCP packets with source port from 1026 to 1125 :
This time, load balacing occur between Gi2 and Gi3 !
An interesting command permits to know in advance the outgoing interface selected by CEF algorithm for each packet by specifying source IP address, source port, destination IP address (and destination port but here we include only source ports in CEF load-sharing algorithm) :
! Verified in captures above
R1#show ip cef exact-route 192.168.100.1 src-port
1026 192.168.200.1 192.168.100.1 -> 192.168.200.1 => label 19TAG adj
out of GigabitEthernet2, addr 192.168.10.2 R1#show ip cef exact-route 192.168.100.1 src-port
1029 192.168.200.1 192.168.100.1 -> 192.168.200.1 => label 19TAG adj
out of GigabitEthernet3, addr 192.168.20.2 |
Load-balancing between labeled and unlabeled paths
Finally to prove Luc de Ghein's assertion we disable MPLS on Gi3 and send the same stream :
R1(config)#interface gigabitEthernet 3
R1(config-if)#no mpls ip R1(config-if)#end R2(config)#interface gigabitEthernet 3 R2(config-if)#no mpls ip R2(config-if)#end R1#show mpls interfaces Interface IP Tunnel BGP Static Operational GigabitEthernet2 Yes (ldp) No No No Yes R2#show mpls interfaces Interface IP Tunnel BGP Static Operational GigabitEthernet2 Yes (ldp) No No No Yes GigabitEthernet4 Yes (ldp) No No No Yes R1#show mpls forwarding-table 192.168.200.1 detail Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 19
19 192.168.200.0/24 0
Gi2 192.168.10.2 MAC/Encaps=14/18, MRU=1500, Label Stack{19} 000C29610C7A000C2952B1488847 00013000 No output feature configured
No Label 192.168.200.0/24 0
Gi3 192.168.20.2 MAC/Encaps=14/14, MRU=1504, Label Stack{} 000C29610C84000C2952B1520800 No output feature configured Per-destination load-sharing, slots: 1 3 5 7 9 11 13 15 |
In contrary to Luc de Ghein's assertion, MPLS load-balancing occur between labeled and unlabeled paths !
Conclusion
At the edge of the MPLS cloud, load-balancing is effective between labeled paths and between labeled and unlabeled paths. We coud not verify Luc de Ghein's assertion « If a prefix is reachable via a mix of labeled and unlabeled (IP) paths, Cisco IOS does not consider the unlabeled paths for load-balancing labeled packets. ».
Notes
- When the MPLS is enabled on an interface, the LSR periodically sends (hello interval) hello messages.
The hello messages (UDP source port = port destination = 646) are sent to the 224.0.0.2 address (All Routers on this Subnet)
The LSR ID and space ID label are indicated in each message hello.
A transport IP address is also indicated. - An LSR can not establish an LDP session with a neighbor router if the transport address announced by this one isn't present in its routing table!
- A single LDP session is established between two LSR if the generated labels have global meaning for the router (per-platform label space)
- The LDP session is established on TCP port 646
- On IOS XE 03.14.01.S, IOS Release 15.5 (1) S1, you can not choose the transport address.
The "mpls ldp discovery transport-address" is not available