Equal-Cost Multi-Path Routing Definition
Equal Cost Multi-Path (ECMP) is a network routing strategy that enables packets of traffic from the same flow or session—traffic with the same destination IP address and/or source—to transmit across multiple best paths with equal cost. This technique fully utilizes bandwidth along links to the same destination that would otherwise remain unused, increasing throughput and load balancing traffic.
Any time routing technology forwards packets, it must decide which next-hop path to use. To make a decision, the device takes packet header fields that identify flows into account.
ECMP is a method for routing traffic based on the principle of identifying and using next-hop paths of equal cost based on hash ECMP algorithms and routing metric calculations. The network gives multiple best paths of the same cost the same metric values, network cost, and preference. The ECMP process through the routing table then identifies a set of routers, an equal-cost multi-path (ECMP) set, and each is an equal cost next-hop address towards the destination. ECMP and most routing protocols may be used together because ECMP demands a local, per-hop decision about only the next hop destination that each router addresses independently.
Although ECMP may increase bandwidth substantially by load-balancing traffic across multiple best paths of equal cost, in practice there can be problems in deployment if the right decisions are not taken.
By default, ECMP load balancing is set as Per-Destination Load Balancing. This kind of load balancing is a type of per-flow load balancing that guarantees that even if multiple ECMP paths are available, packets for a given source and the host pairs for their destination will take the same path.
Per-Packet Load Balancing is also possible. This kind of load balancing determines which path each packet will take to reach its destination IP using a round-robin method. For some kinds of traffic, such as Voice over IP (VoIP), this is inappropriate, because the sequence in which packets arrive is important.
Equal-Cost Multi-Path Routing (ECMP) FAQs
What is Equal-Cost Multi-Path Routing (ECMP)?
Equal-cost multiple path (ECMP) in networking deploys multiple equal-cost routes to load balance traffic (sessions) to the same destination. This process takes the place of selecting and adding one of the ECMP routes from the routing table to its forwarding table and ignoring the other routes unless there is failure along the selected route.
Enabling ECMP functionality on a virtual router also allows the firewall to use all available bandwidth to the same destination efficiently and leave no links unused. It also shifts traffic dynamically to another ECMP member should a link fail, rather than taking time to select an alternate route or waiting for the routing protocol or RIB table to do so—all reducing downtime.
For routes to be considered equal they must be the same in terms of routing protocol and cost. For example, an OSPF route and a static route are from different sources, so they are not considered equal for ECMP load sharing. Only the best route is installed in the routing table if two routes from the same protocol are unequal.
ECMP hashing prevents out of order packets by functioning on a per-flow basis. ECMP hashing does not keep a record of flow states or hashes to each next hop. ECMP hashing does not record these and does not guarantee the equality of traffic sent to each next hop. All packets with the same source and destination IP addresses and ports always hash to the same next hop.
As a per-hop decision limited to a single router, ECMP can be used to load-balance traffic over multiple paths with most routing protocols and may provide substantial increases in bandwidth. Various routing protocols, including Border Gateway Protocol (BGP) and the Open Shortest Path First (OSPF), allow ECMP routing.
How Does ECMP Work?
ECMP software features enable the open shortest path first protocol (OSPF) to add routes with equal costs and multiple next-hop addresses to a given destination on the routing switch in the forwarding information base (FIB).
In an OSPF domain for a particular destination network, there are several types of ECMP routes:
- Intra-area routes lead through the same OSPF area to the destination
- Inter-area routes lead through another OSPF area to the destination
- External routes lead through another AS to the destination
Multiple ECMP next-hop routes cannot mix intra-area, inter-area, and external routes.
Load balancing distributes traffic across links based on various parameters. ECMP per-flow load balancing uses Layer 3 routing information to distribute the packets. When the router identifies multiple paths to a destination, it updates the routing table with multiple entries.
With per-flow load balancing, the router can use many routes to achieve load sharing across many source-destination host pairs. Even if multiple paths are available, packets for a particular source-destination host pair take the same path, while traffic streams for different pairs usually diverge.
The benefits of per-flow load balancing are mainly the cost-effective, elastic scaling. However, ECMP load balancing also has its drawbacks. For example, neither per packet load balancing nor label load balancing are supported. And BGP multi-path, with or without PIC Edge, is not supported with ECMP.
ECMP vs LACP: What’s the Difference?
ECMP and link aggregation control protocol (LACP), a layer 2 technology, are both related to load balancing. LACP helps provide redundancy in case of a network failure. And although both attempt to achieve redundancy, they do so in different ways at different points in the network.
Which is optimal depends on where the organization must achieve redundancy. For example:
To avoid blackholing traffic with configuration errors use PAgP; if that’s unavailable, ECMP can be used. Use ECMP where both ends are not managed. But where speed of detection and repair of bugs on devices and link failures is more critical, use LACP, which identifies and repairs failures more quickly.
How to Configure ECMP
ECMP is supported by multiple routing protocols. For routing IPv4 traffic use OSPFv2. In an OSPFv2 network, configure ECMP by specifying the number of equal-cost paths the router should choose with the maximum-paths command. This traditional value has been set to 4, but newer Cisco IOS versions allow users to configure up to 32.
For routing IPv6 traffic use OSPFv3, which retains most of the underlying OSPF functionality. Use the maximum-paths command to define ECMP inside the OSPF process, as ECMP configuration remains mostly the same.
Each router in the network will need to be configured one by one because maximum-path values apply to each router individually.
Border Gateway Protocol (BGP) also supports multi-path capabilities, but BGP unlike ECMP, BGP demands two commands, for eBGP and iBGP peerings, to achieve this multi-path routing capability.
Does Avi Offer ECMP Load Balancing?
Yes. Avi Vantage can manage virtual service load balancing capacity by dynamically scaling it out or in on additional or fewer Service Engines (SEs). The primary SE for the virtual service coordinates traffic flow distribution amongst the secondary SEs by default, including itself.
Avi Vantage can take advantage of Contrail’s ECMP support on OpenStack with Contrail, and manage the orchestration of ECMP routes as part of virtual service placement.
Home ECMP functionality at the upstream edge router such as Juniper MX or the Contrail vRouter on the host hypervisor. Learn more here.
For more on the actual implementation of load balancers, check out our Application Delivery How-To Videos.