Bonding (Link Aggregation)
Bonding (Link Aggregation)
Section titled “Bonding (Link Aggregation)”TL;DR (Quick Start)
Section titled “TL;DR (Quick Start)”For the impatient: create a basic LACP bond.
# Create 802.3ad LACP bond/interface bonding add name=bond1 mode=802.3ad \ slaves=ether1,ether2 transmit-hash-policy=layer-2-and-3
# Add IP address/ip address add address=192.168.1.1/24 interface=bond1Verify:
/interface bonding monitor bond1Expected output: mode: 802.3ad with active-ports: ether1,ether2 and valid partner-sys-id.
Overview
Section titled “Overview”What this does: Bonding aggregates multiple Ethernet interfaces into a single virtual link, providing higher aggregate bandwidth and/or failover capability. Traffic is distributed across member links using configurable hash policies.
When to use this:
- Connecting servers with multiple NICs (NAS, virtualization hosts)
- Switch-to-switch uplinks
- Router-to-router connections
- Redundant WAN links with automatic failover
- High-availability network designs
Critical Understanding: A single TCP connection cannot exceed the speed of one link. Bonding benefits come from multiple simultaneous flows being distributed across links.
Prerequisites:
- Minimum 2 Ethernet-like interfaces
- Switch with LACP/LAG support (for 802.3ad/balance-xor)
- Same speed/duplex on all slaves (required for 802.3ad)
Bonding Modes
Section titled “Bonding Modes”| Mode | HW Offload | Single Flow Speed | CPU Usage | Use Case |
|---|---|---|---|---|
802.3ad | Yes | 1 link max | Low (4-6%) | Enterprise with LACP switches |
balance-xor | Yes | 1 link max | Low | Static LAG without LACP |
active-backup | Yes | 1 link only | Low | Pure failover |
balance-rr | No | Can exceed 1 link | Very High (~95%) | Max throughput (TCP issues) |
balance-tlb | No | TX: multi, RX: 1 link | Moderate | No switch config needed |
balance-alb | No | Better than tlb | Moderate | No switch config needed |
Common Mistakes
- Don’t expect 2x1G bond to give 2 Gbps for a single file transfer - that’s not how bonding works
- Don’t use
balance-rrfor production TCP traffic - packet reordering causes severe performance issues - Don’t use ARP monitoring with LACP - replies may arrive on single port due to peer’s hash policy
Configuration Steps
Section titled “Configuration Steps”Step 1: Choose Your Mode
Section titled “Step 1: Choose Your Mode”802.3ad (LACP) - Recommended for most scenarios:
- Industry standard, interoperable
- Hardware offloaded on CRS3xx/CRS5xx/CCR2xxx
- Requires LACP support on connected switch
active-backup - For pure failover:
- No switch configuration needed
- Works with multiple unconnected switches
- Use ARP monitoring for end-to-end detection
Step 2: Create LACP Bond (802.3ad)
Section titled “Step 2: Create LACP Bond (802.3ad)”/interface bonding add name=bond1 mode=802.3ad \ slaves=ether1,ether2 \ transmit-hash-policy=layer-2-and-3 \ lacp-rate=30secs \ link-monitoring=miiProperties explained:
transmit-hash-policy: How traffic is distributed (see Hash Policy section)lacp-rate: LACPDU exchange frequency (30secs standard, 1sec for fast failover)link-monitoring: MII monitors local link state; use for LACP
Step 3: Configure Remote Switch
Section titled “Step 3: Configure Remote Switch”Cisco example:
! Configure ONLY on Port-channel, not individual portsinterface Port-channel1 switchport trunk encapsulation dot1q switchport mode trunk
interface GigabitEthernet0/1 channel-group 1 mode active
interface GigabitEthernet0/2 channel-group 1 mode activeStep 4: Add to Bridge (for HW Offload)
Section titled “Step 4: Add to Bridge (for HW Offload)”For CRS3xx/CRS5xx/CCR2xxx hardware offload (see Bridge Configuration for details):
# Create bridge with HW offload/interface bridge add name=bridge1 vlan-filtering=yes
# Add bond to bridge (not individual slave ports)/interface bridge port add bridge=bridge1 interface=bond1 hw=yes
# Verify offload/interface bridge port print detail where interface=bond1Expected: hw=yes and hw-offload=yes
Active-Backup Failover
Section titled “Active-Backup Failover”For pure failover without aggregation:
/interface bonding add name=bond-failover mode=active-backup \ slaves=ether1,ether2 \ primary=ether1 \ link-monitoring=arp \ arp-ip-targets=192.168.1.254 \ arp-interval=100msWhy ARP monitoring: MII only detects local link down. ARP monitoring verifies end-to-end connectivity to the specified target.
Transmit Hash Policy
Section titled “Transmit Hash Policy”| Policy | Hashes | Best For |
|---|---|---|
layer-2 | MAC addresses | Direct device-to-device |
layer-2-and-3 | MAC + IP | Router-to-router, multiple hosts |
layer-3-and-4 | IP + ports | Single host, many connections |
/interface bonding set bond1 transmit-hash-policy=layer-3-and-4How it works:
layer-2: XOR of source/destination MAClayer-2-and-3: layer-2 + XOR of source/destination IPlayer-3-and-4: layer-2-and-3 + XOR of source/destination ports
Common Mistakes
- On CRS3xx/CRS5xx/CCR2xxx with HW offload, hash policy is always L2+L3+L4 regardless of configuration
- Hash policy only affects outgoing traffic - each end independently hashes, so distribution may be asymmetric
Link Monitoring
Section titled “Link Monitoring”MII Monitoring (Default)
Section titled “MII Monitoring (Default)”Monitors local interface state via driver:
/interface bonding set bond1 link-monitoring=mii mii-interval=100msDetects: Physical link down, cable disconnect Cannot detect: Remote failures, one-way issues, upstream problems
ARP Monitoring
Section titled “ARP Monitoring”Verifies end-to-end connectivity:
/interface bonding set bond1 link-monitoring=arp \ arp-ip-targets=192.168.1.254 arp-interval=100msDetects: End-to-end connectivity issues Best for: active-backup mode
| Mode | Recommended Monitoring |
|---|---|
| 802.3ad | MII (LACP PDUs validate link) |
| balance-xor | MII |
| active-backup | ARP (end-to-end detection) |
| balance-tlb/alb | MII only (ARP ignored) |
Hardware Offload
Section titled “Hardware Offload”Supported Devices
Section titled “Supported Devices”- CRS3xx series switches
- CRS5xx series switches
- CCR2116, CCR2216 routers
Offloaded Modes
Section titled “Offloaded Modes”| Mode | HW Offloaded |
|---|---|
| 802.3ad | Yes |
| balance-xor | Yes |
| active-backup | Yes |
| Others | No (CPU processed) |
Requirements
Section titled “Requirements”- Use 802.3ad, balance-xor, or active-backup mode
- Add bond to hardware-offloaded bridge
- Remove individual slave ports from bridge
MLAG (Multi-Chassis Link Aggregation)
Section titled “MLAG (Multi-Chassis Link Aggregation)”MLAG allows bonding across two physical switches for chassis redundancy.
Architecture
Section titled “Architecture”Configuration
Section titled “Configuration”Switch A (Primary):
# Create peer-link bond/interface bonding add name=peer-bond mode=802.3ad \ slaves=sfp-sfpplus1,sfp-sfpplus2 lacp-rate=1sec
# Create client bond with MLAG ID/interface bonding add name=client1-bond mode=802.3ad \ slaves=ether1 mlag-id=10 lacp-rate=1sec
# Create bridge/interface bridge add name=bridge1 vlan-filtering=yes
# Add ports/interface bridge port add bridge=bridge1 interface=peer-bond \ frame-types=admit-only-vlan-tagged/interface bridge port add bridge=bridge1 interface=client1-bond
# Configure MLAG (lower priority = primary)/interface bridge mlag set bridge=bridge1 peer-port=peer-bond priority=50
# Tag peer port in all VLANs/interface bridge vlan add bridge=bridge1 tagged=peer-bond vlan-ids=10,20,30Switch B: Same config but priority=100
Common Mistakes
- Each client bond MUST have a unique
mlag-id- this is the most common MLAG mistake - Peer port must be tagged member of ALL spanned VLANs
- Don’t configure
mlag-idon the peer-port bond itself - Use firmware 7.15.3+ or 7.19.3+ for stability
Wireless Link Failover
Section titled “Wireless Link Failover”For 60GHz primary with 5GHz backup:
# Create VLANs on physical interfaces first (avoids ARP leakage)/interface vlan add interface=wlan60 vlan-id=100 name=vlan100-60g/interface vlan add interface=wlan5 vlan-id=100 name=vlan100-5g
# Bond the VLANs/interface bonding add name=wireless-bond mode=active-backup \ slaves=vlan100-60g,vlan100-5g primary=vlan100-60g \ link-monitoring=arp arp-ip-targets=10.0.0.2Verification
Section titled “Verification”# View bonding interfaces/interface bonding print
# Real-time monitoring/interface bonding monitor bond1Expected output for working LACP:
mode: 802.3ad active-ports: ether1,ether2 inactive-ports: lacp-system-id: AA:BB:CC:DD:EE:FFlacp-system-priority: 65535 partner-sys-id: 11:22:33:44:55:66 partner-key: 1Check MLAG status:
/interface bridge mlag monitorVerify hardware offload:
/interface bridge port print detail where interface=bond1# Look for: hw=yes, hw-offload=yesTest bond throughput:
Use Bandwidth Test or Torch to verify traffic is distributed across multiple links.
Common Scenarios
Section titled “Common Scenarios”Scenario: NAS to Router
Section titled “Scenario: NAS to Router”# Router side/interface bonding add name=nas-bond mode=802.3ad \ slaves=ether5,ether6 transmit-hash-policy=layer-3-and-4
# Configure matching LACP on NAS (Synology, QNAP, etc.)Expectation: Multiple simultaneous connections (different clients, different apps) will use multiple links. Single SMB file copy still uses one link.
Scenario: Router to Router
Section titled “Scenario: Router to Router”# Both routers - identical config/interface bonding add name=inter-router mode=802.3ad \ slaves=ether1,ether2 transmit-hash-policy=layer-2-and-3Scenario: Switch Uplink
Section titled “Scenario: Switch Uplink”# On MikroTik CRS switch/interface bonding add name=uplink-bond mode=802.3ad \ slaves=sfp-sfpplus1,sfp-sfpplus2 lacp-rate=1sec
/interface bridge port add bridge=bridge1 interface=uplink-bond hw=yesTroubleshooting
Section titled “Troubleshooting”| Symptom | Cause | Solution |
|---|---|---|
| Single transfer only 1 Gbps on 2x1G bond | One flow = one link | Expected behavior; benefits come from multiple flows |
| Asymmetric speeds (2 Gbps down, 1 Gbps up) | Each end hashes independently | Normal; verify hash policies on both ends |
| balance-rr TCP very slow | Packet reordering causes retransmits | Use 802.3ad or balance-xor instead |
| Active-backup won’t failover | MII can’t detect remote failure | Use ARP monitoring with arp-ip-targets |
| ”No 802.3ad response from partner” | Mismatched config or ports | Verify slave ports, match lacp-rate |
| Bond not HW offloaded | Wrong mode or not in bridge | Use 802.3ad/xor, add to bridge |
| MLAG flapping | Duplicate MLAG IDs | Each client bond needs unique mlag-id |
Performance Reference
Section titled “Performance Reference”| Mode | CPU Usage | HW Offload | Throughput |
|---|---|---|---|
| 802.3ad | Low (4-6%) | Yes | Wire speed (offloaded) |
| balance-xor | Low | Yes | Wire speed (offloaded) |
| active-backup | Low | Yes | Single link |
| balance-rr | Very High (~95%) | No | Sum of links (TCP issues) |
| balance-tlb | Moderate | No | TX: sum, RX: primary |
| balance-alb | Moderate | No | Better than tlb |
See Also
Section titled “See Also”Prerequisites
Section titled “Prerequisites”- Ethernet Interface Configuration - Configure physical interfaces before bonding
- Interfaces Overview - Interface fundamentals and concepts
- IP Address Configuration - Assign IP addresses to bond interfaces
Layer 2 Infrastructure
Section titled “Layer 2 Infrastructure”- Bridge Configuration - Required for hardware offload on CRS switches
- Bridge VLAN Filtering - VLANs on bonded interfaces
- VLAN Configuration - VLAN tagging on bonds
- Switch Ports - Hardware switching and VLAN configuration
High Availability & Failover
Section titled “High Availability & Failover”- High Availability with VRRP - Layer 3 redundancy complement to bonding
- Netwatch - Monitor link health and trigger failover scripts
- Static Routes - Route failover over bonded links
Performance & Monitoring
Section titled “Performance & Monitoring”- Bandwidth Test - Verify bond throughput
- Torch - Real-time traffic analysis per link
- Bandwidth Limiting with Simple Queues - QoS on bonded interfaces
Security
Section titled “Security”- Firewall Basics - Secure traffic on bonded interfaces