BGP over GCP HA VPN

BGP over GCP HA VPN

Overview

Google Cloud Platform (GCP) HA VPN provides high-availability site-to-site VPN connectivity using two independent tunnel endpoints. Each tunnel carries a BGP session via Cloud Router, enabling dynamic route exchange between your on-premises RouterOS network and GCP VPC.

How it works:

• GCP provisions two external IP addresses (interfaces 0 and 1) on the HA VPN Gateway.
• Each interface gets an IKEv2 IPsec tunnel to your RouterOS router.
• BGP runs inside each tunnel using APIPA link-local addresses (169.254.x.x/30) assigned at tunnel creation time.
• Cloud Router (default ASN 16550) exchanges routes with your router over eBGP.
• Both tunnels must be established for full HA; if one fails, BGP continues on the surviving tunnel.

When to use this setup:

• Connecting an on-premises MikroTik to GCP workloads with automatic failover.
• Advertising on-premises prefixes into GCP and receiving GCP VPC routes dynamically.
• Replacing static VPN routes with BGP-managed routing for multi-region or multi-VPC designs.

Note

GCP HA VPN requires IKEv2 with specific cipher requirements. Policy-based (XAUTH/IKEv1) is not supported for HA VPN.

---

Prerequisites

• RouterOS 7.x with the routing package installed.
• A public IP address on your RouterOS router reachable from the internet (GCP initiates IKE from its external IPs).
• Accurate system time — IPsec IKEv2 authentication relies on time-based nonces; clock skew greater than a few minutes causes IKE negotiation failures. Configure NTP before deploying:

  /system ntp client set enabled=yes servers=time.google.com,time.cloudflare.com

• GCP project with a VPC, Cloud Router, and HA VPN Gateway configured in the Cloud Console.
• From the GCP Cloud Console, collect before starting:
  • Two GCP external IPs (tunnel interface 0 and 1).
  • Two pre-shared keys (one per tunnel).
  • Two APIPA address pairs (e.g., 169.254.1.1/30 for GCP, 169.254.1.2/30 for your router) — one pair per tunnel.
  • GCP Cloud Router ASN (default: 16550).
• Choose your own ASN for RouterOS (must differ from 16550; use a private ASN in 64512–65534 if you do not have a public one).

---

Configuration

The configuration has four stages: IPsec (IKEv2) for each tunnel, BGP connections over the resulting tunnel interfaces, FastTrack/NAT bypass rules to prevent silent traffic drops, and route advertisement filters.

Stage 1: IKEv2 IPsec Tunnels

GCP HA VPN requires IKEv2 with AES-256, SHA-256, and DH group 14 (modp2048) or stronger.

# IKE profile — must match GCP requirements
/ip ipsec profile
add name=gcp-ike2 \
    hash-algorithm=sha256 \
    enc-algorithm=aes-256 \
    dh-group=modp2048 \
    dpd-interval=10s \
    dpd-maximum-failures=5

# ESP proposal
/ip ipsec proposal
add name=gcp-esp \
    auth-algorithms=sha256 \
    enc-algorithms=aes-256-cbc \
    pfs-group=modp2048 \
    lifetime=3h

# Peers — replace GCP_IP1 / GCP_IP2 with your GCP tunnel external IPs
/ip ipsec peer
add name=gcp-tunnel0 address=<GCP_IP1>/32 exchange-mode=ike2 profile=gcp-ike2
add name=gcp-tunnel1 address=<GCP_IP2>/32 exchange-mode=ike2 profile=gcp-ike2

# Pre-shared keys
/ip ipsec identity
add peer=gcp-tunnel0 auth-method=pre-shared-key secret="<PSK_TUNNEL0>"
add peer=gcp-tunnel1 auth-method=pre-shared-key secret="<PSK_TUNNEL1>"

GCP HA VPN uses route-based VPN internally, but RouterOS uses policy-based IPsec. The traffic selector must include the BGP APIPA addresses so that the BGP TCP session is encrypted:

# Policies — cover the APIPA /30 pairs so BGP traffic is encrypted
# Tunnel 0: 169.254.1.0/30 (example — use your actual APIPA ranges from GCP console)
/ip ipsec policy
add peer=gcp-tunnel0 proposal=gcp-esp \
    src-address=169.254.1.2/32 dst-address=169.254.1.1/32 \
    tunnel=no action=encrypt

add peer=gcp-tunnel1 proposal=gcp-esp \
    src-address=169.254.2.2/32 dst-address=169.254.2.1/32 \
    tunnel=no action=encrypt

Tip

GCP HA VPN operates in transport mode for the BGP link-local traffic when both sides are configured correctly. If your design also needs to encrypt LAN-to-VPC traffic, add additional policies for those subnets with tunnel=yes.

Assign the link-local addresses to a loopback or dummy interface so RouterOS has a local address to source BGP from:

/interface bridge
add name=lo-gcp-bgp comment="loopback for GCP BGP link-local"

# Tunnel 0 link-local address
/ip address
add address=169.254.1.2/30 interface=lo-gcp-bgp

# Tunnel 1 link-local address
/ip address
add address=169.254.2.2/30 interface=lo-gcp-bgp

Stage 2: BGP Instance and Peer Connections

# BGP instance — set your ASN
/routing bgp template
add name=gcp-bgp as=<YOUR_ASN> router-id=<ROUTER_ID>

# Connection to tunnel 0
/routing bgp connection
add name=gcp-cr-tunnel0 \
    template=gcp-bgp \
    remote.address=169.254.1.1 \
    remote.as=16550 \
    local.address=169.254.1.2 \
    local.role=ebgp \
    hold-time=30s \
    keepalive-time=10s \
    output.filter-chain=gcp-bgp-out \
    input.filter=gcp-bgp-in

# Connection to tunnel 1
/routing bgp connection
add name=gcp-cr-tunnel1 \
    template=gcp-bgp \
    remote.address=169.254.2.1 \
    remote.as=16550 \
    local.address=169.254.2.2 \
    local.role=ebgp \
    hold-time=30s \
    keepalive-time=10s \
    output.filter-chain=gcp-bgp-out \
    input.filter=gcp-bgp-in

Note

keepalive-time=10s and hold-time=30s are tighter than the BGP defaults (60s/180s). This allows faster failure detection at the cost of more control-plane traffic, and it is recommended for IPsec-backed sessions where tunnel flaps can cause silent drops.

Note

eBGP multihop is not required. Because each APIPA pair is a /30 (e.g., 169.254.1.2/30), RouterOS treats 169.254.1.1 as directly adjacent — it is on-link. Standard eBGP TTL=1 works without setting multihop=yes. Only add multihop if you peer over a loopback whose prefix does not include the remote address.

Stage 3: FastTrack and NAT Bypass

RouterOS FastTrack accelerates forwarded connections by bypassing the normal processing chain — including IPsec encryption policies. Without an explicit exclusion, IPsec-bound flows are silently fasttracked past the encrypt step, causing traffic to drop while the SA remains “established”. This manifests as random BGP session drops even when both tunnels show as connected.

# Exclude IPsec flows from FastTrack
# Place BEFORE any existing fasttrack-connection rule
/ip firewall filter
add chain=forward action=fasttrack-connection \
    connection-state=established,related \
    ipsec-policy=in,none \
    comment="FastTrack — exclude inbound IPsec flows"
add chain=forward action=fasttrack-connection \
    connection-state=established,related \
    ipsec-policy=out,none \
    comment="FastTrack — exclude outbound IPsec flows"

If you have a masquerade rule (e.g., for internet access), it will mangle BGP keepalives and tunnel traffic before they reach the IPsec encrypt step. Add a NAT bypass:

# NAT bypass — must be placed BEFORE the masquerade rule
/ip firewall nat
add chain=srcnat action=accept \
    dst-address=169.254.0.0/16 \
    comment="NAT bypass — GCP APIPA BGP traffic"
add chain=srcnat action=accept \
    src-address=<YOUR_LAN_PREFIX> dst-address=<GCP_VPC_PREFIX> \
    comment="NAT bypass — GCP VPC traffic"

Rule order matters. Both the FastTrack exclusion and NAT bypass rules must appear before their respective catch-all rules. Use /ip firewall filter move and /ip firewall nat move to reorder if needed.

Stage 4: Route Advertisement Filters

Control what you advertise to GCP and what you accept from GCP:

# Outbound: advertise only your on-premises prefix(es) to GCP
/routing filter rule
add chain=gcp-bgp-out \
    rule="if (dst == 10.0.0.0/8) { accept }"
add chain=gcp-bgp-out \
    rule="reject"

# Inbound: accept GCP VPC routes, reject default route
/routing filter rule
add chain=gcp-bgp-in \
    rule="if (dst == 0.0.0.0/0) { reject }"
add chain=gcp-bgp-in \
    rule="accept"

Adjust the prefix in gcp-bgp-out to your actual on-premises network. It is good practice to reject the default route inbound unless you explicitly want GCP to provide internet exit.

:::warning RouterOS v7 route advertisement change In RouterOS v7, BGP does not automatically advertise connected or static routes. You must explicitly tell the BGP connection which prefixes to originate using output.network:

# Add your prefix(es) to an address list
/ip firewall address-list
add list=gcp-advertised-prefixes address=10.0.1.0/24 comment="LAN prefix"

# Reference it on the BGP connection (both tunnel connections)
/routing bgp connection
set [find name=gcp-cr-tunnel0] output.network=gcp-advertised-prefixes
set [find name=gcp-cr-tunnel1] output.network=gcp-advertised-prefixes

Routes not in the output.network address list will never appear in gcp-bgp-out regardless of what the filter accepts. This is the most common cause of “BGP is established but GCP receives no routes”. :::

Stage 5: MTU and MSS Clamping

GCP HA VPN tunnels reduce the effective PMTU. ESP-in-UDP encapsulation (NAT-T) typically leaves ~1350 bytes of usable payload. Without MSS clamping, large TCP sessions (HTTPS, SSH bulk transfers) stall silently while small packets work fine — making this look like a random connectivity issue.

Add a mangle rule to clamp TCP MSS on traffic traversing the tunnel:

/ip firewall mangle
add chain=forward action=change-mss \
    new-mss=clamp-to-pmtu \
    passthrough=yes \
    protocol=tcp \
    tcp-flags=syn \
    dst-address=<GCP_VPC_PREFIX> \
    comment="MSS clamp — GCP VPN tunnel"
add chain=forward action=change-mss \
    new-mss=clamp-to-pmtu \
    passthrough=yes \
    protocol=tcp \
    tcp-flags=syn \
    src-address=<GCP_VPC_PREFIX> \
    comment="MSS clamp — GCP VPN tunnel return"

Replace <GCP_VPC_PREFIX> with your GCP VPC subnet(s). clamp-to-pmtu automatically uses the interface MTU rather than a hardcoded value.

Stage 6: Dual-Tunnel Path Preference (Optional)

By default, RouterOS treats both HA VPN tunnels as equal-cost. To designate a primary and a backup tunnel, influence path selection with Local Preference (inbound) and AS-PATH prepending (outbound).

RouterOS v7 routing filters have no matcher for which local BGP address received a route, so per-tunnel policy requires separate filter chains for each connection. Update each BGP connection to reference its own chains:

# Update connections to use per-tunnel filter chains
/routing bgp connection
set [find name=gcp-cr-tunnel0] \
    output.filter-chain=gcp-bgp-out-t0 \
    input.filter=gcp-bgp-in-t0
set [find name=gcp-cr-tunnel1] \
    output.filter-chain=gcp-bgp-out-t1 \
    input.filter=gcp-bgp-in-t1

# Tunnel 0 inbound — higher local-pref = preferred path
/routing filter rule
add chain=gcp-bgp-in-t0 rule="if (dst == 0.0.0.0/0) { reject }"
add chain=gcp-bgp-in-t0 rule="set bgp-local-pref 200; accept"

# Tunnel 1 inbound — lower local-pref = backup path
/routing filter rule
add chain=gcp-bgp-in-t1 rule="if (dst == 0.0.0.0/0) { reject }"
add chain=gcp-bgp-in-t1 rule="set bgp-local-pref 100; accept"

# Tunnel 0 outbound — advertise without prepend (shorter AS-PATH = preferred by GCP)
/routing filter rule
add chain=gcp-bgp-out-t0 rule="if (dst == 10.0.0.0/8) { accept }"
add chain=gcp-bgp-out-t0 rule="reject"

# Tunnel 1 outbound — prepend AS twice (longer AS-PATH = backup path for GCP)
/routing filter rule
add chain=gcp-bgp-out-t1 rule="if (dst == 10.0.0.0/8) { set bgp-path-prepend 2; accept }"
add chain=gcp-bgp-out-t1 rule="reject"

Note

GCP Cloud Router’s BGP policy support is limited — it does not support arbitrary community rewrite or route-reflector topologies. Local Preference and AS-PATH prepending are the primary levers available for path selection across HA VPN tunnels.

---

Verification

Check IPsec Tunnel State

# Active peers — both GCP tunnel IPs should appear
/ip ipsec active-peers print detail

# Installed SAs — expect one inbound and one outbound per peer
/ip ipsec installed-sa print detail

Both tunnels should show state: established. If a peer is missing, the IKE negotiation failed (check the log for cipher mismatches).

Check BGP Session State

# Session state — both sessions should reach "established"
/routing bgp session print detail

# Expected output for a healthy session:
#   state: established
#   uptime: ...
#   prefix-count: ...

Verify Routes Received from GCP

# Routes learned from Cloud Router
/ip route print where bgp

# Prefixes received per session
/routing bgp advertisements print where session=gcp-cr-tunnel0

You should see GCP VPC subnets (e.g., 10.128.0.0/20 for us-central1) in the routing table with the APIPA next-hop.

Verify Routes Advertised to GCP

/routing bgp session print detail where name=gcp-cr-tunnel0

Check output-prefixes — it should match the number of prefixes your output filter accepts.

To inspect the exact prefixes being sent (disabled by default in v7):

# Enable advertisement tracking on both connections
/routing bgp connection
set [find name=gcp-cr-tunnel0] output.keep-sent-attributes=yes
set [find name=gcp-cr-tunnel1] output.keep-sent-attributes=yes

# Dump what was last advertised
/routing bgp advertisements print where session=gcp-cr-tunnel0

Note

keep-sent-attributes=yes stores the last set of advertisements in memory. Without it, /routing bgp advertisements print returns nothing even when routes are being sent.

End-to-End Connectivity

# Ping a GCP VM using its internal IP
/tool ping 10.128.0.x count=5

# Confirm the path goes through the BGP-learned route
/ip route print where dst-address=10.128.0.0/20

---

Troubleshooting

BGP Session Drops When IPsec Re-Keys

This is the most common issue: the IPsec SA expires and re-keys, and during the brief window the BGP TCP session receives no acknowledgement — the hold timer expires, tearing down the session. After re-key, BGP re-establishes but routes are withdrawn and re-added, causing traffic disruption.

Diagnosis: Correlate IPsec and BGP log events:

/system logging add topics=ipsec,!packet
/system logging add topics=route,bgp
/log print where topics~"ipsec|bgp"

Look for phase2 established entries followed closely by bgp session ... closed.

Fixes:

1. Tighten keepalive/hold timers (already done above with 10s/30s). Ensures BGP detects the issue faster and re-establishes sooner.
2. Match IPsec lifetimes between RouterOS and GCP. GCP HA VPN defaults: IKE phase-1 = 36000s (10h), ESP phase-2 = 10800s (3h). Set the same in the RouterOS proposal:

   /ip ipsec proposal set [find name=gcp-esp] lifetime=3h
   /ip ipsec profile set [find name=gcp-ike2] lifetime=10h

3. Verify traffic selectors include APIPA addresses. If the IPsec policy does not cover 169.254.x.x, BGP TCP is sent unencrypted and GCP drops it.

Session Stuck in Active

BGP is trying to open a TCP connection but failing. Check:

# Is the APIPA address reachable?
/tool ping 169.254.1.1 src-address=169.254.1.2 count=3

# Is the IPsec policy matching?
/ip ipsec policy print detail

If the ping fails, the IPsec tunnel is not established or the policy is not matching. Re-check the IKE profile and peer address.

IPsec Established But No Traffic (BGP or Data)

This is the most reliably confusing failure mode: /ip ipsec active-peers print shows the peer as established, yet BGP drops and pings through the tunnel fail. Multiple root causes produce identical symptoms:

1. FastTrack bypassing IPsec encrypt (most common)

If FastTrack is enabled without the IPsec exclusion from Stage 3, forwarded packets are accelerated past the encryption policy. The SA is valid, but packets leave unencrypted and GCP drops them.

Verify FastTrack is the cause — temporarily disable it:

/ip firewall filter set [find action=fasttrack-connection] disabled=yes

If traffic resumes, add the separate ipsec-policy=in,none and ipsec-policy=out,none FastTrack exclusions from Stage 3 and re-enable FastTrack.

2. Masquerade rewriting BGP source addresses

A srcnat masquerade rule firing before IPsec rewrites the packet source, so the resulting traffic no longer matches the IPsec policy selectors. Add the NAT bypass from Stage 3 before the masquerade rule.

3. SPI mismatch after a failed re-key

Clear and reset:

/ip ipsec installed-sa flush
/ip ipsec active-peers print

4. Traffic selector mismatch

GCP may use a wildcard selector (0.0.0.0/0) internally. RouterOS policy must match the actual traffic flows. If you added a LAN-to-VPC tunnel policy, confirm it matches in both directions.

Check policy hit counters — a policy with 0 bytes matched is not being hit:

/ip ipsec policy print stats

5. DPD triggering premature teardown

If DPD probes are failing due to asymmetric routing, increase dpd-interval or disable DPD (dpd-interval=disable-dpd) temporarily to isolate the cause.

One Tunnel Keeps Dropping While the Other is Stable

With dual HA VPN tunnels, RouterOS may egress BGP traffic via the wrong tunnel if routing is ambiguous. Keepalives sent on tunnel 0 arrive at GCP from tunnel 1’s SA — GCP considers them invalid and closes the session.

Cause: RouterOS resolves the APIPA next-hop via the routing table. If both 169.254.x.x addresses are on the same loopback bridge, and no explicit routing marks distinguish the tunnels, the kernel may choose either interface.

Fix: Ensure each BGP connection uses a unique local.address that is bound exclusively to the corresponding IPsec policy. The IPsec policies defined in Stage 1 already enforce this — verify that each policy covers exactly one APIPA pair:

/ip ipsec policy print detail where peer=gcp-tunnel0
# Confirm: src-address=169.254.1.2/32, dst-address=169.254.1.1/32 only

If the loopback has both APIPA addresses, RouterOS may still choose the wrong source address when the routing table resolves tunnel 1’s APIPA via the same interface. Consider using two separate loopback bridges (one per tunnel) to make each address unambiguous:

/interface bridge
add name=lo-gcp-t0 comment="loopback tunnel 0 APIPA"
add name=lo-gcp-t1 comment="loopback tunnel 1 APIPA"

/ip address
add address=169.254.1.2/30 interface=lo-gcp-t0
add address=169.254.2.2/30 interface=lo-gcp-t1

Routes Learned but Not Installed

/ip route print where bgp and !active

If BGP routes are received but not active, a more-specific static or connected route may be preferred. Check:

/ip route print where dst-address=10.128.0.0/20

If a static route wins, remove it or lower BGP distance to 10 on the connection.

Route Filter Applied but Has No Effect

In RouterOS v7, filters must be explicitly attached to the correct BGP connection fields (input.filter for inbound policy and output.filter-chain for outbound policy). Attaching a filter to the wrong field silently has no effect.

Verify the chains are attached to the correct fields. If you enabled Stage 6, each tunnel should point to its own per-tunnel chains:

/routing bgp connection print detail where name=gcp-cr-tunnel0
# Confirm:
#   input.filter: gcp-bgp-in-t0
#   output.filter-chain: gcp-bgp-out-t0

Also verify the filter chain names match exactly (case-sensitive):

/routing filter rule print where chain~"gcp-bgp"

Automating Recovery After SA Rekey

When an IPsec SA re-keys and the BGP session does not automatically recover, the fastest fix is to cycle the BGP connection so it re-establishes a fresh TCP session over the new SA. Doing this manually works but requires monitoring. Use Netwatch to automate it:

# Netwatch — detect when tunnel 0 BGP peer is unreachable and cycle the connection
/tool netwatch
add host=169.254.1.1 interval=15s timeout=3s \
    up-script="/routing bgp connection enable [find name=gcp-cr-tunnel0]" \
    down-script="/routing bgp connection disable [find name=gcp-cr-tunnel0]" \
    comment="GCP tunnel0 BGP recovery"

add host=169.254.2.1 interval=15s timeout=3s \
    up-script="/routing bgp connection enable [find name=gcp-cr-tunnel1]" \
    down-script="/routing bgp connection disable [find name=gcp-cr-tunnel1]" \
    comment="GCP tunnel1 BGP recovery"

When the APIPA next-hop becomes unreachable (SA dropped), Netwatch disables the BGP connection, stopping retransmission storms. When the SA re-establishes and the ping recovers, Netwatch re-enables the connection for a clean TCP handshake.

Note

The APIPA ping only succeeds if the IPsec policy is matched and the SA is active. A successful 169.254.x.x ping confirms end-to-end IPsec liveness, making it a reliable trigger for BGP connection cycling.

Alternatively, flush stale SAs immediately after detecting a stuck session:

/ip ipsec installed-sa flush
# BGP sessions will re-establish once new SAs are negotiated (~5-10s)

This is the same manual fix — automating it removes the need for intervention.

Both Tunnels Down Simultaneously

GCP HA VPN guarantees 99.99% availability only when both tunnels are configured and operational. If both are down:

1. Check GCP Cloud Console → VPN → Tunnel status.
2. Verify your router’s public IP has not changed (common with ISP DHCP).
3. Check for firewall rules blocking UDP/500 and UDP/4500 from GCP’s external IPs.

/ip firewall filter print where dst-port=500 or dst-port=4500

---

See Also

• BGP Peering: eBGP and iBGP Configuration — RouterOS BGP fundamentals
• BGP Route Filtering with Communities — fine-grained prefix control
• IPsec — RouterOS IPsec reference
• CHR: Installing on Google Cloud Platform — running RouterOS CHR as a GCP VM