bgp-troubleshooting

name	BGP Troubleshooting
description	BGP session state, missing routes, prefix policy, default-originate, next-hop-self, route reflector — decision trees and checklists

PREREQUISITE: Before using this skill, you MUST have already run get_interfaces(device) and get_bgp(device, "summary") on the target device. If sessions are not Established, go directly to the Session Checklist below — do not read the full skill from the top. If all sessions are Established, proceed with the relevant symptom section.

BGP Troubleshooting Skill

Scope

BGP session state, missing routes, prefix policy, default-originate, next-hop-self, route reflector. Covers: AS1010 (E1C, E2C), AS2020 (X1C), AS4040 (IAN), AS5050 (IBN).

All BGP sessions in the current topology are eBGP. The iBGP/Route Reflector section below is included as generic troubleshooting guidance for completeness.

Defaults: Keepalive 60s · Hold 180s · AD 20 (eBGP) / 200 (iBGP) · Weight 0 (received routes) / 32768 (locally originated)

Start Here: Session State

When a BGP neighbor is not Established:

get_bgp(device, "summary")

State	Root Cause	Fix
Established + count	Healthy	—
Idle	TCP failing or admin shutdown	Check reachability, check neighbor X shutdown
Connect	TCP SYN in progress	Reachability issue — peer unreachable or TCP 179 filtered
Active	TCP connect failed, retrying / listening for incoming	Check ACL on TCP 179, check update-source, ebgp-multihop for non-direct eBGP
OpenSent	OPEN sent, waiting reply	Usually transient, else hold-timer mismatch
OpenConfirm	OPEN exchanged, awaiting KEEPALIVE	Usually transient — if stuck, check hold-timer or capability negotiation failure
Idle (Admin)	neighbor X shutdown configured	Remove shutdown

Session Checklist

Run get_bgp(device, "config") and verify in order. Stop at the first mismatch — fix it first, then re-verify. If the issue persists after the fix, start a new investigation cycle for any remaining problems. If config output is ambiguous (e.g., no explicit timers shown but sessions are flapping), run get_bgp(device, "neighbors", neighbor=<ip>) to see negotiated timers directly.

Note: This checklist checks AS numbers before timers, which differs from CLAUDE.md Principle 6's generic ordering (timers before AS). In BGP, AS mismatch causes immediate OPEN rejection — it is more fundamental than a timer mismatch, which is negotiated after OPEN exchange.

1. AS numbers correct? (local-as on each side matches the peer's remote-as)
2. Keepalive/hold timers match on both ends? Defaults: keepalive=60s, hold=180s. Mismatch → root cause found, stop here. Fix direction: restore the non-standard side to defaults — never change a correctly-configured peer to match a misconfigured one. neighbors shows actual negotiated values — if they differ from defaults, check config for explicit timers statements on the local device. Trust the operational data; do not dismiss non-default negotiated values as parser artifacts.
3. Neighbor IP reachable? ping(device, neighbor_ip, source=update_source_ip) from the configured source address.
4. update-source set correctly? Required for iBGP sessions using Loopback addresses — verify both sides agree on source IP. (Not applicable in current topology — all sessions are eBGP with physical interface peering.)
5. ebgp-multihop configured? Required for non-directly-connected eBGP peers. Missing → stuck in Active.
6. Authentication match? MD5 password (key and type) must match on both sides if configured.

Query Reference

Query	Returns	When to use
summary	Neighbor list, state, uptime, prefixes received	First check — verify all sessions are Established and prefix counts non-zero
table	Full BGP table with path attributes (next-hop, AS-path, local-pref, MED, origin, weight)	Route missing, wrong best path selected, or next-hop investigation
config	BGP process config, neighbors, route-maps, prefix-lists, address-family, RR config	Verify neighbor config, policies, route-reflector-client, update-source
neighbors	Per-neighbor detail: negotiated timers, capabilities, address families, reset reasons	Timer mismatch diagnosis, capability issues, session flapping investigation. Use neighbor=<ip> on IOS to scope to a single peer.

---

Symptom: Session Established but Zero Prefixes

When get_bgp(device, "summary") shows a session as Established but with 0 prefixes received:

1. Address-family activation: Check get_bgp(device, "config") for neighbor <ip> activate under address-family ipv4 unicast. On IOS-XE, neighbors defined under the global router bgp process need explicit activation under the address-family to exchange IPv4 prefixes.
2. Outbound filter on peer: The remote peer may have an outbound route-map or prefix-list denying all prefixes. Check get_routing_policies(peer_device, "route_maps") on the sending side.
3. No routes to advertise: The peer may have no routes matching its network statements or redistribution config. Check get_bgp(peer_device, "config") and get_bgp(peer_device, "table").

---

Symptom: Session Flapping / Reset Reasons

When get_bgp(device, "neighbors", neighbor=<ip>) shows repeated resets:

Reset Reason	Root Cause	Action
Hold Timer Expired	Keepalives not arriving in time — aggressive timers, congested link, or CPU overload	Check timers in config — non-default values (e.g., keepalive 3 / hold 9) make sessions fragile on any jitter
Notification received (cease)	Peer sent administrative notification	Check peer for neighbor X shutdown or max-prefix limit reached
Peer closed session	TCP reset from peer	Check peer's BGP process health and interface stability
No route to peer	IGP route to peer's update-source lost	Check IGP adjacencies and get_routing(device, prefix=<peer_ip>)

Key diagnostic: Compare "hold time" (negotiated) vs "Configured hold time" in neighbors output. If the negotiated hold time is under 30 seconds, any network jitter will cause flapping. Restore to defaults (keepalive 60 / hold 180) on the non-standard side.

---

Symptom: Missing Routes

When a route should be in the BGP table but isn't:

get_bgp(device, "table")
get_bgp(device, "config")

Route Presence Checklist

1. Route in BGP table? (> = best, * = valid, = not valid/no best path)
2. Next-hop reachable? → get_bgp(device, "table") — inspect the next-hop field. If the next-hop is an eBGP peer's IP (external address), iBGP peers may have no IGP route to it. Fix: apply neighbor X next-hop-self on the eBGP edge router so it rewrites the next-hop to its own interface before advertising to iBGP peers. Verify reachability: get_routing(peer_device, prefix=<next_hop_ip>).
3. Outbound policy blocking advertisement? → get_routing_policies(device, "route_maps") and get_routing_policies(device, "prefix_lists") — check for deny clauses on the sending device. Remember: route-maps have an implicit deny at the end — any prefix not explicitly permitted is dropped.
4. network statement matching exact route in RIB? (must be exact — not aggregate unless aggregate-address is configured)
5. redistribute configured? Check metric-type and route-map filters
6. Route in RIB but not in BGP? → next-hop unreachable, route not valid (* missing from table output)
7. Own AS in received AS_PATH? BGP silently discards any route whose AS_PATH contains the receiving router's own AS number (loop prevention, RFC 4271 §9.1.2). Symptom: route visible in the sending peer's table but absent from yours — no NOTIFICATION is sent. Check get_bgp(device, "table") for the prefix; if absent, confirm via the peer that its path does not traverse your AS.
8. Well-known community blocking propagation? A route tagged NO_EXPORT (0xFFFFFF01) MUST NOT be advertised outside the AS; NO_ADVERTISE (0xFFFFFF02) MUST NOT be advertised to any peer. Either community causes a route to be silently withheld from outbound updates — route is present locally but invisible to peers. Check communities in get_bgp(device, "table") output for the affected prefix (RFC 1997 §3).

---

Symptom: Default Route Missing

When ISP should be sending default but it's missing:

get_bgp(device, "table")    → look for 0.0.0.0/0 with a valid best path (>)
get_bgp(device, "summary")  → confirm session to ISP peer is Established

Diagnosis

• Default missing: ISP may have default-originate only conditionally (e.g., default-originate route-map) — check ISP config
• ISP policy: ISPs in this topology filter customer-advertised defaults inbound (they send but don't accept)
• Peer IPs: Get ISP peer IPs from INTENT.json → each edge router has different ISP-facing IPs

---

Symptom: iBGP Routes Not Propagating (Route Reflector)

Generic guidance — not applicable to the current topology (all sessions are eBGP).

When iBGP routes are missing on a peer:

get_bgp(device, "summary")    → verify iBGP session to RR is Established
get_bgp(rr_device, "config")  → verify peer has route-reflector-client configured
get_bgp(device, "table")      → check if route exists but isn't best path

RR Checklist

• iBGP routes not propagating → confirm RR has neighbor X route-reflector-client for all clients
• Next-hop still set to originator's IP (not RR) → client needs next-hop-self or IGP must reach originator
• Check RR cluster-id if multiple RRs exist — RRs intended to be in the same cluster must share the same cluster-id (RFC 4456 §8). Different cluster-ids create separate clusters: inter-cluster loop detection does not apply, risking route duplication or suboptimal paths

---

Symptom: Wrong Best Path Selected

When a route exists but the wrong next-hop is being used:

get_bgp(device, "table")      → see all paths for a prefix
get_bgp(device, "config")     → check for weight, local-pref, or AS-path manipulation

Best Path Selection (11-attribute order)

1. Highest Weight (Cisco-local, not advertised)
2. Highest Local Preference (iBGP, default 100)
3. Locally originated (network/redistribute/aggregate)
4. Shortest AS-path
5. Lowest Origin (IGP < EGP < Incomplete)
6. Lowest MED (same AS neighbor only — RFC 4271 §9.1.2: MED is only compared between routes received from the same neighboring AS; routes from different ASes are not MED-compared unless bgp always-compare-med is configured)
7. eBGP over iBGP
8. Lowest IGP metric to next-hop
9. Oldest eBGP route
10. Lowest BGP Router ID
11. Lowest neighbor IP

---

Verification Checklist (Post-Fix)

• get_bgp(device, "summary") shows all neighbors as Established
• get_bgp(device, "table") shows expected prefixes with > (best path)
• Default route 0.0.0.0/0 present from all ISP peers on each customer edge router (E1C, E2C, X1C)
• get_routing(device) shows BGP default route in RIB
• After any policy change (route-map, prefix-list): policy changes do NOT take effect on existing routes without a soft reset. The clear ip bgp command is in the FORBIDDEN set and cannot be executed via MCP tools — advise the operator to run clear ip bgp <neighbor> soft in or clear ip bgp <neighbor> soft out manually on the device.

---

References: RFC 4271 (BGP-4, FSM §8, best path §9.1.2, NEXT_HOP §5.1.3, MED scope §9.1.2) · RFC 4456 (BGP Route Reflection, cluster-id §8) · RFC 4760 (Multiprotocol Extensions) · RFC 1997 (BGP Communities, well-known §3)