Resilience

Most agents fail silently on the first API error. Chapter 9 gives each failure a specific recovery path — and only gives up after exhausting every option.

The Big Picture

Chapter 9 handles failures in the LLM API itself — rate limits, auth errors, timeouts, context overflow.

Solution: a 3-Step Retry Chain. When one step fails, the error moves outward to the next recovery step.

STEP 1: Auth Rotation
  [primary-key] → [backup-key] → [emergency-key]
  key hits error → cooldown → try next key
       │
STEP 2: Overflow Recovery
  context too long?
  → truncate tool results → compact history
       │
STEP 3: Tool-Use Loop
  standard while True + stop_reason (same as Chapter 1–8)
       │
  success → return
  failure → classify → route to the appropriate recovery step

Three Key Concepts

1. classify_failure() — Route Exceptions to the Right Layer

type FailureReason int

const (
    Unknown   FailureReason = iota
    RateLimit
    Auth
    Timeout
    Billing
    Overflow
)

func classifyFailure(err error) FailureReason {
    // LLM APIs don't always return typed errors — parsing the message string is the most reliable approach across providers
    msg := strings.ToLower(err.Error())
    switch {
    case strings.Contains(msg, "rate") || strings.Contains(msg, "429"):   return RateLimit
    case strings.Contains(msg, "auth") || strings.Contains(msg, "401"):   return Auth
    case strings.Contains(msg, "timeout"):                                 return Timeout
    case strings.Contains(msg, "billing") || strings.Contains(msg, "402"): return Billing
    case strings.Contains(msg, "context") || strings.Contains(msg, "token"): return Overflow
    default:                                                               return Unknown
    }
}

2. ProfileManager — Key Rotation with Cooldown

func selectProfile(profiles []*Profile) *Profile {
    now := time.Now()
    for _, p := range profiles {
        if !now.Before(p.CooldownUntil) { // now >= CooldownUntil → key is available
            return p // first available key
        }
    }
    return nil // all on cooldown → exhausted → fallback models
}

3. Retry Limits — Not Infinite

maxRetries := min(max(24+8*nProfiles, 32), 160)
// scales with number of API keys — more keys = more retries worth attempting before giving up

There’s a hard cap. If all retries are exhausted → raise the error up. Never loop forever.

Multi-Provider Tiered Fallback

For maximum resilience, configure fallback across providers:

Tier 1: Anthropic Claude    → primary, best quality
    ↓ rate limit / balance exhausted
Tier 2: Gemini / OpenRouter → cheaper, still cloud
    ↓ all cloud APIs down
Tier 3: Ollama (local)      → free, always available

Quality drops per tier. Responses from smaller or local models may be less accurate or miss nuance — they keep the agent running, but with reduced capability. For agents requiring strong reasoning, keep the primary provider as the preferred choice. Local models are a last resort, not a replacement.

Context Overflow Recovery (in depth)

Chapter 3 (ContextGuard) and Chapter 9 handle overflow at different points in time:

Attempt 1: normal call
    ↓ overflow error
Attempt 2: truncate oversized tool results
    ↓ still failing
Attempt 3: compact — LLM summarizes 50% of history
    ↓ still failing
→ route to Step 1 (try a different key/model)

Chapter 3’s ContextGuard tries to prevent overflow before it happens. Chapter 9’s recovery step handles the case where the API already returned an overflow error — it’s the reactive fallback.

Takeaway

• 3-Step Retry Chain: auth rotation → overflow recovery → tool-use loop
• classify_failure() — router — different errors handled differently
• Key rotation + cooldown — graceful degradation when one key has issues
• Always have a retry limit — no infinite loops
• Multi-provider fallback — last line of defense when all keys are exhausted

Cross-References

• Chapter 3 (Context Management) — ContextGuard proactive overflow prevention
• Chapter 8 (Delivery Guarantees) — write-ahead queue that pairs with resilience on restarts
• Chapter 1 (Tool-Use Loop) — the inner loop that Step 3 wraps