Chapter 7 (Autonomy) introduced background agents that can send messages. The problem: if the process crashes while sending → the message is lost forever. Chapter 8 (Delivery) solves this with a write-ahead queue (write the message to disk before attempting to send it).

The Big Picture

The flow has two stages: first the message is split into chunks and written to disk, then a background runner picks it up and sends it.

Agent reply / Heartbeat / Cron
       │
chunk_message() ← split to platform limits
       │
DeliveryQueue.enqueue()
  1. Write to .tmp file
  2. fsync() → data on disk
  3. os.replace() → atomic  ← WRITE-AHEAD
       │
DeliveryRunner (scans every 1s)
  ├── success → ack() → delete file
  └── failure → fail() → backoff → retry
                  └── 5x failed → failed/

Note: os.Rename() is the Go equivalent — atomic rename/replace on POSIX systems.

The core rule: write to disk first, then send.

Three Key Concepts

1. Atomic Write — Crash-Safe

tmpPath   := filepath.Join(queueDir, fmt.Sprintf(".tmp.%d.%s.json", os.Getpid(), id))
finalPath := filepath.Join(queueDir, fmt.Sprintf("%s.json", id))

f, _ := os.Create(tmpPath)
f.Write(data)
f.Sync()              // force OS to flush to physical disk — not just memory
f.Close()
os.Rename(tmpPath, finalPath) // atomic rename
Crash moment Result
Before fsync Orphan .tmp file, harmless
Before replace Same
During os.replace Atomic on POSIX — either old or new file, never partial

After a crash, the pending files in the queue directory survive. The runner picks them up on restart.

2. Retry + Exponential Backoff + Jitter

Retry 1: 5 seconds   ± 20%
Retry 2: 25 seconds  ± 20%
Retry 3: 2 minutes   ± 20%
Retry 4: 10 minutes  ± 20%
Retry 5: → failed/

Without jitter, all retries fire at the same moment — flooding a service that’s already under stress. Adding a random offset spreads them out.

3. Recovery Scan — Messages Survive Crashes

func (r *DeliveryRunner) Start() {
    r.recoveryScan()   // check for pending files from before crash
    go r.run()
}

Messages already enqueued before a crash → automatically processed on restart. No manual intervention.

What Chunking Is For

Different platforms have different message length limits (Telegram: 4096 chars, Discord: 2000 chars). The chunk_message() step splits long responses before enqueuing, so each delivery attempt stays within limits.

Chunking should be format-aware — don’t split in the middle of a code block or sentence. A split at the wrong place breaks readability on the receiving end.

Takeaway

  • Write-ahead — write to disk first, then send → crash-safe
  • Atomic write — tmp + fsync + os.Rename() → file is never partially written
  • Exponential backoff + jitter — smart retry, prevents thundering herd
  • Recovery scan — on startup, always check for pending messages from before crash
  • After Chapter 8 (Delivery), the agent loop is reliable end-to-end