Sessions & Context Guard

Without session persistence, every restart wipes the conversation. This chapter adds the two components that fix that: a store that survives restarts, and a guard that prevents the LLM from choking on a conversation that’s grown too long.

The Big Picture

In Chapter 1–2 (Tool Use & Structured Output), restarting the program wipes the conversation. Chapter 3 adds two layers:

• SessionStore — persist messages[] to disk (JSONL)
• ContextGuard — protect against context window overflow

Start
  │
  ▼
load_session() ── read JSONL ──> rebuild messages[]
  │
  ▼
[LOOP] User input
  │
  ▼
guard_api_call()
  ├── Attempt 1: normal
  ├── Attempt 2: truncate tool results
  └── Attempt 3: compact history (LLM summary)
  │
  ▼
save_turn() ── append to JSONL ──> disk
  │
  ▼
[end_turn] Print

Three Key Concepts

1. SessionStore — JSONL Append-Only

One event = one line of JSON:

{"type":"user",       "content":"hello",          "ts":1234}
{"type":"assistant",  "content":[...],             "ts":1235}
{"type":"tool_use",   "name":"bash", "input":{...},"ts":1236}
{"type":"tool_result","content":"output...",       "ts":1236}

Why append-only?

• Never rewrites the file → can’t get corrupted
• Restore = read all lines from top, rebuild messages[]

Analogy: A notebook where you can only write at the bottom — never erase or edit.

2. ContextGuard — 3-Stage Overflow Protection

Every LLM has a maximum amount of text it can process in one call, measured in tokens (roughly 1 token per word). This limit is called the context window.

LLMs have token limits. If messages[] gets too long → API error.

Attempt 1: normal call
    ↓ fails (overflow)
Attempt 2: truncate oversized tool results
    ↓ still fails
Attempt 3: compact — LLM summarizes 50% of old history
    ↓ still fails
raise error

This all happens inside a single guard function. You call it instead of calling the API directly — it handles the fallback chain for you.

Stage 3 — LLM summarizes old history:

compacted := []map[string]string{
    {"role": "user",      "content": "[Summary] " + summary},
    // summary is generated by asking the LLM: "Summarize this conversation in 3-5 sentences"
    {"role": "assistant", "content": "Understood."},
    // ... recent messages kept intact
}

3. Short-Term vs Persistent Memory

JSONL file        — persistent memory (on disk)
    ↓
load_session()    — read JSONL, rebuild messages[]
    ↓
messages[]        — short-term memory (lives in RAM, lost on restart)

When a session starts, messages[] is not empty — it’s already populated from JSONL. This is how production agents “remember” previous conversations.

Where to Store Sessions

For most personal or small-team agents, JSONL is sufficient and keeps things simple.

If you’re building an agent used by many users simultaneously, you’ll need a shared store so all instances can access the same history. For a personal agent or small team, JSONL on disk is plenty.

Takeaway

• JSONL — one event per line, append-only, corruption-resistant
• load_session() — how persistent memory enters messages[]
• ContextGuard — 3-stage safety net when context fills up
• Compact — LLM summarizes old history, keeps context lean