When marketers ask an LLM to generate a series of blog posts, create a full‑site audit, or orchestrate an Odoo‑to‑WordPress content pipeline, the intuitive fix is to use a model with a larger token window. Real‑world studies show this approach quickly hits diminishing returns. Attention in a transformer scales quadratically (n² pairwise relationships), so each added token eats into a finite “attention budget.” As the window grows, recall degrades, especially for the earliest instructions that define the overall goal.
In practice, a typical long‑horizon task may involve 50+ tool calls—API queries, file reads, or code edits—while the input‑to‑output token ratio can exceed 100:1. Every observation stays in the prompt, pushing the original goal toward the middle of the window where the model’s recall is weakest. The result is not a model bug; it is an inevitable outcome of unmanaged context.
Mechanism 1 – Context Budgeting and Offloading
The first line of defense is a harness that decides what never enters the prompt. Deep‑Agents, for example, offloads any tool response larger than 20,000 tokens to disk and replaces it with a file path plus a short preview. When the accumulated session context reaches 85 % of the model’s window, older write and edit calls are truncated to pointers, preserving only the reference to the already‑saved file. Only after these rules are exhausted does the system fall back to summarization.
Claude Code follows a similar budget: it caps auto‑memory at the first 200 lines (≈25 KB) and defers full tool schemas until they are explicitly requested. After compaction, any re‑read file over 5,000 tokens is inserted as a path reference rather than raw content. This architecture mirrors a sub‑agent pattern where a coordinator spawns isolated research agents; each sub‑agent explores large data sets in its own window and returns a concise, structured result to the main workflow.
Mechanism 2 – Smart Compaction
When offloading alone cannot keep the window under control, the harness performs a targeted compaction. Compaction means summarizing the near‑full conversation into a new, smaller context and restarting the loop with that summary. The key is to explicitly name what is retained. Deep‑Agents adds dedicated fields for session intent, artifacts created, and next steps, ensuring the core objective survives the reduction. Claude Code’s compaction prompt preserves architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs, then re‑reads the five most recent files to keep fresh context.
OpenAI’s Responses API offers server‑side compaction with a configurable threshold, returning an opaque, encrypted compaction token that must be passed unchanged into the next call. This makes compaction a first‑class engineering artifact rather than a hidden setting, allowing marketers to maintain a reliable content strategy across long‑running automation runs.
Mechanism 3 – Todo‑State Recitation
Even with perfect compaction, the goal can drift between summarizations. A lightweight way to keep the objective in the model’s recent attention is to maintain a mutable todo.md (or similar) file that is rewritten each turn. Each rewrite appends the current plan to the end of the prompt, effectively reciting the goal on every step. Manus demonstrated that a simple todo list reduces drift on tasks averaging 50 tool calls.
LangChain initially shipped a write_todos tool by default, later making it optional after evaluations showed a modest cost trade‑off. The recommendation remains: enable todo recitation for long, multi‑step content pipelines, especially when using less capable models or when the UI can surface progress to human operators.
Mechanism 4 – Persistent Memory Across Sessions
After a task finishes, useful artifacts—project‑level guidelines, SEO keyword maps, or Odoo integration scripts—should persist for future runs. Claude Code re‑injects a project‑root CLAUDE.md and auto‑memory after every compaction. Amazon Bedrock’s AgentCore stores events and runs background extraction strategies so a coordinator can recall prior findings without re‑searching.
However, persistent memory is not free. An ETH Zurich study found that LLM‑generated context files increase inference cost by 20‑23 % on benchmark tasks. The recommendation is to keep persistent files concise (under 200 lines) and to load heavy reference material only on demand, preserving the attention budget for active content generation.
Key Takeaways for AI‑Powered Content Workflows
Shallow loops overflow and lose goals; the harness—not the model—provides the fix.
Budget first: offload large tool results (>20 K tokens) and truncate old edits once 85 % of the window is used.
Compaction must be explicit about retained information; include intent, artifacts, and next steps.
Todo‑state recitation keeps the goal in recent attention, but it adds token overhead that should be weighed against performance gains.
Persistent memory improves continuity across runs but can raise inference cost; keep files short and load them lazily.
