Before and after managed context: Building AI agents that actually scale

The core problem: Context at scale

Every AI agent story starts the same way (but most of them end earlier than expected):

Stage 1: One document

Give a frontier model a single document and it shines. To be fair to the state-of-the-art: on a single document, today’s defaults are genuinely good. Upload a PDF to a frontier model API and it doesn’t just extract the text, it also renders each page as an image and sends both to the model. Charts, layouts, and stamps are actually seen. This is how demos are built and where expectations get set, since the answers are sharp and the agent appears intelligent. 

Stage 2: One large, complex document

Things start breaking. That page-by-page visual mode has hard limitations: page limits, size limits, a cost that repeats on every question. Past them, processing degrades: charts stop being parsed, sections get skipped, a definition on page 3 gives meaning to a table on page 40 and nothing connects them anymore. Answers become partially correct. This is worse than being wrong, because partial looks complete.

Stage 3: Many documents

Now the agent must understand relationships between sources: cross-references, shared entities, which version supersedes which. A figure proposed in one document and finalized in another is one story to a human, but two facts the default stack must reason over. Retrieval may find one without the other, conflicting information gets treated as equally valid, and the same entity can appear under different names without being reconciled. The agent has access to the information, but not necessarily the context needed to understand how it fits together.

Stage 4: Enterprise scale

• Multiple types of unstructured data: PDFs, email threads, support conversations, call transcripts, CRM notes, contracts, tickets.
• Multiple departments with their own vocabulary.
• Multiple agents reading the same data for different tasks

The same information gets processed again and again, token consumption grows, latency increases, and different agents can reach different conclusions from the same underlying data. At this point, the problem is no longer whether the model can understand the information. It’s whether the enterprise can give every model and agent the same complete, consistent, governed context at scale.

But it isn’t only the data that escalates. The questions escalate too. While “What was the Baltic Dry Index in Week 25?” works at scale “List every vessel sold to Greek buyers across all reports” breaks at Stage 3. It’s not the model, it’s the question type that changed: from point lookup to enumeration, aggregation, reconciliation, timeline assembly. Scale breaks agents on two axes at once: more information to understand, and more complex reasoning required across it.

Raw data isn’t context

Organizations assume that if an agent has access to the data, it has the context. But access and understanding are not the same thing.

Raw business data is fragmented: one insurance claim’s story is spread across two case files, an adjuster’s summary, and fourteen months of email between the parties. It’s repetitive and aliased: the same payment appears in three places under three different labels: is that one payment or three? It’s written in a dialect the model has never been taught: your industry’s acronyms, your internal terms, your shorthand. And a surprising amount of it is trapped in layout: dense tables and charts that don’t survive text extraction.

To be precise about what modern agents do: they don’t literally re-read raw files on every question. A well-built agent parses documents once with the tools it has (PDF to text or markdown) and reads from those parsed artifacts afterwards. But what gets persisted in that step is a format conversion, not understanding. The parsed mirror is flat text: charts degraded or gone, table structure flattened, relationships between sources represented nowhere.

So every question still forces the agent to re-derive the semantic work (which sources matter, how entities connect across them, which figure supersedes which, whether two mentions are the same deal) and that understanding is thrown away the moment the answer is delivered. The next question starts from zero. The next agent starts from zero.

Without managed context, an agent persists the bytes once but re-derives the understanding every time it’s asked a question.

A connector doesn’t fix this. A connector gives the agent a door. Managed context tells it which room matters, what is true inside it, and what it is allowed to touch.

Why the obvious fixes fall short

1. “Just use a bigger context window.”

Cost then scales with corpus size on every question, recall degrades in the middle of long inputs, and a window is still not a join. After all, reconciling a figure across three sources requires a relationship, not adjacency. Re-reading the library is a strategy, just not one you can afford per query.

2. “Just add more connectors.”

Connectors multiply doors. They do nothing about what’s behind them: the fragmentation, the aliasing, the domain dialect, the trapped layout. More access to raw data is more raw data.

3. “The default preprocessing pipeline will handle it.”

Once a document exceeds the single-document limit, the system may switch modes behind the scenes. Instead of sending the complete document directly to the model, it may extract the text, divide it into chunks, create embeddings, and use an opaque retrieval mechanism to decide what the model sees. Users have little visibility into when this switch happens, how retrieval works, or what information is lost, including page visuals, table structure, and relationships between sections.

This makes performance difficult to predict and debug. A document that works today could behave differently after a platform update changes the underlying preprocessing or retrieval logic. A managed context layer provides a more reliable alternative: a visible, versioned representation of the data that can be inspected, tested, and debugged. These failures are rarely caused by model quality. They occur because the representation given to the model has already lost essential meaning. The first lever is therefore not the model, it is the context.

Three things drop out in that switch. The page images are gone: the index is text-only, so the charts the model could see at Stage 1 are now invisible. Table structure is gone: rows and headers are flattened into flowing text. And relationships were never there to begin with: the index holds independent fragments with a filename and a page number attached. The only notion of “related” left is similar in embedding space.

Context as AI infrastructure

Managed context means transforming business information (documents, emails, conversations, transcripts, notes, tickets) into representations designed for AI reasoning. This is the way feature stores once transformed raw data into something ML models could actually consume. Instead of exposing agents to raw data, you provide context that is:

1. Semantically organized: Grouped by meaning, not by file or channel boundaries.
2. Self-contained where possible: A unit of context carries what’s needed to interpret it.
3. Linked where necessary: Cross-references and relationships are explicit, not implied.
4. Enriched: With metadata, entities, and your domain’s vocabulary resolved up front.
5. Scoped: Small, curated, task-relevant datasets instead of one giant index.
6. Queryable: Inherently tabular facts live as typed, structured data an agent can query with completeness guarantees, not fragments it can only sample.
7. Managed and versioned over time: Context changes; agents should never observe a half-updated world. New context becomes visible all-or-nothing.

How to build context agents can reason over

• Model the source, don’t just chunk it. For documents: sections, tables, charts, footnotes. These are typed elements that preserve reading order and references. For conversations and email: threads, participants, and turns, and not undifferentiated text. The page-3 definition stays attached to the page-40 table; the reply stays attached to the question it answers.
• Extract the tabular truth. Entities, amounts, dates, parties pulled into structured tables, whether they came from a broker report, an email negotiation, or a support call. Enumeration and aggregation stop being search problems and become queries, with completeness by construction.
• Link like a knowledge graph, organize like a tree. Explicit relationships for cross-references and cross-source identity: the settlement amount proposed in an email and finalized in a summary document is one thread, not two hits.
• Ground domain terms at ingestion. Industry acronyms and internal vocabulary are resolved once, when the data is processed and not guessed at question time.
• Curate scope per use case. A small dataset that is all relevant beats a huge index that is mostly noise. Structure lets you bring only the right information into scope for each task.
• Index the evidence. Every answer should be traceable to the exact rows, passages, and messages that produced it. Search returns passages; managed context assembles evidence.
• Preserve contradictions instead of resolving them with their scope and authority attached.
• Version atomically. Each refresh of the context is a generation; agents switch from one complete world to the next, never a mix.

Consider what this means for a real case: an insurance claim whose truth lives in two case files, an adjuster’s summary, and fourteen months of email. Managed context is what makes “the $3.8M in the email and the $3.8M in the summary are the same payment, labeled differently” a stored fact rather than something every agent must rediscover, or miss.

Managed context is necessary. How your agent accesses it decides the rest.

Fixing the context is the first lever. The second is the architecture your agent uses to consume it.

We tested this on a corpus of weekly shipping-market reports, publicly available from Intermodal and Star Asia. We ran the same six questions across three setups, all on frontier models at medium effort, with repeated runs and every enumeration answer verified by hand against the source PDFs:

• Frontier agent over raw documents: The strongest baseline: a state-of-the-art agent pointed at the folder of PDFs. No retrieval, no managed context.
• RAG over managed context: Top-k retrieval over high-quality context units, one-shot answer.
• MDP: An agent over managed context: it plans, queries the structured tables, follows links, and checks its own coverage before answering.

Start with the first column, which is the strongest alternative. Its problem isn’t only accuracy; it’s that you can’t count on it. On a corpus of a few dozen documents, an agent can still brute-force the problem. In its best runs it spent minutes reading everything and answered almost perfectly. Re-run the same questions and it answers in a few seconds from a single report, returning two Turkish sales out of six. Same agent, same files, but different sessions, different answers. It parses the files once, but what it parses is flat text. In all our runs, the agent never once looked at a chart and the semantic work of deciding what to read and how far to dig is redone on every question, with nothing guaranteeing coverage and nothing signaling which kind of run you got.

And this is the easy setting: in our broader tests, as the corpus grows, especially with many similar-but-different documents, like consecutive editions of the same report, the plain agent’s accuracy and run-to-run robustness degrade further, while MDP stays consistent even in those adversarial scenarios.

RAG’s weakness is architectural rather than stochastic. It’s genuinely good at point lookups, but on distributed questions – “list all”, “how many”, “what total” – it hits a ceiling that no context quality can remove. The model answers from the k retrieved units, and the system can’t know what it didn’t retrieve. Two failure modes follow:

• Partial answers. The Greek-buyers list came back with five vessels. It looked complete; nothing signals that four are missing. A business user ships that number.
• Confident false absence. “The provided documents do not contain information regarding vessels beached at Alang”, while 45 vessels and 435,694 MT sit in the corpus.

MDP’s results were the consistent ones. The same answers run after run, with a single wobble in one of five runs. That’s not the model trying harder; it’s the architecture. Completeness comes from querying tables extracted once, not from how much reading the model happens to do today. And when MDP does miss (the fourth VLCC), the miss is traceable to a row in an extracted table. Fix the extraction once, and every future question benefits. A raw-documents agent’s miss has nothing to fix; you can only re-run and hope.

Both patterns belong in the same platform, chosen per question: retrieval for point lookups and semantic questions in seconds; the agentic pattern for enumeration, aggregation, reconciliation, and timelines in minutes. One context, built once, giving the same answer tomorrow that it gave today.

What changes when context is managed

• Accuracy and consistency on the questions that matter. Not just point lookups but the enumeration, aggregation, and timeline questions your analysts actually ask.
• Lower cost per question. You pay for structuring once, at ingestion, and amortize it over every future query, instead of re-deriving (and re-paying for) understanding each time.
• No reprocessing. The same managed context serves every agent, every access pattern (retrieval and agentic), and every use case downstream.
• Debuggability. When an answer is wrong, you can see which rows, passages, and messages produced it. Not “trust me, it was in chunk 7.”
• More deterministic outputs. The same query over the same tables returns the same rows.
• Scalability. Adding sources grows your tables and your graph. It doesn’t grow your failure modes.

And one thing weak systems can’t do at all: handle disagreement. Two sources can disagree without either being universally wrong. Finance and Sales define “revenue” differently. A global policy and a regional policy coexist. The CRM reflects the official account status while Slack reflects what the team currently believes. A contract defines the rule; an approved exception changes its application. A weak system silently selects one result. A managed context system preserves the disagreement, its scope, and its authority, and lets the agent reason about it.

The trade-offs: Structuring costs compute at ingestion. Agentic answering costs minutes on hard questions. And for a one-off corpus with purely semantic questions and nothing that repeats enough to deserve a schema, a plain retrieval setup is a fine choice. The trade is worth it exactly where businesses live: recurring data, recurring questions, answers people act on.

Wrap up

Before: an agent with a door to your data, re-deriving its understanding from scratch on every question: fast, confident, and silently incomplete.

After: context managed once so it is organized, linked, enriched, queryable. And an agent that chooses how to work it: retrieval when you need to point at a fact, agentic querying when you need every fact, with evidence attached either way.

The difference between a demo and a system was never the model. It’s the context you give it and what you let the agent do with it.

Flexor turns your documents, emails, and conversations into managed context your agents can actually reason over. See it on your own data.