Multi-Agent Architectures That Share a Web Retrieval Layer

The thesis is simple: once multiple agents pull from the same web retrieval layer, the decisions made at that layer, how it caches, ranks, ages out stale content, and shapes what gets handed back, set the ceiling for every agent that depends on it. Get the shared layer wrong and no amount of prompt engineering downstream fixes it. Get it right and the whole system improves at once, because a gain at the retrieval layer is a gain multiplied across every agent that touches it.

What multi-agent systems look like in production and where retrieval fits

The 2026 agent stack, as described in the analysis on codingwithroby.substack.com, runs six layers deep: Agent Surface, then Orchestration/Runtime, then Memory, then Knowledge/RAG, then Tools/MCP, then Models/Inference, with Observability and Governance wrapped around the whole thing. Shared web retrieval sits at layer four. It's below the orchestration layer that decides what gets done and above the model layer that does the reasoning. Retrieval isn't bolted onto an agent anymore, it's infrastructure that sits between planning and inference.

In production, this usually plays out as a planner agent breaking a task into pieces, then handing those pieces to specialist agents that execute them. LangGraph, which hit version 1.0 in October 2025, runs this pattern at scale at companies including Uber, LinkedIn, and Klarna. On the communication side, MCP handles how agents call tools, while A2A (which absorbed the older ACP protocol in August 2025) handles agent-to-agent coordination and has since spread to more than 150 supporting organizations. Most teams still build their actual coordination logic at the framework layer, not by hand-rolling protocols.

Here's where it gets expensive without a shared layer: each specialist agent, left to its own devices, fires off its own web fetches. Asking five agents to research overlapping parts of the same problem produces five sets of near-duplicate search calls, five different sets of context, and five different bills. ROMA, a 2026 architecture out of arXiv research, breaks large tasks into subtask trees that run in parallel across agents; in that shape of system, a shared retrieval layer is the only way to avoid redundant fetches ballooning cost and latency.

A shared retrieval layer is not a shared memory store, a distinction that trips up a lot of designs. Retrieval is the live pipeline that fetches from the web and assembles context on demand. Memory, the kind handled by systems like Zep or Mnemoverse, is what persists across sessions, the durable record of what happened before. Conflating the two leads teams to build a single blob that does neither job well.

The retrieval quality problem every agent in the system inherits

Production data cited by datarmatics.com shows naive vector search fails on roughly 40% of real-world queries. That means in a simple RAG setup, nearly half the retrieval calls hand back context that doesn't actually answer the question. One agent eating that failure rate is a bad answer. A whole population of agents drawing from the same underbuilt layer means that 40% failure rate is on the entire system at once, not on some fraction of it.

Most retrieval tooling wasn't built for this job. Traditional SERP APIs return titles, URLs, and snippets of maybe 150 to 300 characters, which is plenty for an SEO analyst tracking where a page ranks, but nowhere near enough for a model trying to reason about the actual content of a page.

The opposite failure mode, dumping full scraped pages into the context window, isn't better. It buries the signal in navigation menus, ad copy, and boilerplate the model has to wade through, which both degrades the quality of the answer and burns tokens on text nobody needed.

Then there's the propagation issue: when agents share a retrieval or memory layer, a single stale or low-quality entry doesn't stay contained. The same 2026 agent stack research that maps out the six-layer architecture also flags this directly: when agents share a retrieval or memory layer, a single poisoned entry doesn't stay contained, it spreads to every agent that queries that store. The fix isn't more retrieval, it's governed retrieval, meaning provenance tracking and validation built into the layer itself rather than left to each agent to sort out on its own.

A second, subtler failure appears specifically at the population level. Research on Multi-Agent Transactive Memory (MATM), out of Carnegie Mellon with a UC Berkeley co-author (arXiv:2606.19911, June 2026), points out that agent-generated trajectories, the record of how an agent actually solved something, typically get thrown away after one use or kept only by the agent that produced them. New agents spun up later end up re-solving problems that were already solved, because nothing preserved the work. That's a retrieval design failure just as much as a bad chunking strategy is.

Put together, these failures don't add up across a multi-agent system. They multiply, because every agent's reasoning depends on every other agent's retrieval quality feeding into a shared context.

Why agentic RAG changes what a shared retrieval layer must do

Traditional RAG runs a fixed pipeline: retrieve once, generate once, done. Agentic RAG hands the agent control over that process. The agent decides which tool fits the moment, whether that's keyword search, semantic search, an API call, or a database query. It decides when to retrieve, before answering, mid-generation, or as a verification step after a draft answer. And it decides whether what came back is actually good enough, looping back to retrieve again if not.

2026 research on the topic frames this as a real shift, not a minor upgrade: RAG moves from a passive pipeline into something closer to an active agent that plans its own retrieval, inspects the evidence it gets back, and checks its own work. Multi-hop RAG is the clearest expression of this, breaking a query into pieces and chasing them down sequentially or in branches, aggregating what comes back. In practice, each additional hop adds latency, so teams typically cap the number of retrieval rounds before returns diminish.

For a shared layer, this changes the job description. It's no longer enough to answer a uniform batch of fetches on a schedule. The layer has to handle agent-driven retrieval that varies in shape and timing, often several agents hitting it at once with different intents, and it has to hold latency and quality steady under that load. That means the layer needs to expose enough structure, metadata about source, freshness, and relevance, for agents to direct their own retrieval intelligently rather than getting handed a pile of raw results and left to sort it out.

This is happening while RAG adoption itself climbs fast. Enterprise designs incorporating RAG rose to 51%, up from 31% the year before, and search volume tied to production RAG systems grew more than 60% year over year through 2025 and into 2026. The shift toward agentic patterns isn't happening in a lab, it's happening on top of systems that are already scaling hard.

The technical decisions at the shared retrieval layer that determine every agent's ceiling

Hybrid retrieval is the baseline now, not an optimization. Dense-only vector search has lost on nearly every public benchmark run in 2024 and 2025, including BEIR, MTEB, and Anthropic's contextual retrieval work; combining BM25 keyword search with dense embeddings through Reciprocal Rank Fusion consistently beats either one alone. The combination improves recall by up to 17% over dense-only search while adding less than 6 milliseconds of latency, which is a trade worth making at a shared layer, since a recall failure there doesn't cost one agent, it costs all of them. Stacking a cross-encoder reranker on top adds another 5 to 15 points of MRR to hard retrieval sets.

Chunking decisions made once at ingestion carry through the whole system. Semantic chunking, splitting text along meaning rather than a fixed character count, consistently outperforms fixed-size chunking on retrieval benchmarks. Anthropic's contextual retrieval approach goes a step further, attaching document-level context to each chunk before it gets embedded, and reports 49% fewer retrieval failures against naive chunking. Both of these are decisions made exactly once, at the shared layer, that then compound with everything built on top of them.

Reranking has gotten cheap enough that skipping it is the harder decision to defend. Cross-encoder reranking API calls run around $0.025 to $0.050 per million tokens and can lift top-k chunk quality by 20 to 30%. Running that once per retrieval call at the shared layer lets every agent downstream benefit from the same rerank, instead of each agent paying for its own.

Caching is where the multi-agent case actually pays for itself. Agents researching overlapping problems tend to issue queries that are semantically close even when the wording differs, and a shared layer that catches that with semantic caching (matching on intent rather than exact text) avoids redundant fetches across the whole population. A cache that's fast but stale poisons every agent that hits it, so cache invalidation, TTLs and re-fetch triggers, needs to be treated as a core piece of the design, not something patched in later.

Freshness matters beyond caching, too. A model's training data is a snapshot; it can't stand in for current information, and a shared retrieval layer exists specifically to bridge that gap. The stakes are higher here than in a single-agent setup, because one stale fetch at the shared layer contaminates every agent that reads it. Zep, in active development as of mid-2026, tackles a related version of this problem for memory with a temporal knowledge graph that tracks when a fact became true and when it stopped being true. The same idea belongs in retrieval: provenance and time-validity aren't nice extras, they're infrastructure.

Then there's the question of how content gets shaped once it's retrieved. Research on long-context models shows attention isn't distributed evenly across a long context window, models pay noticeably more attention to the beginning and end than to the middle. Anthropic's own guidance on long-context usage tells developers to put the most important information at the start or end for exactly this reason. A shared layer that just dumps raw content back leaves every agent to solve this positioning problem on its own, inconsistently. A layer that returns ranked, structured content solves it once, in the infrastructure, where it belongs. Context windows have grown large across leading models, but that hasn't erased the need for careful retrieval. It's changed the trade-off from "what fits" to "what's worth retrieving versus what should just sit in context."

Embedding and vector store choices that belong at the shared layer, not in each agent

The embedding model is a single-decision, whole-system choice. OpenAI's text-embedding-3-large scores 64.6 on MTEB at roughly $0.13 per million tokens and is the well-supported default across most tooling. The smaller text-embedding-3-small costs about a sixth as much and holds up fine when paired with a reranker. Alibaba's Qwen3-Embedding-8B currently tops the MTEB multilingual leaderboard at 70.58, and Cohere's Embed v4 along with Voyage's retrieval-tuned models hold their own for multimodal or domain-specific work.

What matters for a multi-agent system is that this choice gets made once, centrally. If different agents embed with different models, their retrieved chunks live in different vector spaces and simply aren't comparable, which breaks the entire premise of a shared layer.

Vector store choice follows scale, not preference. Under a million vectors on an existing PostgreSQL stack, pgvector is the practical answer. Under 100,000 vectors for prototyping, Qdrant or ChromaDB fit. At production scale with managed operations, Pinecone or Weaviate are the common picks. Whatever the choice, a sparse index needs to sit alongside it, Elasticsearch, OpenSearch, or even a lightweight BM25 index over the same chunk set, because that's what makes hybrid retrieval possible. Skipping the sparse index means the 17% recall gain hybrid search offers is not available to the system.

Graph RAG is a pattern worth watching as it matures. Microsoft Research's GraphRAG, published by Edge et al. in 2024 under an MIT license, builds knowledge graphs out of documents so a system can reason across entity relationships rather than just matching on semantic similarity. That's relevant wherever a multi-agent system needs to trace how things relate to each other, not just find text that sounds similar.

Federated RAG covers the edge case where centralizing knowledge isn't an option at all, whether for privacy, security, or infrastructure reasons. The "shared layer" still exists in this 2026 direction, but it's distributed across boundaries without ever pooling the raw data in one place.

How a shared retrieval layer enables agents to learn from each other's work

The MATM research out of Carnegie Mellon and UC Berkeley (arXiv:2606.19911, June 2026) makes a case that's easy to overlook: agent trajectories, the actual record of how an agent solved a task, are reusable procedural knowledge. Most systems throw that knowledge away after a single use or lock it inside the agent that generated it. MATM proposes storing these trajectories at the population level instead: producer agents contribute what they've done, consumer agents retrieve it later to skip work that's already been solved.

The results, tested on ALFWorld and WebArena, showed that agents retrieving from a shared memory store of this kind improved task performance and needed fewer interaction steps, without any coordination or joint training required between the agents involved. Search engines built their value by indexing what humans produced so other humans could find it. MATM applies the same logic to agents, indexing what agents produce so other agents can find and reuse it. That's retrieval infrastructure built for AI consumers rather than retrofitted from something built for people.

A related 2026 architecture, CORAL, builds on this by running long-lived multi-agent systems that improve themselves over time through shared, persistent memory and asynchronous execution. Reported improvement rates run 3 to 10 times higher than fixed evolutionary-search baselines, and the gain comes from the population-level store of trajectories, not any single agent getting smarter on its own.

The design implication is straightforward, if easy to miss when a team is focused on shipping the first version: a shared retrieval layer shouldn't just be built to serve the agents running today. Built well, it accumulates value as more agents run through it and leave more solved work behind. At that point it functions as an asset the whole system draws down over time, not just a query endpoint.

Security and governance requirements that arise specifically from shared retrieval

Shared retrieval concentrates risk the same way it concentrates value. A single poisoned or degraded entry in a shared store doesn't stay contained to whichever agent's fetch introduced it, it spreads to every agent that queries that store afterward. That means the attack surface of a shared retrieval layer isn't just its own surface, it's effectively the combined attack surface of every agent sitting downstream of it.

The NSA published security design guidance in 2026 covering AI-driven automation that uses MCP, aimed at exactly this category of risk: systems where agents call tools and retrieve information through a shared protocol layer, and where a compromise upstream can ripple through everything connected to it. The practical response isn't to treat retrieval as a convenience layer that can be secured later. It has to be governed from the start, with provenance tracked on what gets stored, validation applied before content enters the shared pool, and clear boundaries on which agents can write to the store versus which can only read from it. A retrieval layer shared across a growing population of agents is, functionally, a piece of core infrastructure, and it needs the same discipline applied to any other system that everything else depends on.