Architecting the Internet of Agents

IoA Needs Interoperability, Not More Silos

Internet of Agents Stack Reference Architecture

It’s probably unsurprising to regular readers of this Substack that Agentic AI is evolving fast, but without shared protocols for communication, tool use, memory, and trust, these systems will remain siloed. To unlock their full potential, we must build an open, interoperable stack — an Internet of Agents.

Today’s agents can reason, plan, and act autonomously within bounded environments. But real-world tasks cut across platforms and domains. That demands agents who can collaborate, delegate, and share knowledge across organisational and technical boundaries. Like the early internet, we’re missing the protocols, and without them, scale and composability will stall.

---

The Interop Problem: Closed Agents, Fragmented Ecosystems

AI agents operate efficiently within their native ecosystems, like Microsoft, NVIDIA, OpenAI, Meta, Google, ServiceNow, Salesforce, or frameworks such as LangChain, PydanticAI, or LlamaIndex, but their integrations across ecosystems and platforms remain costly, brittle, and non-scalable. Their toolchains, memory models, comms protocols, and access layers are tightly coupled to their execution environments. There's no shared grammar for agents to delegate tasks, invoke tools, or share memory across boundaries.

We're in the pre-TCP/IP era of agents: powerful local nodes, no scalable network.

---

Enabling Cross-Agent Collaboration: Core Interop Dimensions

Interoperability isn’t a single spec – it’s a stack. Here are the key architectural dimensions where we need standards or shared patterns:

1. Tool Use & Invocation
   Standardised, agent-aware tool interfaces built upon existing protocols like OpenAPI, JSON-RPC, or GraphQL. This could involve agent-centric enhancements to enable reasoning about tool semantics, capability descriptions, and dynamic binding at runtime.
   Example: Model Context Protocol (Anthropic), Agent Communication Protocol (IBM/BeeAI)

2. Agent Communication & Messaging
   A protocol layer for agent-to-agent messaging, task delegation, and negotiation. Needs standard message schemas, status codes, and handshake patterns.
   Example: AGNTCY (Cisco, LangChain, LlamaIndex, glean, Galileo), Open Voice Interoperability (Linux AI & Data Foundation)

3. Authentication, Identity & Trust
   Decentralised identities (such as W3C’s DID standard) could provide signed claims, robust access control mechanisms, and Zero Trust-inspired frameworks. This area remains highly experimental but crucial.

4. Memory & Context Sharing
   Shared embeddings, vector memory formats, and cross-agent context windows. Could involve portable context objects or standardised episodic memory APIs.

   Example: Letta’s stateful agents

5. Knowledge Exchange & Inference APIs
   Protocol extensions beyond RAG to enable multi-agent collaborative inference, where agents transparently query each other’s knowledge bases, exchanging reasoning steps and provenance details.

6. Economic Transactions
   Agents will require standardised frameworks for economic interactions, such as micropayment systems, metered billing APIs, escrow services, and agent reputation scores.
   Example: Stripe’s Agent Toolkit is a great prototype.

7. Governance & Policy Compliance
   Embeddable policy engines (e.g. Rego/OPA-style) that allow agents to reason over compliance constraints, licenses, and regulations. Crucial for cross-domain agents.

8. Discovery & Capability Matching
   Registries, schemas, and semantic matching for agents to find and collaborate with others dynamically. Think DNS + service metadata for agents.

9. Error Recovery & Conflict Resolution
   Shared patterns for fallbacks, retry semantics, conflict resolution (e.g. optimistic concurrency control across agents), and escalation paths.

   Still highly experimental.

---

Why This Matters

Without shared abstractions, the agent ecosystem will remain fragmented. AI engineers will be stuck writing brittle integrations, agents will hit capability ceilings, and the vision of general-purpose autonomous systems will stall.

Building this stack is not optional – it’s the prerequisite for scalable, composable, autonomous systems.

Some foundations are being laid. But the real work – defining agent-native protocols, runtime boundaries, state management models, and trust layers – is just beginning.

The Internet of Agents won’t be built by accident.
It’ll be engineered by us.