MCP’s token problem is real.
Traditional Model Context Protocol implementations are bleeding money. Before your AI agent even starts solving a problem, it’s already burned through 55,000 tokens just describing which APIs exist. At Anthropic, some enterprise setups hit 134,000 tokens of pure overhead. That’s not efficiency. That’s a tax on every single request.
The problem is stupidly simple: the system loads every tool definition upfront, regardless of whether the agent will actually use it. All 58 tools from GitHub, Slack, Sentry, Grafana, and Splunk get dumped into the model’s context window as massive JSON payloads. Most of them are irrelevant to the current task. None of this matters.
“Traditional MCP implementations often inject large JSON payloads into the model context, which increases token consumption and reduces efficiency.”
This is where Code Mode enters. And it changes the equation entirely.
What’s Actually Different About Code Mode?
Code Mode doesn’t load tool definitions upfront. Instead, it lets the model generate code that calls tools on demand. The LLM searches a registry of available APIs, pulls the schema only for what it needs, writes Python code to invoke the right endpoint, and executes that code in a sandboxed environment. The result comes back. Done.
The efficiency gain is obvious: no context bloat, no hallucination risk from irrelevant tool descriptions, and dramatically lower token consumption. But the real insight nobody’s talking about? This approach trades context window size for execution intelligence. The model isn’t just describing what it could do—it’s actually doing it.
And that requires a sandbox.
Why You Can’t Just Execute LLM Code Directly
Here’s where the acerbic reality hits. Letting an AI model generate arbitrary Python and run it on your production server is a fast track to compromise. File access. Network misuse. Privilege escalation. System takeover.
OpenSandbox—Alibaba’s open-source platform now listed in the CNCF Landscape—solves this by creating an isolated execution environment. The generated Python code runs inside a container with restricted filesystem access, network controls, resource limits, and process isolation. The sandbox acts as a moat between the model’s intentions and your actual infrastructure.
This isn’t paranoia. It’s architecture.
The flow looks like this: startup discovers all available OpenAPI specs and loads them into a registry. Request arrives. System searches for relevant tools by metadata. The LLM inspects the selected tool’s schema via get_schema. The model generates Python code that correctly invokes the endpoint. That code gets sent to the sandbox through execute. The sandbox runs it in isolation, handles the HTTP request to the actual system, and returns the raw result. The LLM converts that into a human-readable response.
Three core tools make it work: search, get_schema, and execute. That’s it.
Is This Actually Better Than Traditional MCP?
Yes. But with caveats.
For enterprises with hundreds of APIs and massive tool registries, Code Mode eliminates the token tax. A 90% reduction in context overhead isn’t theoretical—it’s what happens when you stop preloading every single tool definition. At scale, that’s a real cost savings and faster inference.
But here’s what won’t make it into Anthropic’s marketing slides: Code Mode introduces latency. An extra round trip to the sandbox, code generation, execution, and result parsing takes time. For latency-sensitive applications, traditional MCP—bloated as it is—might still be faster if you’re hitting the same tools repeatedly.
Also, not every environment needs this level of optimization. If you’re running a narrow set of APIs (say, five tools consuming 15K tokens total), the engineering complexity of sandboxing and dynamic tool invocation might not be worth it.
The Bigger Picture: Context Efficiency as a Competency
What’s interesting is that this isn’t just MCP optimization. It’s a pattern. As models get bigger and token windows expand, the temptation is to throw everything into context. Anthropic is essentially saying: stop doing that. Be intentional about what the model sees.
Code Mode forces that intentionality. You can’t lazy-load 100 tool definitions anymore. You have to think about discovery, relevance, and what the model actually needs to solve the problem at hand.
This matters because context window size is a vanity metric. Real efficiency is about signal-to-noise ratio. And Code Mode improves that dramatically.
For .NET and C# developers implementing this in enterprise settings (which the original author has been researching), the pattern is worth studying. The underlying principle—generate executable code instead of injecting static definitions—scales beyond APIs. It could reshape how agents interact with databases, infrastructure, and internal tools.
The OpenSandbox Question
One last thing: OpenSandbox is relatively new to most developers. It’s solid (CNCF-approved, multi-language SDKs, Docker/Kubernetes support), but adoption isn’t mainstream yet. If you’re implementing Code Mode in production, you’re betting on a platform that’s still building out its ecosystem.
That’s not a dealbreaker. It’s just a reality check.
The win here is real: MCP without the token waste, tool calling that’s actually executable, and a sandbox pattern that doesn’t sacrifice security for speed. But implementation requires more infrastructure than traditional MCP. It’s the right solution for the wrong problem if you’re not hitting the token overhead issue yourself.
🧬 Related Insights
- Read more: Your Access Tokens Are Probably Broken (And Nobody’s Telling You)
- Read more: Azure Kubernetes Service: Why Your Cost Optimization Strategy Is Probably Broken
Frequently Asked Questions
Does Code Mode work with every API? As long as the API has an OpenAPI specification and is accessible via HTTP, Code Mode can discover, schema-inspect, and invoke it. The sandbox needs network egress rules configured to reach your target systems.
Will Code Mode replace my existing MCP setup? Not necessarily. If your tool registry is small and token consumption isn’t a bottleneck, migrating to Code Mode adds complexity without benefit. Evaluate based on actual token overhead and latency requirements.
Is OpenSandbox production-ready? Yes—it’s in the CNCF Landscape and supports enterprise deployment on Docker/Kubernetes. But ecosystem maturity and community support aren’t at the level of mainstream tools yet.
