If you’ve shipped an LLM-powered product, you’ve lived this nightmare. The model streams tokens perfectly. The prose is flowing. Then someone asks for a visualization, and suddenly you’re choosing between three bad options: buffer the entire response (goodbye streaming), parse incomplete JSON (fragile), or train the model on custom syntax (expensive). mdocUI and Markdoc for LLM streaming UI exist because this problem is systemic, unsolved, and—once you see the fix—obvious.
But here’s the thing: the solution isn’t about inventing new tech. It’s about borrowing the right existing patterns and adapting them for a use case their creators never imagined.
The Problem Nobody Talks About (But Everyone Faces)
Let’s be concrete. Your LLM generates this:
Here's your revenue data:
{
"type": "bar",
"labels": ["Q1", "Q2", "Q3"],
"values": [120, 150, 180]
}
That JSON block arrives token-by-token. The first { lands, then more characters trickle in. A traditional parser sees incomplete data—it’s not valid JSON. So you have two moves: wait for the closing brace (kills streaming), or try parsing partial JSON (good luck when you hit syntax errors). Both suck.
Or the LLM tries JSX:
<Chart type="bar" labels={["Q1", "Q2", "Q3"]} values={[120, 150, 180]} />
Except models get confused about when to use JSX props versus HTML attributes. They forget closing tags. The < character appears in normal text, making your parser ambiguous. You end up spending 200 tokens just describing syntax rules, and the model still gets it wrong half the time.
Some teams invent custom DSLs—[[chart:bar:Q1=120]] or similar. Now you’re training the model on a format it’s never seen, burning tokens on instruction overhead, maintaining a parser, and hoping it sticks.
“The opening sequence is something you would never expect in normal prose, standard markdown, or fenced code blocks. A streaming parser can detect it without lookahead or backtracking.”
There’s a better way. It already exists. It’s just not designed for this.
Stripe’s Documentation Tool Wasn’t Built for LLMs—Until Now
Markdoc is a documentation framework Stripe created years ago. It extends markdown with a tag syntax that looks like this:
Here's your revenue data:
[% chart type="bar" labels=["Q1","Q2","Q3"] values=[120,150,180] /%]
Revenue grew 12% quarter-over-quarter.
[% button action="continue" label="Show by region" /%]
This isn’t new. It’s battle-tested. Stripe uses it. Cloudflare uses it. It’s in the training data of modern LLMs. Which means—and this is critical—your model already knows how to write it without instruction overhead.
Three things make this tag syntax perfect for streaming:
First: the [% %] delimiter is unambiguous. You’ll never see it in normal text, markdown, or code blocks. A streaming parser detects it immediately, no lookahead, no backtracking.
Second: models already know it. No custom format to explain. No token budget burned on syntax rules. The model writes it correctly.
Third: prose and components coexist. The LLM writes markdown and drops components wherever they fit. The parser separates them as tokens arrive. No mode switching. Clean.
The Architecture Is Lean
mdocUI borrows Markdoc’s tag syntax—nothing else. The team built their own streaming parser from scratch.
The flow is simple: LLM tokens → Tokenizer → StreamingParser → ComponentRegistry → Renderer.
The tokenizer is a character-by-character state machine. Three states: IN_PROSE, IN_TAG, IN_STRING. As tokens arrive, it separates components from readable text.
The ComponentRegistry validates tag names and props against Zod schemas. A malformed tag doesn’t crash your app—it hits an error boundary instead.
The Renderer maps AST nodes to React components. Every component is theme-neutral (currentColor, inherit, no hardcoded hex values). Swap any component with your own design.
It’s restrained. No bloat. No unnecessary abstraction layers.
Does It Actually Work? (Yes. But It’s Alpha.)
mdocUI ships 24 components out of the box: chart, table, stat, card, grid, tabs, form, button, callout, accordion, progress, badge, image, code-block, and more.
The library is currently at 0.6.x—alpha territory. The API is stabilizing. The team is actively working on Vue, Svelte, and Solid renderers. A Vercel AI SDK bridge is coming. So is a browser devtools for AST inspection and a VS Code extension for syntax highlighting.
Why This Matters (And Why I’m Bullish on the Approach)
Here’s what strikes me: this isn’t a novel invention. It’s disciplined borrowing. The developers looked at an existing, proven tool (Markdoc) and asked: “What if we made this work for streaming?”
That’s how good infrastructure gets built—not by inventing everything, but by taking existing patterns and adapting them for new constraints.
The streaming AI space is full of one-off solutions. Custom parsers for one company’s API. Bespoke renderers. Format creep. What mdocUI does is create a standard that’s already familiar to both models and developers.
If this gets adoption, it could genuinely simplify how teams build generative UI. That’s not hype—that’s just solving a real problem elegantly.
The catch? It’s alpha. The ecosystem is thin. Vue, Svelte, and Solid support are on the roadmap, not shipped. If you need something production-hardened today, mdocUI isn’t it.
But if you’re building something with streaming LLMs and you care about the experience—the lack of buffer freezes, the clean parsing, the fact that your model already knows the syntax—this is worth watching.
What’s Next
The playground is live at https://mdocui.vercel.app. The GitHub repo is open. Docs are thorough. The team is explicitly welcoming feedback and PRs.
If the approach scales, we could see mdocUI become the de facto standard for generative UI in streaming contexts. Not because it’s the most feature-rich. Because it’s the least painful to reason about.
That’s the kind of infrastructure that wins.
🧬 Related Insights
- Read more: A Production Bug Made 73% More Revenue Than Any Feature We Built. Here’s What It Taught Us.
- Read more: Running LLMs on Kubernetes? Your Infrastructure Doesn’t Protect You From Prompt Injection
Frequently Asked Questions
What is Markdoc and why does it matter for LLM streaming?
Markdoc is Stripe’s documentation framework that extends markdown with custom tag syntax. It matters for LLM streaming because the tag format (like [% component ... /%]) is unambiguous, models already know how to write it from training data, and it parses cleanly as tokens arrive—no buffering required.
Can I use mdocUI in production right now? mdocUI is in alpha (0.6.x). The core streaming parser is solid, but the API is still stabilizing and ecosystem support (Vue, Svelte, Solid renderers) is incomplete. It’s safe for early adoption if you’re okay with potential breaking changes. Production deployments should wait for 1.0.
How does mdocUI compare to using JSON or JSX for components?
JSON requires buffering to parse (breaks streaming). JSX confuses models about props vs. attributes and introduces parsing ambiguity around < characters. Markdoc’s tag syntax avoids both problems: it’s unambiguous, models know how to write it, and it parses incrementally as tokens arrive.
