← All posts
EngineeringOctober 11, 2026

Routing one conversation across many APIs

How Relay keeps full context intact while fanning a single thread out to models with wildly different context windows.

One thread, six providers, six different APIs, six different context windows. The routing layer's job is to make that invisible.

Every model receives the same conversation shaped for what it handles best: names prefixed so models know who said what, history trimmed oldest-first when context runs short, and the user's latest message always intact.

Streaming adds another wrinkle — each provider streams differently, so Relay normalizes everything into one event format before it reaches your screen.

Boring infrastructure, magical result: it just feels like one conversation.