AI Expert

Guides/Vendor & routing

Vendor & routing strategy

One provider is simplest until outages, SKU gaps, or unit economics force a router layer. Treat routing as a maintained product: parity evals, backoff logic, and contract SLAs per backend.

Decision flow

Deciding single vendor versus multi-provider routingNeed resilience / failover?YesNoMulti-provider routerhealth checks + backoffStill need price arbitrage?Tiered routingcheap model ↔ escalation SKUContracting: latency SLOs, data residency,identical API surface — ops burden explodeswithout glue-layer ownership.Single vendor works until outages or SKU gapshurt revenue — then invest in observability-first routing.
Treat the router as a product — versioning prompts per backend matters as much as picking models.

Common drivers

Why routePatternTradeoff
Outage / regional unavailableAutomatic failover to mirror SKU or degraded mode messageMust reconcile eval parity across vendors
Cost vs quality tiersRoute simple intents to small model; escalate on confidenceNeeds intent router eval — misroutes annoy power users
Capability gap (e.g. vision)Secondary provider only for multimodal pathsTwo billing relationships + latency variance

Cost of sophistication

Relative burdens — size against your on-call rotation and ML platform maturity.

Integration vs ongoing ops burden (schematic)

Budget headcount for parity testing whenever two backends answer the same PRD prompts.