Guides/Tool & integration design
Tool & integration design for LLMs
Agents fail loudly when integrations behave like one-off scripts. Treat tool calls as API contracts: validate arguments, execute with budgets, and translate failures into responses models can reason about — instead of raw stack traces destined for confused users.
Call lifecycle
Schema & ops checklist
| Design lever | What good looks like | PM smell |
|---|---|---|
| Narrow tool surface | Fewer functions with crisp names beat giant generic “doAnything” RPC | Support tickets cite wrong tool picked — widen docs before widening surface |
| Deterministic args | Enums, bounded strings, max lengths — reject early with actionable messages | Silent truncation causes wrong writes — surface validation to the model path |
| Idempotency tokens | Safe replays on timeouts — critical for payments, tickets, emails | Duplicate side-effects after retries — missing idempotency story |
| Timeout budget | Fail fast; return partial progress markers the model can narrate | Hung threads & angry users — align SLAs with streaming UX guide |
Reliability compounding
Side-effect mishap pressure vs integration hygiene (schematic)
Each layer removes failure modes agents cannot prompt away — especially under retries.