CH 03 · Tool Design · 08/18
Read as one pageBatching
When the model commonly needs N of something, give it a tool that does N at once rather than forcing N separate calls. "Fetch these documents" (a list) beats N× "fetch this document." Batching cuts round-trips, which cuts latency and loop iterations. Cap the batch size in the schema to prevent abuse.
Finished this idea?
Your learning progress stays in this browser.