Tool use and structured generation

A decoder can restrict each next-token distribution to tokens allowed by a grammar and the already generated prefix. This can make malformed outputs less likely or prevent them within the supported grammar, but compatibility depends on the implementation and supported schema features. JSON syntax is only the first layer. A syntactically valid object can omit a necessary field, contain an impossible date, or name a resource that does not exist. Keep each kind of validation visible rather than calling every failure a formatting issue.

JSON Schema can describe types, required fields, enumerated choices, and other structural constraints. Merely listing a property does not require its presence; the schema must declare that requirement. Reject or explicitly handle unknown fields according to the interface contract. Cross-field business rules may need ordinary code, such as ensuring an end time follows a start time. A schema-valid amount can still exceed an account's limit, and a schema-valid identifier can still refer to another user's private resource.

Expose a narrow tool registry with names, argument types, bounded result sizes, and explicit error responses. Parse the model output into typed application data, validate it, and perform authorization using trusted user and workload identity. Do not let a model-supplied user identifier determine whose permissions apply. Retrieved text and tool responses can contain adversarial instructions; carry them as untrusted observations rather than allowing them to redefine the application's available capabilities or permission checks.

Read-only search and a payment operation have different failure consequences. For side effects, use the confirmation or approval policy required by the application and scope each operation to a resource and amount. Assign idempotency keys and persist execution state so a network timeout does not cause an unbounded duplicate action. A timeout can mean the result is unknown, not that the action failed. Reconcile with the authoritative service before retrying an operation that may already have committed.

Toolformer studies learning when and how to use tools, but a learned calling policy does not replace an application's correctness boundary. Evaluate argument accuracy, authorized execution, task completion, and recovery from unavailable tools. A tool invocation can return HTTP success while producing irrelevant search results. Conversely, a deliberate refusal to perform an unauthorized request can be a successful system outcome. Define expected behavior for each case before assigning scores.

Log a trace with proposal, validation result, authorized operation, execution identifier, result summary, and final response. Redact secrets and private content according to the logging policy. Give errors a bounded, machine-readable form so the model can correct a field without seeing a stack trace or internal credentials. Limit attempts, tokens, elapsed time, and tool response size. When the budget is exhausted, expose a clear unresolved state rather than fabricating a successful result from a partially completed workflow.